Meeting 2hrs

ok, meeting time monero-project/meta #640

1. greetings

hello

Hi!

Hi

Grüezi

Hi all 🙂

Howdy

hi

2. let's do updates next, what has everyone been working on?

I have mostly been working on BCH projects this past week. Not much to report on Monero.

Dipping my toes into chain analysis, I guess.

me: I worked on vtnerd's review of my multisig address PR, and discussed wfaressuissia's multisig tx construction PR with him (now available here: monero-project/monero #8114 ).

So is this ready?

I did a bit more view tag testing to squeeze out optimal gains as per UkoeHB's feedback on the PR, and got back to some decoy selection algo work

I've been working on making my blockchain analysis simulator widely available: for not only Linux, but also Mac & Windows. Rucknium has reported, that the basic functionality can be executed under his dev OS: Mac. I will need at least half of January, to be able to deliver the remaining pieces of already working code.

On decoy selection work:

- I put together a very raw code snippet that demonstrates a way to implement one component of deterministic ring selection, specifically how to map randomly generated values from a seed into the gamma CDF (or any other distribution). It's a bit raw to share at this point I think, but the needle is moving there.

- Reviewed the PR to stop selecting duplicate outputs across rings when constructing a tx (monero-project/monero #8047)

- Shared intuition behind a parameter choice for how wide bins should be in wallet side binning (I think the narrower the bins, the more effective the bins at hindering timing analysis, but more prone to someone spamming many locked outputs)

I'm also working with a friend to write a CCS proposal for her, since she could do a lot of things 1) Cheaper 2) In parallel

ArticMine: yes, although reviewers should contact myself, luigi1111, wfaressuissia, or selsta if they want more details about the issues fixed in that PR (they will be disclosed publicly sometime after the PR is merged).

Thanks

I am quite at a loss to see how your PR and 8114 relate, or depend on each other, or whether they solve different problems

im currently trying to log on the web version of this because it will be way easier to communicate:currently on my phone

rbrunner: they solve different problems. My PR is for making a multisig wallet, this new PR is for making multisig transactions.

I see. Would it make sense then to test 8114 in a similar way that I tested your PR, just to check whether "everything still works"?

im here to get feedback on my idea I proposed two weeks ago, and follow up regarding a signal processing issue to estimate hadh rate

hash*

rbrunner: yes I think so

Ok, thanks

maxwellsdemon[m]: can you remind us about your idea? for the logs too

jberman[m]: does that PR cause issues if the number of available outputs is too low, so cross-ring duplicates are unavoidable?

im back

had to reset my password...

maxwellsdemon[m]: can you remind us about your idea? for the logs too

sure, the idea was simple: adaptively adjust threadcount while mining to maximize performance metrics, such as CPU/power supply wear and tear or energy management

I got the idea from mining on my laptop, where it quickly rose the temperature of the CPU. I found out pretty quickly it wouldnt be viable unless i regulated the threads using some feedback from the machine

in my case I considered temperature, but power supply draw would also be a consideration - it really comes down to the information available from the sensors that can be installed or utilized on your systenm

Recalling the discussion, it was thought that this would help people who arent "serious" miners, but probably wont be a benefit to people who are more involved unless it can demonstrate improved efficiency

Is this a project you want to work on?

then it was proposed that I look into improving the estimation scheme to ascertain the network hash rate

May I suggest giving an option for an abstract external signal? This would be for example the accumulated power in your solar battery.

In my view a main goal of maxwellsdemon 's proposed work would be to increase the hash rate of hobbyist miners. As we move into tail emission next year, squeezing as much hash rate out of honest hobbyist miners will become increasingly important.

maxwellsdemon[m]: (More accurately: improving the difficulty adjustment algorithm)

... which is quite controversial, because I think many see it as working just well enough to "never touch a running system"

1) an external signal is viable, but the issue with that is you have to have some kind of uniformity in the hardware so the firmware and software are compatible with the control algos, 2) I doubt it would increase hash rate so much as prevent someone from burning out their machine and maybe save some energy, perhaps increasing the energy efficiency, 3) yes youre correct this was to improve the difficult adjustment

algorithm. My understanding is the current method using a moving window average, which has issues with transient oscillations and delays

1) It's as simple for the user as writing a mapping like 80% power -> 40% cores, 40% power -> 20% cores

2) It's better to mine on one core all the night, than 2 cores for half the day and 0 cores in the night, as this is not perfectly scallable

3) Shameless autopromotion - perhaps my simulator will be of some help, once it's ready. I will take this use case into account.

Well, if it increases the energy efficiency that could bring more hash power anyway since mining would become more economically viable for some hobbyist miners -- electricity cost is a concern, of course.

I would also consider focusing this work instead on XMRig as it is always the recommendation for someone caring more about mining.

Or at the very least seeing if the work can be ported to XMRig as well if viable.

my background is in control theory and mathematical modeling, so wherever my skills would be most useful.

maxwellsdemon[m]: I'm a control engineer as well. Mathematical modeling is my hobby, so I'll ping ya :)

lol nice

Yes, I would assume that maxwellsdemon 's software would be bundled with or connected to xmrig since it's the most widely-used, and I think most efficient miner.

<UkoeHB> "jberman: does that PR cause..." <- I imagine yes, it would make it more difficult to construct rings when there are fewer outputs available. Perhaps it makes sense to only prevent duplicates across rings when there are sufficient spendable outputs in the chain

Personally, I doubt that reducing the number of threads would actually increase efficiency. If anything, it would lower it, because hashrate is pretty much directly proportional to the number of threads, but the power consumption isn't - because you have the power consumed by the mining itself which is proportional, but also a fixed amount of power consumed by the rest of the system to stay on (motherboard, ram, disk, other stuff)

i see what you mean

So for optimal efficiency, the target is to use as many threads as possible to "spread out" that fixed base power cost

jberman: It would be good to get estimates on this. I could help with that.

Something high like 10,000 like the sanity check uses when submitting a tx to a node with sanity check enabled might make sense (github.com/monero-project/monero/bl…tonote_core/tx_sanity_check.cpp#L84)

As for interacting with xmrig, it could be easily done even via some external script that dynamically changes the config file. All you'd have to do then is implement the logic

merope: Yest for the last part, but desktop PCs are not the only miners out there. This doesn't apply that much for the low powered machines, where the "static cost" is relatively lower.

Mining is not all that scallable to threads, unfortunately.

But improving the difficulty adjustment algorithm would be have a much bigger impact on mining and the network as a whole

jberman[m]: yeah that might work

maybe that is a better problem to focus on then

Making sure that blocks times are more consistent (especially in the case of big hashrate swings) would improve the user experience for the people sending transactions, and would (ideally) make the network less susceptible to hashrate attacks

Of course, such a critical change would be subject to very careful review and testing before being released on mainnet via a hardfork

Before the meeting runs out, does anyone have questions/comments on other topics? Perhaps from the agenda or otherwise.

i have a reply but i will hold off

Let me point out a key challenge with determining a difficulty adjjustment algorithm: It's not only an engineering challenge. There is also game theory mixed in since player, i.e. miners and malicious entities, can themselves react to any changes in the system.

jberman: 10,000 units of what, exactly?

I think the important thing is to define the problem clearly, define some requirements/goals to solve the problem, and get some data regarding the signals of interest to start designing the estimation scheme

Rucknium[m]: Spendable outputs in the chain. There are currently 40mn+ spendable RingCT outputs in the chain so wouldn't impact spending RingCT outputs, but some pre-ringCT outputs might have smaller sets (and on the switch to Seraphis, this would be something to consider too)

my thoughts are that we should test the scheme on a model first, de-risk as much as possible in a safe environment, before even thinking about testing on a more realistic platform

On wallet side binning, my view is starting to shift that it may be challenging to get everyone on board with it for next fork, considering there are fairly subjective parameter choices in the algorithm and it's a fairly significant change to the decoy selection algo as is. It's starting to feel like I should roll up some of the intuition from there into pushing forward on deterministic binning (issue 84), which is the more long

term direction we're headed anyway. Curious if others have thoughts on that

jberman: Remind me, what happens under your code when the random selection picks a ring member that is a duplicate from another ring in the same tx?

maxwellsdemon: BCH has done some work on an "improved" difficulty adjustment algorithm with ASERT:

Also some Monero forks with much smaller hashrates use some different algorithm; but as I said, any change here will be a *tough* sell :)

Though sell to whom? Devs, or the general community?

Rucknium[m]: In yorha-0x's PR (monero-project/monero #8047), they simply re-select an output when one has already been seen/used in another ring. Which actually now just strikes me as a potential issue in that PR (this comment monero-project/monero #8047#discussion_r769368957 may be applicable to the general case too)

I think mostly the dev community. The general community probably does not care enough.

Or were you asking about how wallet side binning handles duplicates Rucknium

jberman[m]: I think it would be ok for you to shift focus. Ultimately it is your judgement call, since you are the 'project owner' (so to speak).

jberman: Not binning. The duplicate outs issue.

I'd say the DAA is roughly on the same level of "importance" as the transaction protocol, in terms of its critical impact on the system - and there is always interest in making the tx protocol better/safer (with all the review and testing that it comes with). So why would we not apply the same level of scrutiny for other parts of the system?

Rucknium: I read through the links you provided and the glaring concern I have is I dont see and clear mathematical analysis for the scheme they are using. I feel like these write ups are lacking in detail and that concerns me

If someone comes up with a solid, thoroughly reviewed proposal for a better algo, why not adopt it?

For example, just because the signal oscillates doesnt mean that is necessarily wrong. Further, any FIR filter you implement (a moving average is the most simple type) will have oscillations. That is something you cant avoid

UkoeHB: makes sense, will keep thinking on it

Well, you will have people to agree that it is "better". And there is indeed the "never touch a running system", as I mentioned, to many it works "well enough" now

Some key bits of info needed: In a typical week, what's the difference between the peak and trough of difficulty? The most concerning thing about a DAA would be if it opens the system to a 51% attack more easily by virtue of the fact that it creates a large spread between speak and trough.

Not to discourage anybody, of course. Just to be prepared if walking gets rather stiff :)

merope: My general understanding is A) the current algorithm doesn't have any fundamental weaknesses, B) there is no one 'championing' algorithm research (surae researched this iirc, but no recommendations for a new algorithm were made).

maxwellsdemon: I'm not saying that their DAA is necessarily a good one. I'm saying that at least they have thought through some of the issues, so maybe something can be learned from what they've done.

regarding the DAA, forgive my ignorance, but isnt this "just" controll theory? you have target and need to controll the environment such that the target is met?

Isnt this researched already by kybernetics and robotics and so on? shouldnt there already be a perfect controller which solves this?

atomfried[m]: there is an adversarial dimension not present in normal control theory

^

Yeah, people who try to topple the robot over :)

ahhh i see thank you

atomfried: No, it is not just control theory. It is also game theory, which complicates things quite a bit.

We are running into the end of the hour. Any last minute questions/comments?

Rucknium: I have a ready made csv of block height and difficulty (and nonces, but you can ignore those). I can send it to you, and I'm sure you have better tools for evaluating the data

endor00: Nice. Please do send it.

rbrunner: A key thing is we don't even have a good idea at this moment of the risk that the current DAA is exposing us to with respect to 51% attacks.

atomfried: control solutions are unique to the system dynamics. Control theory isnt simple except for noncritical applications like home thermostats

So to say that "it's not broken so don't fix it" is not really correct since we don't know if it is broken.

It would be nice if we had a simulator to test the algorithm on first

it is very risky to not take that approach - i wouldnt do it

maxwellsdemon[m]: i know that thats why a said "just" lol my uncle is professor in controll theory but i have no understanding of it so i thought i would just ask why all the controll theory cannot be applied to the DAA

I am not clear how the BCH apprach would help here. The problem that BCH has is wild variations in hashrate due to using the same ASICs as BTC

maxwellsdemon: we have test networks that allow us to try this stuff

Monero does not have this issue

We are on the "BTC side" of that issue

A test network isn't good enough since the behavior of the miners would not be the same as on mainnet

But block times can become quite skewed during/after sudden hashrate bursts

ArticMine: Yes, I'm not sure that anything in the BCH approach would help us. It's the only case where I've seen someone take a close look at the DAA, so at least it's good to be aware of it.

We need some type of model, otherwise, in my opinion, whatever solution we come up with is just fancy guessing. We need an understanding of the system dynamics to proceed with a meaningful solution

Especially in the latter phase, when the spike is gone and now the difficulty has to adjust down, but it's taking longer because blocks take longer to show up because the difficulty is still high but the hashrate isn't anymore

maxwellsdemon: Right, right. Building such a model would be part of the task of developing an improved DAA.

Rucknium: I would recommend splitting them because that by itself is a big effort and potentially has many uses

<merope> I know but with BCH trading at ~0.009 BTC and using the same algorithm as BTC this is one stress test on the DAA

UkoeHB: seth mentioned on twitter that seraphis might allow for the removal of the 10 block conf requirement

from what i understood its not possible

can you confirm

maxwellsdemon[m]: i am pretty sure that those two things could be nice bachelors or masters thesis for controll theory students...

Yeah I may have misinterpreted an earlier convo, would also like confirmation.

atomfried[m]: sgp_: Get MAGIC grants on it lol.

Rucknium: it would be, finished school too early for crypto to be a legitimate academic discussion :(

This is actually mechanism design, with control theory nested within it. It _is_ definitely a complex problem, and for now we have a very rough "heuristic"

well, if there is a means to get paid regularly (say monthly), I certainly welcome the challenege

maxwellsdemon[m]: i think this is actually a nice topic because it applies more to controll theory than to blockchain-crypto-buzzwordy stuff

no, it is still no

yup, MAGIC Grants is here to help :)

UkoeHB: thought so. would tx chaining allow for spending an unconfirmed output once the 10 blocks are mined? allow it to queue until its ready to spend?

bbqcore[m]: yes you can queue up txs, however the cost is whoever has a queued tx will know the real spends of that tx

UkoeHB: so a non starter privacy wise

10 block confs remain as is

yes, it is only useful when dealing with trusted parties (or when the real spend is already known, like in atomic swaps)

UkoeHB: so in theory this could allow a wallet to add auto-splitting of incoming XMR to a set amount of outputs

bbqcore[m]: why would you need tx chaining for that?

since it would essentially be a self spend and you would be trust yourself

when the 10 blocks are over, you still have more work to do to complete your tx (i.e. construct membership proofs)

btw the meeting is over, thanks for attending everyone

UkoeHB: because it would queue the unconfirmed incoming tx automatically before 10 blocks are mined

so once its confirmed it splits them into X amount of outputs back into your wallet for future spending

bbqcore[m]: why not just wait for the 10 blocks before making your new tx? The advantage of chaining is you can send a partial tx to another person while the funds being spent are locked in the 10 block window. If there is only one person, there is no advantage.

I have to get going, been a good discussion, thanks. We can follow up later

to make future spending easier with more outputs available

say i set my wallet to auto split any XMR i receive into 5 equal outputs once the original tx is confirmed

I feel like there might be a misunderstanding here. Afaict tx chaining does not reduce or meaningfully reorder the actions you need to take, to split outputs.

UkoeHB: youd have to wait for the splitting tx, but it would be queued automatically with no user input

Right now you need to: A) see outputs sent to you, B) wait 10 blocks, C) construct and submit txs. In tx chaining, it would be: A) see outputs sent to you, B) wait 10 blocks + make partial tx to split outputs, C) finalize and submit the txs.

UkoeHB: ok so it doesnt help in creating a tx beforethe tx gets confirmed

using those unconfirmed outputs

not for this use-case

gotchya

not a big deal as monerujo is adding a manual splitting feature, just curious as to what seraphis makes possibe

jberman: Walk me through exactly what a wallet and daemon would do in the case of finding that there is a duplicated output, under the (evolving) proposed change in PR#8047.

I guess what I am concerned about is a large number of requests being submitted (or computed) as a wallet searches for non-duplicate outputs

In the general case:... (full message at libera.ems.host/_matrix/media/r0/do…e3bd0bbb7a485e37192adad3bea45baa7bf)

In this PR:

- yorha-0x adds a way for wallets to not add duplicates *across* rings, rather than just within a single ring

- this is done by including a cache that keeps track of outputs that the wallet will request from a node *across* rings, and not re-adding duplicate outputs to the set of outputs to request from a node

- it seems to have issues as highlighted in the PR, but ideally if the wallet selects a decoy that has already been used in another ring in a transaction (or will be a real output in another ring a transaction), the wallet will simply re-select another decoy using the algorithm it uses to select decoys

jberman: Thanks! So, how many requests does the wallet send to the daemon, in a typical case and in a case where duplicates are encountered?

And what is the 10,000 sanity check about, that you mentioned earlier? If there are not 10,000 valid outs to choose from, what happens, and why?

> So, how many requests does the wallet send to the daemon, in a typical case and in a case where duplicates are encountered?

The number of requests to the daemon isn't impacted by duplicates. You can determine duplicate-ness without making a request to a node. It requests ~1.5x the number of decoys that will ultimately be included, just in case some are locked

And the daemon will not send back duplicate outputs when they are initially requested?

So the challenge is for the wallet to arrange the decoys in a configuration so that there are no duplicated outputs across rings?

1st question: an honest daemon will if duplicates are requested

2nd question: yep, and this is more of an implementation detail tbh I would think

> And what is the 10,000 sanity check about, that you mentioned earlier? If there are not 10,000 valid outs to choose from, what happens, and why?

When you submit a tx to a node and the request includes a sanity check on the tx (not required), then the node does same basic sanity checking on the transaction's rings, one of the checks being that if there are 10,000 valid outs to choose from, at least 80% of all outputs included across all rings are unique. I assume the intuition behind this was presumably that it would be strange (and highly unlikely) if there were a large

number of naturally occurring duplicates across rings when there are 10,000 valid outs to choose from, and if this were to be the case, it would indicate there may be some other issue going on with the wallet's selection algorithm, and so the tx should be rejected and the wallet should re-select just to be sure it's doing things properly

This sanity check actually happens in the wallet-side code as well. If this sanity check runs in the wallet and it fails 3 times, the wallet automatically throws away the constructed rings and re-selects decoys

And how does the decoy selection algorithm avoid telling the daemon which ring members are the real spends, by omission? (I'm thinking that in a naive implementation of what's going on, the daemon can just subtract the set of decoys that it sent to the wallet from the total set of ring members it gets from the wallet after the wallet constructs, signs, and sends back a tx.)

Correction: in the wallet, if the sanity check fails client-side, it automatically throws away the constructed rings and re-selects decoys. If it fails 3 times, the tx fails to construct and the user gets shown an error

Maybe Zero to Monero covers my questions 🤔

no don't think so

Simplified: before requesting, the wallet sorts the outputs by ID before making the request for output public keys, and then upon receiving output public keys from the daemon, makes sure the real output's public key is included among those requested. This check is naive though, and there are probably other ways for a malicious node to identify real spends, so to be certain you're not revealing your real spend to a node, it's best

to run your own node

By ID you mean index, I assume?

yep

The mental block I'm having is that the wallet needs to know some things about the current state of the blockchain to pull decoy indices from the gamma distribution, correct?

So the daemon has to tell it some things before it can request the public keys by sending their indices

Before selecting decoys, the wallet requests the cumulative distribution of ALL outputs from a node, that gets returned in a fairly large array that could e.g. look like this: [2, 5, 15... N]. Each index in the array corresponds to the number of cumulative outputs in that block. So in that array, there are 2 outputs in the first block in the chain, 3 outputs in the second block, 10 outputs in the 3rd block... etc. all the way up

to N which is number of outputs in the chain up to the final block at the time of tx construction

(Hang on it'll take me a sec to explain how the gamma gets applied using this distribution)

So that array has some 2.5 million elements at this point, then?

Should be 1 million something (since RingCT started), which works out to something like 10mb

Simplified on how the cumulative distribution array (referred to as `rct_offsets` in the code is used to select outputs using the gamma distribution:... (full message at libera.ems.host/_matrix/media/r0/do…6d58b6865a973f1959f5ddb977481649e22)

So the point in the process in which a fix to the "duplicate outs" problem would be implemented is right before the wallet requests the public keys from the daemon (which is at the point that your bullet points above stop), correct? So by the time that you start requesting public keys, the issue has already been resolved (hopefully)

Yep that's right

Ok thanks. So is there a good reason for the technique that constructs the rings to not just randomly order the decoys, then re-index them [1, 2, 3,...,J]. Then take [1,...,10] for the first ring, [11,...,20] for the second, [21,...,30] for the third, and so forth? There would be no need to re-roll a random ordering in that case to avoid duplicates.

It needs to re-roll to avoid duplicates either way, since you don't know what the gamma will spit out until you roll. So if it spits out a duplicate, you need to re-roll

I've updated the addressing scheme proposal for Seraphis: gist.github.com/tevador/50160d160d2…cfc6c52ae02eb3d17024#6-wallet-tiers

For tier 1, the key k_x could be stored in the local daemon and when the wallet reconnects, the daemon hands over a list of potential outputs with pre-calculated spend keys. The wallet can simply throw away the outputs that don't match any keys in the hashtable. Basically instant sync.

Thanks tevador. For next meeting I'd like to focus on Seraphis address schemes and hopefully reach some kind of decision (or get closer, maybe narrow down the choices to 2 or 3).

So everyone better come prepared!

(that means read this monero-project/research-lab #92 and tevador's gist)

tevador: would you mind adding a short note about how to migrate from existing wallets (normal, view-only, hardware) to your scheme?

Nvm you basically have that. It looks like view-only wallets would just stop working for new outputs.

Yes, the spend key holder will need to provide new keys. Reusing the pre-fork view key as one of the new keys is not advisable due to different capabilities.

Basically, sone users might have been using view-only as a Tier 2 and some as Tier 1. Let them decide post-fork which of the new tiers is closer to the intended use case.

Btw is there a reason you flipped the order of keys in the address?

Yes, it made more sense for the encoding. The key K3 can be implicit from the signature.