Ive always wondered if there would be a way to enforce randomized output selection to mitigate potential attacks like this, but i dont really know how it would be feasible

yeah to be honest this is why I had assumed that, just because monero is so focused on built in privacy

and fungibility

cooldiggs[m]: I have a sketch of a method: Enforce a particular probability distribution at the consensus level via hypothesis testing. Make the p-value 0.05 or something.

So 1/20 rings will fail the random test ? How is that enforcing ? :)

How would one determine the correct distribution though? That part seems to be the biggest hurdle in such a system I think

because if there was a way to have built in randomized decoys, this would throw a wrench into the aforementioned attack

which is arguably one of the strongest attacks against Monero, currently

What attack do you have in mind ?

Yes, it would. The wallet would do the test, too, so it would know not to send a transaction that would fail. If a given selection of the wallet fails the test, just draw again.

a poisoned output or a knacc

cooldiggs[m]: The Monero reference wallet already proposes a "correct" distribution: gamma with a certain specified rate and shape parameter.

It does not. You could make a wallet that always chooses a predefined set of 10, plus the real input. It'll likely pass the test.

What does not do what?

I agree that ideally you'd do a pooled or joint test across, like, all the mxins that are going into all the transactions. Not sure how that could be done though.

The statistical power with 11 inputs is very low anyway. This will start to become important with larger ring sizes.

It (The wallet would do the test, too, so it would know not to send a transaction that would fail.) does not (enforce random rings).

FWIW the daemon does such a test (though a very lenient one) by default.

If you reject too much, your distribution also changes from the one you started from.

I suspect something like a gaussian would start decreasing its variance. Not sure what a gamma would do...

Actually I might be wrong on that point, nevermind.

I guess it depends on how exactly you reject.

I agree that a wallet that was deliberately coded to make its transactions stand out could repeatedly pull the same inputs as mixins for every transaction (or, if not the same mixins, then the same mixin age relative to the time of each attempted transaction). I am thinking, though, that as the distribution becomes more important, there may be some way to enforce it at the consensus level just like ring size was enforced. Also, the lack

of minimum ring size enforcement in the past created an externality that harmed the privacy of other Monero users. I am very early in thinking about this, so there may be unsurmountable problems.

"If you reject too much, your distribution also changes from the one you started from." I'm not sure that this is true.

Ah, just had to read further on. Yes, well it may not be true.

Is there any documentation on creating a raw transaction given you have the ingredients to transaction (i.e pub keys, priv keys, outputs etc) ? Im interested in trying to implement something like that in C++ for my own learning

I know you can get the wallet to save a raw transaction for later use/sending, but to my understanding that just returns the tx_blob which is a hexadecimal string of an already generated transaction

I think you might be right about that, moneromooo , although the effect seems small.

I wrote a simulation with 1 million transactions, 11 inputs each drawn from gamma with shape = 1. And then reject a tx if a K-S test gave p < 0.05

A pooled k-s test of all the tx inputs strongly rejects (p < 2.2e-16) that the resultant "sample" of inputs have an age distribution matching the original Gamma distribution. Hoever, when my sample size is smaller, the rejection isn't as strong....

The variance of this set is on the order of 0.99 (depending on the seed set for the simulation), when it "should" be 1. Mean is about 0.997 when it should be 1.

(For completeness, skewness of the sample is on the order of 2.01, when it should be 2)

cooldiggs[m]: are you trying to figure out how monero works by reading the codebase?

simulation code: gist.github.com/Rucknium/fc67e38b3ff8664d0f1cfe8a08c1eaad

I have read the codebase for the most part as my guide, and its given me a lot of insight though I was just wondering if there was documentation I wasn't aware of involving constructing a tx from scratch

construct_tx_with_tx_key function in cryptonote_tx_utils (i think its there) has a lot of helpful information regarding that though

The original cryptonote papers/standards as well have been helpful but they are kind of old and somewhat outdated

cooldiggs[m]: web.getmonero.org/library/Zero-to-Monero-2-0-0.pdf

Havent seen this before, thanks for sending, will take a look now

to make a tx from scratch... if you don't care about matching all of monero's serialization details it shouldn't be impossible

serialization isnt really all that important, as this is just me trying to understand exactly how xmr transactions work and being able to implement it myself, sort of a learning exercise

even if you care about serialization it's still not impossible :)

construct_tx_with_tx_key is probably going to be the most complete "documentation" that exists

Looking for a wallet.keys file that had funds in the 2017-2018 time range to run some experiments re: the selection algo bug (attempting to reproduce j-berman's method j-berman/monero #2 on a set of transactions from that time).

Obviously it can & should be empty now, not trying to claim your Monero V / Classic, if that's a concern just claim it first.

If anyone can donate an old wallet, I'd appreciate it!

<luigi1111w> "construct_tx_with_tx_key is..." <- Haha fair enough, and being honest its a pretty complete function overall and you can tell what it does

Its just when you get to the nitty gritty like generating key images among other stuff it gets a little hazy, although im sure someone experienced in cryptography + C would be able to pick it up easily

if you are wanting to dig into the EC side of things yeah that's a different story

best to treat it like a black box unless you are really interested :)

I would hope one day to understand it enough to write documentation about it, though it'll likely take me a century ahaha

that stuff is mostly ref10 I think

luigi1111w: Cryptography interests me in general so I love learning about it, but the C implementation of all that stuff is like an arcane language

the simple python stuff is probably a better place to start

instead of optimized C

For sure, python is easier to get code out on but personally speaking Im more familar with C++ since it was the first language I learned and I love it dearly

no I mean to look at

Python makes me angry sometimes haha but it has its many advantages

luigi1111w: Oh right you make a good point there

miniero or whatever its called is pretty easy to read and iirc its mostly python

yup

it's a little outdated though I think

but the EC stuff is unchanged

or mostly

I would be suprised if its changed all that much since ed/ec curve stuff is mostly a standard to my understanding but it still gives a good baseline for what is actually happening

yes indeed

Sadly the C code is filled with weird functions like ge_p1p1_to_dontthinkaboutit and it scares me haha

But one day I hope to understand it and write some basic docs about it

yeah that's why I don't recommend starting there :P

Aahhh but I love c/c++, your right though since I havent made much progress in terms of understanding, python will give me some of the divine knowledge so I can understand the other divine knowledge

I see it as a challenge though so its fun to waste time on it, Monero has such an interesting codebase

well a lot of it is optimization

Its interesting as well to compare it to the cryptonote codebase since I remember that being the OG back in the day, seeing how its changed and improved really is a testament to the community and dev team

there are talented coders working on it for sure

luigi1111w: Makes sense considering the implementation, ed curve stuff is supposed to be lightweight according to my understanding

I believe "fast" is in the tagline

cooldiggs[m]: the weird ge_... functions are converting curve points between different representations

edward you sneaky devil

you made things fast but now all the code is scary LOL

> <@esk6v:matrix.org> Looking for a wallet.keys file that had funds in the 2017-2018 time range to run some experiments re: the selection algo bug (attempting to reproduce j-berman's method j-berman/monero #2 on a set of transactions from that time).... (full message at libera.ems.host/_matrix/media/r0/do…22c656ae90f260a949958ff9dc333b88adb)

I think the little comments (where they exist) like X = aG + cD or whatever is far more useful in understanding how the function relates to [constructing a transaction, or most targets] than trying to read the actual function

<jberman[m]> "> <@esk6v:matrix.org> Looking fo" <- Suppose a user made a transaction in 2018 with the same applicable conditions (>=2 blocks from lock time expired & from a block with fewer than 100 outputs), would the real output be identifiable?

No, doesn't look like the code from that time has the same issue

Here's the code I'm looking at: github.com/monero-project/monero/bl…/src/wallet/wallet2.cpp#L6845-L6887

It gamma-selects a block offset, which can end up 10 or higher just fine. It also has this line here that extends the window from which it can select an output, which would bring the window up to the 10th block: github.com/monero-project/monero/bl…/src/wallet/wallet2.cpp#L6876-L6877

<jberman[m]> "It gamma-selects a block offset," <- Got it, thank you

More detail: the gamma was introduced circa Aug 2018 (monero-project/monero #3528). Before then wallets used a different algo that could pull outputs from the whole chain, but a "guess newest heuristic" could work sometimes with that algorithm. This paper could give more insight on that: arxiv.org/pdf/1704.04299

jberman: can I PM you?

sure. the "guess newest" heuristic isn't definitive if that wasn't clear. it would still be a guess

<moneromooo> "It can't be any value, it has to..." <- Hey, another dumb stupid question

If the key_image is deterministic, is the value that determines the key image from key_image.data (which is a char[32] according to ec_point) since it borrows from ec_point?

Im trying to understand where it gets its value from, since in cryptonote_tx_utils.cpp after generate_key_image I print the key image object and it returns a 64 character "string" (I know its likely not a string object, but it returns 64 character hex surrounded by < and >)

Is key_image.data where it derives the actual key image? Im assuming since it borrows from ec_point that ec_point.data is essentially a representation of a valid point on the ed22156 curve (given the base point of course)

Im asking this silly question because if I print out the image object after generate_key_image in src/cryptonote_core/cryptonote_tx_utils.cpp I get this (around line 335):

[wallet 448pHQ]: transfer 4AdUndXHHZ6cfufTMvppY6JwXNouMBzSkbLYfpAV5Usx3skxNgYeYTRj5UzqtReoS44qo9mtmXCqY45DJ852K5Jv2684Rge 2... (full message at libera.ems.host/_matrix/media/r0/do…c3d96b90f35047eb60a826a464ee6161389)

The code I used for this after generate_key_image is this simple rubber ducky code:

std::cout << "Tx key image is: " << img << " and the type of it is " << typeid(img).name() << std::endl;

std::cout << "Key image data is: " << img.data << std::endl;

std::cout << "Replace key image: " << std::endl;

Very basic I know haha

But since the key_image.data doesnt seem to be actual chars im guessing it is a representation of an elliptic curve point, which to my understanding would be either the pub key/priv key, and in essence is just a very long number when you break it down to its mathematical core

That is, if key_image.data is where the actuual key image value is retained from, I can only guess it is from that considering Im not sure where else the key_image actual object would come from

The key_image object confuses me because I realize it is deterministic, but Im not really sure what determines the final value

I would assume the key_image.data is the integer value of the key image which is then converted to an actual key image after it is scalar multipled and converted into special bytes and such (as in generate_key_image), but Im not really sure why key_image.data is an array of chars, I mean if the chars are deterministic to the actual value I feel like it would make more sense if it was say a uint32_t or something like that

I dunno, Im pretty stupid and easily confused by this implementation of key_image since thats primarily what I am stuck on if I were to use the current monero codebase and its functions, the paper someone linked me earlier is very helpful but it still is conceptual in a way and doesnt deal with the core implementation, which is one of the things I am interested in

<cooldiggs[m]> "Im asking this silly question be" <- Also keep in mind, this is just info I generated on a private testnet, for testing purposes so I dont really know if that potentially effects how its calculated

the key image is just a point

Right, but what im asking is that point defined in the code as key_image.data? Or is it a different variable under key_image

Since key image seems to only extend to ec_point to my understanding, and ec_point is just a char[32] array

Unless there is something im fundamentally misunderstanding with the key_image object which is likely since I am a dumbass lol

Though maybe im thinking about this the wrong way

Because generate_key_image starts off with two ge_point variables, and then is manipulated by the pub/priv key

void crypto_ops::generate_key_image(const public_key &pub, const secret_key &sec, key_image &image) {... (full message at libera.ems.host/_matrix/media/r0/do…39af591f1f0aa5c01a7b192252b56896c58)

Now that I think about it, its probably all derived from the pub/priv key since before this function in cryptonote_tx_utils key_image is an object called img that isnt defined as anything

I should stop drinking and do more thinking LOl

* I should stop drinking and do more thinking LOL

keyimage = xHp(P)

Im just overthinking it, I think

yeah it only contains the result (the point), nothing else

Ill look at the code and papers more before I ask more questions haha

One last question, I still dont understand how key_image.data fits into this, well i can guess as I assume that is the curve point value itself, maybe

Well actually maybe I should figure that on my own, since the code can probably tell me somewhere

I just dont see it mentioned much within the implementation

the value is just that 64 hex chars shown in your dump

So to clarify, the actual point itself is represented by a an array of 32 chars? (char[32]?)

yes

Ah ok, sorry for the long stupid walls of text I am tired and just wasnt sure haha

luigi1111w: Got it, and since it is deterministic is entirely dependent on the ec point data

all the "mathy" stuff is done after decoding it

Ah I see, ok, I understand how it is deterministic now, sorry for my excess of dumb questions and walls of text and such, I dont understand easily haha

it's deterministic, but it IS the ec point data

(at least in its encoded form)

Right, so its deterministic based on the ec data just like the actual math function would be, just in moneros special cool kid format

I think I understand now, this gives me enough information to test and play around

Thank you luigi1111 and moneromoo for answering my questions, you guys are lifesavers and very helpful : D

it's not monero's format afaik

Im guessing then its something borrowed from supercop or one of the other libraries used in monero?

Supercop documentation is sparse so i cant really find much about it haha

yeah all the ed25519 libraries use the same encoding I think

Or perhaps ref10, but that has lots of comments thankfully

luigi1111w: Ah good to know

maybe not all but it's standard

a standard gives me a better starting point than say wild west C cryptography impl. haha

gl and gn

Goodnight! Thanks for you guidance and such, hope you sleep well

And hopefully I dont dream of bit shifts and encoding...scary stuff

I'm finally very close to p2pool testing phase. Details on reddit: reddit.com/r/Monero/comments/p9apyd…ro_decentralized_mining_pool_p2pool

Hello, is there an eta for a new monero version? There were two bugs discussed recently. One related to the selection of decoy outputs and one related to some possible overflow if a certain average transaction count occurs. Will those be fixed in the new release?

Yes, soon

nice, thx :)

TM

FWIW both are minor, you don't really need an update in a hurry.

May be a dumb question: why don't centralized mining pools include the payout outputs in the coinbase tx also?

because monerod can't do this and they don't have qualified developers to make custom block template creation code

but they could totally do this too

Why cannot monero do this ?

get_block_template creates miner reward with 1 output

*Monero RPC API can't do this

Oh. OK.

Seems that API should accept params to enable this. Saves KB's of data from payout tx's and prevents deducible spents making their way on chain, like here: xmrchain.net/tx/98f7f72e6ce347b20bb…751813de984700e5e145a01265b0a1525/1

pretty sure the API doesn't support sending to a subaddress either

yes, get_block_template doesn't allow subaddresses

it's technically possible, but it will add additional 32 bytes per output, doubling the size

so I don't allow it in p2pool as well

2160 outputs * 38 bytes = already ~80 Kb

FWIW I'd consider having consensus rules for the share chain to not have the smallest payout threshold possible.

It might sound like a lot, but these 2160 outputs would otherwise have been regular transactions, so...

there is no smallest payout threshold

Then "these 2160 outputs would otherwise have been regular transactions" seems false, no ?

Not sure what you mean

Regular pool would pay 2160 miners anyway, using regular transactions

if these 2160 miners move to p2pool, they'll save space on the blockchain

Paying dust 10 times vs paying quantized 1 time.

Maybe I'm wrong, but I imagine miners are going to get paid more often with a share chain with no threshold.

So back to dust outputs.

they do get paid with every found Monero block

minimum amount is 1/2160 of block reward. It could be a bit less if pool hashrate kept rising after miner found a share

OK, the small length of hte share chain might fix that...

btw transactions combining N inputs take less space than N 1input/2output transactions, so it's not that bad

also, if a miner has a significant % of pool hashrate, he will get paid more in each payout, so it won't be dust

Sure, the existence of non dust does not prive the non existence of dust though.

From now on, my reports will only be available here:

I think jberman can run some simulations to compare miners on a regular pool paid with 16 output transactions vs miners in p2pool... after he's done with gamma-distribution :)

I have a feeling it's still more efficient in terms of used blockchain space

It'll depend on the payout threshold for those pools.

lowest I've seen on pools is 0.001 XMR

so only 2 times more than on p2pool

OK, that's not bad then.

you can sort pools by min payout there

I imagine most people take pool payouts much less frequently but Idk; maybe we could ask a pool operator

well, p2pool can't pay out more than 1 block reward by design, so it has to pay with every block

.merge+ 3672

Added

.merge- 3672

Removed

sech1: would it not be possible for the share chain to keep state of which miners are owed what?

not that it's necessarily the better design choice, just curious

what do you mean? Each share chain block has miner wallet address

oh, you mean payout out only fixed number of "top earners" and remembering other balances?

ya

I tried to do this but couldn't figure out the fair payout scheme - low hashrate miners were always underpaid in my simulations

the simplest example - if some share goes out of PPLNS window without getting paid, you could remember this share difficulty and add it to another share of the same miner

naive approach which makes low hashrate miners underpaid severely

I tried several different payout formulas to fix this but didn't succeed

interesting, got it

the requirement of paying out exactly the block reward every times complicates things even further

which can't even be predicted beforehand because of transaction fees added to the base reward

ya I see the problem

hi I am new here . is it possible to view history

On Matrix, yes

How do I get in touch with Luigi or whoever pays out the CCS proposals?

just message me