.merge+ 8163 8161

Added

I'd like to chat about adding a key to the monerod rpc "get_info" call. The idea is to provide remote node operators the ability to return an arbitrary string via a runtime environment variable MONERO_NODE_INFO. If that's better discussed via a github issue please lmk (I have a patch ready if that's the case lol)

With respect to the fees question I have update MRL issue 70 and referenced pr 7819 with the change in growth rate of ML from 2 to 1.7. I trust this will allow us to proceed with this.

See my comment above ArticMine I will incorporate it into my comment on issue 70

jberman[m] did you do the variant of the test where you comment out the "ge25519_has_small_order/ge25519_frombytes_negate_vartime/ge25519_is_on_main_subgroup" bits?

i think that's most of the overhead

jberman[m] and i think those are all with libsodium, and not sandy2x right?

if we do decide we need the "ge25519_has_small_order/ge25519_frombytes_negate_vartime/ge25519_is_on_main_subgroup" checks, we can do that after the view tag matches instead of before

UkoeHB: Great job with Seraphis!

Hi everyone I have a question regarding the system of view keys and key images. I've read that "because view-only wallet doesn't have key images, it can't see outgoing transactions." When we use a hardware wallet to export the view key, then how the wallet can guess our balance? Does the hardware wallet export key images at the same time it exports the view key?

k.w.x.m.r: When a hardware wallet signs a transaction (ie spends) it must include the key image. The online, view only, wallet can just save the key image then. So, as long as there is only one online wallet, there is no more need for additional communication.

k.w.x.m.r: But an online wallet will, obviously, know all of the recieved outputs and know if it is missing any key images. When it finds an output for which it does not have a key image, it needs to request that key image from the hardware wallet. I imagine this is done every time you use you connect you hardware wallet.

Ok thanks!

But why the wallet just says exports viewkey and doesn't mention we are also exporting key images?

I assume it's for making it easier for users and don't have to ask 2 kind of permission, right?

UkoeHB: On your Monero Talk interview you seem to doubt that seraphis will be useful to force exchanges to reveal their balance. May I ask why? As far as I understand since it will be possible to have view wallets that reveal full balance then why an exchange won't be able to reveal its xmr on a cold storage address? Thanks

kwxmr[m]: what kind of company wants to publicize all their internal finances?

I mean from a technical point of view, is there anything that would prevent them? Because otherwise it will just a matter of pressure from the community and in this case that a huge game changer for xmr

s/I mean from a technical point of view, is there anything that would prevent them? Because otherwise it will just a matter of pressure from the community and in this case that a huge game changer for xmr/UkoeHB: I mean from a technical point of view, is there anything that would prevent them? Because otherwise it will just be a matter of pressure from the community and in this case that's already a huge game changer for xmr/

I guess not? If your only criteria is 'cant spend with the key you publicize'. You can do literally everything else with the seraphis/jamtis view-balance key.

This is absolutely huge! Thanks for confirmation

/

<knaccc> "jberman and i think those are..." <- They were with libsodium right. Working on the rest, took me a bit to get set up with rewritten libsodium files :)

jberman[m] thanks, i'm on the edge of my seat :)

knaccc: sandy2x is for x25519 and only works on sandy bridge+ cpus (no arm64)

vtnerd have you seen this? research.nccgroup.com/2020/09/28/fa…e-symbol-and-an-x25519-speed-record

also, sandy bridge is now 11 years old. since this is only for curve25519, could we include sandy2x and use it where suitable CPUs are available, and fall back to ref10 elsewhere?

can you send xmr multiple times from the same input or is there a limitation on that?

if there is a limitation, does the change transaction show up as a new transfer in the wallet RPC?

WillMorrison[m] inputs are always completely spent, and you get change left over

knaccc: does the change stay in the same address or is there a way to direct it to a certain address?

there are wallet rules, i'm not sure but i think it's normally sent to the first subaddress in the account

Oh ok great

thanks for the help

yes, sandy bridge is has been out a while, but I pointed at donna64 for arm64 arch

that code will not run on such cpus, and apple has now switched to that arch entirely

and that nccgroup.com link you sent is the first time I've seen that

vtnerd: so referring only to our use of x25519 and not ed25519: would there be any objection to including sandy2x and falling back to donna64?

(falling back for aarch64)

in what context? wallet scanning or verification in general?

vtnerd i was only thinking about using curve25519 for wallet scanning

and we'd need it so that the wallet can build the view tag for the tx in the first place too

what? why would it necessarily be needed for view tags?

or do you mean you plan to switch the protocol entirely during this change?

vtnerd i'm not sure if we're talking at cross purposes. monero doesn't use curve25519 anywhere at all right now, right?

correct

so we're only considering using x25519 for generating and checking view tags, because that could result in a huge speedup

everything else would remain ed25519, including how the one-time output public keys and shared secret are generated

correct, which then (kind of almost) forces everyone downstream (not just wallet2!) to use x25519 in addition to the this new view tag implementation

well the good news is that the view tags don't have to be paid any attention to

or they could just ingore the view tag and compute as normal

you can still scan the existing way

right exactly

and x25519 is so widely used that i can't imagine anyone would have any library issues

actually i take that back

the scalar clamping on x25519 needs to be removed, so we don't break compatibility with our existing ed25519 scalars and x25519 scalars

so we do need to mod the x25519 libraries we use to get rid of clamping

ah yeah I forgot about that one, yeah different points will be computed if that remains

I always avoided ed25519->x25519->ed25519 because it required two conversions, but it sounds like the 2nd conversion only occurs iff the view tag matches

the numbers should work better in that case, but the monero ed25519 isn't compatible as you noted, so eh

vtnerd i only realised yesterday: going from ed25519->curve25519 points is fine, but the sign of the ed25519 point is lost so there is ambiguity if you try to go back

so we'd only go one way

yes, theres that fun aspect (I mentioned above but it was in a bunch of stuff)

oh but see wait, you wouldn't convert back? then that changes the protocol entirely, it forces x25519 on every wallet immediately

i'm saying we don't change the shared secret for the actual output pubkey

only for the view tag

iirc it might be doable because that result does a hash_to_point, so you only need a consistent bit pattern to get the final output address

so if the view tag matches, we then compute the ed25519 shared secret from scratch

ah. that changes a bunch of the numbers that have been calculated, yea dunno

we haven't get gathered enough stats to decide whether to advocate for tx pubkeys to be published as curve25519 points in the first place

but i'm hopeful that when the stats come in, ed25519->curve25519 points looks like a cheap enough operation

yeah, it sounded like jberman was working on that, or did someone else need to pick that up?

he's working on it

Sorry it's taking a bit. I broke my dev env with the rewritten libsodium. Almost there

But in the meantime, in libsodium ge25519_frombytes_negate_vartime is where the first step in the conversion happens, is there something inside that function that is unnecessary?

to chop off some cycles in the conversion to x25519 point?

eh you might be able to chop off the recovery for the ed25519 x-coord in that function since x25519 doesn't compute/use it in the ladder, but I think that might bust the conversion to x25519

jberman yes you can skip that entire if statement that checks ge25519_has_small_order/ge25519_frombytes_negate_vartime/ge25519_is_on_main_subgroup

this is necessary: ge25519_frombytes_negate_vartime

that's your conversion from compressed format to ge_p3

ah yes, you're right

wait nvm I guess you don't need the x coordinate, so you can just 0-out the MSB

i've lost the source code, i will check

though I'm not sure what the 'negate' part is doing

i think it's just checking it's a valid point

since only 50% of points are valid

jberman that's ed25519

yes to go ed25519->x25519 they are deserializing the compressed point first

presumably to check the group element's order

i've lost my link to the libsodium code ethat does the ed25519->x25519

ah yes this github.com/jedisct1/libsodium/blob/…to_sign/ed25519/ref10/keypair.c#L46

so yes, we don't need lines 53->57 there at all

one problem is the deserialization code transforms the byte representation of the y coordinate into a `typedef int32_t fe[10];`, so it might not be straightforward to skip the `frombytes` step

knaccc: line 54 is initializing `A`

i think we can just read it as a field element instead

without any overhead of the full ge25519_frombytes_negate_vartime

if there is any overhead, that is

fe ops are on `fe`-typed variables, which are arrays of int32s

oh ok

you're right, we need the if statement then, but not the ge25519_has_small_order or ge25519_is_on_main_subgroup parts

i'm hoping we save a ton of cycles by eliminating ge25519_is_on_main_subgroup

ok we need this one github.com/monero-project/monero/bl…to_ops_builder/ref10/fe_frombytes.c

so you zero-out the MSB of the compressed point, then pass to fe_frombytes to get the y coordinate as an `fe`

nice

libsodium results at the bottom just commenting out `ge25519_has_small_order` && `ge25519_is_on_main_subgroup`: paste.debian.net/1229121

if I got that right, it looks like ed255519->curve25519 then remove extra ops in scalar mult is ~40% faster than just ed25519 variable base scalar mult :)

I ended up nuking trying to rewrite files in libsodium and just copy pasted what was needed from libsodium

and made sure i copy pasted correctly with a sanity check test

you commented things out in libsodium? I can't tell from your test what you did

In `ED25519_TO_CURVE25519_THEN_SCALAR_MULT_REMOVE_EXTRA_OPS`, I have the exact same function as this (github.com/jedisct1/libsodium/blob/…/ed25519/ref10/ed25519_ref10.c#L363), but commented out `ge25519_frombytes_negate_vartime`

lol

that's the one thing not to comment out :)

then I sanity checked that in a separate test by also calling `crypto_sign_ed25519_pk_to_curve25519` haha

um

isn't that UB? lol

wait sorry left that in, but commented out the others

can you just paste the crypto_sign_ed25519_pk_to_curve25519 function into your code?

and then you can have two versions there

and clearly visible

if (/* ge25519_has_small_order(pk) != 0 || */... (full message at libera.ems.host/_matrix/media/r0/do…85952616b7ac16133b52f524675ef7fa388)

ya

1 sec

you're saying the sanity check was just a repeat of the exact same thing as the non-sanity-check version?

knaccc: /* */ is notation for commenting things out

UkoeHB lol yes even I know that :) sorry, re: the sanity check thing i thought that was just written in as a note manually, but now i see it's written in by the code

the sanity check was just to prove it's copy pasted correctly by also running crypto_sign_ed25519_pk_to_curve25519's and testing the output is the same in a separate test. removed the sanity check there^

Also fully aware this is of critical importance and if someone wants to continue on this in my place who has a better understanding of exactly what each step is doing, totally feel free

so the conclusion is, we can go from 1087ms to 517ms by doing ed25519->curve25519 then scalarmult, or go all the way down to 403ms if we get agreement that curve25519 txpubkey points should be published in transactions instead of ed25519 ones

this is great stuff, jberman, thanks for your efforts

thanks for the ideas :D this is exciting

jberman i guess we need to include the hashes as part of the tests though

to get a proper idea

wouldn't its impact just be the same across each?

yes

i'm just not sure what the impact is

do you have timings already for the hash ops?

sure I'll throw it in, the code is right there

ya but it'll be easier to just add the code in here to show

alongside it what the impact is

whatever is easiest

jberman[m]: paste.debian.net/hidden/fd5a1aca this should work

err maybe a little compile bug on that array lol

jberman btw it'd be awesome if you could isolate just the hash cost, so we can compare hashes

UkoeHB: thanks :)

knaccc: UkoeHB has some good tests set up across cn_fast_hash, siphash, blake2, lemme find it

oh forgot I did blake2

blake3 is way faster than blake2

in here: github.com/UkoeHB/monero/blob/serap…tests/performance_tests/view_scan.h .. I'll copy those in

i recall in testing the hash cost was super insignificant

could've been something to do with the threading structure. but I don't recall a meaningful difference between assuming the output isn't owned by the account and hashing

but ya every inch counts

when i say "in testing" I mean when testing scanning against chain outputs

jberman what are we talking, in terms of milliseconds per thousand keccak256 hashes?

i'm just trying to put it in context

that I don't recall. I also made a lot of changes to the threading structure after I ran those tests, so I'll just need to re-run stuff and will give you the most up to date answer :)