02:20:32 .merge+ 8163 8161 02:20:32 Added 03:12:09 I'd like to chat about adding a key to the monerod rpc "get_info" call. The idea is to provide remote node operators the ability to return an arbitrary string via a runtime environment variable MONERO_NODE_INFO. If that's better discussed via a github issue please lmk (I have a patch ready if that's the case lol) 03:17:05 With respect to the fees question I have update MRL issue 70 and referenced pr 7819 with the change in growth rate of ML from 2 to 1.7. I trust this will allow us to proceed with this. 03:18:37 See my comment above ArticMine I will incorporate it into my comment on issue 70 08:15:48 jberman[m] did you do the variant of the test where you comment out the "ge25519_has_small_order/ge25519_frombytes_negate_vartime/ge25519_is_on_main_subgroup" bits? 08:15:56 i think that's most of the overhead 08:17:33 jberman[m] and i think those are all with libsodium, and not sandy2x right? 08:22:10 if we do decide we need the "ge25519_has_small_order/ge25519_frombytes_negate_vartime/ge25519_is_on_main_subgroup" checks, we can do that after the view tag matches instead of before 11:17:37 UkoeHB: Great job with Seraphis! 11:22:48 Hi everyone I have a question regarding the system of view keys and key images. I've read that "because view-only wallet doesn't have key images, it can't see outgoing transactions." When we use a hardware wallet to export the view key, then how the wallet can guess our balance? Does the hardware wallet export key images at the same time it exports the view key? 11:52:04 k.w.x.m.r: When a hardware wallet signs a transaction (ie spends) it must include the key image. The online, view only, wallet can just save the key image then. So, as long as there is only one online wallet, there is no more need for additional communication. 11:55:37 k.w.x.m.r: But an online wallet will, obviously, know all of the recieved outputs and know if it is missing any key images. When it finds an output for which it does not have a key image, it needs to request that key image from the hardware wallet. I imagine this is done every time you use you connect you hardware wallet. 11:56:15 Ok thanks! 11:56:48 But why the wallet just says exports viewkey and doesn't mention we are also exporting key images? 11:58:07 I assume it's for making it easier for users and don't have to ask 2 kind of permission, right? 12:05:51 UkoeHB: On your Monero Talk interview you seem to doubt that seraphis will be useful to force exchanges to reveal their balance. May I ask why? As far as I understand since it will be possible to have view wallets that reveal full balance then why an exchange won't be able to reveal its xmr on a cold storage address? Thanks 13:13:18 kwxmr[m]: what kind of company wants to publicize all their internal finances? 13:15:13 I mean from a technical point of view, is there anything that would prevent them? Because otherwise it will just a matter of pressure from the community and in this case that a huge game changer for xmr 13:16:41 s/I mean from a technical point of view, is there anything that would prevent them? Because otherwise it will just a matter of pressure from the community and in this case that a huge game changer for xmr/UkoeHB: I mean from a technical point of view, is there anything that would prevent them? Because otherwise it will just be a matter of pressure from the community and in this case that's already a huge game changer for xmr/ 13:17:08 I guess not? If your only criteria is 'cant spend with the key you publicize'. You can do literally everything else with the seraphis/jamtis view-balance key. 13:21:06 This is absolutely huge! Thanks for confirmation 14:22:45 / 18:59:27 "jberman and i think those are..." <- They were with libsodium right. Working on the rest, took me a bit to get set up with rewritten libsodium files :) 19:02:01 jberman[m] thanks, i'm on the edge of my seat :) 21:41:18 knaccc: sandy2x is for x25519 and only works on sandy bridge+ cpus (no arm64) 21:42:55 vtnerd have you seen this? https://research.nccgroup.com/2020/09/28/faster-modular-inversion-and-legendre-symbol-and-an-x25519-speed-record/ 21:45:00 also, sandy bridge is now 11 years old. since this is only for curve25519, could we include sandy2x and use it where suitable CPUs are available, and fall back to ref10 elsewhere? 21:46:02 can you send xmr multiple times from the same input or is there a limitation on that? 21:46:19 if there is a limitation, does the change transaction show up as a new transfer in the wallet RPC? 21:46:48 WillMorrison[m] inputs are always completely spent, and you get change left over 21:47:15 knaccc: does the change stay in the same address or is there a way to direct it to a certain address? 21:48:03 there are wallet rules, i'm not sure but i think it's normally sent to the first subaddress in the account 21:48:32 Oh ok great 21:48:33 thanks for the help 21:50:04 yes, sandy bridge is has been out a while, but I pointed at donna64 for arm64 arch 21:50:23 that code will not run on such cpus, and apple has now switched to that arch entirely 21:51:32 and that nccgroup.com link you sent is the first time I've seen that 21:52:01 vtnerd: so referring only to our use of x25519 and not ed25519: would there be any objection to including sandy2x and falling back to donna64? 21:52:37 (falling back for aarch64) 21:53:09 in what context? wallet scanning or verification in general? 21:53:54 vtnerd i was only thinking about using curve25519 for wallet scanning 21:54:34 and we'd need it so that the wallet can build the view tag for the tx in the first place too 21:54:52 what? why would it necessarily be needed for view tags? 21:55:12 or do you mean you plan to switch the protocol entirely during this change? 21:55:39 vtnerd i'm not sure if we're talking at cross purposes. monero doesn't use curve25519 anywhere at all right now, right? 21:55:51 correct 21:56:22 so we're only considering using x25519 for generating and checking view tags, because that could result in a huge speedup 21:56:53 everything else would remain ed25519, including how the one-time output public keys and shared secret are generated 21:56:59 correct, which then (kind of almost) forces everyone downstream (not just wallet2!) to use x25519 in addition to the this new view tag implementation 21:57:23 well the good news is that the view tags don't have to be paid any attention to 21:57:29 or they could just ingore the view tag and compute as normal 21:57:30 you can still scan the existing way 21:57:33 right exactly 21:58:12 and x25519 is so widely used that i can't imagine anyone would have any library issues 21:58:23 actually i take that back 21:58:58 the scalar clamping on x25519 needs to be removed, so we don't break compatibility with our existing ed25519 scalars and x25519 scalars 21:59:16 so we do need to mod the x25519 libraries we use to get rid of clamping 21:59:32 ah yeah I forgot about that one, yeah different points will be computed if that remains 22:00:29 I always avoided ed25519->x25519->ed25519 because it required two conversions, but it sounds like the 2nd conversion only occurs iff the view tag matches 22:01:03 the numbers should work better in that case, but the monero ed25519 isn't compatible as you noted, so eh 22:01:46 vtnerd i only realised yesterday: going from ed25519->curve25519 points is fine, but the sign of the ed25519 point is lost so there is ambiguity if you try to go back 22:02:00 so we'd only go one way 22:02:52 yes, theres that fun aspect (I mentioned above but it was in a bunch of stuff) 22:03:27 oh but see wait, you wouldn't convert back? then that changes the protocol entirely, it forces x25519 on every wallet immediately 22:04:10 i'm saying we don't change the shared secret for the actual output pubkey 22:04:14 only for the view tag 22:04:34 iirc it might be doable because that result does a hash_to_point, so you only need a consistent bit pattern to get the final output address 22:04:37 so if the view tag matches, we then compute the ed25519 shared secret from scratch 22:05:12 ah. that changes a bunch of the numbers that have been calculated, yea dunno 22:06:17 we haven't get gathered enough stats to decide whether to advocate for tx pubkeys to be published as curve25519 points in the first place 22:06:45 but i'm hopeful that when the stats come in, ed25519->curve25519 points looks like a cheap enough operation 22:06:57 yeah, it sounded like jberman was working on that, or did someone else need to pick that up? 22:07:08 he's working on it 22:12:18 Sorry it's taking a bit. I broke my dev env with the rewritten libsodium. Almost there 22:12:37 But in the meantime, in libsodium ge25519_frombytes_negate_vartime is where the first step in the conversion happens, is there something inside that function that is unnecessary? 22:13:43 to chop off some cycles in the conversion to x25519 point? 22:15:08 eh you might be able to chop off the recovery for the ed25519 x-coord in that function since x25519 doesn't compute/use it in the ladder, but I think that might bust the conversion to x25519 22:15:33 jberman yes you can skip that entire if statement that checks ge25519_has_small_order/ge25519_frombytes_negate_vartime/ge25519_is_on_main_subgroup 22:15:49 this is necessary: ge25519_frombytes_negate_vartime 22:16:02 that's your conversion from compressed format to ge_p3 22:16:22 ah yes, you're right 22:16:35 wait nvm I guess you don't need the x coordinate, so you can just 0-out the MSB 22:16:53 i've lost the source code, i will check 22:16:58 though I'm not sure what the 'negate' part is doing 22:17:10 https://github.com/jedisct1/libsodium/blob/447cd270d993d30bcf1ca436a416d6a052694542/src/libsodium/crypto_core/ed25519/ref10/ed25519_ref10.c#L363 22:17:11 i think it's just checking it's a valid point 22:17:15 since only 50% of points are valid 22:18:01 jberman that's ed25519 22:18:32 yes to go ed25519->x25519 they are deserializing the compressed point first 22:18:42 presumably to check the group element's order 22:19:37 i've lost my link to the libsodium code ethat does the ed25519->x25519 22:20:29 ah yes this https://github.com/jedisct1/libsodium/blob/6d566070b48efd2fa099bbe9822914455150aba9/src/libsodium/crypto_sign/ed25519/ref10/keypair.c#L46 22:21:01 so yes, we don't need lines 53->57 there at all 22:21:07 one problem is the deserialization code transforms the byte representation of the y coordinate into a `typedef int32_t fe[10];`, so it might not be straightforward to skip the `frombytes` step 22:21:45 knaccc: line 54 is initializing `A` 22:21:56 i think we can just read it as a field element instead 22:22:14 without any overhead of the full ge25519_frombytes_negate_vartime 22:22:45 if there is any overhead, that is 22:22:46 fe ops are on `fe`-typed variables, which are arrays of int32s 22:22:54 oh ok 22:23:27 you're right, we need the if statement then, but not the ge25519_has_small_order or ge25519_is_on_main_subgroup parts 22:23:53 i'm hoping we save a ton of cycles by eliminating ge25519_is_on_main_subgroup 22:26:45 ok we need this one https://github.com/monero-project/monero/blob/master/src/crypto/crypto_ops_builder/ref10/fe_frombytes.c 22:27:14 so you zero-out the MSB of the compressed point, then pass to fe_frombytes to get the y coordinate as an `fe` 22:27:53 nice 22:36:56 libsodium results at the bottom just commenting out `ge25519_has_small_order` && `ge25519_is_on_main_subgroup`: https://paste.debian.net/1229121/ 22:37:11 if I got that right, it looks like ed255519->curve25519 then remove extra ops in scalar mult is ~40% faster than just ed25519 variable base scalar mult :) 22:37:24 I ended up nuking trying to rewrite files in libsodium and just copy pasted what was needed from libsodium 22:40:07 and made sure i copy pasted correctly with a sanity check test 22:41:17 you commented things out in libsodium? I can't tell from your test what you did 22:42:21 In `ED25519_TO_CURVE25519_THEN_SCALAR_MULT_REMOVE_EXTRA_OPS`, I have the exact same function as this (https://github.com/jedisct1/libsodium/blob/447cd270d993d30bcf1ca436a416d6a052694542/src/libsodium/crypto_core/ed25519/ref10/ed25519_ref10.c#L363), but commented out `ge25519_frombytes_negate_vartime` 22:42:47 lol 22:43:09 that's the one thing not to comment out :) 22:43:11 then I sanity checked that in a separate test by also calling `crypto_sign_ed25519_pk_to_curve25519` haha 22:43:11 um 22:43:31 isn't that UB? lol 22:43:31 wait sorry left that in, but commented out the others 22:43:41 can you just paste the crypto_sign_ed25519_pk_to_curve25519 function into your code? 22:43:52 and then you can have two versions there 22:43:57 and clearly visible 22:44:08 if (/* ge25519_has_small_order(pk) != 0 || */... (full message at https://libera.ems.host/_matrix/media/r0/download/libera.chat/793be85952616b7ac16133b52f524675ef7fa388) 22:44:39 ya 22:44:42 1 sec 22:45:08 you're saying the sanity check was just a repeat of the exact same thing as the non-sanity-check version? 22:50:18 knaccc: /* */ is notation for commenting things out 22:53:15 UkoeHB lol yes even I know that :) sorry, re: the sanity check thing i thought that was just written in as a note manually, but now i see it's written in by the code 22:54:10 https://paste.debian.net/1229123/ 22:55:16 the sanity check was just to prove it's copy pasted correctly by also running crypto_sign_ed25519_pk_to_curve25519's and testing the output is the same in a separate test. removed the sanity check there^ 22:55:42 Also fully aware this is of critical importance and if someone wants to continue on this in my place who has a better understanding of exactly what each step is doing, totally feel free 22:56:25 so the conclusion is, we can go from 1087ms to 517ms by doing ed25519->curve25519 then scalarmult, or go all the way down to 403ms if we get agreement that curve25519 txpubkey points should be published in transactions instead of ed25519 ones 22:56:37 this is great stuff, jberman, thanks for your efforts 22:57:34 thanks for the ideas :D this is exciting 22:57:51 jberman i guess we need to include the hashes as part of the tests though 22:58:06 to get a proper idea 22:58:36 wouldn't its impact just be the same across each? 22:58:47 yes 22:58:53 i'm just not sure what the impact is 22:59:20 do you have timings already for the hash ops? 22:59:39 sure I'll throw it in, the code is right there 23:00:22 ya but it'll be easier to just add the code in here to show 23:00:36 alongside it what the impact is 23:01:23 whatever is easiest 23:02:10 jberman[m]: https://paste.debian.net/hidden/fd5a1aca/ this should work 23:04:26 err maybe a little compile bug on that array lol 23:11:49 jberman btw it'd be awesome if you could isolate just the hash cost, so we can compare hashes 23:23:28 UkoeHB: thanks :) 23:24:01 knaccc: UkoeHB has some good tests set up across cn_fast_hash, siphash, blake2, lemme find it 23:25:04 oh forgot I did blake2 23:25:23 blake3 is way faster than blake2 23:25:36 in here: https://github.com/UkoeHB/monero/blob/seraphis_perf/tests/performance_tests/view_scan.h .. I'll copy those in 23:26:34 i recall in testing the hash cost was super insignificant 23:27:29 could've been something to do with the threading structure. but I don't recall a meaningful difference between assuming the output isn't owned by the account and hashing 23:28:23 but ya every inch counts 23:29:12 when i say "in testing" I mean when testing scanning against chain outputs 23:31:47 jberman what are we talking, in terms of milliseconds per thousand keccak256 hashes? 23:31:54 i'm just trying to put it in context 23:34:26 that I don't recall. I also made a lot of changes to the threading structure after I ran those tests, so I'll just need to re-run stuff and will give you the most up to date answer :)