10:57 <jberman[m]> If you construct a tx on a particular date and then don't submit the tx, then the ring(s) your wallet constructed will be saved on disk and reused if you spend the input in the future

Woah if didn’t know this, is there no expiration date?

Does it use the old ring even if later combined with other inputs?

That actually may be a plausible explanation for some of the transactions we’ve seen

No expiration date (you can manually erase them). Also with other inputs (why would this have an impact on it ?)

(it's interesting whether audit is finished already or temporary paused due to planned conference)

<fluffypony> "wallet2 assumes that the..." <- Is there a downside to creating a cache for remote node cases?

Feather used this: feather-wallet/monero 089ba9d

^ that's not live, btw. I committed this an hour ago. It builds upon on earlier patch to mitigate output distribution poisoning: feather-wallet/monero fdd5c63

When using the "std=c17 -pedantic" flag, the function getloadavg() is supposedly not defined: "implicit declaration of function", because it's a gnuc function (as I understand). How would you handle this, just ignore the warning (because the code runs in Linux, BSD) or #ifdef some value

how is it related to monero ?

We can use the patch tobtoht made, but we'd rather it be merged upstream ofc. Is there a way to only persist the cache if it's using a non-localhost node?

You want to cache what data ?

And by non localhost, you mean trusted or any random asshole from the internet ?

In our case, it usually means trusted, since it's our nodes or a node a user manually specifies

cache the rct_distribution

currently the first spend on mobile takes a few extra noticeable seconds

ooo123ooo1234567: "Don't ask to ask, just ask", I had to try :)

<sgp_[m]> "currently the first spend on..." <- great, if it's too painful then write and submit patch that will cache it correctly, or use arbitrary patch locally

<ilgatu> "When using the "std=c17 -..." <- gnu.org/software/libc/manual/html_node/Feature-Test-Macros.html, that function is an addition from glibc, it isn't available with strict c standard

it's available with -std=gnu17, but not -std=c17

what's the purpose to use -std=c17 without understanding what it means ?

ooo123ooo1234567: So there's no way to use gnuc functions in a c standard code. Ok.

ooo123ooo1234567: Thanks for your time, btw.

I would not cache stuff that can be total bullshit from some random node. Unless trusted. But we know people, they'd just set any random node as trusted anyway. So probably not even if trusted.

ilgatu: Check xyproblem.info helps too

Or maybe cached per IP, if not Tor.

moneromooo: In the way I implemented this only data up until a checkpoint height is cached and must match a hardcoded hash. The wallet will throw an error if a random node sends bullshit data (anything pre checkpoint height) and will not continue with tx construction.

One issue with this approach is increased maintenance burden, as the hardcoded hash should be updated with each release.

I agree caching non-checked data from random nodes is a bad idea.

ofrnxmr[m]1: Hi, thanks a lot. A new perspective, I used to see it the other way. Again, thanks for your time.

OK, that's a good idea.

Alright, I'll work on a PR.

Thank you

Well, shipping the known good hash to prevent bullshit data is a good idea :D Whether adding such caching in monero is, I dunno. Maybe. I'm kinda torn.

indeed, why not to add another centralized checkpoint for 2MB of data

cake-tech/cake_wallet #395, interesting who are those Monero developers

"interesting who are those Monero developers" Do you mean this as a statement, or as a question?

* cake-tech/cake_wallet #395, it's interesting, who are those Monero developers ?

Well, sethforprivacy is, among other things, a dev, but hardly active on Monero code proper, and SamsungGalaxyPlayer isn't a dev as far as I know

But well known for their "Breaking Monero" series of videos: monerooutreach.org/breaking-monero

er, his videos

I'm not really a dev to be honest

I was talking with jberman and kayabanerve @kayabanerve:matrix.org about this issue because I had a suspicion this was the UX problem I kept running into

They confirmed it was likely output distribution for ring generation and seems they were right

This issue is one of the few constant PITA hurdles for me still in using Monero, and is especially onerous in person (as I was at Monerokon trying to spend and had to wait like 20s before I could even submit the TX)

More jberman than me :) I was there to discuss the effects of arithmetic circuits on it and discuss doing it in Rust :p

Not caching this data is also a bandwidth and IO load on all remote nodes, especially more commonly used public ones like mine unnecessarily.

Though the fix I did only caches it per application lifetime. Not too suitable for deployment to wallets...

Interesting. I have looked around quite a number of times in the wallet2 code for various reasons and never stumbled over this. Doesn't tell me anything at all.

rbrunner: jberman's comment is it's a 10 MB data set

<sethforprivacy> "This issue is one of the few..." <- is it reproducible with fast internet connection and not overloaded monerod ?

I have run into it many times and always notice that the first transaction after wallet open is excessively slow

3s+

Which is very poor UX

25s just now

lol

is it reproducible with monero-wallet-cli ?

Idk, that's not the target audience of the two wallets I opened issues for

We shouldn't base UX assumptions on a CLI wallet.

If mobile wallets all have poor UX when sending transactions it has to be fixed one way or another

25s to create a transaction with my own private full node over 300mb/s internet (through VPN, but very fast VPN) is nuts

sethforprivacy: can you try to reproduce it with my node and cake wallet or whatever wallet you are using? 88.198.199.23:18081

I would hazard a guess most of the delay is IO-bound but will look into more.

selsta: Sure

check with cli wallet is mostly needed to know who to blaim for that delay

if it only happens with one wallet it's possibly a bug in that wallet

selsta: 6s but not sure if it was relying on any cache as I didn't close the wallet

Testing again

1. do end-to-end debug with 1 wallet; 2. check different wallets to localize problem; since 1 is too hard probably, why not to do 2 ?

27s with your node on a fresh wallet open selsta

Testing with monerujo now

how many inputs?

not sure if that makes a difference

idk, just sending a small TX to test, but didn't ever send it

Probably just 1

sethforprivacy: and the wallet software is ... ?

10s with Monerujo

Others were Monero.com wallet (Cake)

1s after initial transaction is constructed/sent in Monerujo

Cake is 10-12s for some reason even after initial construction (but I didn't send the TX yet, so maybe its just dumping that output cache?)

lots of variance with cake, was 4s now

it took me around 2s to generate a tx with the gui and a freshly deleted shared-ringdb so nothing should be cached

25s with Monerujo with your node selsta

selsta: Local node or your remote?

remote

but my remote node is in Europe, and you are from US I think, so not ideal for you latency wise

is it possible to test with monero-wallet-cli ?

selsta: Yeah

ooo123ooo1234567: Sure

same 2 seconds with CLI, will try Cake Wallet

it must be something specific to environment of sethforprivacy

Can no one else test but me and selsta?

What do you see ooo123ooo1234567?

Fwiw, I implemented this in Feather because constructing the first transaction over Tor (which is inherently slow) takes a long time. If you're on a bad circuit downloading 6-7 MB worth of data takes a while.

Anyone test with Cake/Monerujo?

tobtoht[m]: that isn't even 6MB of data

sethforprivacy: I'm just waiting for a test with monero-wallet-cli in your environment

* a test result with monero-wallet-cli

Working on it, had to restore as I dont use cli

Will test with local node, remote node of my own, and selsta's remote node.

ooo123ooo1234567: You can test it too to give us another data point 🙂

If you delete ~/.shared-ringdb and also restart the program there should nothing be cached.

Use selstas node, mine (node.sethforprivacy.com:18089) and one of your choosing

sethforprivacy: restoring wallet with monero-wallet-cli isn't slower than with any of those wallets

* of those mobile wallets

ooo123ooo1234567: ?

No idea what that has to do with anything

I had to restore a wallet with funds so I can test

I'm not claiming its slow?

Please test yourself and report back

* Please test transaction generation w/o cache yourself and

If there's a question about this being slow, it's something I've noticed for a while but never realized what was causing the issue

Think everyone just went "it's always like this so I'll ignore it", I know I did

But seems a problem for everyone I've spoken to when they realize it

yup

Waiting 10-20s to generate a transaction on mobile is really, really shitty UX

So lets find the problem (I think we already did) and find a solution (seems like tobtoht already did) and implement it.

2s with CLI on local node

~3s with my remote node from CLI

on a strong connection with Cake to my own remote node (also with a strong connection) the first signing process only takes about 1s right now. but I definitely have seen it takes 6 or so seconds (usually not more)

unsurprisingly, it's variable based on how quick one can download the data

sethforprivacy: with restart and deleted shared-ringdb?

~3s for selsta node

that sounds normal

selsta: Yes, deleted ~/.shared-ringdb between each wallet start

It's created immediately on wallet open though, not on transaction generation

So it may "feel" faster if its being done on wallet open and not on transaction creation

That's probably the main difference here between Cake and CLI

I don't think that's the case here

creating a file by itself does not mean much

Then why is that folder/DB files created on wallet open w/o transaction being generated?

Created that without me trying any transactions

it's a tiny file that does not make any speed difference

Much smaller than before I deleted it initially, true

Didn't read the size properly 🙂

~/.shared-ringdb doesn't cache the output distribution and the output distribution is only requested when the first transaction is created. It affects a different part of transaction construction.

I just know that it caches something and deleting it doesn't hurt for testing.

was monero-wallet-cli started within the same mobile phone ?

ooo123ooo1234567: No, on another device

Don't have binaries or anything on mobile

Can do that but will be a bit

Same network though

i'm still syncing cake wallet

selsta: Yes, if you create a transaction and don't submit it ~/.shared-ringdb will store rings, which speeds up subsequent tx construction because it can reuse those rings instead of asking the node for outs.

I keep force closing cake and reopening, and the first tx generation often takes 1-2 extra seconds, again on a pretty ideal connection setup for remote trusted node

so who to blaim for that delay ?

I'd like to test with the local distribution cache and see if that improves it :)

not looking to blame, just trying to test improvements to see if they help with UX

i feel like these are two different topics, 1-2s extra vs 25s tx creation

incorrect code is much bigger PITA for users, than 1-2 seconds optimization

on tor and a roaming phone connection it indeed makes sending monero terrible

let me try with orbot to give you an idea

in the worst case correctly implemented cache will be redundant, in the best case it will optimize those few seconds

but it isn't a priority to use another centralized hash for 1-2MB of data instead of correctly implemented cache if it's even possible

Not really sure what you're trying to say.

Do you have a better proposed fix for this? Are you saying it's not a problem? Have you tested this at all?

Still haven't seen any test results from you.

let's firstly find what is causing that delay in your environment

s/find/identify/

sethforprivacy: if it takes 3s with monero-wallet-cli and significantly longer with cake wallet on the same connection than it's unrelated to this cache

Mobile is also using VPN but not sure how that could add that long

Can no one else reproduce this?

ideal test setup would be to try monero-wallet-cli on mobile

Other than sgp to a lesser degree

like i said my cake wallet is still syncing

selsta: Yeah syncing it now

sethforprivacy: any way to ssh into your environment ?

do you know if there's any crowdfunding service based on XMR open for the public beside CCS?

Give you SSH access to my env?

No

lol

otherwise it's impossible to reproduce outside of it

sethforprivacy: great, waiting for it

You can test the same different approaches with the same nodes and gauge how it works for you.

Much more useful if many people test this

81 seconds on first send using Cake + Orbot. 22 seconds on second send

1st send wasn't relayed, and 2nd send was using the same outputs ?

sgp_: holy shit lol

correct, didn't relay. just canceled and started over for second

if it'll make a difference I can try sending the first one and then queuing up second

"25s to create a transaction with my own private full node over 300mb/s internet (through VPN, but very fast VPN)" orbor is quite the opposite of this setup

It's a bandwidth issue. If you reduce the amount of bandwidth required to construct a transaction by 97% then transaction construction is going to be faster. It's really that simple.

s/bandwidth/data/

s/orbor/tor/orbot/

Yeah I'm not really sure what ooo123ooo1234567 is getting at -- are you proposing a different fix or saying this isn't an issue?

We're proving that the worse your network connection the worse the UX for sending because your wallet isn't caching rct_distribution

my node has gig up and I have something like 600 down here so there was only a tiny delay there. but with Tor it took forever, which makes sense

At least thats the idea we have for why it's happening.

sgp_: should be different

s/because/_because/, s/your/the/, s/rct_distribution/rct\_distribution_/

I'll send numbers in a minute then for send 1 output on fresh launch, then second output immediately after

sethforprivacy: what's about wallet synchronization with such slow connection ?

ooo123ooo1234567: Also of course a problem

But if there is an easy fix here (there is not for broader wallet sync) than we need to explore it.

I still have no idea what the goal of your arguments are.

I think they're most against the checkpoint, but maybe we can just enable the cache only for trusted daemons

Here is some data: transaction construction (1 input) without output distribution cache uses 1.58 MB (bytes received only).

Of this, 1.53 MB is needed to obtain the output distribution. (And this will keep increasing as the chain gets longer).

So, a less substantial size than thought. (I looked at memory, but there is some compression going on here).

But, it still makes up 97% of the network traffic required for initial tx construction.

If checkpoints are not the way to go, a compromise is to grab the output distribution as soon as synchronization is finished.

This way the output distribution will typically be available when the user wants to create a transaction.

is 80s normal to get 1.6MB over Tor?

tobtoht[m]: wow, less 2 MB

selsta: no, it depends on setup

It's pretty fast nowadays. 30 kB/s is rare now. I often get like 200+ or so.

It can be slow to setup a good circuit first though.

first output send: 1m40. second output send: 14s

But, also, much less than, say, ten years ago.

obviously there will be variability over tor and I haven't optimized mine at all

tobtoht[m]: checkpoint is a compromise, correctly implemented cache with asynchronous fetching isn't a compromise

> <@tobtoht:monero.social> Of this, 1.53 MB is needed to obtain the output distribution. (And this will keep increasing as the chain gets longer).

* wow, less than 2 MB

on mobile roaming I'm capped at 128kbps so sending does indeed feel like it takes forever currently for first time

just to give an example. obviously everyone's plans are different

ooo: can you describe how your ideal implementation differs from what I proposed?

I'm waiting for a test with monero-wallet-cli from sethforprivacy

on that mobile device with fast vpn

With a fast VPN this isn't much of an issue.

yeah it's like 1-2s only

Almost done syncing

without any code changes required data size reduced from 10MB to 1.5MB, the quickest fix ever

?

Going out of your way to be antagonist to people trying to give useful feedback and improve Monero is not a good look.

An "off the top of my head" calculation over dinner is just that, an estimation

I can't even sync 5k blocks over Tor currently, wonder if the Tor network itself is having issues or I just have to restart a couple times until I get a good circuit

tor is indeed having issues

selsta: Tor DoS is still having an impact IIRC

been under DDoS for like a mo

Really. I haven't noticed :S

that's why all wasabi 2.0 mixes haven't been going though, lol

Then again, I think nothing of waiting a minute for connection to happen from time to time...

22s with monero-wallet-cli on 1st send

2s on subsequent

what kind of phone hardware?

hmm, much larger than I though. What's your down speed and ping?

I'm using a non-ideal VPN server, 30ms ping and 66mbps down

Changed VPN server, 18ms ping, 81mbps down

Testing again

19s first spend, 2s subsequent

subsequent = pressing cancel and transact again?

without any restart

No, sending and then sending again

Actually sending

No restart

18s first spend, 6s subsequent for selsta node

Deleted ~/.shared-ringdb between wallet runs

phone hardware?

SoC?

Pixel 6

So their first-gen Tensor SoC

Cutting edge 🤷

the extra downloaded 1.58MB don't explain the difference you are seeing

Mine is S20+ fwiw, snapdragon 865

What else would be causing this?

Network is good, phone is latest gen, nodes are known-good

I still have 200k blocks on my phone to scan

And subsequent spends are just as quick as expected

It's something being cached

(being cached for subsequent spends and not for initial spends, to be clear)

90% reduction in generation time between first spend and subsequent.

If you want to check what's slow, did you try: sudo perf top (it will show if something's being precalced), strace (with timestamps), or just log level 2 (with timestamps), --no-dns ?

Whatever is being gathered when you try and construct a transaction should be gathered on initial sync, cached, and then updated with each subsequent sync. Shouldn't wait for transaction generation anyways.

moneromooo: it's on a phone

1.58MB data, over 66mbps should be downloaded in less than a second

20s difference simply doesn't add up, in no way

That phone doens't run strace or perf ?

sethforprivacy: does perf top work on a phone?

idk haha

Can try but have never done that before

log level 2 should work

and --no-dns

OK

the timestamps should tell what takes how long

Also if it has gdb, gdb into the process. If it's stuck waiting in http client, it's just waiting for the daemon.

(and the upper levels of the stack should tell you what was requested)

Node on log level 2 or wallet cli?

wallet

node might be helpful too but would check wallet first

calling get_output_distribution manually should also show us how long it takes

DMing to selsta

<sethforprivacy> "Network is good, phone is latest..." <- is it reproducible with `--no-dns` ?

6-8s with --no-dns

For first spend, so much more reasonable but still slow

can you check manually in logs what took most of the time ?

9s according to logs

I'll let selsta dissect them

2022-06-24 18:54:22.168 c8ec46223000 DEBUG wallet.wallet2 src/wallet/wallet2.cpp:3557 Daemon is recent enough, requesting rct distribution

that took 5s

and other 4 seconds ?

transaction <xx> generated ok and sent to daemon, key_images: [<xx> ]

2022-06-24 18:54:31.285

And initial transfer command was at `2022-06-24 18:54:18.204`

can you paste obfuscated log with timings ?

ooo123ooo1234567: Yeah working on that now

selsta: Before or after this line?

A lot to redact lol

tobtoht[m]: wait, i sent it in a confusing way

github.com/monero-project/monero/bl…master/src/wallet/wallet2.cpp#L9807, github.com/monero-project/monero/bl…aster/src/wallet/wallet2.cpp#L10296

what are the timestamps for these two lines ?

ooo123ooo1234567: 18:54:21.963 and 18:54:27.553

no

github.com/monero-project/monero/bl…7.3.2/src/wallet/wallet2.cpp#L10066, where is this msg in log ?

2022-06-24 18:54:27.553 c8ec46223000 INFO wallet.wallet2 src/wallet/wallet2.cpp:10069 Transaction 1/1 <xxx>: xxx weight, sending xxx in 1 outputs to 1 destination(s), including xxx fee, xxx change

this one is less redacted paste.sethforprivacy.com/?616953025…kK3oMkbxrKSrtVfpwRHCHU57nkeH7pTd8Wm

We're not calling get_output_histogram because we're dealing with rct outs, and we're not is_after_segregation_fork. Is there anything else after "Daemon is recent enough, requesting rct distribution" other than the get_output_distribution.bin call that could cause the delay?

get_ringdb_key() key possibly? Looks like it's cached.

cake wallet took less than 3s, in line with monero-wallet-cli and gui for me

on iOS at least

on both home and mobile connection, but i have fast mobile internet here

<tobtoht[m]> "get_ringdb_key() key possibly..." <- Nvm, would have shown up in the logs. And it's called after opening the wallet.

create_transactions_2 - 5.6s, get_output_distribution - 5.08s; with 334KB/s connection and --no-dns

<sethforprivacy> "For first spend, so much more..." <- are there any difference between 1st and 2nd sends without dns ?

s/are/is/

Not sure, will test now

ooo: is that get_rct_distribution() or only the /get_output_distribution.bin call?

8s first spend, 7s subsequent

both 1st and 2nd were relayed ?

yes

network is 20ms ping, 1.55ms jitter, 90mbps down, 180mbps up

Using selsta's node.

what's the ping to my node?

I'm pretty sure we can remove this m_node_rpc_proxy.get_rpc_version() call. There is no daemon out there that doesn't support this. That would save one round trip.

+1

got added 4 years ago

<sethforprivacy> "8s first spend, 7s subsequent" <- is it obvious why dns is so slow there ?

<selsta> "what's the ping to my node?" <- 140-200ms

> <@selsta:libera.chat> what's the ping to my node?

* 140-250ms

I saw 400 at one point when testing

yea, sorry my node is only good for Europe :D

Yeah haha, its expected

<sethforprivacy> "19s first spend, 2s subsequent" <- what's the delay with your own node with --no-dns ?

selsta: monero-project/monero #8404

ooo123ooo1234567: 2-3s

wow, is it enough for good UX without tor ?

Better than 20s

What about no-dns seems to be improving it this much?

might be a problem with your VPN DNS?

I think on desktop you didn't use VPN?

I'm using custom NextDNS, not VPN-based

I don't see any other DNS issues or latency with other apps, browsing, etc.

records from centralized monero dns aren't reachable instantly for some reason

`drill -t txt @8.8.8.8 segheights.moneropulse.org`; `drill -t txt segheights.moneropulse.org`

what's the delay for this dns requests ?

What is `drill`?

Have never seen that before and not in apt

use any other tool for dns requests from terminal

`time (for d in org net co se; do drill -t txt segheights.moneropulse.org &; done; wait )` the wallet is doing 4 requests, maybe one of them is too slow for your environment

43ms vs 52ms

Negligible

s/.org/.$d/

meh thats just cached now

Originals here:... (full message at libera.ems.host/_matrix/media/r0/do…918cfa076293cc5ae03ee6cb44885a9a507)

did you try to from that mobile device ? and without 8.8.8.8 server

s/to//

Sorry no, testing now

<tobtoht[m]> "I'm pretty sure we can remove..." <- monero-project/monero #5600/files, there is a cache for rpc version, it would be good to use it

We don't need the check at all.

indeed, that's true if no one is relying on that error with old daemon

`git grep 'MAKE_CORE_RPC_VERSION'` why not to remove all of them then ?

<sethforprivacy> "Going out of your way to be..." <- all involved people were busy with optimizing minor thing with another centralized checkpoint, it's pretty good look to not accept it

<ooo123ooo1234567> "`git grep 'MAKE_CORE_RPC_VERSION..." <- Sure.

<sethforprivacy> "Sorry no, testing now" <- what was the result ?

"nitter.42l.fr/sethforprivacy/status/1540322475355082754#m" facepalm

an interesting question is how to do development of decentralized project in order to avoid such vulnerabilities

<ooo123ooo1234567> "what was the result ?" <- Had to step away, sorry, might get to it later or tomorrow.