01:22:08 <isthmus> 10:57 <jberman[m]> If you construct a tx on a particular date and then don't submit the tx, then the ring(s) your wallet constructed will be saved on disk and reused if you spend the input in the future
01:22:20 <isthmus> Woah if didn’t know this, is there no expiration date?
01:22:42 <isthmus> Does it use the old ring even if later combined with other inputs?
01:23:48 <isthmus> That actually may be a plausible explanation for some of the transactions we’ve seen
07:30:44 <moneromooo> No expiration date (you can manually erase them). Also with other inputs (why would this have an impact on it ?)
13:24:13 <ooo123ooo1234567> (it's interesting whether audit is finished already or temporary paused due to planned conference)
13:34:42 <sgp_> <fluffypony> "wallet2 assumes that the..." <- Is there a downside to creating a cache for remote node cases?
13:35:00 <sgp_> Feather used this: https://github.com/feather-wallet/monero/commit/089ba9dde963b3d235c956bead679c9865a92e7c
13:42:15 <tobtoht[m]> ^ that's not live, btw. I committed this an hour ago. It builds upon on earlier patch to mitigate output distribution poisoning: https://github.com/feather-wallet/monero/commit/fdd5c639127a5163f739147341bc1b03f8d0a889
13:43:56 <ilgatu> When using the "std=c17 -pedantic" flag, the function  getloadavg() is supposedly not defined: "implicit declaration of function", because it's a gnuc function (as I understand). How would you handle this, just ignore the warning (because the code runs in Linux, BSD) or #ifdef some value
14:07:27 <ooo123ooo1234567> how is it related to monero ?
14:13:52 <sgp_[m]> We can use the patch tobtoht made, but we'd rather it be merged upstream ofc. Is there a way to only persist the cache if it's using a non-localhost node?
14:16:01 <moneromooo> You want to cache what data ?
14:16:52 <moneromooo> And by non localhost, you mean trusted or any random asshole from the internet ?
14:19:39 <sgp_[m]> In our case, it usually means trusted, since it's our nodes or a node a user manually specifies
14:20:55 <sgp_[m]> cache the rct_distribution
14:21:04 <sgp_[m]> currently the first spend on mobile takes a few extra noticeable seconds
14:26:11 <ilgatu> ooo123ooo1234567: "Don't ask to ask, just ask", I had to try :)
14:26:19 <ooo123ooo1234567> <sgp_[m]> "currently the first spend on..." <- great, if it's too painful then write and submit patch that will cache it correctly, or use arbitrary patch locally
14:26:44 <ooo123ooo1234567> <ilgatu> "When using the "std=c17 -..." <- https://www.gnu.org/software/libc/manual/html_node/Feature-Test-Macros.html, that function is an addition from glibc, it isn't available with strict c standard
14:26:54 <ooo123ooo1234567> it's available with -std=gnu17, but not -std=c17
14:27:59 <ooo123ooo1234567> what's the purpose to use -std=c17 without understanding what it means ?
14:30:50 <ilgatu> ooo123ooo1234567: So there's no way to use gnuc functions in a c standard code. Ok.
14:31:07 <ilgatu> ooo123ooo1234567: Thanks for your time, btw.
14:31:26 <moneromooo> I would not cache stuff that can be total bullshit from some random node. Unless trusted. But we know people, they'd just set any random node as trusted anyway. So probably not even if trusted.
14:31:50 <ofrnxmr[m]1> ilgatu: Check https://xyproblem.info helps too
14:35:03 <moneromooo> Or maybe cached per IP, if not Tor.
14:46:31 <tobtoht[m]> moneromooo: In the way I implemented this only data up until a checkpoint height is cached and must match a hardcoded hash. The wallet will throw an error if a random node sends bullshit data (anything pre checkpoint height) and will not continue with tx construction.
14:46:46 <tobtoht[m]> One issue with this approach is increased maintenance burden, as the hardcoded hash should be updated with each release.
14:46:59 <tobtoht[m]> I agree caching non-checked data from random nodes is a bad idea.
14:47:13 <ilgatu> ofrnxmr[m]1: Hi, thanks a lot. A new perspective, I used to see it the other way. Again, thanks for your time.
14:50:48 <moneromooo> OK, that's a good idea.
14:58:26 <tobtoht[m]> Alright, I'll work on a PR.
15:15:42 <sgp_[m]> Thank you
15:47:32 <moneromooo> Well, shipping the known good hash to prevent bullshit data is a good idea :D Whether adding such caching in monero is, I dunno. Maybe. I'm kinda torn.
15:50:14 <ooo123ooo1234567> indeed, why not to add another centralized checkpoint for 2MB of data
15:53:00 <ooo123ooo1234567> https://github.com/cake-tech/cake_wallet/issues/395, interesting who are those Monero developers
16:36:23 <rbrunner> "interesting who are those Monero developers" Do you mean this as a statement, or as a question?
16:39:50 <ooo123ooo1234567>  * https://github.com/cake-tech/cake_wallet/issues/395, it's interesting, who are those Monero developers ?
16:41:19 <rbrunner> Well, sethforprivacy is, among other things, a dev, but hardly active on Monero code proper, and SamsungGalaxyPlayer isn't a dev as far as I know
16:42:49 <rbrunner> But well known for their "Breaking Monero" series of videos: https://www.monerooutreach.org/breaking-monero/
16:43:09 <rbrunner> er, his videos
16:43:13 <sethforprivacy> I'm not really a dev to be honest
16:43:45 <sethforprivacy> I was talking with jberman and kayabanerve @kayabanerve:matrix.org about this issue because I had a suspicion this was the UX problem I kept running into
16:44:05 <sethforprivacy> They confirmed it was likely output distribution for ring generation and seems they were right 
16:47:49 <sethforprivacy> This issue is one of the few constant PITA hurdles for me still in using Monero, and is especially onerous in person (as I was at Monerokon trying to spend and had to wait like 20s before I could even submit the TX)
16:48:05 <kayabanerve[m]> More jberman than me :) I was there to discuss the effects of arithmetic circuits on it and discuss doing it in Rust :p
16:48:26 <sethforprivacy> Not caching this data is also a bandwidth and IO load on all remote nodes, especially more commonly used public ones like mine unnecessarily.
16:48:49 <kayabanerve[m]> Though the fix I did only caches it per application lifetime. Not too suitable for deployment to wallets...
16:49:21 <rbrunner> Interesting. I have looked around quite a number of times in the wallet2 code for various reasons and never stumbled over this. Doesn't tell me anything at all.
16:50:17 <kayabanerve[m]> rbrunner: jberman's comment is it's a 10 MB data set
17:02:42 <ooo123ooo1234567> <sethforprivacy> "This issue is one of the few..." <- is it reproducible with fast internet connection and not overloaded monerod ?
17:04:00 <sethforprivacy> I have run into it many times and always notice that the first transaction after wallet open is excessively slow
17:04:02 <sethforprivacy> 3s+
17:04:11 <sethforprivacy> Which is very poor UX
17:05:24 <sethforprivacy> 25s just now
17:05:28 <sethforprivacy> lol
17:05:41 <ooo123ooo1234567> is it reproducible with monero-wallet-cli ?
17:05:53 <sethforprivacy> Idk, that's not the target audience of the two wallets I opened issues for
17:06:00 <sethforprivacy> We shouldn't base UX assumptions on a CLI wallet.
17:06:14 <sethforprivacy> If mobile wallets all have poor UX when sending transactions it has to be fixed one way or another
17:06:38 <sethforprivacy> 25s to create a transaction with my own private full node over 300mb/s internet (through VPN, but very fast VPN) is nuts
17:06:44 <selsta> sethforprivacy: can you try to reproduce it with my node and cake wallet or whatever wallet you are using? 88.198.199.23:18081
17:06:50 <sethforprivacy> I would hazard a guess most of the delay is IO-bound but will look into more.
17:06:55 <sethforprivacy> selsta: Sure
17:07:14 <ooo123ooo1234567> check with cli wallet is mostly needed to know who to blaim for that delay
17:07:35 <selsta> if it only happens with one wallet it's possibly a bug in that wallet
17:07:48 <sethforprivacy> selsta: 6s but not sure if it was relying on any cache as I didn't close the wallet
17:07:51 <sethforprivacy> Testing again
17:08:24 <ooo123ooo1234567> 1. do end-to-end debug with 1 wallet; 2. check different wallets to localize problem; since 1 is too hard probably, why not to do 2 ?
17:08:44 <sethforprivacy> 27s with your node on a fresh wallet open selsta 
17:08:49 <sethforprivacy> Testing with monerujo now
17:08:56 <selsta> how many inputs?
17:09:06 <selsta> not sure if that makes a difference
17:09:09 <sethforprivacy> idk, just sending a small TX to test, but didn't ever send it
17:09:13 <sethforprivacy> Probably just 1
17:09:48 <ooo123ooo1234567> sethforprivacy: and the wallet software is ... ?
17:10:06 <sethforprivacy> 10s with Monerujo
17:10:13 <sethforprivacy> Others were Monero.com wallet (Cake)
17:10:32 <sethforprivacy> 1s after initial transaction is constructed/sent in Monerujo
17:11:11 <sethforprivacy> Cake is 10-12s for some reason even after initial construction (but I didn't send the TX yet, so maybe its just dumping that output cache?)
17:12:08 <sethforprivacy> lots of variance with cake, was 4s now
17:14:27 <selsta> it took me around 2s to generate a tx with the gui and a freshly deleted shared-ringdb so nothing should be cached
17:14:48 <sethforprivacy> 25s with Monerujo with your node selsta 
17:15:01 <sethforprivacy> selsta: Local node or your remote?
17:15:03 <selsta> remote
17:15:31 <selsta> but my remote node is in Europe, and you are from US I think, so not ideal for you latency wise
17:18:01 <ooo123ooo1234567> is it possible to test with monero-wallet-cli ?
17:18:46 <sethforprivacy> selsta: Yeah
17:18:53 <sethforprivacy> ooo123ooo1234567: Sure
17:21:05 <selsta> same 2 seconds with CLI, will try Cake Wallet
17:22:01 <ooo123ooo1234567> it must be something specific to environment of sethforprivacy
17:22:19 <sethforprivacy> Can no one else test but me and selsta?
17:22:29 <sethforprivacy> What do you see ooo123ooo1234567?
17:22:33 <tobtoht[m]> Fwiw, I implemented this in Feather because constructing the first transaction over Tor (which is inherently slow) takes a long time. If you're on a bad circuit downloading 6-7 MB worth of data takes a while.
17:22:33 <sethforprivacy> Anyone test with Cake/Monerujo?
17:23:02 <ooo123ooo1234567> tobtoht[m]: that isn't even 6MB of data
17:23:55 <ooo123ooo1234567> sethforprivacy: I'm just waiting for a test with monero-wallet-cli in your environment
17:24:03 <ooo123ooo1234567> * a test result with monero-wallet-cli
17:24:07 <sethforprivacy> Working on it, had to restore as I dont use cli
17:24:19 <sethforprivacy> Will test with local node, remote node of my own, and selsta's remote node.
17:24:36 <sethforprivacy> ooo123ooo1234567: You can test it too to give us another data point 🙂
17:24:52 <selsta> If you delete ~/.shared-ringdb and also restart the program there should nothing be cached.
17:24:53 <sethforprivacy> Use selstas node, mine (node.sethforprivacy.com:18089) and one of your choosing
17:26:29 <ooo123ooo1234567> sethforprivacy: restoring wallet with monero-wallet-cli isn't slower than with any of those wallets
17:26:34 <ooo123ooo1234567> * of those mobile wallets
17:26:46 <sethforprivacy> ooo123ooo1234567: ?
17:26:56 <sethforprivacy> No idea what that has to do with anything
17:27:00 <sethforprivacy> I had to restore a wallet with funds so I can test
17:27:04 <sethforprivacy> I'm not claiming its slow?
17:27:19 <sethforprivacy> Please test yourself and report back
17:27:41 <sethforprivacy> * Please test transaction generation w/o cache yourself and
17:28:03 <sgp_> If there's a question about this being slow, it's something I've noticed for a while but never realized what was causing the issue
17:28:26 <sethforprivacy> Think everyone just went "it's always like this so I'll ignore it", I know I did
17:28:36 <sethforprivacy> But seems a problem for everyone I've spoken to when they realize it
17:28:39 <sgp_> yup
17:28:55 <sethforprivacy> Waiting 10-20s to generate a transaction on mobile is really, really shitty UX
17:29:19 <sethforprivacy> So lets find the problem (I think we already did) and find a solution (seems like tobtoht already did) and implement it.
17:31:13 <sethforprivacy> 2s with CLI on local node
17:34:51 <sethforprivacy> ~3s with my remote node from CLI
17:35:06 <sgp_> on a strong connection with Cake to my own remote node (also with a strong connection) the first signing process only takes about 1s right now. but I definitely have seen it takes 6 or so seconds (usually not more)
17:35:46 <sgp_> unsurprisingly, it's variable based on how quick one can download the data
17:35:53 <selsta> sethforprivacy: with restart and deleted shared-ringdb?
17:35:59 <sethforprivacy> ~3s for selsta node
17:36:04 <selsta> that sounds normal
17:36:18 <sethforprivacy> selsta: Yes, deleted ~/.shared-ringdb between each wallet start
17:36:48 <sethforprivacy> It's created immediately on wallet open though, not on transaction generation
17:37:03 <sethforprivacy> So it may "feel" faster if its being done on wallet open and not on transaction creation
17:37:31 <sethforprivacy> That's probably the main difference here between Cake and CLI
17:37:36 <selsta> I don't think that's the case here
17:37:54 <selsta> creating a file by itself does not mean much
17:38:02 <sethforprivacy> Then why is that folder/DB files created on wallet open w/o transaction being generated?
17:38:10 * sethforprivacy sent a code block: https://libera.ems.host/_matrix/media/r0/download/libera.chat/5db1fbf12585e69d4805378b3d7fda55245076bf
17:38:22 <sethforprivacy> Created that without me trying any transactions
17:38:52 <selsta> it's a tiny file that does not make any speed difference
17:38:59 <sethforprivacy> Much smaller than before I deleted it initially, true
17:39:04 <sethforprivacy> Didn't read the size properly 🙂
17:39:22 <tobtoht[m]> ~/.shared-ringdb doesn't cache the output distribution and the output distribution is only requested when the first transaction is created. It affects a different part of transaction construction.
17:39:47 <selsta> I just know that it caches something and deleting it doesn't hurt for testing.
17:39:53 <ooo123ooo1234567> was monero-wallet-cli started within the same mobile phone ?
17:40:03 <sethforprivacy> ooo123ooo1234567: No, on another device
17:40:13 <sethforprivacy> Don't have binaries or anything on mobile
17:40:17 <sethforprivacy> Can do that but will be a bit
17:40:20 <sethforprivacy> Same network though
17:40:23 <selsta> i'm still syncing cake wallet
17:40:42 <tobtoht[m]> selsta: Yes, if you create a transaction and don't submit it ~/.shared-ringdb will store rings, which speeds up subsequent tx construction because it can reuse those rings instead of asking the node for outs.
17:43:08 <sgp_> I keep force closing cake and reopening, and the first tx generation often takes 1-2 extra seconds, again on a pretty ideal connection setup for remote trusted node
17:43:44 <ooo123ooo1234567> so who to blaim for that delay ?
17:44:15 <sgp_> I'd like to test with the local distribution cache and see if that improves it :)
17:44:55 <sgp_> not looking to blame, just trying to test improvements to see if they help with UX
17:45:17 <selsta> i feel like these are two different topics, 1-2s extra vs 25s tx creation
17:45:23 <ooo123ooo1234567> incorrect code is much bigger PITA for users, than 1-2 seconds optimization
17:45:50 <sgp_> on tor and a roaming phone connection it indeed makes sending monero terrible
17:46:17 <sgp_> let me try with orbot to give you an idea
17:46:58 <ooo123ooo1234567> in the worst case correctly implemented cache will be redundant, in the best case it will optimize those few seconds
17:47:28 <ooo123ooo1234567> but it isn't a priority to use another centralized hash for 1-2MB of data instead of correctly implemented cache if it's even possible
17:47:55 <sethforprivacy> Not really sure what you're trying to say.
17:48:11 <sethforprivacy> Do you have a better proposed fix for this? Are you saying it's not a problem? Have you tested this at all?
17:48:17 <sethforprivacy> Still haven't seen any test results from you.
17:49:28 <ooo123ooo1234567> let's firstly find what is causing that delay in your environment
17:49:31 <ooo123ooo1234567> s/find/identify/
17:50:01 <selsta> sethforprivacy: if it takes 3s with monero-wallet-cli and significantly longer with cake wallet on the same connection than it's unrelated to this cache
17:50:27 <sethforprivacy> Mobile is also using VPN but not sure how that could add that long
17:50:39 <sethforprivacy> Can no one else reproduce this?
17:50:44 <selsta> ideal test setup would be to try monero-wallet-cli on mobile
17:50:50 <sethforprivacy> Other than sgp to a lesser degree
17:50:51 <selsta> like i said my cake wallet is still syncing
17:50:54 <sethforprivacy> selsta: Yeah syncing it now
17:52:28 <ooo123ooo1234567> sethforprivacy: any way to ssh into your environment ?
17:52:41 <aog> do you know if there's any crowdfunding service based on XMR open for the public beside CCS?
17:52:48 <sethforprivacy> Give you SSH access to my env?
17:52:48 <sethforprivacy> No
17:52:50 <sethforprivacy> lol
17:52:56 <ooo123ooo1234567> otherwise it's impossible to reproduce outside of it
17:53:23 <ooo123ooo1234567> sethforprivacy: great, waiting for it
17:53:25 <sethforprivacy> You can test the same different approaches with the same nodes and gauge how it works for you.
17:53:32 <sethforprivacy> Much more useful if many people test this
17:58:16 <sgp_> 81 seconds on first send using Cake + Orbot. 22 seconds on second send
17:58:48 <ooo123ooo1234567> 1st send wasn't relayed, and 2nd send was using the same outputs ?
17:59:03 <sethforprivacy> sgp_: holy shit lol
17:59:08 <sgp_> correct, didn't relay. just canceled and started over for second
17:59:43 <sgp_> if it'll make a difference I can try sending the first one and then queuing up second
18:00:14 <ooo123ooo1234567> "25s to create a transaction with my own private full node over 300mb/s internet (through VPN, but very fast VPN)" orbor is quite the opposite of this setup
18:00:35 <tobtoht[m]> It's a bandwidth issue. If you reduce the amount of bandwidth required to construct a transaction by 97% then transaction construction is going to be faster. It's really that simple.
18:00:54 <tobtoht[m]> s/bandwidth/data/
18:00:59 <ooo123ooo1234567> s/orbor/tor/orbot/
18:01:05 <sethforprivacy> Yeah I'm not really sure what ooo123ooo1234567 is getting at -- are you proposing a different fix or saying this isn't an issue?
18:01:29 <sethforprivacy> We're proving that the worse your network connection the worse the UX for sending because your wallet isn't caching rct_distribution
18:01:34 <sgp_> my node has gig up and I have something like 600 down here so there was only a tiny delay there. but with Tor it took forever, which makes sense
18:01:38 <sethforprivacy> At least thats the idea we have for why it's happening.
18:01:53 <ooo123ooo1234567> sgp_: should be different
18:02:08 <sethforprivacy> s/because/_because/, s/your/the/, s/rct_distribution/rct\_distribution_/
18:02:18 <sgp_> I'll send numbers in a minute then for send 1 output on fresh launch, then second output immediately after
18:02:57 <ooo123ooo1234567> sethforprivacy: what's about wallet synchronization with such slow connection ?
18:03:10 <sethforprivacy> ooo123ooo1234567: Also of course a problem
18:03:23 <sethforprivacy> But if there is an easy fix here (there is not for broader wallet sync) than we need to explore it.
18:03:36 <sethforprivacy> I still have no idea what the goal of your arguments are.
18:04:37 <sgp_> I think they're most against the checkpoint, but maybe we can just enable the cache only for trusted daemons
18:11:58 <tobtoht[m]> Here is some data:  transaction construction (1 input) without output distribution cache uses 1.58 MB (bytes received only).
18:12:11 <tobtoht[m]> Of this, 1.53 MB is needed to obtain the output distribution. (And this will keep increasing as the chain gets longer).
18:12:14 <tobtoht[m]> So, a less substantial size than thought. (I looked at memory, but there is some compression going on here).
18:12:23 <tobtoht[m]> But, it still makes up 97% of the network traffic required for initial tx construction.
18:12:43 <tobtoht[m]> If checkpoints are not the way to go, a compromise is to grab the output distribution as soon as synchronization is finished.
18:12:55 <tobtoht[m]> This way the output distribution will typically be available when the user wants to create a transaction.
18:13:22 <selsta> is 80s normal to get 1.6MB over Tor?
18:13:39 <ooo123ooo1234567> tobtoht[m]: wow, less 2 MB
18:13:49 <ooo123ooo1234567> selsta: no, it depends on setup
18:14:11 <moneromooo> It's pretty fast nowadays. 30 kB/s is rare now. I often get like 200+ or so.
18:15:25 <moneromooo> It can be slow to setup a good circuit first though.
18:15:33 <sgp_> first output send: 1m40. second output send: 14s
18:15:44 <moneromooo> But, also, much less than, say, ten years ago.
18:16:00 <sgp_> obviously there will be variability over tor and I haven't optimized mine at all
18:16:15 <ooo123ooo1234567> tobtoht[m]: checkpoint is a compromise, correctly implemented cache with asynchronous fetching isn't a compromise
18:16:32 <ooo123ooo1234567> > <@tobtoht:monero.social> Of this, 1.53 MB is needed to obtain the output distribution. (And this will keep increasing as the chain gets longer).
18:16:32 <ooo123ooo1234567>  * wow, less than 2 MB
18:17:03 <sgp_> on mobile roaming I'm capped at 128kbps so sending does indeed feel like it takes forever currently for first time
18:17:28 <sgp_> just to give an example. obviously everyone's plans are different
18:17:53 <tobtoht[m]> ooo: can you describe how your ideal implementation differs from what I proposed?
18:18:49 <ooo123ooo1234567> I'm waiting for a test with monero-wallet-cli from sethforprivacy
18:18:59 <ooo123ooo1234567> on that mobile device with fast vpn
18:19:30 <tobtoht[m]> With a fast VPN this isn't much of an issue.
18:19:40 <sgp_> yeah it's like 1-2s only
18:24:48 <sethforprivacy> Almost done syncing
18:26:02 <ooo123ooo1234567> without any code changes required data size reduced from 10MB to 1.5MB, the quickest fix ever
18:27:27 <sethforprivacy> ?
18:27:46 <sethforprivacy> Going out of your way to be antagonist to people trying to give useful feedback and improve Monero is not a good look.
18:28:24 <sethforprivacy> An "off the top of my head" calculation over dinner is just that, an estimation
18:28:40 <selsta> I can't even sync 5k blocks over Tor currently, wonder if the Tor network itself is having issues or I just have to restart a couple times until I get a good circuit
18:28:55 <sgp_> tor is indeed having issues
18:28:57 <sethforprivacy> selsta: Tor DoS is still having an impact IIRC
18:29:01 <sgp_> been under DDoS for like a mo
18:31:35 <moneromooo> Really. I haven't noticed :S
18:31:58 <sgp_> that's why all wasabi 2.0 mixes haven't been going though, lol
18:32:10 <moneromooo> Then again, I think nothing of waiting a minute for connection to happen from time to time...
18:35:53 <sethforprivacy> 22s with monero-wallet-cli on 1st send
18:36:06 <sethforprivacy> 2s on subsequent
18:36:30 <selsta> what kind of phone hardware?
18:36:43 <sgp_> hmm, much larger than I though. What's your down speed and ping?
18:37:08 <sethforprivacy> I'm using a non-ideal VPN server, 30ms ping and 66mbps down
18:37:38 <sethforprivacy> Changed VPN server, 18ms ping, 81mbps down
18:37:40 <sethforprivacy> Testing again
18:39:31 <sethforprivacy> 19s first spend, 2s subsequent
18:40:08 <selsta> subsequent = pressing cancel and transact again?
18:40:25 <selsta> without any restart
18:40:45 <sethforprivacy> No, sending and then sending again
18:40:46 <sethforprivacy> Actually sending
18:40:48 <sethforprivacy> No restart
18:42:24 <sethforprivacy> 18s first spend, 6s subsequent for selsta node
18:42:38 <sethforprivacy> Deleted ~/.shared-ringdb between wallet runs
18:42:49 <selsta> phone hardware?
18:42:55 <selsta> SoC?
18:43:02 <sethforprivacy> Pixel 6
18:43:15 <sethforprivacy> So their first-gen Tensor SoC
18:43:21 <sethforprivacy> Cutting edge 🤷
18:43:45 <selsta> the extra downloaded 1.58MB don't explain the difference you are seeing
18:44:10 <sgp_> Mine is S20+ fwiw, snapdragon 865
18:45:47 <sethforprivacy> What else would be causing this?
18:46:05 <sethforprivacy> Network is good, phone is latest gen, nodes are known-good
18:46:18 <selsta> I still have 200k blocks on my phone to scan
18:46:29 <sethforprivacy> And subsequent spends are just as quick as expected
18:46:36 <sethforprivacy> It's something being cached
18:47:29 <sethforprivacy> (being cached for subsequent spends and not for initial spends, to be clear)
18:47:57 <sethforprivacy> 90% reduction in generation time between first spend and subsequent.
18:48:46 <moneromooo> If you want to check what's slow, did you try: sudo perf top (it will show if something's being precalced), strace (with timestamps), or just log level 2 (with timestamps), --no-dns ?
18:48:49 <sethforprivacy> Whatever is being gathered when you try and construct a transaction should be gathered on initial sync, cached, and then updated with each subsequent sync. Shouldn't wait for transaction generation anyways.
18:48:53 <selsta> moneromooo: it's on a phone
18:48:58 <selsta> 1.58MB data, over 66mbps should be downloaded in less than a second
18:49:05 <selsta> 20s difference simply doesn't add up, in no way
18:49:38 <moneromooo> That phone doens't run strace or perf ?
18:49:39 <selsta> sethforprivacy: does perf top work on a phone?
18:49:46 <sethforprivacy> idk haha
18:50:00 <sethforprivacy> Can try but have never done that before
18:50:10 <selsta> log level 2 should work
18:50:17 <selsta> and --no-dns
18:50:33 <sethforprivacy> OK
18:50:50 <selsta> the timestamps should tell what takes how long
18:51:27 <moneromooo> Also if it has gdb, gdb into the process. If it's stuck waiting in http client, it's just waiting for the daemon.
18:51:54 <moneromooo> (and the upper levels of the stack should tell you what was requested)
18:52:16 <sethforprivacy> Node on log level 2 or wallet cli?
18:52:19 <selsta> wallet
18:52:47 <selsta> node might be helpful too but would check wallet first
18:57:18 <selsta> calling get_output_distribution manually should also show us how long it takes
18:57:33 <sethforprivacy> DMing to selsta
18:57:41 <ooo123ooo1234567> <sethforprivacy> "Network is good, phone is latest..." <- is it reproducible with `--no-dns` ?
18:57:58 <sethforprivacy> 6-8s with --no-dns
18:58:18 <sethforprivacy> For first spend, so much more reasonable but still slow
18:58:47 <ooo123ooo1234567> can you check manually in logs what took most of the time ?
19:00:02 <sethforprivacy> 9s according to logs
19:00:08 <sethforprivacy> I'll let selsta dissect them
19:00:46 <selsta> 2022-06-24 18:54:22.168	    c8ec46223000	DEBUG	wallet.wallet2	src/wallet/wallet2.cpp:3557	Daemon is recent enough, requesting rct distribution
19:00:48 <selsta> that took 5s
19:02:05 <ooo123ooo1234567> and other 4 seconds ?
19:02:26 <selsta> transaction <xx> generated ok and sent to daemon, key_images: [<xx> ]
19:02:39 <selsta> 2022-06-24 18:54:31.285
19:04:39 <sethforprivacy> And initial transfer command was at `2022-06-24 18:54:18.204`
19:04:46 <ooo123ooo1234567> can you paste obfuscated log with timings ?
19:05:00 <sethforprivacy> ooo123ooo1234567: Yeah working on that now
19:05:00 <tobtoht[m]> selsta: Before or after this line?
19:05:00 <sethforprivacy> A lot to redact lol
19:05:16 <selsta> tobtoht[m]: wait, i sent it in a confusing way
19:07:04 <ooo123ooo1234567> https://github.com/monero-project/monero/blob/master/src/wallet/wallet2.cpp#L9807, https://github.com/monero-project/monero/blob/master/src/wallet/wallet2.cpp#L10296
19:07:19 <ooo123ooo1234567> what are the timestamps for these two lines ?
19:09:04 <selsta> https://paste.debian.net/hidden/fe717531/
19:11:28 <selsta> ooo123ooo1234567: 18:54:21.963 and 18:54:27.553
19:12:18 <ooo123ooo1234567> no
19:12:41 <ooo123ooo1234567> https://github.com/monero-project/monero/blob/v0.17.3.2/src/wallet/wallet2.cpp#L10066, where is this msg in log ?
19:13:27 <selsta> 2022-06-24 18:54:27.553	    c8ec46223000	INFO	wallet.wallet2	src/wallet/wallet2.cpp:10069	  Transaction 1/1 <xxx>: xxx weight, sending xxx in 1 outputs to 1 destination(s), including xxx fee, xxx change
19:14:50 <selsta> this one is less redacted https://paste.sethforprivacy.com/?616953025dd02397#Cb1QU29rakK3oMkbxrKSrtVfpwRHCHU57nkeH7pTd8Wm
19:21:33 <tobtoht[m]> We're not calling get_output_histogram because we're dealing with rct outs, and we're not is_after_segregation_fork. Is there anything else after "Daemon is recent enough, requesting rct distribution" other than the get_output_distribution.bin call that could cause the delay?
19:21:38 <tobtoht[m]> get_ringdb_key() key possibly? Looks like it's cached.
19:26:45 <selsta> cake wallet took less than 3s, in line with monero-wallet-cli and gui for me
19:27:01 <selsta> on iOS at least
19:30:07 <selsta> on both home and mobile connection, but i have fast mobile internet here
19:31:20 <tobtoht[m]> <tobtoht[m]> "get_ringdb_key() key possibly..." <- Nvm, would have shown up in the logs. And it's called after opening the wallet.
19:37:30 <ooo123ooo1234567> create_transactions_2 - 5.6s, get_output_distribution - 5.08s; with 334KB/s connection and --no-dns
19:38:23 <ooo123ooo1234567> <sethforprivacy> "For first spend, so much more..." <- are there any difference between 1st and 2nd sends without dns ?
19:38:25 <ooo123ooo1234567> s/are/is/
19:38:40 <sethforprivacy> Not sure, will test now
19:39:04 <tobtoht[m]> ooo: is that get_rct_distribution() or only the /get_output_distribution.bin call?
19:40:12 <sethforprivacy> 8s first spend, 7s subsequent
19:40:36 <ooo123ooo1234567> both 1st and 2nd were relayed ?
19:41:42 <sethforprivacy> yes
19:42:34 <sethforprivacy> network is 20ms ping, 1.55ms jitter, 90mbps down, 180mbps up
19:42:39 <sethforprivacy> Using selsta's node.
19:42:51 <selsta> what's the ping to my node?
19:47:21 <tobtoht[m]> I'm pretty sure we can remove this m_node_rpc_proxy.get_rpc_version() call. There is no daemon out there that doesn't support this. That would save one round trip.
19:48:07 <selsta> +1
19:49:11 <selsta> got added 4 years ago
19:54:06 <ooo123ooo1234567> <sethforprivacy> "8s first spend, 7s subsequent" <- is it obvious why dns is so slow there ?
19:55:58 <sethforprivacy> <selsta> "what's the ping to my node?" <- 140-200ms
19:56:00 <sethforprivacy> > <@selsta:libera.chat> what's the ping to my node?
19:56:00 <sethforprivacy>  * 140-250ms
19:56:45 <sethforprivacy> I saw 400 at one point when testing
19:58:00 <selsta> yea, sorry my node is only good for Europe :D
19:58:41 <sethforprivacy> Yeah haha, its expected
19:59:47 <ooo123ooo1234567> <sethforprivacy> "19s first spend, 2s subsequent" <- what's the delay with your own node with --no-dns ?
19:59:59 <tobtoht[m]> selsta: https://github.com/monero-project/monero/pull/8404
20:02:14 <sethforprivacy> ooo123ooo1234567: 2-3s
20:02:27 <ooo123ooo1234567> wow, is it enough for good UX without tor ?
20:05:25 <sethforprivacy> Better than 20s
20:05:34 <sethforprivacy> What about no-dns seems to be improving it this much?
20:05:50 <selsta> might be a problem with your VPN DNS?
20:06:00 <selsta> I think on desktop you didn't use VPN?
20:06:29 <sethforprivacy> I'm using custom NextDNS, not VPN-based
20:06:39 <sethforprivacy> I don't see any other DNS issues or latency with other apps, browsing, etc.
20:06:49 <ooo123ooo1234567> records from centralized monero dns aren't reachable instantly for some reason
20:07:28 <ooo123ooo1234567> `drill  -t txt @8.8.8.8 segheights.moneropulse.org`; `drill  -t txt segheights.moneropulse.org`
20:07:48 <ooo123ooo1234567> what's the delay for this dns requests ?
20:10:01 <sethforprivacy> What is `drill`?
20:10:06 <sethforprivacy> Have never seen that before and not in apt
20:10:13 <ooo123ooo1234567> use any other tool for dns requests from terminal
20:11:15 <ooo123ooo1234567> `time (for d in org net co se; do drill  -t txt segheights.moneropulse.org &; done; wait )` the wallet is doing 4 requests, maybe one of them is too slow for your environment
20:11:16 <sethforprivacy> 43ms vs 52ms
20:11:19 <sethforprivacy> Negligible
20:11:30 <ooo123ooo1234567> s/.org/.$d/
20:11:41 * sethforprivacy sent a code block: https://libera.ems.host/_matrix/media/r0/download/libera.chat/63dfa65976262273be9f8ab90a8e4bfaa5329e42
20:12:14 <sethforprivacy> meh thats just cached now
20:12:43 <sethforprivacy> Originals here:... (full message at https://libera.ems.host/_matrix/media/r0/download/libera.chat/8ead6918cfa076293cc5ae03ee6cb44885a9a507)
20:13:43 <ooo123ooo1234567> did you try to from that mobile device ? and without 8.8.8.8 server
20:13:52 <ooo123ooo1234567> s/to//
20:19:27 <sethforprivacy> Sorry no, testing now
20:20:47 <ooo123ooo1234567> <tobtoht[m]> "I'm pretty sure we can remove..." <- https://github.com/monero-project/monero/pull/5600/files, there is a cache for rpc version, it would be good to use it
20:24:08 <tobtoht[m]> We don't need the check at all.
20:30:44 <ooo123ooo1234567> indeed, that's true if no one is relying on that error with old daemon
20:36:57 <ooo123ooo1234567> `git grep 'MAKE_CORE_RPC_VERSION'` why not to remove all of them then ?
20:44:03 <ooo123ooo1234567> <sethforprivacy> "Going out of your way to be..." <- all involved people were busy with optimizing minor thing with another centralized checkpoint, it's pretty good look to not accept it
20:48:43 <tobtoht[m]> <ooo123ooo1234567> "`git grep 'MAKE_CORE_RPC_VERSION..." <- Sure.
22:04:32 <ooo123ooo1234567> <sethforprivacy> "Sorry no, testing now" <- what was the result ?
22:19:13 <ooo123ooo1234567> "https://nitter.42l.fr/sethforprivacy/status/1540322475355082754#m" facepalm
22:20:03 <ooo123ooo1234567> an interesting question is how to do development of decentralized project in order to avoid such vulnerabilities
22:47:20 <sethforprivacy> <ooo123ooo1234567> "what was the result ?" <- Had to step away, sorry, might get to it later or tomorrow.