17:48:40 I don't feel like he's focusing on worthwhile targets. dunno 17:49:15 I also haven't timed builds recently to see how much his improvements have actually done to speed up builds 17:49:20 anyone else? 17:49:49 on a completely separate topic... our P2P protocol still seems pretty slow. 17:50:21 was wondering why we don't just rip it out and use bittorrent protocol instead. or something similar. 17:50:57 bittorrent has an advantage in that it can receive chunks in any order. it knows where to put them because it has an initial .torrent file with hashes of every chunk. 17:51:20 we could introduce the notion of "superblocks" that are just sets of hashes of N blocks 17:52:10 so for IBD first you grab as many superblocks as possible. and each can also contain the hash of the preceding superblock, for sanity check 17:52:44 once you have those, you can download the actual blocks in arbitrary order, and you'll always know if you're getting valid ones or not 17:56:58 the p2p ccs should have a larger PR soon 17:57:43 even so, I think this idea will improve IBD significantly 17:58:13 The build speedups are really just for fun AFAICT. They don't speed up the common case, except maybe the precompiled headers one. 17:58:14 I'm not sure because I don't think networking is the bottleneck. 17:58:31 Or not significantly. 17:58:46 considering how much faster sync runs with two daemons on the same host, I'm quite sure it is 17:59:08 About syncing, monero needs to have the data in order anyway, so you don't havr that much leeway in ordering. 17:59:29 When syncing did you manually increase out_peers and down_limit? 17:59:37 If you sync and run "sync_info", you can see whether it's waitoing on the network. 17:59:55 moneromooo has a PR open to increase out peers during initial sync 17:59:59 selsta: on localhost, all default params. just add-exclusive-node 127.0.0.1:xxxx 18:00:00 The "moooo" line shows the current state of the blocks queue. 18:00:32 "m" means matches what we need next. o means we got that data. . means we requested that data. 18:00:51 So if it starts with . you're waiting on network. If it starts with m you're not. 18:01:19 It usually starts downloading ahead fairly quickly. 18:01:41 What might help is the rw lock on the blockchain class. 18:04:12 if we also changed the on-the-wire txn format to reference outputs by hash instead of 64bit ID, we wouldn't have that block ordering issue 18:04:35 if the concept of 64bit IDs was purely a local DB-internal affair 18:04:41 We would. 18:05:26 If a tx uses an output and it's not on the chain, you don't know whrether it's valid whether it's an offset or a hash. 18:06:03 sure, for newly arriving txs 18:06:16 but during IBD the node ignores new txs 18:06:33 I sure hope it doesn't. 18:07:06 Oh you mean it skips checks for stuff matching known good hashes ? 18:07:10 yeah 18:07:27 I don't see how using hashed would change anything there. 18:07:32 hashes* 18:08:02 it can't validate anything until it has a complete set of blocks 18:08:24 hyc afaik his new CCS is part time kinda general stuff. The compile time reduction CCS is still ongoing/hasn't been worked on much recently 18:08:27 Yes, 20 blocks atm. 18:08:47 But that's like a couple MB typically. 18:10:20 Anyway, I guess if you get the network as bottleneck, it might be a worthwhile change. It seems like a very large work/gain ratio though. 18:10:31 But do check sync_info first to make sure. 18:10:59 ok 18:11:20 I'll be setting up a new home node next week on a rockpro64 18:11:31 so will test add-exclusive-node across LAN again 18:12:30 That said, the code that decides who to ask for what data is very ad hoc, and is probably far from optimal. I dunno how to improve it though. 18:13:27 ie, when to decide to request from a second peer some data we need soon and we don't have (because that first peer might be very slow, malicious, or the like). 18:13:53 that sounds like something bittorrent nodes have already optimized 18:14:05 If libbittorrent had some code to intelligently do that, it might be a good thing to get (f licence compatible). 18:14:07 dunno if it's worth a look at their code