01:17:09 how would this work? 01:19:39 Also, removing the number of txs from future block hashing blobs still means that we need to save that number for past/existing blocks, so we need it either way for block 202612 08:46:44 @jeffro256: Yeah I know that's why I said it doesn't really change anything :) 08:48:23 @jeffro256: Because the block 202612 hash is an impossible hash to make, the number of txs it states is different to the size of the Merkel tree. 18:02:40 Right, but how does the integer inside the block hashing blob do anything to change this while processing headers? > <@boog900> Because the block 202612 hash is an impossible hash to make, the number of txs it states is different to the size of the Merkel tree. 18:06:48 block 202612 bug was precisely about the number of transactions and Merkle tree 18:09:11 I know that, but what about the precense of the integer 514 inside the header hashing blob does anything whatsoever to help process that? You need either A) the fixed merkle tree hash, or B) the list of TXIDs iin its entirety (what the current code does), then map that into the wrong existing hash. The integer 514 being inside the header hashing blob does nothing 18:11:57 I shouldn't be able to pass a header with the same prev ID and 514 txs and have it map to the existing block ID and PoW hash. It should map iff the content representing by the list of TXIDs is what already existed on-chain in what we refer to using block ID bbd604d2ba11ba27935e006ed39c9bfdd99b76bf4a50654bc1e1e61217962698 18:12:23 Even though that block ID doesn't bind to the last 2 txs in the list 18:13:22 it alows you to compute a blocks hash and just compare that hash against what is block 202612's real hash to swap it out. If the numb of txs was not hashed then that would be broken as you could cut 2 txs off the end to get the bad block 202612 hash. 18:13:57 IMO that's nicer than hardcoding block 202612's blob or double hashing 18:14:14 "With only header hashing info, it is trivially faked" I don't understand this argument. With only header hashing info, merkle hash can also be faked, so what? 18:14:30 I don't understand this whole buzz about removing a single varint 18:14:33 to save 1-2 bytes 18:15:45 Doesn't P2Pool use it? Maybe I'm thinking of DataHoarder's block explorer/mining pool tracker 18:18:54 @boog900: Nope. The number of txs hashed in is vestigial here. What the block hashing code does is hash the whole block blob, which means hashing the entire TXID list, which is what actually lets you catch when the last 2 TXIDs would be swapped out. The number inside the header could be 3, 4293, 37394, 0, 42, etc it does [... too long, see https://mrelay.p2pool.observer/e/t86fi4wLZ1RIbUxU ] 18:20:15 Yeah monerod does the double hash 18:20:22 monero-oxide does not 18:23:51 You mean this, right? https://github.com/monero-oxide/monero-oxide/blob/946ec5f00ff071b129758ee8cba5528539fccfe4/monero-oxide/src/block.rs#L182-L218 18:24:53 This is how I would do it, personally. But yeah, again, the number of txs in the header does absolutely nothing here. The presence of the true merkle hash being here is what matters 18:25:09 no it does 18:25:48 what matters is that the merkle hash is checked against the real merkle hash of the transactions list 18:25:56 this is why merkle hash matters 18:26:05 sech1: exactly 18:26:06 it prevents someone from cutting off the last 2 txs from block 202612 and getting the correct hash causing a collision 18:26:12 tx count from hashing blob is also checked in block verification 18:26:17 so you can't call it "fake" 18:26:24 by your own logic 18:27:38 also, what boog900 said about block 202612 18:27:54 sech1: re merkle hash also being faked: that is absolutelky true. But the difference is that A) the merkle root being here is a cryptographic necessity, and B) no one pretends to use the merkle root as a metric for mempool processing without further context 18:27:57 because block 202612's merkle root was calculated from 512 transactions, not 514 18:28:27 @boog900: The true merkle root being present does that, not the number of txs 18:29:12 It would be equally as secure without the number of txs being present is what I'm saying. If Keccak256 had length extension security flaws, maybe this would be a different convo IDK 18:29:12 It doesn't 18:30:27 sech1: I'm not talking about block consensus verification, I'm talking about block header sync > tx count from hashing blob is also checked in block verification 18:30:54 Of course you can validate it with further context, but with further context, then the field being present is completely meaningless anyways 18:30:56 what about block header sync? 1 extra byte makes it impossible due to high bandwidth? 18:31:05 The real merkle root has 512 txs, we swap the 514 txs merkle root for the 512 one. If the number of txs was not hashed this would be a collision 18:31:47 As someone could provide block 202612 with only 512 txs and get the same hash as the 514 tx block 18:31:51 "the field being present is completely meaningless anyways " meaningless is a personal opinion 18:31:55 sech1: No, but it adds ~2.5-5% bandwidth for no security benefit 18:32:12 no one said it's for security 18:32:53 I have a theory why that field is there in the first place, but I don't have pre-github change history for Cryptonote to confirm it 18:34:39 why are you not talking about minor version byte in the hashing blob then? Also a complete waste of bandwidth! 18:34:51 This is a useless converation 18:36:24 Hashing blob could be just a merkle hash(transactions + header) + 4 bytes for nonce, that's it 18:36:44 let's optimize it to the logical limit 18:37:02 Yeah I do think there are better places to save bandwidth 18:37:31 also, 256 bits for merkle hash seems like a waste too 18:37:35 128 bits will be enough 18:37:45 Having the p2p network use a self describing format is a big one 18:38:21 > <@boog900> The real merkle root has 512 txs, we swap the 514 txs merkle root for the 512 one. If the number of txs was not hashed this would be a collision 18:38:21 My bad for using confusing language. When I say "true" merkle root, I meant the merkle root of the updated tree hash code binding all 514 txs. When I say "existing" merkle rooot, I meant the merkle root of the old code which only bound 512 txs, but is what was used for calculating the existing (non-binding) block ID. The mon [... too long, see https://mrelay.p2pool.observer/e/he3mi4wLektVcWgt ] 18:39:32 sech1: 128 bits is not enough cryptographically to prevent hash collisions 18:40:05 The existing and current merkle tree code will give the same result for the first 512 txs of that bad block fwiw 18:40:19 We have a test in monero oxide for it 18:40:22 The conversation is not useless if we can come to a consensus to make header-only more efficient > This is a useless converation 18:41:19 https://github.com/monero-oxide/monero-oxide/blob/946ec5f00ff071b129758ee8cba5528539fccfe4/monero-oxide/src/tests/merkle.rs#L76 18:41:21 if it's about network efficiency, you'll save more bandwidth if you send many headers in a single packet 18:41:36 one header will waste a lot due to TCP/IP overhead 18:41:41 @boog900: I understand that, but you aren't passing the existing (512) merkle root to the linked function are you? Youre passing the true (514) merkle root, which correctly binds the 514 txs 18:42:06 sech1: That's what I'm prepping in https://github.com/monero-project/monero/pull/10758 18:42:17 @jeffro256: Yeah but then what if a bad actor sends the block across the network with only 512 txs 18:42:43 They would split the network as the block would have the same hash 18:46:46 Okay I see what're your arguing for on that end. That is one valid way to do it, yes. But you can still process the block correctly without the number of txs by rejecting the existing merkle root altogether too 18:52:42 What is header-only sync for? You can only get headers, block IDs and PoW hashes out of it, nothing else. And the total number of transactions in each block, of course (which will be not fake because PoW) 18:56:12 Header only sync is useful because it lets you decide early on the most likely highest-chainwork fork without the need to download a ton of data. 18:57:04 https://lmddgtfy.net/?q=blockchain%20why%20header%20only%20sync%20 click on first result 18:58:07 should I send you google links on how to better compress data? I already see 3 places where your current code is inefficient 18:58:13 can reduce by 2-3 bytes per block easy 18:58:43 instead you try to abolish the legacy field which is useful, only because you want to save 1 byte 19:01:04 For that reason and that also it is being used as a misleading display value 19:01:18 it's not misleading 19:01:35 it's exactly the number of transactions in the block (including the coinbase) 19:02:38 Only when the block template creator is honest 19:02:52 if you want to completely change how PoW hashing blob is constructed, you can get down to 32 bytes per block 19:03:22 "Only" = always, because if they're not, they will lose block reward 19:03:31 block will be invalid 19:05:36 just make PoW hash = RandomX(keccak(block header)), then you will only need to send keccak(block header) aka block id 19:06:07 and the initial block header in full 19:06:28 sech1: not very useful, that doesn't prove the hashes form a chain 19:07:45 A) that is still a vector even if you ignore it. It costs nothing to the template creator beside opportunity B) again, miner-local txs don't lose block reward 19:08:19 I'm pretty sure you can't compress it to fewer than 35-ish bytes per block 19:08:45 I was talking about 37-38 by my estimates 19:09:27 And you need timestamps for non-fixed difficulty calculation 19:09:49 If we're talking about new block header rules for the next hard fork, I'd propose a rule to have a strictly increasing timestamps. 19:10:05 that will backfire 19:10:12 timestamps will run away into the future 19:10:37 No 19:12:05 I'd even go further: strictly increasing timestamps and wtema for the difficulty calculation using only the last block's timestamp 19:12:48 you will have "T,T+1000,T+1001,T+1002,..." if someone malicious make a big jump in their block 19:12:54 not very good 19:13:26 other nodes will have to adjust their real timestamps just to satisfy the new rule 19:14:12 so now instead of one fake timestamp, you have multiple 19:14:34 forcing honest miners to fake their timestamps 19:16:35 So what? The timestamp should only be used for the difficulty calculation anyways. You don't set your watch based on it. At most, it can drift by the FTL, which can be ~10 minutes. 19:18:08 what are the exact rules then? It's either my example, when the max deviation is limited, or it will run away into the future because of malicious miners 19:18:31 it's not a timestamp at this point anymore 19:20:18 FTL of 10 minutes means nodes won't accept blocks with a timestamp more than 10 minutes ahead of their local time. And the purpose of the timestamp is to adjust the PoW difficulty. It can be in arbitrary units as long as it does its job. 19:20:30 if max deviation is limited, then it will open some new attack vectors, because honest miners won't be able to just set their timestamps, and maybe even won't be able to broadcast some mined blocks for ~1 second because their timestamp is too far ahead 19:26:43 FYI, my proposal is not a new idea, it has been researched and your concerns are unfounded. 20:15:40 I wrote up what was discussed in the monero-docs channel. I think using remote nodes at all essentially destroys privacy in a bad way that doesn't seem to have been previously described https://gist.github.com/k-privacy-enjoyer/9684e4b5270cd806a31e3c0819e8f3c1 20:18:46 @kiersten5821:matrix.org: Thanks. Feather wallet does switch remote nodes regularly, making it the best remote node wallet 😍 20:20:04 AFAIK, it switches remote nodes regularly for tx construction and wallet sync. By default, it does multi-broadcast to all nodes on its list simultaneously to reduce the chance of a failed broadcast. 20:23:32 This may be helpful for your computations: https://github.com/Rucknium/misc-research/blob/main/Monero-Fungibility-Defect-Classifier/pdf/classify-real-spend-with-fungibility-defects.pdf "Formula for Accuracy of Guessing Monero Real Spends Using Fungibility Defects" 20:37:57 @rucknium: thanks, i didn't know that and checked, you're right. i added an incomplete section at the bottom about which wallets are safe from this 20:40:29 i probably won't make better estimates as i expect it to still be really bad and we already know the solution, i am just a simple user of the software