09:19:06 @rucknium:monero.social peer selection likely remains a hot topic... Great work*. I'd like to get into your comments about BTC's ban score. I looked into how it is implemented in btc. Also looking for positive signs. Maybe next week ? Kind of meta-topic. 09:20:02 *http://maldomapyy5d5wn7l36mkragw3nk2fgab6tycbjlpsruch7kdninhhid.onion 15:24:25 I am sorry for the late reply, I wasn't available yesterday. I don't have any major points to discuss, but I will join briefly for a quick question. I received some feedback regarding spy nodes and would like to get your input on whether it could be useful. 17:00:37 Meeting time! https://github.com/monero-project/meta/issues/1208 17:00:49 1) Greetings 17:01:10 *waves* 17:01:17 Hi 17:01:24 Hello 17:01:34 hi 17:01:36 Hi 17:01:53 Hello 17:02:07 Howdy 17:02:09 ola 17:03:26 2) Updates. What is everyone working on? 17:04:25 Me: hot/cold wallet changes for after FCMP++, plus a bunch of smaller miscellaneous FCMP++ tasks. 17:04:30 me: updating the scan_tx feature for FCMP++ / an RPC to fetch paths in the tree by global output id 17:05:03 started literature review of +20 papers on anomaly detection, self-organizing trust models and game theory in p2p nets. 17:05:17 <0xfffc:monero.social> hi everyone 17:05:26 me: I have preliminary results from the custom loss function of the neural net learner to evaluate the privacy risk of deploying OSPEAD without a hard fork. I wrote a simple proof for the mixed strategy equilibrium of the adversary-protocol game for subnet deduplication. Continuing to test rbrunner's subnet deduplication implementation. 17:05:43 Me: completed tx construction in lws-frontend, minus some fee bugs. So working on that and completing remaining functions 17:05:50 <0xfffc:monero.social> me: finished our new transaction relay protocol. Just cleaning it up and submitting it in a day or two. https://github.com/0xFFFC0000/monero/pull/60 17:06:35 flip flop: Check Cholex & Ignat (2024) "Sybil Attack Strikes Again: Denying Content Access in IPFS with a Single Computer" https://dl.acm.org/doi/10.1145/3664476.3664482 17:06:53 Cholez* 17:07:22 3) FCMP++ & divisors. 17:07:41 Me. Running fxmp++ testnet very painfully 17:08:23 not ready for stressnet [@rucknium:monero.social](https://matrix.to/#/@rucknium:monero.social) 17:08:26 What's painful? 17:08:35 A lot is broken 17:09:09 A list would be solid 17:09:22 Like, i couldnt send tx until i deleted and resynced the blockchain 4x, ffwd -> wallet is completely broken after a few hundred txs 17:09:35 AFAIK, we still don't have a document from Cypher Stack discussing their progress on the divisor review. It is OK if Cypher Stack wants to keep it non-public for now, but I wanted to check on it Diego Salazar 17:10:19 "i couldnt send tx" -> what error were you seeing? 17:10:29 it segfaults 17:10:40 the wallet or node? 17:10:46 Oh/ in the beginning.. i have the errors saves somewhere. I think i sent to jberman 17:10:50 Wallet 17:10:54 This is what I was afraid of for the public stressnet. I wanted to have things break on the alpha stressnet so we could spam confidently. 17:10:54 Jeffro** 17:11:51 Yes I have it in the DM, I will forward to j-berman. I haven't seen that error yet, but I also haven't tried repro-ing yet in the way that you described. Although this is probably a good chat to have outside of MRL 17:12:49 Ideally, it would be possible to send one tx per second from a single RPC wallet 17:13:01 More like 1 tx per 30sec 17:13:17 (for 8 output) 17:13:25 Well, I guess turns out we made the right call postponing a public stressnet. Also worth sharing, jeffro256 shared instructions on how to run a local testnet for anyone who wants to help iron out bugs like this or get involved with development: https://github.com/seraphis-migration/monero/issues/47 17:13:39 wrong call 17:14:00 ya would've been great if everyone spun up nodes and started complaining about how everything is broken 17:14:05 yes 17:14:22 How else would anyone know everyrhing is broken 17:14:39 We do know now, right? 17:14:49 Yea, because i'm stubborn 17:14:52 Reflecting on divisors, I wonder if MRL expected miracles in a tidy time box. In other words, this level of mathematics requires more time than the quotes given to MRL by the cryptography firms. 17:15:04 Something to consider for the future. 17:15:06 This is because we left out the blinds cache, because divisors is postponed 17:15:51 Spamming has special requirements, which will need to be worked through. The blinds cache would eventually be exhausted 17:16:29 The blinds cache wasn't finished fwiw 17:16:58 I think some reviews went fine within the alloted time frame, perhaps not many ahead-indications that divisors would be so different 17:17:04 As in, jeffro and I still had work to do to harden it for usage 17:17:09 btw @ruck, my "spammer" just sends normal txs in a loop. Not submitting raw txs, nothing pregenerated etc 17:17:22 <0xfffc:monero.social> I think we should have plan for extensive, long test, including stressnet, testnet of fcmp++ + carrot. 17:17:53 Agree with that^ 17:18:05 I wanted to release "my" testnet to public, but until my broken wallet issue is fixed i think its a waste of time 17:18:34 great so right call! 17:18:38 ;) 17:18:42 no 17:18:56 Itbtook like 3 days ans 1000+ tx to hit this condition 17:19:13 It depends on how you structured your tx sending. The cache auto-refills, so it would work in the background. If you spaced out sending, b/c of the 20-minute lock time, you could ostensibly have it re-filling with a high-ish tx volume depending on how fast your machine is 17:19:17 Oh nice, that's good to hear 17:19:29 so we're still waiting on divisors report thing from cypher stack, right? 17:20:04 afaik yes^ 17:20:07 Yes; the meeting log from last week has the details. Strong language :) 17:20:30 indeed. So whats the plan? Do we have a timeline from them? 17:20:39 <0xfffc:monero.social> Sorry for hijacking the meeting. speaking of testing, this is the new sheriff in town: https://github.com/0xFFFC0000/benchmark-project 17:20:40 <0xfffc:monero.social> Running stressnet on your local machine. with how many nodes you want. This is pre-pre-alpha. So very ugly code. But even at this step found a serious thing that I will disclose later. 17:20:42 "Itbtook like 3 days ans 1000+ tx to hit this condition" -> my "Oh nice, that's good to hear" was in response to this btw 17:20:49 AFAIK, the current plan, as last discussed in #no-wallet-left-behind:monero.social , is for jeffro and jberman to keep working on the integration work, which needs to be done regardless of divisors or not, and figure out someone else to write the non-divisor FCMP code, hopefully. 17:20:57 Though yes if you intended to dump it all at once at didn't sufficiently fill up the cache, then it will run out pretty quickly 17:22:32 the extremely large inputs is a separate issue, (for some reason 17000+xmr are locked eventually). but the double spend -> segfault wallet == i cant even resync the wallet. Its nice to find these issues as early as possible 17:22:36 I think we have learned all that we need to from Cypher Stack. The document would have the details, but the details aren't very important right now. IIRC, they estimate maybe 6 more months of effort to actually prove the soundness of divisors (or disprove them perhaps). 17:22:47 My understanding is that you can't give a timeline for something you don't fully know yet how hard it will get, or whether it will work out att all 17:22:55 But in the meantime, the non-divisors backup plan should be pursued. 17:23:17 "figure out someone else to write the non-divisor FCMP code" -> we should be good on this front (we have a strong candidate), but out of respect would request this is taken at my word at this point until I have confirmation 17:23:54 Oh the tension 17:23:57 i think divisors integration (plan a) should be completed before moving on to plan b 17:25:47 anyway, i'm done ranting. Hopefully jeff or 0x can fix the wallets 17:25:53 "(for some reason 17000+xmr are locked eventually). but the double spend -> segfault wallet == i cant even resync the wallet. Its nice to find these issues as early as possible" -> appreciate you testing this, can investigate it further 17:26:18 What's your reasoning, in short, about using the divisors already nevertheless? 17:26:58 I think I agree with rbrunner in that these things can't really be assigned a timeline. If they truly are grid locked at the moment on agreeing on the validity of the security proofs, I really don't see any timeline given holding any weight. This isn't to knock Cypherstack, but that's just how cutting-edge R&D is 17:26:59 And well, don't we do this already in the code that you test now? 17:27:19 (the divisors) 17:27:27 We assume that veridise's ACK, while rushed, was correct, and that CS more in-depth review will agree with it 17:27:55 The code i test now, i think (dont know), is moving towards backup plan 17:28:02 jeffro256: yeah, understood. 17:28:10 With "assume" you mean "we speculate"? 17:28:17 Yes 17:28:21 a completed divisors integration would include a blinds cache and hardened weight analysis, which seems ill-advised to spend cycles hardening, when instead we could work on need-to-be-completed tasks for either path a or b 17:28:23 jeffro256: Yes maybe I self-contradicted myself a little "expecting miracles in a tidy time box....And it will be done in 6 months!" :D 17:30:10 Yes i agree on the common tasks, but not plan b (non-divisor path) 17:30:58 one comment from Diego last wee indicated that CS isn't exactly sure how far they are from having a clear conclusion on divisors. there was a perceived sense of urgency on their part, which made them draw a NACKing temporary conclusion for now. IIUC a total ACK or a total NACK on divisors could be anywhere from a few weeks from now (optimistic path) to 6+ months where only a full 17:30:58 formalization yield clear results, but for now it's all uncertain. 17:31:13 Might be a good idea to wait until somebody went so far into that plan b as to be able to give an estimate of work required 17:31:27 divisors is just an optimization, right? it won't even need a hard fork to plop in? 17:31:35 I mean, if that's only 2 weeks or so, in extreme, no sweat 17:31:39 it would need a hard fork 17:32:09 it's an optimization like BP+ was an optimization over BP 17:33:12 In magnitude, it's more like how BP was an optimization over non-BP RingCT txs, but for verification time instead of tx size. 17:33:19 AFAIK 17:33:55 I think divisors are larger than non-divisor, but non costs more to verify 17:34:08 "Yes i agree on the common tasks, but not plan b (non-divisor path)" ok, well in the immediate term, the candidate working on non-divisors would not be working on anything else for FCMP++, so in the immediate term / until that's implemented, no resources are slated to be pulled away from anywhere else toward it 17:34:26 I would speculate that non-divisors FCMP with safe parameters may cause tx confirmation delays routinely. 17:34:33 How much larger? 17:34:59 Until it's implemented, jeffro and I can continue on common tasks for non-divisors and divisors, and then we can reconvene and discuss the optimal path forward dealing with divisors versus non-divisors 17:35:44 Sounds good to me, in any case 17:35:49 Ahhhhh back in the dark ages when Borromean ange proofs where >6 kilobytes per output 17:36:04 lmao 17:37:13 IIRC non-divisor FCMPs will actually probably be slightly smaller 17:37:15 So I should complete the scaling on the basis of non divisors. It can then be changed later if we decide to go with divisors 17:38:29 As far as we know right now, we can pencil in (without actual figures): non-divisors would likely be slower to verify, smaller in byte size, and faster to construct 17:39:19 I don't want to share figures that turn out to be incorrect, so would probably be best not to assume what they'll be at this point 17:39:24 <0xfffc:monero.social> I have a suggestion, how about having a list of bugs specifically for FCMP++ + Carrot. 17:39:55 Have started to keep track of bugs over here: https://github.com/seraphis-migration/monero/issues 17:39:57 You can use the issue tracker at https://github.com/seraphis-migration/monero/issues 17:40:01 jinx 17:40:46 <0xfffc:monero.social> a separate place, or list that we all can report, and keep track of the status. The reason why I am saying that is (1) I want all to see the status (2) each of us gonna encounter different issue running / testing. 17:41:27 I think, until we have an academic NACK on divisors, we should be building torwars divisors .. 17:41:34 <0xfffc:monero.social> ( that is just a suggstion, don't wanna hijack the topci ) 17:41:55 (for irc, my comment is reply to artic) 17:42:04 is something insufficient with those github issues 0xfffc ? 17:42:11 / keeping track there? 17:43:08 "I think, until we have an academic NACK on divisors, we should be building torwars divisors .." We have an academic NACK on divisors, one of the researchers thinks there is no way it'll work 17:43:13 <0xfffc:monero.social> ( Nah, my second message was supposed to be attached to first one, but you and jeffro beat me and send the link faster) thanks I was not aware keep the FCMP++ + Carrot issues there. 17:43:14 Jberman https://github.com/seraphis-migration/monero/issues/46 17:43:16 This looks like same bug i had on the first few testnets i spun up 17:44:22 Frankly, I don't understand the hurry about going after divisors despite the situation is so foggy right now ... 17:45:04 Especially with jberman and jeffro256 have weeks of non-divisor-related work on their board 17:45:35 It is a simple change to accommodate the transaction weight change. 17:45:36 The question of how to price actual transaction size vs verification time is entirely a different matter, given the likely change in verification time with divisors. 17:46:57 " one of the researchers thinks there is no way it'll work" We just fire that one? :) 17:47:57 science done right 17:48:52 to ease some of your frustration here / explain more of my perspective on why it's also nice to continue tasks on our end before opening up premature code very wide: here is an example of a PR that will lend itself to stronger handling of types across the C++/Rust FFI. The current code's handling is what may have lead to a segfault. Completing the full scope of this PR could solve 17:48:52 the segfault on its own without even needing further digging into its original cause with the current code: https://github.com/seraphis-migration/monero/pull/39 17:50:51 Not to be too critical, but the "divisors" that are being discussed were suggested by Liam Eagen, who also proposed BP++: https://moneroresearch.info/83 17:50:52 Later, sarang reviewed BP++ and thought its claims of soundness were not sufficiently supported: https://moneroresearch.info/217 17:51:31 But now sarang and Eagen are working together in the same firm, AFAIK 😁 17:52:03 I propose we ask author to review his own work 17:52:08 Anything more to discuss now on FCMP divisors? 17:52:45 We've made attempts but Eagen is pretty locked in on what they're doing now 17:53:01 <0xfffc:monero.social> we were really thinking we can do Box in Rust and free in C? 17:53:09 sad but underestandable 17:53:52 It's actually possible 17:53:56 on paper 17:54:02 Barbie was right when she said "Math class is tough" 17:54:06 <0xfffc:monero.social> WRONG 17:54:14 WHY CAPITALS 17:54:29 STOP YELLING AT ME 17:54:57 <0xfffc:monero.social> ( oh it was not yelling, it was trump like. we can take this discussion to private DM if you are interested ) 17:55:24 well it's been a TODO for many months now to not do that so, no 17:55:25 You can for Rust types which implement `Copy` and not `Drop` 17:55:41 lol. So we need countermeasures for heavy non-divisor fcmp++ txs 17:56:54 Rust-C interaction is a -dev discussion. Or #no-wallet-left-behind:monero.social . We still have agenda items. 17:57:06 <0xfffc:monero.social> ( That is such huge leap! There can be a lot of things. but for software as complex as FCMP++ going to deeply integrated into monero code base, I would take such a claim ( from Rust team or any other team ). ) 17:57:30 4) Maldo Map spy nodes research by jhendrix . Tor hidden service link: http://maldomapyy5d5wn7l36mkragw3nk2fgab6tycbjlpsruch7kdninhhid.onion/ 17:57:30 iirc @jhendrix observed that some "new gen" spy nodes can switch to full nodes, it that correct ? 18:00:16 There were reachable nodes, but only for a day. 18:00:55 A quick question about `--ban-list`. I received feedback from another community suggesting that instead of banning nodes, we could only disallow broadcasting transactions to them. 18:00:56 I can see a few advantages to this approach: 18:00:58 - Spy nodes won't know which peer has blocked them. 18:01:00 - We can still use their resources to synchronize with the blockchain. 18:01:35 Would it be useful? 18:01:40 I think that would be OK, but it requires a code change. 18:03:09 A little more like shadow-banning then 18:03:13 IIRC, that idea was considered, together with opt-out "banning" because that type of measure isn't as harsh, so maybe tolerable as opt-out instead of opt-in. 18:03:34 for now they seem to avoid serving data, anyway 18:04:35 I don't know if there is a big value in not informing spy nodes that they are banned from specific nodes. 18:05:16 those nodes still take up connection slots (sybil) 18:05:20 And they might even eventually be able to figure it out statistically, if they are receiving no stem-phase txs from a node. 18:06:06 whats new on de-doubl'ing ? 18:06:15 black hole nodes should be blocked/banned, not just silently allowed to parasite 18:06:20 In testing. 18:06:42 Careful to make nothing worse than before, only better ... 18:07:27 I had a few minor comments on the Maldo Map work: 18:07:28 > By default, every node connects to 12 peers to synchronize the blockchain, but we need to increase that number and recompile the software. To achieve this, we must make a few modifications to the cryptonote_config.h file. 18:07:30 It is not necessary to recompile. This behavior is available through the `in_peers` console command or RPC call or `--in-peers` startup flag. 18:07:49 I must agree on ofrnxmr on this one. Shadowbanning them is taking a potentially genuine connection slot and it seems more like a "political" decision to reassure the few "monero must not hard choose upon people" people. 18:08:05 > All of these represent remote full nodes, we like to call them public full nodes since they provide a service that allows anyone to download the blockchain from them, essentially, it's the same thing. 18:08:06 You can also sync blocks in the initial block download step from an unreachable node. It is just less likely to happen. 18:08:39 Also, iirc boog900 initial tests indicated that Cuprate was slower at syncing when pulling from spy nodes than genuine nodes 18:08:45 so even as a block synchronization service it's not ideal 18:09:13 tbf that was more intuition - these nodes are proxies sending many requests to few nodes 18:09:24 ugh. we just need to get txs over i2p. 18:09:25 ah alr 18:09:39 Let's get into subnet deduplication. 18:09:54 5) Unit test for implementation of subnet deduplication in peer selection algorithm. https://github.com/Rucknium/misc-research/blob/main/Monero-Peer-Subnet-Deduplication/pdf/monero-peer-subnet-deduplication.pdf 18:10:12 thought about a script, using monerod ban + print_pl ... 18:11:29 I'm running this test of the mainnet behavior of rbrunner's subnet deduplication code 18:11:30 https://rucknium.github.io/xmrpeers/reference/peer.selection.test.html 18:11:32 https://github.com/rbrunner7/monero/tree/peers 18:11:38 It is passing the tests, but two notes: 18:11:50 1) A chi-squared goodness-of-fit test is usually used to detect divergence of an empirical distribution from a theoretical/reference when the distribution is discrete/categorical. It seems that when there are lots of zeros in the empirical distribution, the chi-squared test rejects the null hypothesis when it shouldn't be rejected. This problem mostly affects the `gray_list` test because there are so many zeros. Advice online suggested to aggregate categories to eliminate the zeros, but it is best to avoid destroying information when practicing statistics. So I looked for alternative tests. 18:12:03 I tried a couple. A root-means-squared (RMS) goodness-of-fit test with simulated p-values seemed to fix the "problem" of zeros in the empirical distribution. Why? I don't know. I tried to search for information about why it works better, but didn't find anything. Because it works better in practice, I am using RMS to evaluate the implementation. 18:12:26 2) The hypothesis test of selection from the `white_list` produces a "too high" p-value. This is not necessarily a problem, but it is strange. A _low_ p-value (e.g. 0.05, 0.01, 0.001,...) means that the null hypothesis is rejected and the implementation does not behave according to the expected reference distribution. A _high_ p-value may be just a fluke, or it may be that the emp irical distribution conforms "too closely" to the reference distribution than expected by pure chance. 18:12:43 For example, a fair 50-50 coin could be flipped 100 times. Getting exactly 50 heads and exactly 50 tails is the "most likely" outcome, but it would only occur with 8% probability. It would be more likely that there would be an excess of heads or tails, by chance. The p-value on a chi-squared test of this 50-50 empirical outcome would be about 1, the highest possible p-value. 18:13:16 AFAIK, rbrunner is looking into why the initial connection handshakes seem to fail at a high rate. 18:13:39 So, maybe that would be fixed or at least understood when this implementation code is submitted as a PR 18:14:02 Yes, seems that there is something strange in the attempts to find new white peers. Maybe was for years, with the old code already, will know more after this weekend 18:14:20 Related, maybe the number of attempts in a single "batch" should be increased from only 3. 18:14:44 > AFAIK, rbrunner is looking into why the initial connection handshakes seem to fail at a high rate. 18:14:44 could this be because of the amount of unreachable peers being injected into address books? 18:15:17 Those don't become *white* peers just like that, no? 18:16:03 ah I thought we were talking about promotion to white peers, no these would be grey peers. 18:16:28 Gray peers seem to be ok, strangely, with about a 70% success rate to find a new one when "it's time" 18:16:42 It just seems to me that the handshakes fail too quickly. In about one millisecond. _And_, it seems that the first attempt from the gray_list has a much higher probability of success than the next ones in the same "batch". Since the candidates are being pulled with uniform probability from the gray_list, it would make more sense that the problem is with "our node", not the peers. 18:17:23 Will check that, as well. 18:18:13 The whole thing is quite dynamic, but not really complicated, and can be watched with logs and debugger quite nicely 18:19:57 Next things I am going to work on is getting https://github.com/Rucknium/misc-research/blob/main/Monero-Peer-Subnet-Deduplication/pdf/monero-peer-subnet-deduplication.pdf into a form to be published as an MRL research bulletin (the first one in many years). I think the last things I need are a description of the spy node-finding procedure and a description of Monero's peer selec tion algorithm, and its proposed new algorithm with subnet deduplication. 18:20:51 rbunner: There was also the matter of whether to avoid connecting to peers in a /16 that we are already connected to, or in a /24 that we are already connected to. 18:21:21