05:05:03 Last week I posted regarding an idea to write a small C++ module which enables using the Monero network for small scale encrypted communications. The idea was to encode message characters into double precision floating point numbers, which will then be sent as tx amounts into the recipient's sub-address, who after receiving them - decodes them back into a character string . Right now my module does not interact with any Monero processes 05:05:03 - and only uses modules from the standard C++ library. Anyone interested to read more about it, can find it here: https://github.com/dvd280/sparrow-messenger 06:26:03 so best case 1.42Kb per letter and median 0.5 letters per minute 07:09:12 "Last week I posted regarding..." <- This is very cool. For experiment sake, maybe consider adding some kind of start/end character sequence so a wallet would know to interpret the values a message, a meta block that could contain things like return address, name, etc, and then have your message in a content block. 07:10:27 crypto_grampy: can you elaborate? I am not sure what you mean 07:12:40 anonyphi[m]: If your 'messaging protocol' was wrapped in say .000000777's, you could make a wallet that worked as normal, but interpreted values wrapped in a start and end value as text messages. If that makes sense. 07:13:01 To integrate this with wallet functionality there needs to be consensus on it... The module as it stands just automates the encoding/decoding part. The user still needs to send the transactions himself when he's sending, or export the transfers csv him self when he is recieving. 07:13:45 s/him/himself/, s/self// 07:13:49 s/him/himself/, s/self//, s/recieving/receiving/ 07:14:22 crypto_grampy: I do agree that this opens up a large opportunity to add unique functionality to monero 07:15:41 anonyphi[m]: Yep. I'm just saying if you developed it further as a messaging protocol in mind. It would be fun to hack one of the wallets and add something like that in 07:17:34 crypto_grampy: Yeah, I thought of doing it using the monero-cpp library that woodser wrote, but it is currently ready only for linux, so working out the dependencies to make it compatible with other platforms is quite a pain. 07:19:29 The module is intended mostly as a demonstration of how would one go about doing communication on the monero network with minimal fees/costs. It still is quite expensive to send long messages. 08:36:08 anonyphi[m]: Does this demonstration, to keep it simple, still encode 1 letter per transaction, as you described in your first Reddit post about this? 08:38:37 rbrunner: Yes, if the message is n characters long, the number of transactions that the program creates is n+2 (the extra two are separator characters, so that the reciever is able to distinguish between messages 08:39:36 s/rbrunner: Yes, if the message is n characters long, the number of transactions that the program creates is n+2 (the extra two are separator characters, so that the reciever is able to distinguish between messages/rbrunner: Yes, if the message is n characters long, the number of transactions that the program encodes and puts in the EncodedTxs.txt is n+2 (the extra two are separator characters, so that the reciever is able to distinguish 08:39:36 between messages). this means that the EncodedTxs.txt file will have n+2 rows, one per transaction./ 08:43:47 1 character per transaction is the opposite of "communication on the monero network with minimal fees/costs" 08:44:20 more like maximal fees/costs :D 08:46:20 sech1: It is not false, if you have method to communicate through a channel that's secured by 3 GH/s that is cheaper, I would be happy to use it instead. 08:46:42 * channel that's completely private and secured by 08:46:47 just use tx_extra and encrypt stuff there, up to 100 Kb per transaction 08:47:06 actually don't, Monero blockchain is not a messaging app 08:47:30 sech1: Im sorry but it seems like you do not get to decide how people use their funds. 08:48:06 there are talks already to remove tx_extra field in the next hardfork 08:48:11 precisely for this reason 08:49:07 *remove or limit in size drastically 08:49:52 sech1: This does not affect my method... My implementation makes the transaction amounts themselves serve as data bytes. 08:59:21 anonyphi[m]: I would love to hear your opinion about BitMessage and one of its implementations, PyBitMessage? How would you compare that with your Monero-based method? 09:02:31 I readily agree that the BitMessage is *not* secured by gigahashes of mining power like the Monero network, but then I fail to see any possible attacks on it that would such hashpower to defend against 09:02:44 The network seems to be safe without that 09:04:03 rbrunner: main difference between using monero and every other platform, is that with monero you get security that you just can not get in any other protocol. the reason is that if you wanted to max your security - you could send each transaction through a different node on the network. You also do not need a chain of active connections between you and the reciever. This method is asynchronous. meaning that the recieving party does not 09:04:03 need to be online when the message is sent, and the reciving party does not need the sender to be online to read his message. 09:04:29 s/./,/ 09:04:59 s/rbrunner: main difference between using monero and every other platform, is that with monero you get security that you just can not get in any other protocol. the reason is that if you wanted to max your security - you could send each transaction through a different node on the network. You also do not need a chain of active connections between you and the reciever. This method is asynchronous. meaning that the recieving party does not 09:04:59 need to be online when the message is sent, and the reciving party does not need the sender to be online to read his message./rbrunner: main difference between using monero and every other platform, is that with monero you get security that you just can not get in any other protocol. the reason is that if you wanted to max your security - you could send each transaction through a different node on the network. You also do not need a 09:04:59 chain of active connections between you and the reciever. This method is asynchronous, meaning that the recieving party does not need to be online when the message is sent, and the reciving party does not need the sender to be online at the time of reading his message./ 09:05:48 s/rbrunner: main difference between using monero and every other platform, is that with monero you get security that you just can not get in any other protocol. the reason is that if you wanted to max your security - you could send each transaction through a different node on the network. You also do not need a chain of active connections between you and the reciever. This method is asynchronous. meaning that the recieving party does not 09:05:48 need to be online when the message is sent, and the reciving party does not need the sender to be online to read his message./rbrunner: main difference between using monero and every other platform, is that with monero you get security that you just can not get in any other protocol. the reason is that if you wanted to max your security - you could send each transaction through a different node on the network. You also do not need a 09:05:48 chain of active connections between you and the reciever. This method is asynchronous, meaning that the recieving party does not need to be online when the message is sent, and also it does not need the sender to be online at the time of reading his message./ 09:05:51 Would you mind to continue on #monero, as this is now not about Monero development specifically? 09:10:37 np sure 09:13:09 ... 1 char per tx ? Wha the atual fuck ? :D 09:13:35 moneromooo: it is slow, and it is expensive, I admit. 09:14:32 but it is also pretty safe, especially if users make sure to send each byte to a different node on the network 09:22:26 I knew moneromooo would like it 09:35:20 "crypto_grampy: Yeah, I thought..." <- I don't think the monero-cpp library has any dependencies that monero-project doesn't have 09:35:45 getting the build to work on windows might take some tweaking as I haven't tried myself 09:38:04 woodser: yeah Im certain that it is possible, it's just too much work. either way I think I kind of prefer the non intrusive method of not interfacing with sensitive Monero processes directly. 09:38:15 * much work for me. either 09:39:21 perhaps wallet rpc and a library in your preferred language is a better approach 09:41:23 Possibly, my aim was just to show it was possible. people who can do C++ will have no problem understanding my 150~ lines of standard C++ and converting them to more custom solutions if they wish 09:50:41 > However in the case of a core wallet churning continuously through a primary address, both the “recipient” and the “change” output are sent to the same wallet, and may be combined later! Unless the anomaly exhibited extremely precise change control practices, it is likely that in many cases two outputs created by the same transaction would be consumed by the same or subsequent transactions. 09:50:41 What is the change output of a churn if you churn the whole balance? (Regular churn, not this spam attack.) Does it differ if you churn to the same wallet or send all to a fresh new wallet? Any opsec tips for churns? I churn since 2016 but didn't know the above info... 1/11 isn't enough plausible deniability for me (e.g. collusion of sender/recipient), and AFAIK each churn creates a new branch for 11^x possibilities, but please let 09:50:42 me know if I'm wrong. 09:53:06 change output is 0 XMR if you churn everything 09:53:27 it goes back to the original wallet and can still be combined with other outputs in that wallet 09:54:32 Empty input? 09:55:04 yes, 0 XMR. But your wallet can still use it when selecting inputs 09:55:15 When and how would it be selected for a TX if it has no value? Is it random? 09:56:08 afaik wallet selects two random inputs and if they have enough XMR combined then they will be used. One of the inputs can still be 0 XMR 09:59:09 OK thanks. I assume both churn and change has the same timestamp. If you churn quickly (e.g. a few churns with 1 hour between churns), could it be visible in e.g. xmrchain that you are spending the churn and its change? How about one input, 2 output TXs? I will try to find out what inputs I have and how many 0 XMRs are there. 10:03:27 use sweep_single to churn a single input, that way you make sure it's not combined with anything. Also use random time intervals between churns and move the final churn to a separate wallet to avoid combining inputs later 10:06:53 Thanks. I generate a new fresh wallet to receive Monero and move to a separate wallet per use case for spending (e.g. Bitcoin ATM vs. VPS payments vs. Bitrefill). I churn a few times between steps, anything from 1 hour to a few days, no specific or automated times. I use Monero GUI but will download Monero CLI to see inputs and access sweep_single. 10:08:35 The 0 change output goes to a random address, not the sending address. It'd be too confusing. 10:14:19 Is it burned or goes to a random address that is used? E.g. people can receive other people's 0 XMR churn changes and use them in their TXs? 10:21:34 Another question: Would it be recommended to use a fresh subaddress per churn in the same wallet? I only use main addresses right now. 10:26:58 if it goes to a random address then chances of receiving 0 XMR output are astronomically low 10:27:02 Also I guess 0 XMR changes can also be used as decoy inputs in regular TXs? 10:27:37 this phrase is not correct then: "However in the case of a core wallet churning continuously through a primary address, both the “recipient” and the “change” output are sent to the same wallet" 10:28:03 It was here: https://mitchellpkt.medium.com/fingerprinting-a-flood-forensic-statistical-analysis-of-the-mid-2021-monero-transaction-volume-a19cbf41ce60 10:37:36 They can be used as fake outs. 10:56:59 Thanks for the help. Looks like churn is how I expected (churn 1 input, get 1 input back). 11:50:59 If I need to record the time something happens in the Monero codebase with more than 1 sec resolution, is ticks and misc_utils::get_tick_count() the way to go? 11:52:27 See perf_timer.h, it uses those IIRC, and converts to SI. 11:52:45 Or gettimeofday. 11:56:46 Thanks. So if I want to record the exact-enough time a tx entered the pool, so somebody can ask "give me the txs that entered the pool at or later than point in time x" 11:57:20 Those ticks look like the right way to do it, with a datatype of uint64_t then. 11:57:35 Oh, you can set log level to msg.p2p:INFO and you'll get the timestamp for these. 11:57:50 Oh, sorry, programmatically ? Then yes. 11:58:33 If your intent is to know which txes are new, I suggest you use the txpool cookie instead. 11:58:34 Exactly. There are probably a dozen ways to do it, but I try to stay within style and do it "like it should be done". 11:59:11 er, in fact, scratch that. 11:59:44 Yeah, I think I need something really time-based. From wallet2 to connected daemon. 11:59:57 With the time of that daemon, of course. 12:01:14 You might want to have the daemon send the time with that response, or you may miss things. 12:01:33 Yes, that's part of the pattern. 12:01:50 Though sending the time might open the door to fingerprinting, since a clearnet and tor daemons would have the same time offset... 12:02:18 "Here, have pool txs until point x. Ask again with x+1 if you want new ones next time" 12:02:23 So maybe you want to quantize that time a bit. 12:03:10 Quantizing would require asking with x again, thus getting dupes. Not sure which is best. 12:03:40 I see. I suspect that the system should be robust and able to deal with duplicates anyway, so that looks doable. 12:05:30 Since you are here already: If I want to store say one hour worth of tx ids, ordered by such a tick-resolution timestamp, and regularly delete the oldest ones to keep it one hour give or take 12:05:43 What would be a good C++ / STL data structure to do so? 12:05:56 Can find out myself of course, but you probably know just like that :) 12:06:09 map or multimap. 12:06:27 The iterator will be sorted by key. 12:06:39 Alright, thanks, will read up about those two! 12:06:41 multimap if you can have dupes. 12:06:52 why not use two parallel vectors 12:07:05 I guess deque would also work since it's always ends yo're interested in. 12:07:11 Less memory allocation. 12:07:54 What's that "two parallel vectors" idea? 12:08:18 multimap will be less annoying if you need to remove some txes at random places though, which ahppens as blocks get mined. 12:08:47 two vectors, one for ids, one for timestamps, every new tx/stamp pair is pushed into their respective verctors 12:09:35 vectors allow deleting objects within them, although it would require memory reallocation. 12:10:11 also, if you plan on having a constant number of ticks in your container then why not use an array? 12:10:29 i mean two arrays inside some class 12:10:47 Maybe I just submit something and vtnerd will give the absolutely best optimized version in their code review :) 12:11:34 anonyphi[m]: No, the number of things will not be constant, so simple array is probably not a good way to go. 12:46:00 while talking about containers, monero seems to use std::unordered_map, was it ever considered to use a faster hashmap? 12:50:19 std::unordered_map is as fast as the compiler/c++ library that implement it 12:51:09 a "faster" hashmap might not be faster than std::unordered_map on some systems 12:51:25 yes i know, but there exist different benchmarks that the version in the standard libs are not that fast compared to other implementations 12:53:18 is it even a bottleneck in any place? My guess is crypto math and PoW are the 99% of bottlenecks. Not hash maps 12:53:27 for example google sparshhash or googles dense_hash_map outperform the std::unordered_map in a lot of benchmarks 12:53:46 sech1: i am just poking arround if this was ever considered 12:54:15 I'm not a fan of unneeded optimizations. More dependencies on more 3rd-party code that can break 12:54:26 atomfried[m]: aren't there limitations/risks to other hash map strategies? 12:57:24 UkoeHB: that should be evaluated ofc but as far as i know std::unordered_map uses chaining and dense_hash_map uses quadradic probing, a lot of the performance difference comes from the different strategies used i guess 13:01:44 some of them also use sse instructions to speedup some things 13:01:44 https://www.youtube.com/watch?v=ncHmEUmJZf4 15:28:55 The daemon keeps the tx pool on disk and accesses through LMDB, right? Why is that, because the size is open-ended, and/or because you don't want to loose it if the daemon restarts? 15:35:31 size 15:36:02 and sure, it's more efficient network-wise not to lose it on restarts 15:37:28 Ok. But say for a new purpose I would need the pool transactions ordered in a new way, it should be possible to keep their hashes in RAM in the right order and then pick the transactions themselves from the DB? 15:38:01 Or would it be altogether better to create a new index for that order in the DB? 15:39:29 IMO you should always assume your data will grow larger than RAM so yes, a new index in the DB would be best 15:39:47 but what do you have in mind? 15:40:23 I try my luck with this here: https://github.com/monero-project/monero/issues/7571 15:40:44 For that I would need chronological ordering of pool txs, according to the time they entered the pool 15:40:59 Not yet sure how they are ordered now in the DB however 15:41:10 i dunno either, mooo wrote that 15:41:32 Doesn't a new index open up all kinds of upgrade problems? 15:42:43 how so? 15:43:07 if you're merely adding a table, older versions will just ignore it 15:43:47 So a daemon that includes that logic needs a check "Index exists already?" and if not creates it first? 15:44:40 Auto-upgrade, so to say 15:44:44 that would work yes 15:44:59 just like the other auto-upgrades it already does 15:45:14 i see txpool is keyed by txn hash 15:45:48 so yes you would need a new table keyed on timestamp, containing hash 15:46:57 Ok, thanks. Looks like a lot to learn for me if I really go down that route ... 15:48:04 I can write that specific code if you want. not a big deal. 15:48:14 it'd be up to you to wire it in to wherever you need to use it 15:49:04 Sounds an interesting offer. Let's see how this develops, I am only in the investigation phase now. 15:49:18 ok, keep me posted 15:49:50 Sure thing! 15:50:59 No upgrade problem, the lmdb open call creates the table if it does not exist. 15:51:24 moneromooo: looks like txn receive_time is a uint64_t, but resolution is only 1 second 15:51:34 The txpool object already keep an in-RAM index of txes sorted by fee/byte. 15:51:49 that might be annoying for lots of txes arriving in the same second 15:52:11 If you need better resolution, I have no problem with bumping resolution. 15:52:34 That one might need a db upgrade though, to convert txpool_meta. 15:52:49 I guess it will depend on what the wallet needs. there's no problem using DUPSORT in LMDB to allow multiple records per timestamp key 15:54:50 with an eye toward eventually handling thousands of txs/sec, we would prob want to bump rez up finer than millisecond at least 15:54:56 Maybe a first step might be to make the fee/byte index a lmdb table too. Easiest job first :) 15:58:52 I would use gettimeofday() to retrieve microseconds, and shuffle so 44 bits of seconds OR'd with 20 bits of microseconds 15:59:47 that gives headroom for 4096 * 70 years 16:01:09 Hmmm ... if you take that with the thousands of txs/sec serious I think the way wallet2 works now would probably break down anyway, for every sync time it fetches the whole tx pool. 16:01:40 that can be fixed some other day 16:01:56 :) 16:03:03 "The txpool object already keep an in-RAM index of txes sorted by fee/byte." Looks tempting for me like an easy way out, I make a second RAM-only index, with the same "right" ... 16:06:28 anyway, after your work, wallet2 wouldn't need to do that any more would it 16:06:47 it would only need to fetch txns newer than the last time it looked 16:06:57 Yeah, probably 16:07:56 all of this still implies that users should just always have a long-lived wallet-rpc running all the time 16:08:14 and UIs should just talk to that. would save so much of the UX badness everyone complains about. 16:09:45 have it on home network, listening on i2p, and it'd be accessible from anywhere, no port forwarding issues 16:32:00 In an ideal world ...