17:51:29 <***> Buffer Playback... 17:51:29 [21:53:41] jberman[m]: editing messages in matrix causes spam on the irc side 17:51:29 [21:54:18] ah, ok sorry. I'll stop 17:51:29 [02:20:17] Thanks for the context, jberman, very helpful! 17:51:29 [10:37:18] Are there any examples/documentation on how to use ZeroMQ RPC with monerod? 17:51:29 [10:41:10] or is ZMQ not maintained at all? 17:51:29 [11:22:46] Is there an archive for past chatperusal? 17:51:29 [14:47:43] sech1 : is it maintained 17:51:29 [14:47:49] *it is 17:51:29 [14:48:14] theres some docs about PUB/SUB in `docs/ 17:51:29 [14:48:34] and https://github.com/vtnerd/monero-lws has a working example of RPC and PUB/SUB 17:51:29 [14:49:08] fwiw, someone just contacted me about some CCS work, and gave them several options and they (may) be adding msgpack support to zmq 17:51:29 [14:49:48] I wanted to see the perf numbers first ... but zmq is json only and the frequent bin->hex conversions are a decent penalty among other things 17:51:29 [14:50:52] it'd also be nice to spit spit out schemas automatically, but describing how todo that task to others is difficult enough 17:51:29 [14:51:14] but src/serialization/json_object.cpp is currently the one way to tell the structure of the json 17:51:29 [14:51:31] for both PUB/SUB and RPC 17:51:29 [14:54:02] https://github.com/vtnerd/monero-lws also uses zeromq but its abstracted a bit 17:51:29 [14:54:12] it uses both RPC and PUB/SUB Now 17:51:29 [14:57:15] vtnerd thanks. I'm mostly interested in using zmq to replace daemon polling when solo mining with xmrig. zmq has "new block on main chain" notification as far as I can see from the code 17:51:29 [14:59:35] correct, updating xrmrig was on my todo list, but if you did it awesome 17:51:29 [14:59:46] there's two block notification modes, one "light" and one "full" 17:51:29 [15:00:04] yes, polling is inefficient - it introduces delay between monerod getting new block and xmrig starting to mine on top of it 17:51:29 [15:00:12] it's on my todo list for next week 17:51:29 [15:00:47] https://github.com/monero-project/monero/blob/master/src/rpc/zmq_pub.cpp#L140 17:51:29 [15:01:09] I only need "light" notification, it seems. Then xmrig can just call getblocktemplate RPC 17:51:29 [15:01:33] yes, you get block height, previous_hash, and an array of hashes 17:51:29 [15:01:52] the array is in case of reorg it could (but not yet) send all of the blocks at once 17:51:29 [15:02:18] there's some code elsewhere in the daemon that I punted on that, but the made the schema support it 17:51:29 [15:02:25] *but then made 17:51:29 [15:03:34] I only need block height and previous hash to know if I need to call getblocktemplate 17:51:29 [15:06:14] yup, I made that mode for mining basically 17:51:29 [15:06:42] let me know if you need a "heartbeat", because one thing that is annoying with ZMQ is that it doesn't "out-of-the-box" tell you when disconnects occur 17:51:29 [15:07:15] I think there might be an option that does this though, dont remember 17:51:29 [15:07:45] otherwise you end up with "longer" polls (why hasn't a block happened in 5 mins?) or the daemon pushes something out every 30 seconds 17:51:29 [15:07:52] I have no idea how zmq work. Is it just a json protocol? Then xmrig has built-in support for json and can handle disconnects 17:51:29 [15:08:13] zmq is agnostic to the protocol format, so its whatever we want which is currently json 17:51:29 [15:08:35] I guess I'll find out when I start implementing it 17:51:29 [15:08:41] the zmq protocol itself is binary and has a length field for transmitting arbtirary data 17:51:29 [15:09:23] the PUB/SUB requires setting an option on the socket to "subscribe" to a topic 17:51:29 [15:09:44] the one requirement per the protocol is that this string must be present at the beginning of messages 17:51:29 [15:10:29] so the message as ZMQ provides will be `json-minimal-chain_main: {...}` where `{...}` is a JSON object 17:51:29 [16:52:25] Community meeting in 8 mins to discuss CCS and other things: 17:51:29 [16:52:25] https://github.com/monero-project/meta/issues/584 17:51:29 [17:24:30] boy19: it definitely is. Take a look at https://github.com/monero-ecosystem for example 17:51:29 [17:30:44] Hi! I was wondering if there was any conversation regarding code licensing for the CCS? I am waiting for them to be licensed before importing them into our vertcoin-project repository. As for the original dev's approval, I heard back from rehrar who believed it would be added soon and that it's up to the Core team to add it. I haven't seen xiphon around but can email them to join a conversation and I'm not sure who Devin is. 17:51:29 [17:30:44] https://repo.getmonero.org/monero-project/ccs-front/-/issues/5 17:51:29 [17:32:10] Devin is an employee of one of fluffypony's companies (globee I think), so you want to ping the pony. 17:51:29 [17:34:12] That projects collect a number of repo from different community members. Take a look at whatever catches your interest 17:51:29 [17:42:12] sure. Depending by what you would like to help with, you could open an issue or ping woodser here 17:51:29 [17:44:14] definitely, yeah 17:51:29 [17:49:54] boy19: You will be fine with git. My only knowledge in git hahaha: fork the repo, git clone, do your thing, git commit, git push. git diff for edits, git status for added/removed files. Something bad happened: search for "Git Tools - Rewriting History" (then git push -f) hahahaha 17:51:29 <***> Playback Complete. 17:53:01 Okay, I emailed xiphon the link to the ccs-front issue regarding licensing. 17:53:07 ping fluffypony 17:53:10 Could you possibly ask Devin to weigh in here regarding licensing for the CCS? https://repo.getmonero.org/monero-project/ccs-front/-/issues/5 17:55:01 I still need to learn how to use github, with that considered, is it still realistic that I can contribute? 17:55:02 Aloha, 17:55:02 I was trained as a programmer, worked in web design with PHP/SQL for a few years, but havent coded in 10 years. Is it possible for me to contribute if I try really hard? 17:55:02 thanks for the encouragement. 17:55:02 however, how can I find a task to work on? 17:55:02 the projects are so big. Cant someone tell you what to do? 17:55:02 ErCiccione: Say https://github.com/monero-ecosystem/monero-java for example, 17:55:03 is there a way I can request to contribute? 17:55:03 ErCiccione: Thank you, what section should I read first? 17:55:04 can someone say "write this function...." and I go away and try and do it? Is such a structure possible? 18:06:43 boy19: Sorry I am somewhat similar to your state right now. I can only do Python well right now (finding time to learn other languages like C++ and Java). I am also finding ways to contribute in. For now, in repositories, go to its "issues". Those are the repository's "problems." 18:08:47 coinstudent2048: maybe we should study together. 18:08:47 Oh, sorry for being a ass. Im just lonely. 18:09:02 Advanced python? If so, this is a good issue: https://github.com/monero-ecosystem/monero-python/issues/73 18:15:16 "write this function" is possible in some cases, but usually not. 18:15:56 At first approximation, it's possible in some specific self contained domains, like crypto (write a fast function that takes keys and does whatever). 18:16:30 Or small bits really. But non trivial stuff typically requires more "interaction" with other code. 18:16:59 Unit tests are also well suited to that, but I think most parts that can be unit tested fairly easily already are. 18:17:34 Most other stuff tends to be about modifying existing code. 18:18:38 Though I'm talking about monero itself here mostly, maybe easier to find something like that where ErCiccione pointed to. 18:20:19 t-900-a: well not very advanced, but I'll check this one and try to work on it! Thanks! 18:20:39 moneromooo: Thanks for your reply. 18:20:39 The key thing is, if you dont know what to do and you want to contribute, finding a starting point seems daunting 18:21:14 What I usually suggest (and it's what I did myself) is to use the software, find something small that doesn't work quite right, and fix it. 18:21:41 But it's modifying existing code typically. 18:21:49 thats a good idea, that would mean downloading a desktop or mobile app right? 18:22:01 Yes. 18:22:22 clearly crypto is a the future, so we need to be here 18:32:20 boy19: Since you already have experience (web design right?), I think you just need to get updated in the tech. I also find that daunting, but hmm... sorry I don't know what to advice, so I'll just say good luck! 18:34:26 You can check here for Monero web development (e.g. Patreon, online stores, web wallets): #monero-punks:libera.chat I'm also a full-stack webdev including PHP/SQL. 18:35:59 anarkiocrypto: coinstudent2048 thanks guys 18:36:19 anarkiocrypto: you must be loving it with the current global situation, programmers are like kings now 18:37:40 I am doing the same type of remote work for crypto since 2016. 18:38:26 Increased corona statism is a negative for everyone, but this probably isn't a topic for Monero Dev chat. 18:42:02 moneromooo: Do you have non trivial things in mind that would require large changes across core code? I’m talking to vtnerd about the aforementioned tasks he had in mind (zeromq supporting msgpack instead of json for performance + auto generating json-schemas) and thinking of including them + whatever else would be valuable in a CCS proposal once I’ve got a strong understanding of the scope 18:44:07 boy19: afaik moneropunks also has forum.moneropunks.org . Ok I'll be out now! 18:50:15 Non trivial stuff requiring large changes.... Well... Not off the top of my head but I'm sure there are a number of these things :) 18:50:29 Reading github issues is sure to find a few. 18:51:57 Oh, yes: making the wallet cache a lmdb file would be nice. Someone said htey'd do that a few months ago, no news since. 18:52:12 And using a rw lock for the Blockchain class instead of a mutex. 18:52:39 That last one is probably best left to someone with good knowledge of the monero code though. 18:53:36 Ah, vtnerd had suggested authenticated encryption for the wallet cache. 18:53:37 I've got decent knowledge of it :) will check these out as well 18:53:44 OK :) 18:54:54 The daemon RPC is also not really thread safe. 18:55:04 I'm not sure how much of a problem it is in practice though. 18:55:27 Stuff like getinfo can get you inconsistent data, technically. 18:55:42 Like, read curent height, block gets added, read current difficulty. 18:56:24 In fact, this one's trivial to fix. 18:56:32 If we want to fix it. 18:56:40 We probably do I think... 18:58:22 If you have two NICs, there's an old issue about being able to select which NIC to use for outgoing data. 19:00:44 wallet_api also seems to have thread related issues according to https://github.com/oxen-io/oxen-core/pull/1466 19:02:45 eh, getinfo should be using a single read txn around all of its steps 19:02:57 it shouldn't matter if a block gets added while it's executing 19:07:32 this function here, ya? https://github.com/monero-project/monero/blob/master/src/rpc/core_rpc_server.cpp#L458-L538 19:11:25 looks like it 19:12:00 also, ideally there would be no blockchain mutex or rw lock. Just use the DB txns to control concurrency 19:12:40 you're saying a single lmdb read txn, right? some of those functions inside on_get_info are reading from memory looks like, not touching lmdb 19:12:43 the blockchain_db implementation could use a mutex or rw itself if it didn't support txns. Both BerkeleyDB and LMDB support txns so no need for it here 19:13:33 yes, but those items they're fetching from memory appear to be unrelated to the blockchain anyway. like connection counts 19:16:23 so yeah, there ought to be a block_rtxn_start() call near the beginning and an rtxn_stop() call near the end of that 19:19:25 I guess that function has now been wrapped in db_rtxn_guard 19:19:34 so many layers of wrappers, geeze 19:34:32 this seems simple enough 19:35:37 thank you guys :) 19:36:41 will look into all the above 21:05:04 sech1, xmrig is using daemon polling? O_O 21:12:55 perhaps we should add a "how to get started coding on monero" in the initial contributing header in the readme. all it currently does is link to contritbuting.md. moneromooo , do you mind if i paraphrase your words if i ever get around to it? 21:15:01 Sure. 21:18:28 gingeropolous as far as I know, even pools use daemon polling 21:44:26 there was no zmq stuff until recently 21:47:47 so the workaround was the process notification, usually a signal 21:48:11 jtgrassie's mining pool software was one of the few that supported such a thing before zmq was added 21:55:39 .merges 21:55:39 -xmr-pr- 7349 7748 7753 7754 7761 7762 7764 21:56:26 .merge- 7349 21:56:26 Removed 22:10:05 hrmm... chained tx churning ... 22:11:23 anyone know a specific spot of contribute with php? 23:22:52 this might be a stupid question, but why benefits are there of lmdb over the existing implementation? 23:23:15 and also, why not sqlite? 23:26:08 * this might be a stupid question, but what benefits are there of lmdb over the existing implementation? 23:27:46 we don't use sqlite, we do use lmdb already 23:29:26 what uses lmdb currently? From the previous conversation, it seems like the wallet cache does not use lmdb atm 23:29:32 the blockchain 23:31:11 I see. is sql in general not appropriate for all monero related stuff? I haven't used lmdb before, am curious about what its benefits are 23:37:20 performance is important 23:40:19 currently some custom serialization code is used for the wallet cache as far as I know 23:42:14 ringdb also uses lmdb (to name something specific to the wallet) 23:44:43 sqlite could be used for the wallet but we don't have any dependency on it currently so it doesn't really make sense if we can just use lmdb, sqlite for the blockchain would result in horrible performance 23:50:15 what does the data structure "look" like? trying to wrap my head around how a key value store is sufficient for something as complex (to me) 23:50:27 * what does the data structure "look" like? trying to wrap my head around how a key value store is sufficient for something as complex as monero (to me)