00:03:06 the whole project is built around async calls and libuv event loop, so I'm not so sure :) 06:57:37 Hi All, I was reading about Monero and mining and came across p2pool. The concept seems pretty good atleast with my limited understanding. One statement somewhere in their website stumped me. They support only primary addresses. Why does it matter, I mean an address is an address no? 06:59:51 no 07:02:57 Okay that was succinct 😀. Let me progress a little more on mastering monero and get back. 07:03:44 https://monero.stackexchange.com/questions/8756/is-mining-monero-to-a-subaddress-supported 07:03:54 plenty of info on stackexchange 07:04:01 just search for subaddress 08:26:11 Thank you hyc 09:34:07 I wonder if it would be worthwhile to make calcpow a binary rpc instead of json. ideally it should be as low latency as possible. 09:40:39 rpc vs binary will make <0.01% difference, hash calculation is the slowest 09:41:01 btw I wrote the code for p2pool during breakfast, will be testing it today :D 09:41:30 and p2pool has good support for json rpc and nothing for binary 10:56:38 hyc https://github.com/SChernykh/p2pool/pull/78 10:56:54 it will complain about calling uv_default_loop() from the wrong thread every time 10:57:15 I'll to fix it later because it's not thread safe 10:57:22 but you can test already 12:02:41 hyc https://github.com/SChernykh/p2pool/pull/78 should be good for testing now 14:34:23 nice I'll give it a whirl 15:31:48 this might be quite neat for running many instances of p2pool indeed 15:36:49 yes, my second p2pool instance failed to allocate huge pages even though my server has 64 GB RAM. Memory fragmentation and I don't want to reboot it 15:39:22 slightly higher latency probably 15:39:34 also does it do the check via RPC on light mode? 15:40:03 cause if I feed many p2pool instances to single monerod node, I'm afraid it might not keep up on light mode 15:40:32 monerod uses light mode by default 15:41:42 hyc said something about adding a command line or ENV var option to monerod to fix it 15:44:03 also this RPC API makes it possible to run p2pool+monerod on a 1 GB Raspberry Pi :D 15:58:22 hm. p2pool just hanging at startup 15:58:42 oh wait n/m 16:00:19 yeah something not moving 16:01:57 all the hasher threads waiting for rpc completion 16:02:15 cond_wait 16:03:44 hmm, there's more than one thread? 16:03:54 I might need more than 1 cond var :D 16:03:54 yes 16:03:59 doh 16:04:28 prob going to have to serialize the entire thing, I don't think monerod supports more than 1 outstanding rpc on a connection 16:04:43 each request creates a separate connection 16:04:51 ah 16:05:21 I don't see anything weird on the monerod side 16:05:39 but if each request uses its own resources then yes, each thread needs its own condvar 16:07:13 would be nice to avoid the overhead of creating/destroying mutex&condvar on each request 16:08:50 hyc https://github.com/SChernykh/p2pool/commit/cbd3d7dd888761ffda702a54b918566a90004d3d 16:09:02 like this for now 16:09:16 overhead is little compared to RPC delay 16:12:30 and after initial sync, it will calculate PoW only when receiving new sidechain blocks (or Stratum shares with enough difficulty), so it's not so many calls 16:12:46 ok, running again now 16:14:39 still something stuck 16:14:54 no active pools, stop mining - so xmrig is not getting a reply from p2pool 16:15:35 2021-11-20 16:15:19.9117 Util background jobs running: 16:15:35 P2PServer::handle_incoming_block_async (3) 16:15:35 StratumServer::on_share_found (1) 16:17:04 still stuck in cond_wait. any chance the rpc on monerod side isn't returning a valid message? 16:17:11 guess I'll try tcpdumping a bit 16:21:53 p2pool would show something if JSON response was invalid 16:25:55 hyc I added logging of JSON requests/replies 16:27:04 you could also try to move "MutexLock lock(m_loopMutex);" to the very top of RandomX_Hasher_RPC::calculate() to serialize everything 16:27:08 it will be easier to debug 16:31:55 I don't see calc_pow in the packet trace at all 16:32:06 maybe the req isn't actually getting flushed to the network 16:34:06 and tcpdump on loopback interface should be seeing it all 16:34:42 if it's not even sent, probably event loop is not running. Check p2pool log for "RandomX_Hasher event loop started" and "RandomX_Hasher event loop stopped" 16:36:00 yeah, says it started 16:36:17 4 requests are logged 16:36:32 nothing else 16:37:17 4 connections established too. dunno why I don't see it in packet traces 16:39:47 ok, try to add some logging in JSONRPCRequest::on_connect(). Like LOGINFO(1, "JSONRPCRequest::on_connect"); 16:43:49 is it possibly sending garbage? I'm seeing some stuff that doesn't look likvalid rpc traffic 16:44:08 maybe a cmd buffer goes away before it got sent? 16:45:24 it's copied in JSONRPCRequest constructor 16:45:37 it wouldn't work with other RPC requests if it wasn't copied 16:46:11 ok. well I see the other requests like get_version get_miner_data 16:48:33 Strange. It worked on Windows. I'll check Linux tomorrow 16:55:53 ok. I filtered on loopback interface, port 18081. calc_pow is not in the capture at all 16:59:56 I see get_info, get_version, get_miner_data, get_block_header_by_height, get_block_headers_range, no other requests 17:02:15 event loop doesn't work for some reason 17:03:06 last attempt: try to remove ", &m_loop" from https://github.com/SChernykh/p2pool/blob/calcpow/src/pow_hash.cpp#L457 17:03:14 it will fall back to the default loop then 17:03:22 ok 17:03:25 it's not thread-safe, but it should work 17:05:06 it complains 17:05:07 2021-11-20 17:04:44.7250 Util uv_default_loop() can only be used by the main thread. Fix the code! 17:05:15 as expected 17:05:20 but does it send requests? 17:05:39 still seems stuck. i didn't capture pkts, lemme try again 17:07:33 nope. not in packet trace 17:11:29 hm, it may be working now\ 17:11:46 I'm seeing results, yes 17:12:30 massively delayed 17:14:37 delay before sending request or when waiting for response? 17:14:45 2 minutes btw first request and first response 17:14:53 I think it's delay before sending request 17:17:55 add log to JSONRPCRequest::on_connect in json_rpc_request.cpp. It should connect immediately, not after 2 minutes 17:18:40 LOGINFO(1, "JSONRPCRequest::on_connect"); should be enough to see it in log and figure out delays 17:19:46 it does connect immediately 17:19:54 it doesn't send request tho 17:20:38 maybe uv_tcp_nodelay doesn't really work 17:20:46 I see a connection open on port 47240 -> 18081 at 17:11:59 17:21:17 the request on port 47240 happens at 17:14:04 17:21:39 but do you have log in on_connect? It will show when p2pool actually gets notified about established connection 17:21:54 ok I'll add that 17:24:11 I see lots of on_connect logs 17:25:55 https://paste.debian.net/1220209/ 17:28:14 it looks like it only moves forward after receiving new miner data 17:30:23 and after that it works fine 17:30:44 still seems a long delay between logging the request and actually sending it 17:31:37 17:24:44.2795 request sent, 17:24:44.3041 response receive 17:31:39 not long 17:31:58 are you sure that is the response to that request? 17:32:36 no 17:32:46 ok, it's the previous one 17:32:57 monerod seems to not accept multiple connections 17:33:16 netstat shows 4 established 17:33:58 but it serializes them 17:34:35 I guess. that's still not the real problem tho 17:36:23 does this always use a new connection per RPC? can it reuse the same connection? 17:41:15 ohh I think I know what's happening. libuv loops are really not designed to work in more than 1 thread 17:42:26 the loop is just waiting on a set of handles, and m_loop is waiting on just 1 async handle (m_shutdownAsync) 17:42:48 and it stays asleep even if there's a pending connection request (uv_tcp_connect) 17:43:05 it'll start processing it only after it's woken up (by anything) 17:43:19 this is why it works with uv_default_loop, but with huge delay 17:43:28 new Monero block probably triggers uv_default_loop 17:43:52 fix coming soon 17:43:59 ok 17:44:53 need uv_async_send() ? 17:44:56 yes 17:46:51 hyc submitted the fix, try again 17:48:18 ok building 17:49:54 there it goes :) 17:52:11 you can use uv_async_init with a NULL callback 17:54:44 ok 17:57:56 overall doesn't seem to bad. job acceptance is about as fast as before but occasionally spikes if monerod was busy 17:58:55 share accept times reported by xmrig ... 18:03:12 The current code is still not thread-safe (RPC call adds more handles to loop's list and it can interfere with the loop processing TCP packets and traversing that same list) 18:03:32 I'll rewrite it to use async callback since async is there already 18:03:39 ok 18:08:13 freeing up 512MB of RAM on this rockpro64 is good ... 18:09:51 ahh, I just made it serialized 18:09:57 then I don't need to juggle with async calls 18:10:19 whatever works 18:10:43 this way when JSONRPCRequest::call modifies loop's handle list, the loop is guaranteed to not be doing anything 18:10:56 also I can use only 1 cond var again 18:11:06 cool 18:13:58 can you test again with latest? 18:14:09 ok lemme grab it 18:15:51 running. it's going 18:16:11 nice 18:21:28 yeah looks good