16:01:13 <Rucknium[m]> I'm starting to collect mempool data and I saw something interesting. Maybe this is already a known fact.
16:02:52 <Rucknium[m]> It looks like mining pool operators update the block template that they send to client miners only once: when the previous block has been found. If true, that means that any transactions broadcast on the network (and confirmed by a mining pool) have to wait two blocks for their first confirmation, not one block.
16:03:40 <Rucknium[m]> That would mean that transactions are on average taking 2 minutes longer to confirm, on average, than the ideal with an instant template update.
16:05:01 <Rucknium[m]> p2pool updates its block template every 10 seconds. In a quick spot check, p2pool's found blocks do seem to have been updated with new txs more frequently.
16:06:10 <Rucknium[m]> This also means that pool operators are, individually, leaving money on the table. It's not a lot of money because tx fee revenue is low for now.
16:07:43 <Rucknium[m]> I wonder if this is the default behavior of jtgrassie 's monero-pool software. Can it be changed?
16:08:09 <Rucknium[m]> This is all based on preliminary data analysis.
16:13:06 <sech1> p2pool updates whenever a new sidechain block is found (10 secnds on average), and it has a forced update after 30 seconds
16:13:18 <sech1> nanopool force updates templates every minute
16:14:44 <sech1> you can just run xmrig, connect to a pool of your interest, and watch the last number in "new job" lines in console - it has the number of transactions in the template
16:15:34 <sech1> and if you run with fixed difficulty, you can also check how often pool sends you new jobs
16:17:04 <sech1> p2pool also has 10 second delay for every new transaction before it gets included, to account for tx propagation delays
16:20:57 <Rucknium[m]> sech1: Thanks for the info. Do you think my hypothesis about most pools is correct?
16:21:22 <sech1> yes, I noticed it before too
16:23:14 <sech1> big pools can't send new jobs very often, they have 10s of thousands of miners connected
16:23:27 <sech1> not because of the traffic, but because of the load on DB
16:23:34 <sech1> each job for each miner needs to be saved
16:24:46 <Rucknium[m]> Looks like we can get a big improvement in tx confirmation times if pools change their block template update behavior. It's in their self interest to do so, too.
16:24:47 <sech1> not just miner wallet, but each connected xmrig
16:24:48 <Rucknium[m]> Yes I was wondering about the computational/bandwidth cost
16:26:23 <sech1> p2pool has an advantage, it doesn't need to save anything and it doesn't need to handle everything in one database
16:27:38 <sech1> it can also force send new job to miners if an especially fat (high fee) transaction is received (this logic is not implemented yet)
16:28:41 <sech1> which is logical because whoever pays high fee, wants it to be mined fast
16:31:25 <Rucknium[m]> Deviations from economic rationality like this make me slightly worried about the dynamic block size and fee mechanisms that we have in place.
16:32:14 <sech1> Pools just use whatever transaction picking algorithm is in monerod
16:33:01 <sech1> p2pool uses its own algorithm which is a bit more advanced, but it always makes better (higher block reward) choices
16:34:43 <sech1> and it tends to settle at 301500 bytes block size when mempool is full (+1500 bytes over median)
16:35:10 <Rucknium[m]> Is the logic for either algorithm documented anywhere?
16:35:19 <sech1> no, it's only in the source code
16:35:38 <sech1> but the general logic is "sort by fee per byte and select the highest first"
16:36:40 <Rucknium[m]> The difficult part is to decide when to start pushing the block size up and then through the block reward penalty threshold.
16:37:19 <sech1> monerod just keeps adding as long as the block reward increases
16:37:52 <sech1> p2pool tries to shuffle and replace other transactions to hit that sweet spot of block size and get higher reward
16:38:32 <sech1> there's also an algorithm that finds an optimal solution (it's a discrete knapsack problem), but it's too slow for real use
16:53:21 <Lyza> I can confirm that some mining pools update block template more often
16:53:50 <Lyza> MoneroOcean in particular -- I think they have customized the pool software to a good degree
16:56:28 <Lyza> or maybe I'm dumb and looking at the wrong number ignore me, looks like 1 update per block, never noticed
19:16:12 <DataHoarder> Rucknium[m]: code segment that implements it in p2pool which is lightly commented fair enough https://github.com/SChernykh/p2pool/blob/4bb198280674e4448eeee9160236127f678a6f1f/src/block_template.cpp#L263-L552
19:17:02 <DataHoarder> to test the vs the optimal one #define TEST_MEMPOOL_PICKING_ALGORITHM 1
19:26:48 <Rucknium[m]> Thanks!
21:25:41 <jtgrassie> Rucknium[m]: "I wonder if this is the default behavior of jtgrassie 's monero-pool" <- No, not quite
21:26:49 <jtgrassie> monero-pool updates the template when either the network height changes or on a configurable timer
21:27:26 <jtgrassie> which is typically set to 30 seconds, though can be whatever the pool op desires
21:45:56 <Rucknium[m]> jtgrassie: Thank you. I tried to find a config option in the code, but couldn't. Can you tell me where a pool operator can set the timer?
22:00:48 <jtgrassie> Rucknium[m]: my appologies, as I just mentioned is not pushed. The public version is just fetching every 120s or on height change
22:02:16 <jtgrassie> A pool op can trigger a new template by sending a signal (SIGUSR1) which can be done on a cron job or whatever.
22:06:45 <jtgrassie> It's not for optimizing tx fees though as new jobs won't get created unless there's a height change