cache generation time on HDD makes it no go

it's too slow even on an SSD

I had another idea of pulling random data from blockchain (some random block number + corresponding tx_pruned from that block) at the beginning of each RandomX hash, and attaching this data to the final Blake hash. This way CPU and I/O can go in parallel, and HDD should be fast enough to be done in 10-15 ms

*random block number -> data from block and tx_pruned tables

I tried this, it's too slow. It's better to select a random block and a some random txs separately.

miners will have to run fast NVMe drivers or straight up loads of RAM

whatever, just select some random chunk of data from either table

so it could be done in one I/O operation

wouldn't that penalize miners on older hardware though, thus giving the upper hand to large farms/servers/high-end hardware

miners would have to run a RAM disk or some other db structure than LMDB

miners mine in fast mode, so they have 1-2ms to read the data on each thread

or LMDB with lots of RAM so that it never has to hit the disk

Actually, typical gaming PC is 10-20 kh/s, so any NVMe capable of 10-20k IOPS will work in theory

selecting 1 block blob means several random disk accesses with LMDB

with proper buffering of completed hashes

or rather queueing of completed hashes until their db data is available

Wait, what about pool miners? Or even people running multiple miners to a single node/p2pool?

they would need their own copies of the blockchain

Wouldn't the network traffic/latency kill the performance?

Right, but if you have to pull random data for each hash, then you'll have to wait for the node to respond

So you're adding internet latency in there, no?

even "slow" NVMe drives can do way above 100k IOPS in 100% read mode: techpowerup.com/review/kingston-nv1-1-tb/4.html

100% read mode is what will happen during mining, right?

there will be some writes as new blocks are added to the db

99.9% read then :D

probably

read in the beginning of each hash -> calculate hash normally -> queue the final Blake data until the db read is finished

if I/O is done asynchronously, it shouldn't even affect the hashrate that much

the problem is that miners could skip hashes that hit blockchain data they don't have

of course db in RAM will be much faster anyway

then we can do db read in the end

more queueing, still the same hashrate

you'd want to force at least 1 RandomX program before selecting the block, maybe more

what's the size of `blocks` and `tx_pruned` tables now?

around 10 GB together, give or take

ah, so should be easy to just load them to RAM in xmrig

no need for fancy I/O

LMDB basically does this, but it works even if you don't have enough RAM

but fragmentation

miners could easily have a dedicated db with just the two tables

probably not even a db, just two files growing

because new blocks usually are added to the end

you'd need to store at least the offsets where each blob starts, some primitive database

but then, pools would just need to send new pruned blobs to miners, less than 300 Kb per block, right?

plus the initial download

but it's still a lot of bandwidth for pools

you proposal required 256 Mb every 64 blocks, so 4 Mb per block which is more than with this approach

actually, even with the original proposal, pools could just send updates every 2 minutes (pruned blobs)

*every block

initial download can be done via torrent that updates every day or week

yes, but miners would need to keep the 10 GB database

that's a botnet killer

yeah, but the point was to cripple pools

the whole idea is that each miner must have blockchain data. It turns out that it's only 10 GB and pools can send updates every block which is not as much bandwidth as initially thought

I wonder who would provide the initial download to new miners

initial download via torrent

pool can just seed from their servers

with a pruned node, you can just run it and it will sync by itself

or pruned node, yes

so pool doesn't even need to provide it

if every miner ran a node, that would be a big win

<sech1> "pool can just seed from their..." <- wouldn't that just open up the pool to an easy ddos?

miners will eventually end up with "leech" node that just gets data from real nodes and updates these two tables

less resources, less disc space used

And not just the pool, but also every miner who contributes would end up getting caught in it (assuming that their upload bandwidth is much smaller than a typical server)

so something like xmrig doing the hashing and "monero-db-miner-sync" updating the files in realtime

most likely setup in the end will be 1 real node per miner (they'll run own node for reliability) + all PCs in their network will run "leech" nodes to sync

so pools will still be possible, but miners will have to run nodes

your test was with a cold disk cache, so it's not a fair comparison to a RAMdisk setup

keep in mind that a RAMdisk setup requires application to manage shuttling cold/hot data in and out of RAMdisk

I'm not even sure it would kill botnets

who runs their node from a RAM disk?

whereas LMDB just needs to access blocks and let FS cache handle things

the amount of data being sent every 2 minutes is not that big, botnet can handle it with clever data distribution

my point is, LMDB will always be more efficient than any other solution

except when most of the database file is dead weight

on an active node you can assume most of the interior of the Btree is cached in RAM

therefore most data seeks will only incur 1 IOP

I measured an average "read amplification" factor of 8 with a standard pruned db file

that sounds like Linux default readahead

e.g. 2 GB actually read from the disk to read 256 MB of blob data

it always reads 64K for any 4K access

perhaps, but with a defragmented db, the amplification factor went down to ~2

so I'm assuming you read a page from the disk and most of the data in it is from other tables

except that we don't interleave data from separate tables onto a single page

logical page maybe, but the actual hardware page may be larger

and txn blobs will always be much smaller than 4K anyway

large-sector devices still only use 4KB per sector instead of 512B

SSDs nowadays use 8 or 16 KB pages

that's a pretty strange result

I don't think the OS cares, it will access VM-page sized blocks

SSDs cannot read a smaller unit than a page

and they typically cannot erase a smaller unit than 128 pages

that's not true. they can't *erase* a smaller unit than a page

they erase blocks, which are groups of pages

will have to hunt down a data sheet later

anyway, the read amplification you measured only makes sense if you used a cold cache, so internal btree pages needed to be read

the whole measurement is for about 500 000 reads (average blob size ~500 bytes)

and then used a warm cache for the "defragmented" DB

no, the defragmented db was also cold

if you know your SSD page size is 16KB you might try rerunning a test with LMDB pagesize set to 16K

I guess that would require rebuilding the db file

yes. mdb_dump / mdb_load will do it

of course, that will also defragment the data

it's true that the data is interleaved since it's written on the fly as blocks or txs arrive. but there is no interleaving within pages. so the fragmentation should only result in random seeks, nothing more

it shouldn't result in excess reads per request

So my OS page size is 4K, but the SSD page size is 16K. That explains the read amplification.

that's kinda bad. does x86 even support 16K VM pagesize?

HDDs will still be fine with 4K. I think even SMR drives are still 4K

what model SSD was that?