-
spectrehosting[m
Won't clog this channel with the actual post, but if anyone's interested in getting paid in Monero to work on a privacy-focused cloud computing startup check out #monero-community-dev:monero.social
-
someoneelse49549
Could anyone tell me their though about replacing LMDB by HSE (
github.com/hse-project) to store the blockchain in the monero node. (performance speaking, do not talk about effort i don't need to sleep).
-
selsta
I doubt anyone will be able to guess performance differences but I don't think there's much interest in replacing LMDB.
-
xfedex[m]
LMDB is extremely slow especially for hard drives. I would love to see Monero switch to a faster database such as LevelDB. This is a good blog post by Mozilla which compares the two databases:
-
xfedex[m]
-
xfedex[m]
According to
github.com/mykmelez/kvbench, LevelDB is much faster than LMDB for reading and writing, and it uses less disk space; the disadvantage is that opening the database is much slower for LevelDB
-
UkoeHB
"LevelDB is widely noted for being unreliable and databases it manages are prone to corruption." -> from your mozilla link
-
monerobull[m]1
lol
-
selsta
xfedex[m]:
github.com/laanwj/bitcoin/tree/2016_04_mdb bitcoin uses leveldb and an experimental lmdb implementation showed over 40% speedup in sync
-
selsta
and optimizing for HDD instead of SSD in 2023 is exactly the opposite of what we should be doing lol
-
xfedex[m]
It's kinda annoying that monerod syncs 0.3 blocks per second on an HDD thought
-
xmrack[m]
Buy an ssd. They are < 100
-
moneromooo
Wow. That's very rare! Only 100 people can ever get a whole SSD!
-
plowsof11
lol
-
DataHoarder
Just fit the blockchain in RAM, it's not that big! Then push it to HDD at the end
-
Rucknium[m]
RavFX is doing some interesting experiments with fast syncing to HDD with SSD acting as a cache:
m-h-o-l.org/index.php?article=hdd_sync
-
sech1
IIRC some aggressive OS-level caching settings can help with syncing on HDD
-
ofrnxmr[m]
-
ofrnxmr[m]
ArticMine @ArticMine:libera.chat: hyc:
-
hyc
don't have my tor browser handy at the moment
-
hyc
but the usual stuff I would do: run monerod async, no fsyncs at all
-
hyc
set /proc/sys/vm cache params: writeback 10 minutes
-
hyc
dirty_ratio 60% or more (I usually use 90%)
-
hyc
dirty_background_ratio 60%
-
hyc
that will keep most writes in RAM, and OS will flush whenever things get full
-
RavFX[m]
Yeah, if you have a lot of ram ( >=32GB )
-
RavFX[m]
adjusting the vm cache work
-
RavFX[m]
Except that this cache is gone after a reboot.
-
RavFX[m]
It's fine if you sync one shot then leave monerod running full time (at least, for now)
-
hyc
same strategy isn't available on Windows, the cache tuning params onlu existed in win2k. they were still present in winXP but no-opd
-
hyc
also not viable for windows simply because the OS isn't that reliable, and a crash will leave a corrupted file
-
RavFX[m]
I did have corrupted monero lmdb on Linux after a crash, two times so far. Probably file system & vm cache settings dependent.
-
ofrnxmr[m]
hyc: this is the clearnet link
-
hyc
yes, need a FS that guarantees ordered writes, like ext3 / ext4
-
hyc
this is already noted in the LMDB docs. all known since LMDB was created in 2011.
-
RavFX[m]
Yep.
-
RavFX[m]
I have XFS everywhere here. Not testing new CPU/RAM overclocking when monerod is running
-
hyc
ofrnxmr[m]: where?
-
ofrnxmr[m]
-
ofrnxmr[m]
Sorry. Forgot the link 😅
-
hyc
heh
-
hyc
using a persistent cache is great, yeah
-
hyc
Optane was quite promising on those lines
-
hyc
too bad Intel axed it. the write endurance never lived up to claims.
-
hyc
You should already have read the benchmarks across different filesystems
symas.com/symas-lmdb-tech-info
-
hyc
xfs doesn't do so great
-
RavFX[m]
pas *
-
RavFX[m]
Wrong keyboard...