-
moneromooo
Unless you're explicitely enabling it, syncing historical blocks will not check PoW.
-
sech1
only up to a certain block height, then it checks everything
-
spackle_xmr[m]
I am starting work on a view-only wallet scanning service in preparation for Seraphis. Before spending significant time/money on the effort, I want to ask for critiques on the project and the viability of this approach to it.... (full message at
libera.ems.host/_matrix/media/r0/do…c6a1a0caf11e31e4b4eee41b8600b3ce89f)
-
sech1
LAMP stack? nginx + php-fpm is much more efficient. Also, server scanning collects too much data about real transaction inputs, you need to think how to ensure it's not saved anywhere in the default config. Diskless server support maybe?
-
sech1
*abour real spent outputs
-
spackle_xmr[m]
I will start looking into diskless servers and nginx. I definitely want to do whatever makes sense, especially from an information security perspective.
-
spackle_xmr[m]
I mention LAMP because I've used it in the past, but I'm happy to shift to something else to improve the project. One of the benefits of starting this now is that it gives me time to learn.
-
BusyBoredom[m]
Since your goal is to dilute the market of wallet scanners, I think you LAMP stack choice is fine assuming you don't run into any performance issues.
-
BusyBoredom[m]
You might also consider docker support for easy setup. If hitting your performance goals proves difficult, Docker will also give you the flexibility to adopt a more complex build process without hurting your developer-experience too much.
-
spackle_xmr[m]
Sounds like a plan. I'll get to reading, and I'll throw the initial setup online using nginx and docker.
-
spackle_xmr[m]
The worthiness of the general concept is my primary concern, and I'm encouraged to see that there haven't been any immediate objections to the general plan. If anyone would take a dramatically different approach to the problem (or has concerns about what I've outlined) I am all ears.
-
moneromooo
hyc: do you have a nice way to ask lmdb how much data is written per commit (I assume abort writes nothing or close to) ?
-
moneromooo
Also, if you have contemporaneous transactions, and one writes while the other needs to keep a view of the old data, does this result in a lot more writes ? I'd assume not as just the "branches" need to be duplicated, not the leaves, but I'd like confirmation.
-
hyc
there's no hooks to track writes per commit, no
-
hyc
and no, other transactions have no bearing on the volume of writes
-
moneromooo
-
moneromooo
It does seem like a hell of a lot.
-
moneromooo
Just keeping up, I'd expect a fair amount of reads, but substantially fewer writes.
-
hyc
yeah, saw that. remember that by default now we're doing an mdb_env_sync once per second
-
hyc
because of windows unreliability...
-
hyc
so this isn't giving the OS as much opportunity to do lazy writeback
-
moneromooo
Would this account for 400 GB in 120 hours ? A sync just writes stuff it'd have written anyway (except if it was then changed), no ?
-
hyc
ordinarily, in fully async mode, you would expect a percentage of dirty written pages to be reused in closely following write txns
-
moneromooo
If keeping up, I would expect write+deleted or write+changed to be not much.
-
hyc
so the total volume that needs to be flushed over a long period of time would be smaller than the total volume of writes per commit, due to the same dirty pages being used more than once
-
hyc
but since we do frequent syncs, that page reuse doesn't really happen
-
moneromooo
OK. Do you have a setting suggestion for --db-mode to see if that is the reason for the high usage ?
-
hyc
I usually suggest fast:async:1000000
-
hyc
but it also won't make much difference if there's not enough RAM to cache more dirty pages
-
moneromooo
OK, I'll post that. Thanks.
-
hyc
if they can run with `/usr/bin/time -v` that will give full stats from getrusage()
-
hyc
you'd be looking for the pagefault counts to decrease with a looser sync-mode
-
hyc
on my phone with 4GB RAM it made basically no difference tho
-
hyc
your reply has a typo on last line. should be --db-sync-mode not --db-mode
-
moneromooo
ty, fixed