07:04:55 Unless you're explicitely enabling it, syncing historical blocks will not check PoW. 07:06:07 only up to a certain block height, then it checks everything 11:36:51 I am starting work on a view-only wallet scanning service in preparation for Seraphis. Before spending significant time/money on the effort, I want to ask for critiques on the project and the viability of this approach to it.... (full message at https://libera.ems.host/_matrix/media/r0/download/libera.chat/af371c6a1a0caf11e31e4b4eee41b8600b3ce89f) 11:52:09 LAMP stack? nginx + php-fpm is much more efficient. Also, server scanning collects too much data about real transaction inputs, you need to think how to ensure it's not saved anywhere in the default config. Diskless server support maybe? 11:53:00 *abour real spent outputs 12:01:41 I will start looking into diskless servers and nginx. I definitely want to do whatever makes sense, especially from an information security perspective. 12:01:41 I mention LAMP because I've used it in the past, but I'm happy to shift to something else to improve the project. One of the benefits of starting this now is that it gives me time to learn. 12:18:27 Since your goal is to dilute the market of wallet scanners, I think you LAMP stack choice is fine assuming you don't run into any performance issues. 12:18:27 You might also consider docker support for easy setup. If hitting your performance goals proves difficult, Docker will also give you the flexibility to adopt a more complex build process without hurting your developer-experience too much. 16:54:23 Sounds like a plan. I'll get to reading, and I'll throw the initial setup online using nginx and docker. 16:54:23 The worthiness of the general concept is my primary concern, and I'm encouraged to see that there haven't been any immediate objections to the general plan. If anyone would take a dramatically different approach to the problem (or has concerns about what I've outlined) I am all ears. 17:48:00 hyc: do you have a nice way to ask lmdb how much data is written per commit (I assume abort writes nothing or close to) ? 17:51:12 Also, if you have contemporaneous transactions, and one writes while the other needs to keep a view of the old data, does this result in a lot more writes ? I'd assume not as just the "branches" need to be duplicated, not the leaves, but I'd like confirmation. 18:13:02 there's no hooks to track writes per commit, no 18:13:49 and no, other transactions have no bearing on the volume of writes 18:14:46 OK. I'm asking because of https://github.com/monero-project/monero/issues/8189 18:14:53 It does seem like a hell of a lot. 18:15:23 Just keeping up, I'd expect a fair amount of reads, but substantially fewer writes. 18:15:24 yeah, saw that. remember that by default now we're doing an mdb_env_sync once per second 18:15:41 because of windows unreliability... 18:16:03 so this isn't giving the OS as much opportunity to do lazy writeback 18:16:41 Would this account for 400 GB in 120 hours ? A sync just writes stuff it'd have written anyway (except if it was then changed), no ? 18:16:44 ordinarily, in fully async mode, you would expect a percentage of dirty written pages to be reused in closely following write txns 18:17:00 If keeping up, I would expect write+deleted or write+changed to be not much. 18:17:23 so the total volume that needs to be flushed over a long period of time would be smaller than the total volume of writes per commit, due to the same dirty pages being used more than once 18:17:52 but since we do frequent syncs, that page reuse doesn't really happen 18:18:24 OK. Do you have a setting suggestion for --db-mode to see if that is the reason for the high usage ? 18:18:41 I usually suggest fast:async:1000000 18:19:01 but it also won't make much difference if there's not enough RAM to cache more dirty pages 18:19:08 OK, I'll post that. Thanks. 18:19:49 if they can run with `/usr/bin/time -v` that will give full stats from getrusage() 18:20:13 you'd be looking for the pagefault counts to decrease with a looser sync-mode 18:20:55 on my phone with 4GB RAM it made basically no difference tho 18:30:28 your reply has a typo on last line. should be --db-sync-mode not --db-mode 18:31:04 ty, fixed