-
gingeropolous
it got sigkill. wtf
-
gingeropolous
so i had 52 lines of stuff like this: [Thread 0x7ffff4dafa00 (LWP 104520) exited]
-
gingeropolous
Program terminated with signal SIGKILL, Killed.
-
gingeropolous
The program no longer exists.
-
gingeropolous
did it get OOM killed?
-
gingeropolous
ima try running it with --max-concurrency 4
-
gingeropolous
it *is* a 32 thread cpu with only 16 gb of ram
-
upp
im kind of in a perpetual lifelong state of shock at the programming convention of just pretending that there is infinite memory, and if your program exceeds what memory the system *actually* has, then this is totally fine, we should just let the computer crash and catch fire and force the user to pull the plug and start over, and all of that is totally fine too
-
upp
WTF kind of convention is that?
-
upp
computers have existed for how many decades now and we are still doing that
-
gingeropolous
its v0.17.3.0_
-
gingeropolous
compiled on ubuntu 20
-
gingeropolous
wtf. "the program no longer exists" again
-
gingeropolous
this time there were only 23 threads
-
upp
my chain was a few years old and i synced fine recently, i guess you have to start at 0 to get this crash?
-
gingeropolous
dude i got daemons running all over the place. this is the first time i've seen this
-
gingeropolous
i mean, i've got xmrig mining monero on all threads at the same time
-
gingeropolous
but this isn't any different than any other setup
-
gingeropolous
ugh, i know there's a way to log memory usage instead of just starting at top
-
gingeropolous
well there's always a bash loop with free -h....
-
gingeropolous
oh i guess i'll google it
-
upp
i think you can track a specific process with top?
-
gingeropolous
yeah but im trying to dump it to a log file so i can crockpot it
-
gingeropolous
i got it
-
Yin-Dawg
what was the issue?
-
gingeropolous
i mean i figured out my log situation to watch the memory :)_
-
gingeropolous
my guess is there's a memory leak somewhere. if i can get this to reproduce quickly ima try a git bisect
-
gingeropolous
because what are SSDs there for but to shred
-
Yin-Dawg
LOLs
-
Mochi102
SShreD
-
selsta
gingeropolous: right, sigkill seems like the OS is killing the process and not a crash
-
selsta
check with htop if you have enough resources, but currently there is no known memory leak in v0.17.3.0
-
gingeropolous
yeah. so according to my bash loop of ps, monerod got up to 50% ram utilization and was then killed
-
gingeropolous
im trying it on a different box
-
gingeropolous
with the same amount of ram
-
gingeropolous
alright, 0.17.2.3 up to 49.4%
-
gingeropolous
58.6 ....
-
gingeropolous
i mean im running log level 3. i guess i could test if high logging is responsible but ...
-
gingeropolous
and killed
-
gingeropolous
on v0.17.2.3 . wtf
-
gingeropolous
or i guess i stop the mining process. yeah lets do that
-
hyc
how much RAM is in that box? did you allocate 2GB of hugepages for the miner?
-
selsta
I currently run 5 monero daemons + tor and i2p with 6gb ram usage
-
hyc
hugepages are a separate memory allocation. they're not pageable, they're not interchangeable with other memory.
-
hyc
so allocating them is like removing 2GB from the machine, they're unavailable for any other use.
-
gingeropolous
nope, wasn't the mining either
-
gingeropolous
16 gb in the box
-
gingeropolous
i honestly just think its been a while since ive tried a fresh sync
-
gingeropolous
ok, got killed on another box with 16 gb ram
-
gingeropolous
ubuntu 20
-
gingeropolous
huh, shot up to 76.4% usage before it got killed. perhaps i need to log memory usage more than every 10 seconds
-
gingeropolous
or maybe because i usually run with systemd the memory gets managed better? but then again, thats always with an old chain, never a fresh sync
-
gingeropolous
"your node is 7.9 years behind" ..... oh time
-
rbrunner
Feeling old already? :)
-
moneromooo
You can try applying wfaressuissia's serialization patch. It cuts down a lot of memory usage.
-
gingeropolous
i mean, im generally not affected. but noobs doing fresh syncs on your average computer might be encountering this
-
gingeropolous
wow. so it seems it was the log level
-
hyc
maybe the logger itself has a memleak
-
gingeropolous
log level 3 and 4 cause ever increasing mem usage
-
hyc
would be nice if we could maybe do some sort of annual rollup txns so we would no longer need the preceding year's worth of blocks
-
gingeropolous
they were in screen sessions. i wonder if it happens with it being daemonized sans screen
-
hyc
no idea how that would work since we still need th txos
-
hyc
if your screen session has a ridiculously large scrollback history setting, that could've been it
-
hyc
but then you should've seen the memory being attributed to screen, not to monerod
-
gingeropolous
its default scrollback. didn't know i could mod that :)
-
hyc
you can mod everything
-
hyc
ctrl-A : scrollback 10000
-
gingeropolous
lulz. the one box is on wifi and the data stream is screwing up other wifi things in the house
-
hyc
I always switch away to an idle shell
-
hyc
so the terminal contents doesn't spew across the network until \I want to take a look
-
gingeropolous
gotta close the windows so you don't let the wifi out
-
hyc
i only use organic free range wifi
-
rbrunner
"organic free range wifi" ?
-
moneromooo
Organic: does not leach harmful chemicals in the air where you can breathe it in; free: as in software; range: because wifi if you're within half a meter from your router sucks.
-
moneromooo
Maybe you're missing "free range", it means "allowed to roam" for poultry. I guess it might be a UK thing only.
-
rbrunner
I am mostly puzzled about that "organic" part.
-
moneromooo
Following a set of rules that basically say "don't be a jerk to the environment".
-
» moneromooo might have missed a joke here
-
rbrunner
Ah, I think I finally got the joke. It's about "don't let the wifi out" from ginger
-
rbrunner
hyc does not have a problem there, this wifi is indeed always roaming free, and even organic on top.
-
ndorf
free range is a thing in the US, too
-
moneromooo
With a name like that, I'd expect something to do with long rifles in the US ^_^
-
ndorf
haha, that too!
-
gingeropolous
ok, well the wifi one got to 51% with default logs
-
gingeropolous
and once i stopped sync my music works again.