afungible[m]: we looked into the issue a while ago, it's unclear if this is a bug in RandomX code or not, hyc and sech1 think it's a bug in macOS SDK

it looks like the cache doesn't get invalidated but we do call `sys_icache_invalidate` as explained here: developer.apple.com/documentation/a…-in-time-compilers-to-apple-silicon

it worked fine before macOS 13

there's also no segfault at a specific place, since it's cache / JIT related the crash always happens at a different place

I've begun to think it never crashed with the old SDK because building with the OLD SDK just always disables the new API feature

so even on newer Macs it always uses the old code

<sech1> "KDF must be resistant to..." <- Right, hence the “multiple rounds” I mentioned. It seems like it would be slightly easier to screw up key derivation when mishmashing a bunch of hash functions together instead of applying one known very secure fast hash function over and over again, which is quite a common practice.

<selsta> "there's also no segfault at a..." <- Thanks for the explanation. It could be that it uses the same internal API call at different points in the code, reason it crashes at different places points. Then we'd want to find which API that might be. Just a guess.

hyc selsta I just realized the current RandomX code doesn't follow Apple's docs strictly: developer.apple.com/documentation/a…-in-time-compilers-to-apple-silicon - sys_icache_invalidate() must be called _after_ pthread_jit_write_protect_np and we do it _before_

hyc can you check if it fixes the issue?

by the way, XMRig does it in the correct order: github.com/xmrig/xmrig/blob/master/…/common/VirtualMemory_unix.cpp#L113

wait, RandomX master branch doesn't even call sys_icache_invalidate() anywhere, lol

@hyc tevador/RandomX #259

I'll try it

sech1: we use the gcc __builtin___clear_cache() which calls sys_icache_invalidate() on MacOS

same in clang codebrowser.dev/llvm/compiler-rt/lib/builtins/clear_cache.c.html

yes, but it was called in the wrong place

I have the same issue with p2pool, let's see if he replies: SChernykh/p2pool #217

sech1: xmrig master still crashes

I don't recall ever reproducing the problem locally. randomx-benchmark works here on master

damn apple

so it's not only the correct order

hyc: you have to update to macOS 13

ah. I'm on 12.0.1

in any case, that sys_icache_invalidate() must be there according to Apple's own docs

so this PR makes sense

we can put all blame on Apple after this :P

I have collected these miscellaneous functions in my seraphis library and don't know where to put them: github.com/UkoeHB/monero/blob/serap…src/seraphis_crypto/sp_misc_utils.h . Any suggestions? Just leave it where it is? It can't go in `src/seraphis` because the dependency goes `misc utils -> src/seraphis_crypto -> src/seraphis`.

Also, I'm planning to start PR-ing stuff out of my seraphis lib to trim it down. The PRs will build off each other but won't impact any production code, so the PR stuff will look 'dead' until seraphis is fully online (except some multisig stuff could be used to improve existing multisig - mainly key image proofs). So now is the time to decide whether to go all-in on seraphis and allow changes to master.

For a preview, my initial PRs are looking to be: blake2b, x25519, twofish, crypto/generators.*, sp_transcript, sp_hash_functions, sp_generator_factory, sp_crypto_utils, sp_misc_utils, dual base vector proof (for multisig stuff), a bunch of multisig utilities (these will take ages to get reviewed, best to start asap)

Basically if no one stops me, I will start making PRs lol

Make those before someone can stop you :)

lol gotcha

UkoeHB: Where will your pull requests be to look at?

on the main repo?

Thanks

selsta: actions/runner-images #6002