.merges

what are we thinking about these others

7760

8426?

selsta: about your question yesterday: yeah you’re correct, OpenSSL 1.1 has a lot of the features we need for forwards compatibility without deprecating the older stuff

<spirobel[m]> "can you post a link to the repo?" <- spirobel: Here is the repo: github.com/luigi1111/xmr.llcoins.net/blob/master/site/js/crypto.js

It seems it does not implement the `_free` function: `function _free() { }`

.merge+ 8426

Added

luigi1112: i would say let's include it

it's been a year now and it's more well tested than any other patch and it significantly improves node connectivity

<jeffro256[m]> "selsta: about your question..." <- The "without deprecating the older stuff" part gives me hope, that I can continue running nodes on all 3 networks 🙂

gingeropolous: master-beta contains other patches but it's included in there

<TrasherDK[m]> "I'm curious. How does this fit..." <- Please see my link above. It seems the crypto.js does not implement the `_free` function.

Hmm. `function _free() {}`

Latest commit monero-gui:... (full message at libera.ems.host/_matrix/media/r0/do…3ae38d8f03e552b24f4011a54a43b624223)

* Latest commit monero-gui:... (full message at libera.ems.host/_matrix/media/r0/do…d0d72e00fbb95b6e1040342820433e99e10)

BigmenPixel[m]: yes, it was merged accidentally

it will be fixed tomorrow once we update the submodule l

or later today

i added it to the merge queue because i didn't want to forget it in the release but forgot to write a do not merge for now comment

<selsta> "BigmenPixel: yes, it was..." <- Okay, thank you.

the patch is correct but requires us to update the monero submodule to v0.18

now that 7760 has been decided can we tag/branch today?

yds

yes*

once luigi wakes up and has time

nothing left to decide at this point

ledger users will have to continue using the old version due to their useless version lock, so that's something we have to warn about

<selsta> "once luigi wakes up and has time" <- Is luigi the author of github.com/luigi1111/xmr.llcoins.net/blob/master/site/js/crypto.js?

he maintains the site but no idea if he wrote he code

the

luigi1111: luigi1112 Can you take a look at the github.com/luigi1111/xmr.llcoins.net/blob/master/site/js/crypto.js? It seems it does not implement the _free function: `function _free() { }`, which causes a bug:... (full message at libera.ems.host/_matrix/media/r0/do…31f1bb59786749044f1a598df2bf17fc5c9)

<selsta> "nothing left to decide at this..." <- whats the status of the rest of anon/ooo

* of anon/ooo's ccs work?

what status? there is a second rewrite patch that's not reviewed yet, but i have it in the master-beta branch as it works well

was all the work completed/PRd? or were 7760/7999 the final bits?

.merges

"was all the work completed/PRd?" Well, how would we know?

I guess these 4 need to be merged, then branch and tag?

there was way more than 3 months of work done

sech1: yes

as soon as luigi wakes up

I have a Mac VPS now you can SSH into to debug the RandomX issue. Though at this point it's unclear if it's an OS bug or not.

still trying to get everything setup.

selsta: i see only 1st milestone was completed so wondering how much work is left on the p2p layer and if it will even happen given the recent issues dealing with ooo

I'm not sure what to debug on Mac there, it's clearly a CPU cache invalidation not working

any way to confirm that? wouldn't this also cause issues with other programs? so far everything works fine apart from randomx / monero

i assume something like a browser would also show issues

r4v3r23[m]: there is still a lot of daemon code left that needs work

RandomX tests sometimes passing and sometimes not passing is a clear indicator

Always call sys_icache_invalidate(_:_:) before you execute the machine instructions on a recently updated memory page. On Apple silicon, the instruction caches aren’t coherent with data caches, and unexpected results may occur if you execute instructions without invalidating the caches. It’s also safe to call the sys_icache_invalidate(_:_:) function on Intel-based Mac computers, where the function does nothing.

I assume we do that already? Otherwise it wouldn't have only showed up after the OS update.

I'm just surprised that I find nothing about this on Google and this sounds like a large issue.

probably because RandomX is the only program that overwrites code pages rapidly

There is a second issue that wallet sync crashes after a while, but I assume that wouldn't be related to cache invalidation?

I don't think there is any JIT code in wallet sync code

yep

trying running monero tests now with MONERO_RANDOMX_UMASK=8, let's see

"sys_icache_invalidate" xmrig calls it, and __builtin___clear_cache should also translate to it

right, if that wasn't getting called it would already be failing on older MacOS

does anyone know of any websites that make heavy use of WASM? hard to imagine they hadn't tested their browser against that, but you never know

the other possibility is they've introduced some other syscall that must be used in advance to prepare for JIT use

willshu[m]: yeah I didn't write that part, patches welcome

hyc: developer.apple.com/documentation/a…-in-time-compilers-to-apple-silicon

nothing here

I noticed we don't add MAP_JIT to all mmap calls but that also didn't help

yeah, the other mmap calls are only used for data, not to hold instructions

ok, so the apple docs haven't changed at all

hyc: so interestingly starting the intel version with rosetta works fine

both monero wallet sync and xmrig benchmark

ok, false alarm. monero wallet sync still crashes, but randomx works fine

I wonder if freeBSD behaves the same way. macos and tcp/ip support seems really flaky

hyc: now this makes it even weirder, the xmrig arm binary also works fine with jit

hm, then that ought to mean we can find something xmrig did differently

but i tried xmrig self compiled and it crashed

but afaics the only thing was the explicit sys_icache_invalidate

oh

then a compiler flag is missing

just like what sech1 pointed out SChernykh/p2pool #189

but for some other rev, not cortex-a53

but wouldn't that compiler flag not also be missing on their release bins? they use the same cmake file

hmmm

if same flags, then different compiler version?

it's a beta version of xcode so could be compiler bug

can that cause a different hash on every run?

still can't imagine how this work

compiler bugs can break anything, who knows

so for our release binaries it shouldn't matter since we use an older sdk

can you try building xmrig with the old sdk?

or randomx

just to run its tests

yes

all pass, at least with intel version of rosetta

apart from [90] Cache initialization: AVX2 ... SKIPPED but that's expected as rosetta doesn't support AVX2

i can try compiling clang myself and see if i can reproduce the issue

arm version of randomx-tests also pass so definitely compiler error

ok, that's good to know

hmmm. can you try just compiling to asm source and comparing the output of both compiler versions?

for which file?

good question. I have no idea where a compiler bug could cause this exact problem

I guess jit_compiler_a64.cpp to start

trying to compile clang13 first, which i know doesn't have this issue

clang master did crash

which version is apple using in macos 13?

clang14

might be worth opening a bug report on that even tho we haven't nailed it down. maybe someone else has already hit similar

if it doesn't show up with clang13 i can bisect the issue

cool

also crashes with clang13 hmmmmmm

doesn't make sense

maybe it's random, try to run the version that passed a few more times

i even managed to do randomx-benchmark for 1M hashes

and it was green?

yes

then it definitely works

i meant xmrig benchmark

actual crash, or just wrong result?

wrong result

so same binary worked once, failed once?

it does pass the sdk with -isysroot paste.debian.net/hidden/5f2d6d5d

no, self compiled always fails, compiled on macOS 12 seems to work

compiled with clang13 on macos12? then that would imply some broken header file in SDK

I should be able to download an older SDK

lol specifying the old SDK solves it

macos12 sdk works?

11.3 was the latest here github.com/phracker/MacOSX-SDKs/releases/tag/11.3

ah

12 works too, 13 fails

so, no compiler bug, just an apple bug

might try comparing nm output between the two binaries, see if all library calls are the same

U ___clear_cache

is missing on the broken binary

oh wait but it does have _sys_icache_invalidate instead hmm

wait, have to undo my changes to the repo

nm output looks identical, apart from different addresses sometimes

might be best to ignore this for now and see if it's fixed in the next beta

well, if you can send feedback on the beta, might be good to draw attention to it

if nm output is basically identical, can't imagine what actually broke. would have to be a #define or a template or somesuch

I'll still try to find the exact SDK that first introduced the issue

.merges

luigi1111: i think we can branch and tag

k

just these 4?

LFG

I wonder if the gitian cache really needs to differentiate releases

it's rebuilding all dependencies now, for monero-linux-0.18 instead of using what was built for monero-linux-0.17

seems unnecessary

* depends cache

Hmm, all merged? If I build right now from latest master, will the binaries be exactly the same as after branch and tag (assuming no more commits to the branch)?

I think they should be, not sure

I'm building master now anyway

branch should have same commit hash so would expect to be identical

Commitd IDs should be the same

yeah

but the version string won't be identical

so the hashes also won't match

oh, right

master vs v0.18-release

hmm, then I'll tag v0.18 in my cloned repo and build :D

using the xcode 14 SDK on an older macOS version doesn't seem to cause any JIT related issues, so it's seem to be a combination of OS and SDK bug

that's bizarre

writing a bug report now but I'm doubtful they will look at it

in randomx source, comment out the __isOSVersionAtleast() function

since you're building natively, it will use the platform SDK's copy instead

the __builtin_available() check got a lot more complicated on newer SDKs. which is kind of the opposite of what you'd want, but so it goes

same with it commented out

ok. so it's definitely them and not us ;)

branching?

yesy

what was the MacOS SDK url for gitian builds?

not necessary anymore

nice

I'm already building from my cloned repo with the tag :D

not sure it will be the same binaries

branch is up

can tag now or soon

ah, my build will have version string v0.18.0.0-master so no, not the same

waiting for the real tag then

when you checkout a tag it only writes the tag, not the branch

so should be fine

ok

luigi1112: let's wait for tests to finish

:thumbs_up:

my current build on master paste.debian.net/1247318

I'll stop this and switch to the branch

without tag it will still be the wrong hash

if you create the tag yourself it should be good

good point

2022-07-15 17:24:24.996 I Monero 'Fluorine Fermi' (v0.18.0.0-release)

running on my server now

this is from my repo with tag

still building other binaries

ok restarted build with tag

selsta luigi1112 tests finished

ok let's tag

I'm already finishing the build, will publish hashes soon. But those are preliminary, I'll start a new build with the official tag later

this is what I have so far paste.debian.net/1247325

Mac still building. using my own tag

you have wrong file names

monero-aarch64-linux-gnu-b6a029f22.tar.bz2 should be monero-aarch64-linux-gnu-v0.18.0.0.tar.bz2

and hashes don't match what I have so far

doh

ok then I'll start over

I tagged in my forked repo and building with the tag from the same commit ID, so my hashes should in theory be correct

my hashes so far: paste.debian.net/hidden/7b6c43d2

ah, I needed to create an annotated tag, but I only created a plain tag

my hashes: paste.debian.net/hidden/2c2ada14

waiting for the official tag now

ok I fixed my tag and now my hashes match yours sech1

are you available luigi1111 to do the tag?

also why do 3 luigis exist suddenly

what's the name of this thing

v0.18.0.0 I think

Fluorine Fermi

wait, let my copy paste it

I already pushed

Fluorine Fermi

feel free to correct and we can do a classic force push

oh indded

yes you have a typo

sorry what

what would we do without force push

Typo where?

I already started a build

I started a new build too, but since the tag name hasn't changed, should just get the same result again

yeah, my linux builds just finished, same hashes as before

I just realized I deleted all build folders to make a new build so I'll need to wait until the build is finished to get my gitian.sigs :D

yep, getting the same hashes too

when is the ring size increase to 16?

in about 1 month

<selsta> "in about 1 month" <- thank you. I was trying to find information but only found an article stating that it was planned for tomorrow

we pushed back the date

I understand. thank you. I was even looking in the code and see ring size is 16 in some files

build finished, hashes were the same: paste.debian.net/hidden/2c2ada14

it seems that libboost_atomic-mt is a dependency of libboost_filesystem-mt. does that mean monero should also link against libboost_atomic-mt?