to be clear, my "yep" was to kayaba's explanation. every view tag is independent and is constructed on a per-output basis. even if 2 outputs have the same view tag, it doesn't yield any useful about the outputs, it's just random chance. a single wallet can create a view tag of any value between 0 and 255 for each output when constructing a tx (each view tag in the tx is independent), and a wallet still has to check every single view tag

when scanning

Hi there! Does anyone know if there is any software that can scan every new block and tell me if there is any output (with amount) going to a list of subaddresses I provide (along with the primary address and view key related to them)?

Reading the docs, I see it can be done with wallet-cli (to create wallet file restoring from primary address and view key) and wallet-rpc (to open and close wallet and get txs). But as I want to automate this, I would need to have one wallet-rpc running for each primary address, and query all of them every minute. It don't think it will work with 1k addresses

Perhaps monero-lws

I havent played with it extensively, but it does scan multiple address and view key pairs.

But its meant to be used with wallets. I imagine there to be some functionality to parse the data without a wallet

It doesn't support subaddresses yet

Thanks, I'll take a look at it

here's the current proposal to support subaddresses in monero-lws/light wallet servers in general; I think that would enable exactly what you're looking for: monero-project/meta #647

Was there a large testnet reorg in the last few days?

With "large" meaning 1005 blocks or something similarly dramatic?

Or maybe a fork of testnet which one of my daemons chased down for that many blocks?

I synced testnet two days ago and I don't see any reorgs in the log

Hmmm, strange.

My situation: I have two machines where I claim that testnet is running identically, daemon just compiled from master just this morning, and synced to top of testnet without problems

I have a wallet that opens and refreshed without problems on one computer

On the other it balks and tells me it wants to do a reorg 1005 blocks deep which is larger than the allowed maximum

Currently out of ideas where the difference and/or the problem could be :)

I found a little more: Running daemon and monero-wallet-cli without any special options beside "--testnet" works

If I bind the daemon to the static public IP of the machine like so: monerod --testnet --rpc-bind-ip <ip number> --confirm-external-bind

And then instruct the wallet to use that like so: ./monero-wallet-cli --testnet --daemon-address <ip number>:28081

Then the wallet reacts with, probably expected: Warning: using an untrusted daemon at 80.254.179.107:28081

But also gives an unexpected error:

Error: refresh failed: refresh error: reorg exceeds maximum allowed depth, use 'set max-reorg-depth N' to allow it, reorg depth: 520. Blocks received: 0

Possibly there is a reorg, but a smaller one ? When using an untrusted daemon, the wallet quantizes its current height, so will request earlier blocks.

(so a daemon cannot tell a returning wallet from that info)

Not sure I understand what you mean. The wallet is currently synced to almost top of chain, the daemon follows the chain normally

The wallet misses only a few blocks now

Where would a reorg sneak in here?

I always add --trusted-daemon when I connect to my nodes

Hello, is Monero Wallet RPC thread safe? Or should the RPC calls be synchronous?

it should be thread safe

I mean that the wallet will pretend its height, when starting is something like H - H % 1024 or so.

So if it needs to reorg for, say, 5 blocks, it might end up asking for 1005 blocks.

Yeah, but why should that be suddenly a problem?

I think the wallet RPC is thread safe because it handles one connection at a time.

Because there might be a reorg.

But it's single threaded right?

Try "status" in the wallet. It might give you a block hash.

I narrowed down further: The external IP seems to play no role, simply telling the CLI the daemon is untrusted triggers my problem

like so: ./monero-wallet-cli --untrusted-daemon --testnet

Yes, "one connection at a time" implies that, for "interesting" threads.

I tried hitting the wallet rpc with multiple parallel requests and shit went wrong

OK, that's odd. Maybe someone made it handle several calls at once then. Pretty sure there's a comment in hte source saying "don't do that".

Then I decided to make it synchronous which seemed to behave better

Another wallet wants to go back 153 blocks now, and again tells that this is too much.

Stnby[m]: Seems like a lot of rpc binding libs allow this to happen, we should probably place those mutexes in our lib to make it safe

A third one 898.

How could they all hit the same reorg problem?

Did you use them all at about the same time ?

Probably not, no

How probably ?

I check

Within a day or so of each other.

No. I guess that also the reason they all want to go back a different number of blocks.

Because their caches do stand at different heights now

One is almost at top, the other two different numbers of weeks back

The comment's still there about threads. Maybe the underlying code now does a listen/accept/handoff which would be unfortunate. I don't think it's super likely though. So maybe the problem is elsewhere...

Did you run these wallets off a temporarily branched chain (ie, when using --fixed-difficulty etc for testing) ?

No, and they still sync properly, I just have to leave out " --untrusted-daemon" for the CLI wallet ...

Did you keep the old caches ?

Probably could fish them out of some backup.

Then change "status" or other to print the height and hash of the top block they have. Compare with the chain.

If it's on the chain, it looks like a bug. If it's not on the chain, it looks like a reorg.

print_block in monerod remembers old orphans, usually.

I synced now one wallet to top, without " --untrusted-daemon ". Started again with " --untrusted-daemon ", and now it want to reorg 555 deep.

All my alt chains are 1 block long. Very nice...

Then it is almost certainly a bug.

Somebody around with current master compiled who could check running the CLI wallet with " --untrusted-daemon " with that, with any wallet?

I have done a lot of dumb things in the last 5 years of Monero, who knows :)

I confirm p2pool.io/u/4daa3896be2dbbc1/fail.png

Ok, so it's not me.

It works perfectly fine with --trusted-daemon

Can somebody please inform the CEO?

Monero is broken :)

BTW I'm testing p2pool + testnet right now - SChernykh/p2pool #153

If it's only a problem with untrusted nodes, it's not a problem :)

Lol

so I have the wallet and testnet node at hand

But still, I'll try to have a look.

Are you looking at it sech1 ?

Are there any devs in here that have experience with monero-javascript?

Reorg error? No, I have no idea what to look at.

OK, I will then.

Plus I have to go now

I'm testing p2pool during testnet fork

rbrunner: monero-project/monero #8331

(tests not run yet, but I don't expect they'll fail (but then I never do))

Thanks, that was fast. Is this an old error then, nothing to do with recent changes?

Since 2020 I think.

So not too recent, but not too old.

I wonder why I only hit that today. Before I was probably using an early 0.17 version.

Maybe because you usually use your local node ?

You could bisect I guess, maybe something else "hid" this before.

I caused that "untrusted" situation on my Monerotech.info webwallet by using the external IP maybe starting 1 year ago.

Anyway, I will try the fix.

Already fixed? Nice

Works.

Looks like it already broke right after the patch in 2020.

That might mean close to nobody uses untrusted nodes :)

Or, more likely, people mindlessly add --trusted-node for those.

Or just stop using monero.

I admit I do this:P

Stop using monero ? :D

Add --trusted-node. But I run my own nodes

in wownero cli it makes you pass "--this-is-a-malicious-spyware-node" before adding any remote node

Sounds like something I'd add.

UkoeHB: do you have a prefered wording for "multisig is disabled, enable with set x=1" ?

i'm working on a monero node running on kicksecure (kicksecure.com) on top of a debian 11 vps... you guys have a great day

Hey! Want to start contributing to/working on projects in the space. I'm a CS sophomore in India good with C, C++ and Rust. Do tell me where and how I can start!

very possible... i am a web designer, system admin, highly skilled in html/css, bash scripting, and gimp

You can look around at what floats your boat. There's monero, the GUI, mymonero, monero-lws, haveno, and more.

C/C++ means monero I guess.

If you want to hack on monero itself, it's easy: biuld the source, use it, find something that's a bit broken, not good enough, etc, that annoys you, and fix/improve it.

I suggest minimal changes (ie, no reformatting or changes just because you like it better the new way). Separate patches by self contained change if possible.

Nice! Will do that. Any other projects using a C/C++ codebase other than Monero core?

There's a CONTRIBUTING.md file with that kind of guidelines.

The GUI has some C++, but it's mostly boilerplate IIRC. The meat is JS.

monero-lws is also C++.

Yep got it. Hack around and find out and if I get a solution worth implementing at scale with proper security (with no reformatting or major changes), submit a PR?

Right.

Great, that works very well. Thank you. 🙏

Major changes is fine once you're comfortable with the code. It's more like... no unnecessary changes.

Especially the "my editor reformatted the whole file" nightmare :D

Will take some time to get comfortable with the code itself. Still in the learning process, but enjoying every bit due to the community. Probably the first time I've seen non gatekept research unlike universities etc where you require credentials

You can also look at github for issues, see if one of them looks easy and something you'd want to work on.

moneromooo: Haha! Definitely not :D

moneromooo: Don't necessarily need easy. Willing to be a long term contributor and grind on the problem if it has the potential to teach me a lot!

Well, you gotta start with easy bits. I did.

If you don't, you run the risk of losing interest.

Up to you of course.

Oh! Would love to hear your journey if you could elaborate :)

Well... I just wanted to contribute, so I looked at something easy to start with, that's all.

I read code, and I found something that looked buggy, if memory serves.

So I patched it. Hopefully less buggy now.

And as I read code, I got more comfortable with it, what's where, what does what, etc. Allowing me to work on more complex stuff.

Pretty standard really.

i'm learning python now, to create gui front-ends for running bash script... the masses need the gui... make it very pretty in gimp... large scale adoption is the key for monero's future... payment interfaces running as apps on mobile

Anyone managed to statically build monero, with `BUILD_GUI_DEPS` on?

Ah. I used to dream of some generic GUI maker that'd run --help on programs and auto generate a GUI from that.

On mac m1*.

quila[m]: what are you trying to do?

a generic gui maker... interesting to say the least

selsta: I am want to build `libwallet_merged.a`. I managed to do in on Big Sur, but since Monterey I could not do it anymore.

Pipe dream I think. You'd end up with something verbose and not really usable in practice.

quila[m]: did you test release-v0.17 or master branch?

Paste errors to paste.debian.net and post the URL here.

selsta: Yes, I have.

moneromooo: Sure, let me grab the logs.

I mean which branch did you test? both?

Which of the two ?

there were some changes in master branch a while ago

true... a gui for me is python executing script, nice logo, minimal, very clean css interface

I usually just do with the release branch, sorry.

It's fine. That branch ought to build.

I have also tried to change the arch to `apple-latest`/`apple-a14`/`apple-m1`.

Let build it again to share the logs.

selsta: do you run `make release-static` or `release-static-mac-x86_6` to build on M1? (btw I tried both)

Monero builds nicely, only GUI_DEPS on causes the error.

on another tip... i created my first nft, sized for a twitter banner, as the musk-rat bought twitter and only a matter of time before nft format allowed as banner images there... going price: 10 xmr ;) made a screenshot and put up on one of my web pages if anyone wants to see it... very colorful :) alt-web.xyz

<moneromooo> "Paste errors to paste.debian.net..." <- There you go: paste.debian.net/1240950

The error seems to be: clang: error: the clang compiler does not support '-march=x86-64'

If I build it with arch `apple-latest` it can make it to 50% or somthing.

Unlcear where from due to -j10.

Can you try with -j1 so we see where it comes from ?

<moneromooo> "There's a CONTRIBUTING.md file..." <- I guess this: github.com/monero-project/monero-site/blob/master/CONTRIBUTING.md

I don't see anything setting x86-64 specifically for GUI deps.

moneromooo: Sure. Do you want me to change the arch to `apple-latest` also?

Or just same command?

Try same for now.

moneromooo: paste.debian.net/1240953

Hmm. Looking at the Makefile, release-static does assume x86_64.

Doesn't explain why the rest of moneor would build though.

Yes...

I don't use release-static on Mac

Fails in the first file with the "top level" biuld system. You sure the rest of monero built ?

These are the archs I got.

Oh, maybe only with apple-latest then ? If so, try with apple-latest nw please.

apple-m1 seems nice too.

Or you could wait for hyc, who built for m1, I never did so I'm blind here.

You can set ARCH=default

Weird, I don't know what I did overnight but none of those are progressing now.

Same error.

Let me try `apple-a14` and `default`.

You might need to remove CMakeCache.txt from your build tree, cmake is a bit of an ass when it comes to changing settings.

```clang: error: the clang compiler does not support '-march=apple-m1'```

Try default or native then as selsta said.

When you have something working, a new make target for M1 can be added :)

`default` seems to have gone through. 50% now.

moneromooo: it failed at 92% paste.debian.net/1240955

Looks like it just needs extra libs to link against.

How this `default` works behind the scenes? It seems to corresponds to `arm64`.

EVP_MD_size sounds like openssl.

event_* is probably libevent

sech1: you had the libevent problem recently, no ? Fixed it ?

Building it with `native` results in the same error: paste.debian.net/1240960

My libevent to have been built on `arm64`. Same for openssl.

quila[m]: did you install it with homebrew?

selsta: Yes...

Just to clarify, I have managed to build this in the past on Mac x64 Big Sur. It has then failed on mac x64 Monterey and now M1 Monterey.

I even innocently copied the `.dylib` from the old machine to this one but it seems Rosetta don't like it.

You might have to compile static openssl yourself? Not sure.

I do have the openssl static libs:

```/opt/homebrew/opt/openssl@3/lib/libssl.dylib: Mach-O 64-bit dynamically linked shared library arm64```

(static lib is also here, just can't run `file` on it)

try to install openssl⊙1

Let me try that.

and remove @3

Ok, I am building now with openssl1.1.

moneromooo I fixed it by compiling on another machine (I was able to fix libevent undefined symbols, but there was another library that gave linker errors)

nettle library: undefined reference to `nettle_rsa_public_key_init' and so on

<selsta> "and remove @3" <- Same error: paste.debian.net/1240964

sech1: are you on M1?

no, it was an Ubuntu x64 VM

Hummm... It seems I will have to investigate further. If anyone if manages to build statically with `BUILD_GUI_DEPS` ON on M1 chip let me know. I will do the same if I find out.

Not sure if anyone from cake wallet is in the room.

That could help*

and compiling without build_gui_deps works?

selsta: Yes, it does.

That IS really odd since build_gui_deps is just an extra wallet lib. The binaries get build regardless.

quila[m]: how did you enable build_gui_depy?

selsta: I updated the line 1244 of CMakeLists.

can you double check by setting it to OFF on line 1244 to see if it compiles?

Sure, doing it now.

Ok, I get the same error. With `BUILD_GUI_DEPS` OFF. It seems it just `STATIC=ON` related. moneromooo sorry for the misleading info earlier, I probably ran some other command.

Yes, I can totally see that static would do that.

So you just need to find out what static libs link to those libs that aren't linked in.

I ran `debug-static-all` and got same problem: paste.debian.net/1240968

Then detect and add them to the link line, either through the lib that depend on them (ie, if libX wants libY, detect libY and "set X_LIBRARY $X_LIBRARY $Y_LIBRARY" or so) or through EXTRA_LIBRARIES if it's "most programs need that system lib", like librt, libdl etc.

For an example of this, see how libpgm is detected and added to libzmq, since libzmq is sometimes built with pgm support.

That's why pkg-config was made, but cmake NIH obviously.

It seems I don't have `nghttp2` installed on my machine.

You can add it to the deps list in README.md. If there's none for Mac yet, you can add it.

Hopefully it's just that. Do you recognize `_event_base_dispatch` function?

Likely libevent/libevent2/

Or any of these:```undef: _event_base_set... (full message at libera.ems.host/_matrix/media/r0/do…cacfb623643414859e13430380778de573d)

grep in /usr/include (or wherever on mac)

Ok, I can see problems here.

My internet might be off more than on for the next hour or so, good luck

Cool, thanks for the help.

Brew didn't install static libs for `libevent`.

that's why I thought you might have to compile the dependencies yourself

<selsta> "that's why I thought you might..." <- I assumed it was good practice to brew recipes include static and shared libs... But `monero/contrib/brew/Brewfile` is definitely missing `nghttp2`.

I need to come back to this later. Thanks selsta and moneromooo for the help.

I didn't catch everything in the scrollback. what's he trying to build on M1?

release-static

I'll give it a shot. unfortunately the apple version of compilers deviate from gcc/clang normal target names

gcc/clang want aarch64-apple, apple compilers want arm64-apple

that's one of the things I had to account for in #8312

hyc: paste.debian.net/1240964

that were the openssl linking errors

I just did make release-static and got

24%] Building CXX object contrib/epee/src/CMakeFiles/obj_epee.dir/byte_slice.cpp.o

clang: error: the clang compiler does not support '-march=x86-64'

something in CMakeLists.txt is still assuming darwin only has x86-64

hyc: you have to edit ARCH=x86-64" to default in Makefile

make release-static has the ARCH still hardcoded

ah I see. I changed it to native for now

.merges

.merge+ 8315 8308 8307 8266

Added

jberman[m]: did you run unit tests with monero-project/monero #8330 ?

CI has timed out, seems it got stuck somewhere

hyc: something else, jeffro256[m] implemented your comment suggestions, can you take another look? monero-project/monero #8322

ok

I kinda don't see the need for reorganizing sha256 into its own source & header files

that strikes me as gratuitous churn

anyone else have opinions there?

so I just ran into that nghttp2 symbols unresolved etc

A lot of those epee patches seem pointless, but since I don't contribute much anymore, I'm mostly keeping my mouth shut about it.

it's still basic principle of collaborative dev: minimize churn. do the least amount of changes needed to solve the problem, and nothing more

unless you decide up front that you're going to tackle a major refactoring

which afaik nobody here is trying to do

hyc: refactoring just for the refactoring is bad aim too

it leads to a lot of redundancy

this 8322 touches 14 files just to update the sha256 function

that's nuts.

<moneromooo> "A lot of those epee patches seem..." <- like instead of fixing some problems inside, someone decide to polish front door

hahahaha

A nice front door invites people to look inside. Pretty simple concept tbh

so libunbound requires -lnghttp2 and -levent

^ as inviting as a venus flytrap :P

hey at least they are trapped fixing bugs, instead of running away (which a lot of people do when they see uninviting code)

Well, since rewriting stuff to the format you like can involve unwittingly adding bugs... it's a neverending churnfest :)

I suppose differentiating between "I like this better" and "this is really bad code" can be a fuzzy line.

There was a recent patch for example, where some atomicity was mistakenly lost, and the patch's purpose was to have better atomicity.

(though the underlying intent was actually right, using std::atomic so you would not forget to call atomic read/write)

IIRC there were hangers on in the patch too, the "I like it better that way" type. Shrug.

Well, surely there is some balance between code quality improvements and in-place changes. In-place changes lead to accumulated technical debt, which makes it harder to find bugs and easier to create bugs with future changes.

And also harder to evaluate if new changes are buggy or not.

Right. And different people put the line in different places :)

I'm at one extreme, I'll readily admit.

But hey, I was young too. I once reformatted stuff to remove stray tabs in a code base. That's when I really learnt about conflicts ^_^

I mean, *really* learnt.

Like, learnt that they do not come up once and then never again. Unless you never use your VCS's history.

UkoeHB: I don't believe in any kind of code quality. It always about reading code until you understand what is going on and then something can be changed.

How many comments / code quality patches are needed to make code verification / audit easier ?

If you can afford to spend shitloads of time on code, I'd agree. I don't agree in practice though.

Final code verification implies that everything is untrusted and should not be verified outside of context.

ooo123ooo1234[m]: yeah, but why the hell would some want to read code if it looks like shit? Even if the reader is qualified and competent.

Hey yall, sorry I'm late

I saw that yall were reviewing the SHA256 PR

It's just an enormous waste of time to translate shit code into comprehensible logic.

I want to spend all my time thinking about the logic, not tracing out some bs execution flow.

We would have to rewite that SHA256 code anyways because OpenSSL 3.0 doesnt support it anymore but I figured it it was going to be overhauled anyways, might as well make it look clean

I don't see how the particular source file its in affects its cleanness or code quality

What do we use SHA256 for btw ? I know of the update check, anything else ?

Because we have a giant 2K+ file called "util.cpp" that is largely undertested and unorganized

If it is just that, we can just use keccak instead, since it doesn't have to be compatible with anything else.

UkoeHB: any short example ?

Sorry 1.4K

Hmm. Merge mining (can be changed, not too late), trezor (probably hard to change).

It's used also in merge mining I believe

... does anything actively merge mine with Monero right now?

... I'd guess Wownero?

* guess Wownero if I had to pick a project to do so?

ooo123ooo1234[m]: github.com/monero-project/monero/bl…240a4/src/wallet/wallet2.cpp#L13504 . Figuring out if `update_multisig_rescan_info()` introduced a nonce-reuse bug was a waste of time that could have been avoided with a better design (i.e. code quality).

I was going to say nothing. I didn't knopw about wownoero.

That is interesting.

Yeah, me neither

as far as I know they don't merge mine

IMO, Monero shouldn't have any merge mining code. At all. If another project wanted to setup some merge mining system, that should be completely on them.

Does townforge merge mine with Monero is it a seperate chain?

Tari was the one who wanted to merge mine. Townforge also will.

And I don't care to submit a PR removing the TX Extra byte, as it increases extra decode rate to have it, but if we're saying merge mining somehow has compatibility requirements...

Merging that code means any project can reuse it, which means more hash rate going into monero.

In theory anyway.

Isn't the code in question simply a cleanup of the sha function solely used for updating? How does this feed into merge mining anyways?

MM just uses sha256 because it was there and I needed a hash function I think. Not sure why I picked this one and not keccak.

I would have to look deeper into it, but I believe that that the merge mining code uses it it validate the intgerity of the merge mined blockchsind

(hence "can be changed, not too late").

Tari is already proposing proxying the Monero mining RPC for its needs.

Considering nothing actively uses this, it personally sounds like MM should be moved to keccak256 at least, or we should drop code from it and leave Tari to augment their proxy to replace the dropped code.

Just to be perfectly clear, the MM code in Monero. It's solely indexing data and providing RPC methods, right? None of it actually affects node operations?

Right.

IIRC.

It's been a while now...

sha256 is also used to verify the checkpoints.dat file , but that could easily be changed

It was meant as an incentive to other chains to start using MM.

But nobody seemed to care so far.

Probably too busy doing stuff for that ethereum house of cards before it crashes down under its own weight :D

I can see this changing with tail emission

MM with Monero could be seen as an option

Does anyone know the comparison of cryptographic security between keccak and SHA256? Because we only "need" one good cryptographicly secure hash function

keccak is the basis for SHA3

I would say it's superior to all SHA2 algos in every meaningful way

I still prefer Blake2b :( But that's a derailment for another day :p

Was this a SHA3 candidate ?

But yeah, keccak is effectively SHA3. It's just minor padding changes from the submitted contestant to the released version that causes people to still call it keccak.

@hyc that's what I figured since it was a SHA3 finalist , so maybe we could start to transition away from SHA256?

Blake2 and Keccak were both candidates. Keccak had lower performance but was more distinct and provided redundancy in case SHA2 was broken IIRC.

blake2 was also designed specifically to be fast on CPUs

Keccak was chosen and just had the padding changes I don't think affect its security in any meaningful way.

We should drop SHA2 though

So... if openssl drops SHA256, does it mean it doesn't support any suite that uses SHA256 ? Or did it just drop it from its public API ?

Just the suite of APIs we were using

You can still use it thru the EVP API

they totally revamped their hashmech API

<UkoeHB> "ooo123ooo1234: github...." <- A better design already known to a reader. Untrusted code verification will be always as hard as this (en.wikipedia.org/wiki/Boolean_satisfiability_problem). Even if this state is reached at some point, with addition of anything new full verification will be needed again. The only way is to maintain knowledge about code in memory. Only code of dead project is easy to review.

hehe

quila[m]: this makes the static build succeed on M1 Mac

ooo123ooo1234[m]: how about a design where copies of nonces are never made, so even the question itself doesn't have to exist?

sha3 uses exactly the same keccakf1600 function in the main loop, the only difference is the initial message padding

ooo123ooo1234[m]: plus, my complaint was about TIME, converting code into a comprehensible mental model takes time, and it takes less time if the code is cleanly written. Saying you don't want to write clean code is like saying you want to waste people's time (when clearly you are obsessed with other people wasting your time, quite the double standard).

I think this is an issue of "bandwidth". The code "cleverness" has to be high enough that it's good and able, but low enough than enough people can work with it. Too clever and only very few people can handle it, and it'll end up untrusted because few people can verify it. Too low and it's full of gunk.

You want a sweet spot.

And of course I've been advocating for keeping some of the gunk because churn in itself adds some gunk despite the apparent gunk removal...

Bandwidth should not be a bottleneck. You really want to avoid bottlenecks, of which there are many.

"Bandwidth" is probably not the best term here. For the purposes of this argument, I define it as "number of people times their ability to go through and grok the code".

Another bottleneck is the number of independent people being able to read and understand the code. Or at least parts of the code, realistically.

It's a balancing act, sadly. No magic bullet.

There are so many simple things you can do to speed up reading code. Good naming, functions that do only one thing, small comments that help you orient yourself when re-reading code, whitespace that helps the reader parse language syntax. And advanced stuff like minimizing raw loops and conditionals, very judicious use of inheritance, strict interfaces and well-understood pre/post conditions, etc.

UkoeHB: Do you know any code that meets all of this but is vulnerable ?

ooo123ooo1234[m]: Where did I saw clean code is invulnerable? I said clean code is easier to read. Maybe you should re-read my reading rant.

It was not claimed you said that. It's just about shifting a probability distribution.

UkoeHB: Does "easier to read" mean easier to verify correctness ?

Any code more complex then "hello world" has bugs/vulnerabiliteis, it's inevitable. The scale and severity of these bugs can be restricted by good coding/testing/reviewing practices.

Easier to understand means easier to verify correctness.

It means easier to find people willing to verify correctness, and less cost for anyone who wants to verify correctness.

because time = money

But "verify correctness" for large enough code is practically impossible since the combinatorics are just too large. It's *really* hard to keep a large state in one's mind, especially when it changes and rolls back.

UkoeHB: There must be at least 1 example of code that was written in accordance with all of that subjective standards, audited but turned out to be vulnerable

And that's what's needed to *verify* correctness.

or redundancy added by clean code standards helped to hide vulnerability

ooo123ooo1234[m]: you could ask, eg, trail or bits or similar for a bit of code where they found a vuln and the code was really clear/nice/etc.

s/ or / of /

ooo123ooo1234[m] just read articles at pvs-studio.com/en/blog/posts/cpp - plenty of examples

People like that would likely remember such cases, as htey're their biggest successes.

on topic of clean code with vulns: pvs-studio.com/en/blog/posts/cpp/0935

ooo123ooo1234[m]: surely there are many such examples. Are you going to ask for a statistical comparison between vulnerable shit code and vulnerable clean code? Maybe I should ask YOU for that statistic, since it's pretty easy to assume that shit code is more likely to contain bugs (i.e. not just conceptual errors, but literal software bugs that can be exploited).

For me, "clean code" is also code that doesn't trigger compiler warnings at highest warning level and is regularly checked by static analyzers. It helps to catch 90% of coding errors.

then come automatic unit tests and integration tests that catch 9% more bugs. 1% is left which are really tricky logic errors or thread races etc. - but no simple bugs.

also AddressSanitizer, ThreadSanitizer and UB Sanitizer are really helpful

As long as -Werror is not used. What a shit idea that one was...

-Werror works fine in p2pool. But I had a luxury to keep in like that from the start.

*keep it

there's a lot of subjectivity about what is readable and comprehensible

Hence the "please don't rewrite to how you like it better".

if you want to understand why something was done the way it was, the code itself may not be obvious

and so you may have to trawl thru the commit history to understand the evolution of the code and why

so what is *objective* is the fewer changes you make to the code, the easier the history is to grok

hyc: don't know any example where code itself isn't obvious

* itself isn't <del>obvious, * obvious</del> enough

these are pretty well established principles in colalborative software development

I find it harder to read other people text than code

Did you every look at the IOCCC ? Some of that stuff is really clever.

ooo123ooo1234[m]: e.g. n wallet code there are some warnings for unused vars. why do they exist? were they put in for a yet-to-be-implemented feature?

or left over from a previous code removal "cleanup" ?

only the commit history can explain

hyc: sure, there is subjectivity. There are also very obvious things that few people can disagree with. For example, you don't write a book with font size 3 if you want people to read it.

Oh, that one's mine IIRC. There was something I needed to get back to, and removing the warning would just lose the reminder. Never did :/

#1 rule in a project must be "change only what is necessary and nothing more"

ooo123ooo1234[m]: Isnt enough for what?

And tbh I doubt I'll get to it. If someone else wants to fix it...

ooo123ooo1234[m]: did you mean "I find it harder to read other peopls's written language than other people's code" ?

e.g., these are the guidelines we've used in OpenLDAP openldap.org/devel/programming.html

(and yes, it's a good example of that, heh)

this one I think

ooo123ooo1234[m]: so what, everyone should be like you (a C++ compiler or something?)? is that a realistic way to get anything done?

Yes. That one would just sweep something under the carpet. Very good example of mindless kowtowing to compiler.

ooo123ooo1234[m]: I am going to assume you are a competent coder who would like to make monero better here.

another rule of open source development is you never get to dictate the style of a project you join in-progress. you always adopt the existing coding style.

This is in a way similar at a high level to free software. There are constraints in the licence, so how can it be free ?

The point is that the freedom is that of the software, not of any particular user. And of users as a whole over any particular user. Though this is subject to interpretation and I'm not sure this is the right way of saying this, but anyway:

hyc: Or, depending on the license, fork it. Put in the work. And let the merits of your ideas speak for themselves.

This is my take on it, not the definite trust of course: In order to build a piece of software, I think you want the highest "bandwidth" of work on it. That bandwidth improves with the numer of people working on it, and also with the competency of people working on it.

wernervasquez[m]: sure, that's always an option

However, different people have different abilities. You can't improve everyone's competency.

if you want to contrinbute a patch to the linux kernel, you must code to the linux kernel coding style.

you can fork the kernel and reformat if you really want to, but that would be stupid

If people work on the same codebase, they have to settle down at roughly the lowest common denominator of competency, or the people who don't understnad the code will drop off.

So you either have to drop "cleverness" to include more people, or increase cleverness to drop them off and make up numbers for "good code".

This can to some extent be solved by having people work on only part of the code. Like, say, clever cookies work only on clever crypto, where people like me won't have to understand it.

Why the fuck am I doing explaining what's obvious. It's patronizing. I'll shut up.

I thought you were doing fine.

it seems a lot of newer contributors don't understand

But I know for a fact it's hard to dumb down your code. Some people I find useless, and it's difficult to constantly try to explain things agian and again.

But you need to do that or end up essentially on your own and you'll achieve... not much.

If you're after self development, it'll work fine. If you're after building something great others can help you with, you'll crash and burn.

And I'm here in the middle I think. I see shit people around, and people like sarang who are able to grok all the crpto and still write the C++ to go with it. So I think I see both sides.

So if you want the project to grow, and not just you, you need to allow others - up to a point. IT's not a kindergarten project.

And yes, that means others will just be blind to obvious stuff. Apologies to everyone who thought I was a dunce for asking obivous stuff avout crypto repeatedly.

Yeah that's pretty well put

But if you try and let others be able to grok your stuff, it'll grow the project faster than you could have done alone.

Especially once you decide to fuck off :D

If you look at the biggest, most popular libraries and codebases, thing that are actually used by millions on a daily basis, it's not the code that it the cleverest, it's the stuff that has the the most numbskulled, dead-easy API to use

I'm feeling conflicted saying that and writing some code that's not the easiest to read though :D

(sorry guys)

that's reality. some things are just hard.

Anyway. That explains why sometimes it's just best to kick someone in the butt out of here if they persist in being assholes.

if it's too hard to understand and you don't want to put the effort in, fine, nobody's forcing you. move on to some other project if you want.

Just not worth the bother. Try not to be that guy,

That's okay honestly, but I think what has been missing so far is someone to come clean it up for the rest of us plebians

hyc: There is a tradeoff at somepoint though. Unless you goal is to work alone.

the goal is to solve problems. some problems are just hard by their nature.

Well, it's always a balancing act, isn't it... Sometimes shepherding newbies is just more trouble than you get back.

I probably won't be able to contribute to critical code for a long, long time, I'm just not that confident in my abilities yet, but what I think is currently missing is like a two prong approach that a lot of companies take on where you have some people write the code and others that make it easier to grok

seriously. if this were easy everyone would be doing it...

That seems... dangerous...

jeffro256[m]: are you talking about regular for-profit companies ?

^ yeah, shades of the debion openssl RNG bug

Heh. Exactly.

There are risky changes, and there are not risky changes. Moving functions out of a bloated file is not risky.

open source s/w dev is not like commercial s/w dev. there are no assigned roles

Maybe not risky, but conflicty as hell.

@ooo123ooo1234 yeah

@hyc yeah there are no "assigned" roles but some people tend to focus on some things, and that's alright you gotta play to each other's strengths and we can use each other to patch each other's weaknesses

No one has to write perfect code the first time

"bloated file" is also subjective. if you only need to look inside that file once every 10 years because some upstream dependency changed their API, it's hardly a problem worth spending time on

maybe you don't need to look in that file, but any newcomer learning the codebase probably needs to look in it

The issue with size is compile time. If you read code, where hte code is is mostly irrelevant, no ?

large files are incredibly hard to navigate

If someone needs to read the source of util.cpp on a regular basis, something is terribly wrong.

the only way I can get around wallet2 is ctrl-f for things I already know about

Like, I created wallet2_cc.cpp for new stuff in my fork, but that's just for memory usage.

they should only need to read util.h to make use of its functions

Would you... not search for stuff if the file was, say, 500 lines ?

that's like saying someone needs to read the source of malloc() on a regular basis

Reading malloc as an exercise in working out why something that should be simple is fucking complicated :D

Or... printf...

it's a utility, you just use it and move on...

moneromooo: it's not about the usefuless of searching, it's that searching is the ONLY way to navigate

there is such a thing as ctags

and if you don't know what you're looking for, it's doomed

OK. I guess we just have different ways to work. I always git grep and / search.

@hyc I tend to agree with UkoeHB here, you are a wizard and have spent a lot of time in the code. But for the rest of us, we can't see it as "just a utility:" because we don't know what ir does and its mostly undocumented and unorganized

So file size is irrelevant for me.

moneromooo: the same

hyc: I know how you feel dero (scam). And I am not promoting or defending it as a project. But speaking solely about the code base, have you ever read their re-implementation of monero's stuff in Go? I think it is objectively better code.

It just creates a lot of mental friction for someone trying to understand the codebase at large

jeffro256[m]: that's just the wrong approach

you can't understand the codebase at large.

"better code" is a subjective category unless we define some metrics

you find one piece and work from there

and in any piece of code - the header file is the interface. if you need to look beyond that, there's something wrong

"one piece" <- assuming the codebase is nicely split into independent pieces and not just one huge cpp file :P

with the interface

moneromooo: for me, knowing the relative location of every function and piece of logic makes it easier to get things done

util.h is 259 lines, at least 30% comments

so it seems like not a big deal to grok, as someone who wants to use it

For me, file size is not a big ussue as well. I've seen huuuuge files in commercial codebases I worked on (I'm talking about > 50,000 lines)

OK. I have trouble understanding why it helps, but I acknowledge it does.

I almost never read monero's headers, always look at the source functions to figure out what they are doing first.

hyc All but one utility function in util.h has not a single comment

jeffro256[m]: you're exaggerating, quite a bit

Ah well theres a few at the top actually but that still leaves 16 undocument functions

I basically default to looking at source for most things (some of the ringct.h and crypto.h stuff is usable).

Sorry I saw the ones between the class declarations but still

bool disable_core_dumps(); - you should be able to tell from the signature whether it's a function you care about or not

I should have called it discd so I could have added a comment.

The purpose of the project matters. If you are trying to write the most efficient miner or database, nobody really cares if the code is readable by a junior dev. They just was speed and correctness.

LOL

<wernervasquez[m]> "The purpose of the project..." <- it's unreachable goal - different people have different experience / abilities / ...; What's the goal to tune code readability for someone who don't know it if such people can't even verify correctness of their changes ?

<UkoeHB> "I almost never read monero's..." <- code is the most detailed source of info, what kind of detalization is ok for function header ? different cases require different detalization

in the worst case function header should comment with it's source above

hence it's the most detailed comment

I did it few times at work

hehe

* header should have a comment with

C++ particularly sucks because so much of implementations are in headers anyway

you wind up needing to read header and source for anything you're trying to debug

ooo123ooo1234[m]: the point was about navigating large code files and using stuff from them. It’s not as simple as looking in the header.

UkoeHB: What did you use for source file navigation last time ? vim / some IDE ? Do you use advanced navigation features ? Does it help ?

ooo123ooo1234[m]: the purpose of Monero is a private cryptocurrency that any person can use. How is writing code that any person can read for their own satisfaction, for the cryptocurrency they use, not also a good goal?

ooo123ooo1234[m]: I mainly use the find-all feature in sublime text which works pretty good (I suck at remembering shortcuts, so vim won’t work for me).

But I don’t use that for stuff whose location I already know.

Location is just an easier way to navigate for me (and conceptualize) than named lookups.

There is no ‘what’s that big generic file that has what I want? what’s the exact name to find it?’

ooo123ooo1234 why not put all of src/common into one header file and make all the functions inline? That would make it easier to CTRL-F it

@hyc @selsta UkoeHB I pushed to the Sha256 PR btw with a smaller test file and I wrap EVP_MD_CTX in a class so it never leaks

<hyc> "github.com/monero-..." <- Thanks for the help. I have tested this and got less errors now:... (full message at libera.ems.host/_matrix/media/r0/do…4da0e830f877f7d93699b1451bfa6460b0d)

Also btw the evp.h file is not going to be a one-off file for sha256sum. There's other parts of the codebase that are deprecated by OpenSSL 3.0 but which the EVP-equivalent isnt

> > <@hyc:libera.chat> monero-project/monero #8332... (full message at libera.ems.host/_matrix/media/r0/do…e0ecef164e1fa28557c4dc9f5cbe0b8b554)

quila[m]: strange, I didn't get those errors. those symbols are from openssl

that's a system library on macos. On mine it's actually libressl 2.8

hm. no I've got homebrew openssl 1.1.1l in /usr/local/lib

hyc: There are too many environments to support all of them by default. It's enough to maintain at most one cross platform environment for all targets.

It's just waste of time to figure out what decomposition of headers/libs is being used by some env

dunno which conv you're responding to

but yes, that's why we use gitian for official reproducible binaries. single env for all targets.

hyc, "It's easier to make a correct program run fast than to make a quick program run correctly!"

cool thesis

"openldap.org/devel/programming.html" from here

yes

we've lived with that philosophy for ~24 years on the project

and also happen to perfectly conformant with protocol specs and world's most efficient implementation

correctness comes first.

<hyc> "hm. no I've got homebrew openssl..." <- I have set the FLAGS to use openssl 1.1, but got a problem with boost lib now.... (full message at libera.ems.host/_matrix/media/r0/do…4e14f68121c2c9042381aae822b3392150c)

icu to be more precise.

Testnet fork is tomorrow. If I want to help debug the switch I should git pull from master today?

hyc: how can I set multiple `CPPFLAGS`? Would something like `export CPPFLAGS="-I/opt/homebrew/opt/icu4c/include:$CPPFLAGS" work?

garth: yes

quila[m]: why do you want to change CPPFLAGS? that won't make any difference here

strange that my homebrew libboost doesn't have this problem

quila[m]: at this point it seems like more trouble than it's worth

the gitian build for M1 Mac in #8312 works

I have no idea what's different about your homebrew install that's breaking these library dependencies

not worth my time to help you figure that out when I've already created a reproducible build environment for M1 that everyone can use

No problem. Thanks for the help anyways. I will investigate further tomorrow.