is there an explaination for Monero's transaction structure?

is Monero working with something like UTXOs like BTC????

There an explaination for Monero's transaction structure somewhere on monero.stackexchange.com. Monero works with UTXOs, yes.

mj-xmr[m]: I don't see the point of a testing framework. Doing `time make` with cleared / disabled ccache is as objective as it gets.

That's what 99% of users will compile and is what should be optimized for.

I don't think you're right on the 1st one.

2nd one: It's for developers, not users. Users compile once a quarter. They don't care.

What is unobjective about doing `time make`?

The documentation says it all.

Short answer: many things.

I will remeasure monero-project/monero #7217, because I might have done something wrong in not enabling tests.

how does ninja have an influence on cmake? do you mean make?

Yeah. That's a valid finding.

Thx

Although it's cmake which creates these Makefiles, so...

how can I integrate monero payments in my online shop, I use nodejs ?

Maybe "monero integrations" has what you want.

Plugins for some online shop software.

ty I'm gonna check

Re: 7217, please do if you wish, especially since my framework takes some 4-8 hours to complete.

Re: ninja: it replaces make and Makefiles entirely

Re 7217, the expected result is, that the core tests themselves compile ~40-50% faster. So if you can prove it in a matter of 2 hours, it's all fine.

The problem with ninja is, that it's incompatible with some exotic improvements, like icecream. But not everybody uses all the improvements at the same time.

The most important part is, that it works with ccache.

moneromooo: do you remember what you meant with this comment? monero-project/monero #6097#issuecomment-549979567

I had to reorder some code to fix... a timing info leak IIRC. In turn, this undid something I had fixed before, which I believe was a subtle race condition with the txpool.

It was pretty subtle IIRC, and so I needed time to think about how to refix it after the move, and I never got to it. I do not remember the details of the race.

I don't think I'm likely to ever get to it now.

ok ty, i guess i'll let the other decide if we keep the unused variable

At this point, since it was to remind me, it could be replaced by a comment if people really hate it.

Makes sense.

mj-xmr[m]: i just checked and ram disk is negligible for measurement on my system. the difference is < 1% for single core compilation

OK. Actually for OSX the RamDisk option is still disabled, since I can't test it properly. But for Linux it's fine so far.

mj-xmr[m]: interesting that your recommended command ends up being slower monero-project/monero #7217#issuecomment-1144767782

Hmmmm. Then either I'm reading it upside down, or you wrote it upside down.

I meant RAM disk, -DBUILD_SHARED_LIBS=ON and -DSTRIP_TARGETS=ON

it's either -DBUILD_SHARED_LIBS=ON or -DSTRIP_TARGETS=ON that slows things down

aaah.

strip targets is probably extra work, not sure why you enable it

Well it's a mixture.

Exactly

but it's irrelevant for measurement

For this reason:

- Strips symbols off the targets, in order to spare the space on RAMDisk, that would otherwise lead to memory fragmentation or worse: swapping

And when you strip in both runs, the negative influence zeroes out. For example, imagine what interference of the same time your simple `make core_tests (with unity)` and `make core_tests (without unity)` has caused, let alone your `make -j8`

read: thread starvation.

I didn't do -j8?

hmmmm. Were you testing it already?

Perhaps I sleep too little lately because of a bit too long meetings in Matrix.

that previous test run a couple days was incorrect because i didn't compile tests in the first place, that's why i redid it

Ah. Thanks for the sanity check.

literally.

is there any reason in particular why `get_address` requires that the subaddress index(s) supplied must be >= the number of labelled subaddresses?

wallet2.h: size_t get_num_subaddresses(uint32_t index_major) const { return index_major < m_subaddress_labels.size() ? m_subaddress_labels[index_major].size() : 0; }

wallet_rpc_server: THROW_WALLET_EXCEPTION_IF(i >= m_wallet->get_num_subaddresses(req.account_index), error::address_index_outofbound);

my use case is that I don't want to 'pollute' my wallet using `create_address` when the part of my program which runs after the create `create_address` is fallible. Basically for each item I am trying to get a subaddress - ideally using the id off the item as the subaddress index

HenryHollingwort: are you sure it's >= ?

but that says throw an exception if >= ?

ah derp i meant to say must be <=

it has to be < probably

basically I'd like to use `get_address` for subaddress index say 65 - when I haven't created subaddresses for indexes up to and including 64

wow i'm struggling with my inequalities tonight

the CLI wallet has this address one-off command

I'm not sure if RPC also supports this

[wallet 9ssWyM]: address one-off 0 200

Address at 0 200: BdcYFby3sYXfgBiuv2RWHDUhsoi2v1pETgZLViSbqmox3BFmoJuyyxj9pMkqiH7LBHKyify1wip6Fgf4ymn4JUX3DVNdiQ5

yep that is perfect

I might have a go at adding this to the wallet RPC because AFAKT it doesn't support it?

if you add it a PR would be nice :D

my project can just generate a new address each time it tries to do something - which isn't a roadblock it's just not ideal and then I can have a go at implementing the RPC command

great :)

mj-xmr[m]: so far I don't see the point of your "test framework", none of the things you listed have a measurable impact (>1%), apart from BUILD_SHARED_LIBS

And that's exactly why we need the framework.

Why? Because it tells a different story.

and I don't want to be accused of "charlatanism" as usual.

And a general remark: I'm not a rookie. Apart from these ill-fated Boost PRs, which were created BEFORE I could do any measurements, from now on whenever I PR something, it does have a measurable impact. Otherwise I wouldn't PR it in the first place.

So exercise the possibility, that the problem may be within your measurements, maybe?

lol

What? Not a chance?

I feel like you are overcomplicated something simple. Measuring build time is as easy as it gets. And I gave you the benefit of the doubt by testing every problem you have listed in your test framework (RAM disk, RAM cache influence, dynamic linking, etc.) and all of this has been negligible, as I expected. The only thing that has made a difference is BUILD_SHARED_LIBS but for setting that you don't need a complicated test framework.

Well, thanks for helping anyway. Appreciate it.

*I

and I'm not saying that the things you have listed have 0% impact on build time, they do have an impact probably in my testing it's negligible (<1%), so I'm not sure if it's important to concentrate on it now

but*

<1% would be not worth it, I agree.

basically we have to decide if we want to test with or without BUILD_SHARED_LIBS

Do you mean running the tests on GH? Then only static, because this is closest to the user experience as possible.

Anything else that a Developer typically wants is using approximations of all these User's comfy settings, in order to maybe execute a bit slower, but build a lot faster.

So in the Dev's case - shared.

afk

the main reason static builds are slow is because we use LTO. for debug it's sufficient to turn that off if we want faster static linking

but sure, using shared libs could be faster still, since binaries won't need relinking

1) I think static builds are fast enough, even with LTO, if they're done on GH, using ccache. Limiting header interdependencies helps even more, since ccache would be using smaller, more reusable chunks of independent blobs

2) Yes. Especially if one's machine has a slow disk.

FWIW, ccache does not cache link outputs, just compiled and preprocessed outputs. Unless it changed when I wasn't looking, which is possible.

No, it doesn't (or never did). I'm just saying that on GH, the caches are being stored in a relative abundance, since the each new run of master updates and extends the cache.

and GH is the typical use case for an statically linked binary + LTO

So that advantage of ccache there somehow regularizes the losses made by LTO

Hey there. Wall of text incoming, apologies in advance. :)

For those of you that don't know, I am with RINO. Due to our products nature, we depend on multisig and thus are quite concerned about it's current state. We try to help out where we can, and issued a while ago a $10k bounty for fixing multisig in the upcoming hardfork.

As already dicusssed in the dev meeting, another audit of PR #8149 is advisable. We decided that RINO would commission this review with Inference AG (did Monero audits for CLSAGs, Thorchain, and possibly others). After checking with the folks over at the research lounge, UKoe and the Haveno guys, we decided to put the promised $10k bounty towards this audit. In the interest of time and efficieny, RINO will pick up the rest of the tab

as well.

UkoeHB and I met with the auditors and defined the scope of the audit. We addressed some concerns with their Thorchain audit; UkoeHB had some nice suggestions wrt to the scope in order to prevent these kind of oversights in the future. Overall we were satisfied that their audit can improve the trust in PR #8149.

The contract has been signed and the audit will start on Monday. UkoeHB agreed to collaborate with the auditors to help them in their work - thank you, your work is much appreciated! I hope to receive the final report the week after. Once the audit is finished, we'll share the report with the community.

A few questions remain to be discussed.

1) We're obviously not sure what the audit will yield, so it's unclear to which extend it'll improve trust in the multisig system and whether it's deemed sufficient to enable it again.

2) Several people brought up the idea to audit the whole multisig system (via a CCS proposal, most likely). The main issue here is that the multisig system badly designed, badly implemented, and lacking a formal specification. We discussed this potential "big" audit with the guys from Inference AG and it became clear that neither the scope nor the goals are defined enough to estimate cost/effort. Also, Ukoe doesn't seem to see the

necesscity of one big audit over smaller, incremental fixes/audits (correct me if I am wrong). Overall I would suggest to put this idea aside until we have the results of the audit of #8149, and then rediscuss.

3) The current implementation of #8149 will break multisig transaction building pre-hardfork. Not sure how big of a problem that is, but this differs from how hardforks usually work in Monero I think. Ukoe already proposed to fix the PR, but some questions around testing remain.

ErCiccione suggested that we get together to discuss the multisig situation, maybe already this weekend, at the latest after we receive the auditor's report.

#2 sounds like quite a lack. We should get an actual spec written

and as for badly designed and badly implemented, while writing the spec we should take the opportunity to fix the design.

it sounds like it's all a mess but if the audit from #1 says "doesn't make things worse" I guess that's the best we can expect

I don't understand #3. if the 8149 patch is deployed in the next hardfork then it doesn't matter what its pre-hardfork behavior is, does it?

re #3: just my concern from an operator's perspective. Deploying the new version pre-hardfork (as one is supposed to do) yields a broken software (can't build txes)

also, any multisig users are desperately waiting for a patch, hardfork or no. The issue prolongs this wait by a month.

I am obviously biased though

I see, yeah, it'll be unusable for a month.

but obviously you can keep on using the current release then, if you need it up till the hf occurs.

re #2: the question is: who is willing to write such a spec? someone would essentially need to a) reverse-engineer the code, b) derive the design, c) fix the design, and then d) fix the code. Lot of work, and I am not sure who can do it. The auditors usually only work on c) and d).

who understands what the problems with the current design are?

> <@arnuschky:matrix.org> 3) The current implementation of #8149 will break multisig transaction building pre-hardfork. Not sure how big of a problem that is, but this differs from how hardforks usually work in Monero I think. Ukoe already proposed to fix the PR, but some questions around testing remain.

>

Multisigs are disabled as of master without a cli flag. I don't personally see this as an issue.

Theoretically it's definitely the cleanest solution thought - especially as every new fix or audit will do some of the reverse engineering again, and then kinda throw away the result.

hyc: I think Ukoe knows best. The anonymous author of #8149 had lots to say on the topic, but nobody seems to be able to rate the truth of it.

hyc: I personally think the biggest practical issue has nothing to do with the algorithm and is instead the use of unvalidated rpc routes to step it

UkoeHB mentioned that he knows of 2 issues that need fixing, but as I said he doesn't seem to see the necessity of a "big" audits beyond fixing those issues

it's pretty hard to define the boundary of a big audit

(UkoeHB let me know if I am talking nonsense here, working on the basis of my notes but might have misunderstood)

I can provide notes, comments, and outlines about how multisig should work. I don't have time, energy, or enthusiasm to write a spec. Nor do I want to be sucked into doing work that someone else says they will do (like this damn 8149 pr).

Why is the current non-spec not good enough as a spec ?

rbrunner: 8149 only works on testnet

Ah, you mean, it works only if already forked?

yes

That I did not notice :)

Yeah, that's a bit unfortunate ...

Operators have the choice of updating before the hardfork, and not being able to build transaction, or wait to update after the hardfork and be inoperable in the meantime. Either way, it's a forced downtime, which isn't nice.

Well, but the way things currently look it may well be that #8149 does not make it into the release anyway?

For lack of final review, at least

I'm working on an update so it works with mainnet

<moneromooo> "Why is the current non-spec..." <- Not sure. A full-spec seems to be a precondition of a "full" audit, which was discussed in the community.

UkoeHB: awesome. so maybe we should drop this point then

rbrunner: That's what we're trying to fix with the upcoming audit - though probably not in time

Yes, release is currently planned for June 16, if I remember correctly. And a bit of testing beforehand would probably be nice :)

I have also a PR on the way that may miss that deadline, although of course much less important

MAGIC was also working on audit prep to some degree, though it looks we got beaten to the punch. If there is later a full audit... We'd still be happy to help there :)

since multisig will be disabled by default anyway, I think 8149 should be able to go in

I think so as well. If UkoeHB manages it to get working on mainnet pre-fork in the next few days, merge the damned thing. It works.

It can't be worse than what we have :p

So with disabling by default, I see absolutely no reason it shouldn't be optimistically included in the next release

Either it is broken and we need to update or it's fine and...

Though I'm not advocating for merging the current PR. koe would need to comment is ready, albeit unaudited, under these criteria

Think positively: Bad people don't have to go to extraordinary lengths and try to find exploits in Monero multisig. Make an NFT, people will throw money at you, finished :)

moneromooo: I'm trying to debug core tests but it doesn't want to read the input commands: Failed to parse arguments: unrecognised option '--generate_and_play_test_data --filter *multisig*'

ok got it: "args": ["--generate_and_play_test_data", "--filter=*multisig*"] damn syntax

You probably also want \*multisig\*, it won't do what you want if you have a file matching *multisig* in ..

Also, it looks like it's somehow parsing it as a single option for some reason. Did you put the whole inside "" ?

this is in vscodium's handy debugging framework - it's a json file that passes stuff to lldb (so it has annoying syntax nuances I guess)

at the command line you need escape characters

arnuschky[m]: rbrunner ok pushed a new commit that should work on master/stagenet (core tests and functional tests do not cover this)

Wow, that was fast.

Will try tomorrow with stagenet

im not a zoomer... born too soon?

<UkoeHB> "arnuschky: rbrunner ok pushed..." <- oh wow, that was fast! thanks, we'll go test the shit out of it and get started on integration work (monero-js and monero-cpp afaik)

<arnuschky[m]> "UkoeHB and I met with the..." <- "We addressed some concerns with their Thorchain audit" is it private info ?

<arnuschky[m]> "As already dicusssed in the..." <- Inference AG didn't do audit for CLSAGs.Advisor of Inference AG (JP Aumasson) was involved into CLSAG audit, but not actual workers of Inference AG. Check names of Thorchain audit and CLSAG audit.

<arnuschky[m]> "I think Ukoe knows best. The..." <- Indeed, nobody

<arnuschky[m]> "I think Ukoe knows best. The..." <- if you all stopped pretending that you understand something in area where you don't, everything would be much simpler. I think you will just learn hard way that you've thrown away 10+k $

hahahaha

<rbrunner> "I think so as well. If UkoeHB..." <- Are you advocating for another blind merge ?

ooo123ooo1234567: " is it private info ?" -> Failure to consider validating preconditions as in-scope (which it should be). I had them add a clause to the scope about preconditions, so if anything goes wrong with the audit that falls under that clause, there won't be any ambiguity.

<UkoeHB> "ooo123ooo1234567: " is it..." <- What's about already prev audit ? Are they going to compensate somehow their big overlook in prev audit ?

s/What's about already prev audit ?//

ooo123ooo1234567: Which was...?

kayabanerve[m]: gitlab.com/thorchain/tss/monero-sig…master/src/wallet/wallet2.cpp#L7285

I don't trust people that blindly approve PRs, but these even claimed to do proper audit and failed. I can't understand it.

I was more asking what in that function was broken which they missed

And while I know they reviewed thorchains edits, which would've been over the historical multisig, if their scope was just the edits...

Though I wouldn't be surprised to hear Thorchain has bad code, I've been saying so myself for a while and submitted a few crits to their WIP XHV work.

I also wouldn't be surprised to hear an auditor missed something. Plenty do. I'm curious what specifically they missed, as I did know they had an audit for the edits to monero for thorchain

<kayabanerve[m]> "I also wouldn't be surprised..." <- You're lowering quality bar when saying "missed something. Plenty do.". I'm sure there is certain reason that audit was shitty. But I can't debug it since I don't have access to info

So you said they had a "big overlook" and cited this function definition. Unless you can say what they overlooked, you can't say they overlooked something

Their audit report is public

I'm also not justifying them because I'm saying a lot of auditors do make mistakes. I'm saying there's a lot of bad auditors and I wouldn't be surprised to hear they're one of them, because statistically...

"monero-project/research-lab #100" how would you describe progress on this ?

stopped / abandoned / slowly progressing / undefined ?

<kayabanerve[m]> "I'm also not justifying them..." <- someone here is doing the next audit with that Inference AG, while you're saying "I'm not justifying them", "there's a lot of bad auditors"; I can't understand this inconsistency

Not a priority for me. I've previously tried letting you know why your conversations are seen as frequently antagonistic and unproductive. To try again, if we're discussing X, and have somewhat opposing view points, and then you ask about Y which is a lack of progress, it feels like an ad hominem attack in bad faith. You truly need to contextualize your statements.

You haven't provided any evidence they're bad

There are a lot of bad auditors. I wouldn't be surprised if they were bad. Do you have evidence they are or not?

Without such evidence, I will default to RINO's review and wait to see the audit.

kayabanerve[m]: Did you encounter some obstacle or just stopped due to other tasks with higher priority ?

kayabanerve[m]: I don't see any reason to continue talk about audit, since I don't agree with you and you don't want to debate about it. That's why I've asked about something interesting for me and similar to prev topic and you.

kayabanerve[m]: we discussed it in #monero-research-lounge a week ago or so, gitlab.com/thorchain/tss/monero-sig…master/src/wallet/wallet2.cpp#L7285 is vulnerable to the same issues that are fixed 8149, and they were in scope

Got it. That would absolutely be critical and means they did not handle that competently at all

I assumed their audit was specifically based on the API functions added per the segments I read

kayabanerve[m]: I thought that you're communicating with UkoeHB and knew it already. Hmm, strange

selsta: no, I was right

> The client requested a review of the changes introduced by the thor_monero_signing_parallel

branch in the thorchain/tss/monero-sign repository. The code was forked from github.com/monero-

project/monero at commit cb70ae94.

This was solely the changes introduced by thorchain. It was not their responsibility nor scope to check the actual algorithm

So unless their changes re implemented the underlying algorithms, and that's why they're also vulnerable to 8149 AND in scope...

a scoping debacle, as I have long argued... if scope is properly defined then it's easier to set expectations and identify failures

<ooo123ooo1234567> "Did you encounter some obstacle..." <- It has a large amount of plumbing which is why I wanted to move the proof of concept to halo 2, as the ecc offers elliptic curve ops and hash functions for it. I have multiple other prioritized items, unfortunately

I personally think it's completely fine to say we didn't touch monero code and assume it's fine. We only did these rpc routes

So unless the rpc routes explicitly open issues, I don't believe the audit is at fault

And I'd have to review the routes they added myself accordingly, as it isn't really just routes. It coordinates the process as a whole

<ooo123ooo1234567> "I thought that you're communicat..." <- I did know multisig was critically flawed. The part I was responding to was those issues being in scope

Based on what the audit says however, they weren't

Though I do hate discussing thorchain D:

I'm quite sure they did whitespace changes making it technically in scope but I'd have to look up the chat backlog from -lounge.

but yes discussing this again now doesn't help

things can't get worse with the audit, if it's worth the funds will be seen

selsta: The changed whitespace itself was fine though :p it's just what is directly next to which isn't

Though I do love the technicality

I'll try to read up on that myself. The contract is signed so unfortunately this decision has been made. The monero community can review the audit and debate its competency

I personally believe we should merge when koe agrees, yet before the next release, purely because it can't be worse. The question solely becomes the experimental flag

kayabanerve[m]: How do you prevent internal conflict due to your own project ?

Just interesting, how it works

My potential conflict is a completely separate codebase

I actually am disincentivized from monero having a working multisig lol

*if you want to argue that

It would probably be best to leave the experimental flag until a proper model can be published (if that ever occurs), and the code validated with respect to the model.

kayabanerve[m]: Yes, that's what I'm saying

s/saying/seeing too/

I'm also a monero community member who's submitted exploits to our multisig work and think multisig is incredibly important, who benefits from the theory being understood

Although from one point of view, the monero protocol itself is 'experimental', since there is no formal model for the entire thing.

UkoeHB: "best to leave the experimental flag" <-- agree

kayabanerve[m]: which one ?

My stance is one which ensues it's ready when it's ready and potentially voids the need for yet another release

UkoeHB: I'm waiting to see the immediate audit, but I'm currently leaning towards secondary audits as discussed this morning

I do agree the experimental flag should remain though :) Just want to be verbose and clear on that as it's being brought up I could be working against monero :O

Btw ooo123ooo1234567 speaking of progress, where the hell is your progress report? All I have seen is you being a toxic ass for months.

multi-sig is very important and will be used a lot

UkoeHB: progress report for what task ?

UkoeHB: period of unpleasant talk started from "libera.monerologs.net/monero-dev/20220507#c92654", it's even less than 1 month, so not months

felt like months tbh

<kayabanerve[m]> "I'm also a monero community..." <- do you mean burning bug or something else ?

UkoeHB: is there an issue disclosing here? I think it already was tbh

selsta: It is exhausting :(

No, you being a toxic ass started in December when you ignored my messages about PR 8149 and disappeared. How about progress on anything? I'm not your task manager, why don't you find something useful to do instead of being so unbearable.

UkoeHB: Do you remember at least 1 msg from me which you may consider as toxic without any reason ?

* without very good reason

ignoring is at least less painful, than writing criticism directly, which I've postponed until I've solved problem by myself, so that I'm 100% what I'm talking about

just curious, what happens in such an audit contract when issues are missed?

Generally nothing

kayabanerve[m]: yes, nothing, the same as in any other rewarding system (current monero environment) without verification

* without verification and competition

but if you can verify if an audit is correct, do you need the audit in the first place?

complete*

Wait till we discuss bit flips from radiation :(

Formalizing the algorithm would enable complete verification of the theory. Then an audit is solely on if the algorithm is matched and if it leaks data while doing so

So nothing is ever complete, yet there's a reason such a step was noted and why is relevant

s/is/it's/

kayabanerve[m]: No. Define quality bar for audit. If it's too low then go to do it yourself. if it's too high then ask others to do it. Currently it looks like bar is too low, so it's more profitable to do useless audits, then to rely on them.

we don't know yet if the audit will be useless

wrap CSS

everyone will be using multi-sig

probably just hard to find right company

<ooo123ooo1234567> "No. Define quality bar for audit..." <- I've reviewed worthless audits. PirateChain had one at a conference, which I don't say to accuse piratechain of on purposely getting one, yet as a literal fact. No issues anywhere, not even minor. I will not hesitate to review the audit, continue my own reviews of the code, and discuss it here

But unless we shadow the process entirely, we have to wait to do that. We don't have the audit yet

<selsta> "we don't know yet if the audit..." <- It can't hurt. We can review its competency when we see it. RINO made this decision. It's not a Monero decision. It's solely a discussion we're having.

MAGIC is continuing its work on its platform, and we wanted to host audit funding for multisig, believing it was a solid fit. After this much discussion, it sounds like that consideration shouldn't be striked, not that I'm making any guarantees on behalf of MAGIC, and we should be discussing a second audit by a community agreed upon group

I'd also be curious what UkoeHB: thought of the auditors cryptography talent based on their discussions

kayabanerve[m]: "cryptography talent" facepalm

K, you still haven't posted anything and the most recent discussion here was a scope issue which had them in the clear. I was PMed something that was flawed yet supposedly in scope, and I'll review that when I can, yet I'm out right now and the diff isn't feasible to parse from a phone

kayabanerve[m]: false assumption of safety may hurt

It did sound like they missed something crit, and in scope, suggesting that no, they're not the best part for this, at which point koe could comment if they've changed/assigned competent people this time, or if he believes this audit shouldn't be planned upon and monero needs to continue establishing its own review process, including formalization and an audit.

we don't assume safety based on this audit

It explicitly sounds like this audit will be a discussion point of a much lengthier conversation at this point. The immediate question would be who wants to formalize the multisig protocol

Beyond the pure math of it, I would hope to see data flow also formalized. Certain random values were moved to a hash chain. That's perfectly secure, yet then the question becomes how are we deriving other random values.

Etc

<kayabanerve[m]> "UkoeHB: is there an issue..." <- answer to this ?

this was disclosed

What's the next date for hardfork in case if current master will be released as is ?

6+ months / 12+ months ?

<ooo123ooo1234567> "What's the next date for..." <- Likely when enough or important enough changes require