this Solana attack is insane, especially if it's the result of an unsafe ed25519 lib

thank goodness we did the SUPERCOP exercise

Whts the value of H used in Commitment aG+bH give in x,y form in hexadecimal

It's in rctOps.h IIRC.

Oh nvm it's not. I looked.

Ah, in rctTypes.h.

According to it H is hash of G

Not checked, but it seems strange that’s only an hash.. G and H are EC points, so maybe an hash is applied to G serialization (here it is first additional stuff) to honor NUMS, then a function has to return the second coordinate of H (and here it is second additional stuff)

meeting 1.75hr

h

*

meeting time monero-project/meta #724

1. greetings

hello

Hi!

hey

Hi

hello

hi

2. updates, what has everyone been working on?

This week I implemented bulletproofs in Rust and I will start scanning the blockchain for the bp transactions this weekend. Although my Python code works, it is tooo slow. So I thought that making another implementation in Rust would be more efficient and serai (kayabaNerve) can also profit from that.

Also, I finished my first reading about Seraphis and have some questions about it. But I will let them for the end of the meeting.

OSPEAD, with a focus on forecasting and evaluating out-of-sample risk.

reviewing trezor's hf PR, then planning to move over to reviewing the serialization overhaul (7999/vtnerd's alternative)

me: still working on legacy balance recovery for my seraphis lib (will probably be doing less hours this month, on vacation)

I've been working on the MAGIC grant, I've finished processing all the datasets and now hyperparameter tuning the models. If anyone has free time over the next few weeks I'd love for some constructive feedback on the paper.

3. discussion, anybody have comments/questions?

xmrack[m]: Nice! I will try to in the next 2-3 weeks.

xmrack: Are you ready to share some preliminary results here?

Yea, one second

I have a 100% assembly implementation of the X25519 key exchange using mulx/adcx/adox instructions. Will post some benchmarks later this week. That's probably as fast as output scanning can get with current CPUs.

Hello :)

tevador: very cool, thanks for taking that up

tevador: what's the difference to the supercop lib we use?

I implemented BP+ verification in Rust as well, sorry for being a bit late to comment

selsta: supercop doesn't use mulx, only legacy mul. And this is X25519, which only works on the X coordinate.

tevador: Fantastic. Would the code apply to Seraphis outputs (enotes) too?

I estimate ~30% speed-up for x86 and up to 3x speed-up for ARM64, which also has a full assembly implementation (not by me).

It appears that the AdaBoost random forrest is performing the best when tested on mainnet transactions, with a 19% accuray. The gradient boosted classifier also achieves similar accuracies but with very difffernt feature importances. The results may vary as my mainnet validation dataset only includes ~72 ring... (full message at libera.ems.host/_matrix/media/r0/do…2d1a3baf69a251633477a118328deeb21f1)

Rucknium[m]: the plan is to use X25519 with Seraphis

tevador: 🎉

dangerousfreedom: what are your questions about seraphis?

xmrack: In the machine learning field, is there a typical explanation for why two ML algorithms would have quite different feature importance rankings?

UkoeHB: 1) To make sure I understood, if you could boil down Seraphis to two concepts, what would them be? No ring signatures anymore (usage of linking-tags instead) and separation of this part (membership) from the rest of the transaction?

2) What is state of the implementation? Do we have a minimum wallet or minimum node?

3) Is there already a template of a transaction with all the inputs and outputs like we would see at the json representation of a tx?

The membership proof is still for a specified subset. It's key images which are replaced by linking tags.

1) separation of 'membership proof' and

whoops

"key image" is a colloquial term for a linking tag; it's the same thing

Yeah

1) separation of 'membership proof' and 'ownership proof' into two pieces; more powerful user key structures are possible with the new key image construction

While that is true in purpose, and it's a bit awkward to be changing terminology, I do appreciate highlighting their vastly different implementation.

Actually, the linking tag would be the stealth_address, right?

Rucknium[m]: Yes, each model uses different algorithms which separate the features to the best of their ability and correlate patterns for each of the ring positions. The way they go about doing this could easily cause a difference in feature importance across multiple models. This is mainly why I'm training a few different ones along with a neural network.

These would be the members?

I basically gave up on the linking tag terminology lol

Haha

Eh. We should encourage it

But yes, does take more time to do so :p

the kind of linking tag we have is a key image, so I think it's fine

xmrack: I think medium-term it would be good to estimate confidence intervals for the F1-score on the mainnet transactions since the sample size is small. The typical approach would be bootstrapping as long as the F1-score satisfies the technical regularity condition of being a _pivotal statistic_.

I do have a side comment on Seraphis dev/impl, further inspired by a recent thing I noticed from BP+. BP+ implemented new generators and a new transcript. While I have no idea why (as under batch verification, there should be no benefit to new generators of the same quantity/method, especially since these aren't even batched together). The new transcript is... negative however. It's the same methodology as the previous one

(with all the idiocy it brought), yet with a DST. While I don't mind the DST, it broke type safety because Monero keys aren't type safe :/

2) the core library is mostly done, I just need to finish integrating legacy enotes and add a coinbase tx type; the core library should be well-featured enough to build a wallet from

Hopefully there is a standard and mathematically-proven way to do that with ML. If there is not, I will be Very Upset with the machine learning field :P

It also performed unnecessary operations in its initial parameters, which I can't explain. While yes, I know the BP+ code is settled, and this is -dev commentary, I do believe there is the discussion with Seraphis about ensuring we don't continue such oddities.

Rucknium[m]: Off the top of my head I’m not sure. But I’ll look into this

While UkoeHB is a great developer, and I know they have fixed a lot of issues like this in their work on a new library and new wallet I want to highlight the value of independent implementation for full review, both of theory and practice.

3) there is no serialization finalized - that kind of thing is up to future developers, but the main tx type is here github.com/UkoeHB/monero/blob/serap…b/src/seraphis/txtype_squashed_v1.h

there are several areas where serialization can be optimized compared to the C++ representation, I'll probably have to write some things down for that

I think this will bring much fun because it does not derive from the CryptoNote tx type that now permeates the code from bottom to top

Ok, nice! Thank you for the answers!

UkoeHB as a quick remninder (which you probably know), the cryptonote serialization is separate from the work that jberman[m] is referencing, and is a bit more compact in binary size

kayabanerve[m]: I already forked the BP+ code to use the seraphis transcript system + generator factory github.com/UkoeHB/monero/blob/serap…b/src/seraphis/bulletproofs_plus2.h

vtnerd: I mean serialization in general (of any kind)

all I have is a C++ struct

kayabanerve[m]: I believe switching to seraphis transcripts actually sped things up a bit since blake2b is a bit faster than keccak

for seraphis? a brand new one? probably makes sense to start new but man

yes

I have another point too. Maybe in resonance with what kayabaNerve said.

I'm not happy with the non-canonical points and scalars stored in the blockchain. Although I didn't find any bug that could definitely lead to inflation, these points and scalars can lead to wrong signatures and fungibility issues that will cause trouble for different nodes and wallets implementations, which are inevitable as the project grows. Moreover, I believe we should change the mindset that we have now which is

'trust in the code' to something like 'trust in the math' which is a more robust argument as math proofs are usually less prone to errors than implementations of these math proofs.

I know :p I'm very happy you covered both of these. I'm discussing items of this level of eccentricity in general though. While you're also working hard against it, I'm highlighting how we're individuals in our own ecosystems, and how certainty there...

its probably a great chance to break away from cruft, and finally get some "librarization" going for downstream projects, but hopefully it won't turn into too big of a project

I refuse to write this kind of disgusting mess in my library lmao github.com/UkoeHB/monero/blob/24ef2…42a04ba7/src/ringct/rctTypes.h#L335

The hero we need

yeah and `/src/ringct` is one of the better folders

well kaya the problem has been doing librization meant total pain with compatiability

Iirc you also made it so we don't need to write commitments into the bp to do verification which is great

but since thats being blown up it does make it easier in one sense, but the older code still needs to be present for blockchain verificaiton :/

anyway I;ll stop hijacking this, ukoehb if you need seralization suggestions thats one of my few specialities

the interface I developed could also do tagless (like cryptonote)

vtnerd: This is about librarization. It's about implementation preferences of developers or decisions relevant to an ecosystem which make it hard to move out. I'm commenting independent implementation is one of the highest forms of review and enables commenting on the portability of decisions as well

so most compact

vtnerd: you may want to spearhead that effort, since I don't plan to do anything with serialization

once legacy stuff and the coinbase tx type is ready, I am moving on (after finishing the paper)

I am a bit confused. Why should existing serialization be unfit for sure?

ah - you'll have enough with trying to get seraphis integrated, etc. ?

While portability largely isn't an issue, I'd highlight the rct::key type ambiguity and custom hash to point as examples. I'm mainly concerned about the eccentric preferences which have pervaded monero's history though

it is a massive effort already

After all we serialize all kinds of things already, no?

the existing serialziation for cryptonote is decent actually

its just that the interface I wrote could handle epee/json/(most) cryptonote/msgpack

whereas now there are multiple in the codebase

there are several "tricks" in the existing tx that can't be duplicated easily with a generic library, since it reuses array length values across fields, etc., that is extremely custom

I think it is impossible to go from rct::key -> point with certainty. The other way round is possible. I dont know if it happens in the code.

and as ukoehb will point out, that code is a bit gross and hard to follow due to the custom nature

Making the protocol itself more strict can improve development decentralization. Hopefully there could be less reliance on the actual wallet2 code and more reliance on a protocol specification. (This goes for decoy selection and other forms of defects in tx uniformity IMHO too.)

With every additional wish here right now we cooly add one more person year of work :)

I will finish my CCS by the end of this month and I would like to continue working for the betterment of Monero as I have built in the past 6 months my tools and knowledge about Monero to do so. Specially regarding the security and cryptography behind, as I said, we should really make sure to have only canonical scalars and points stored in the bc. I will propose a new CCS this month and would like to know how I can help.

:)

if ukoehb does rewrites similar to multisig it will help out big time, regardless of any strict spec being written

From my side, I would like to continue working on my monero-inflation-checker (as I still have BP+ to check and also I believe I can improve the content and make more implementations in Rust) and help to implement the necessary functions for Seraphis/Jamtis. Do you think I could help you, jberman, tevador and someone else to implement the necessary stuff for Seraphis/Jamtis? How is the work structured so far? Where is the

TODO list of the project ? :p

fluffy had one somewhere a long time ago

doing what ukoehb is finally doing with txes was on that list

As I mentioned, hightlight every place in the code that now references cryptonote::transaction and mark it for possible rewrite.

I don't think an updated one exists

Then sit back and marvel at about half of all code marked ... :)

yeah idk anything about how tx serialization is handled

yeah thats the one concern I have with the seraphis proposals, its going to be a beast to integrate and will likely take longer than expected

figuring out the core library already takes up my entire attention

which isn't to say that Im against it, just that it will likely be a slog

I don't say UkoeHB did not do good work, maybe there really is no way around this hard brake, but boy will this bring work with it.

*break

oh no trying to get seraphis shoehorned into all parts is already a huge task

yeah billable hours for all!

For all our dozens of devs? Hurray!

vtnerd: it should be possible to completely deprecate wallet2 aside from wallet file migration, so hopefully that helps

yeah it always sounds good until you realize it just means you become overworked

:)

yes, we can move stuff to the side but it always have to be around somewhere sadly

But well, maybe the introductin of RingCT looked similar at start, and we could do it after all

or maybe we just make wallet2 forcibly sweep all coins, and then start fresh

Hiring freezes and layoffs have started in the tech sector. Could be a good time to do some talent scouting.

vtnerd: it won't be necessary, the new library can do full legacy balance recovery + spend legacy coins

That's the spirit.

or will, when I'm done working

ok you are ambitious. clear out your calendar lol

I think that alone will be a very big plus. Thank you for that.

Is there any discussion about changing the crypto libraries (and therefore the serialization)? What if we took a LibSodium or OpenSSL libraries?

dangerousfreedom: no the monero crypto library is too deeply embedded

Maybe that would also be quite risky and would need very, very good testing.

we need the raw crypto library for optimized implementations anyway

I believe we should consider enforcing some encodings at least to be compatible canonical points and scalars for Seraphis. As in my opinion this presents a far bigger risk than quantum computing.

(If we are only planning to change the library for post-quantum cryptography schemes someday)

vtnerd: I think it would be even more work if legacy stuff wasn't fully integrated, because there are balance recovery overlap problems around the transition.

you'd end up in a situation where wallets need an instantiation of both wallet2 and spwallet

dangerousfreedom: yes everything is already enforced to some degree

prime subgroup isn't enforced except where necessary for efficiency reasons, but crypto point/scalar serialization should be enforced everywhere

ok that's the end of the hour, so I'll call it here; thanks for attending everyone; I'm happy to answer any further questions about seraphis in this channel (as usual)

Thank you :)

dangerousfreedom the serialization for crypto points just uses ed25519 standard form. every ed25519 library uses that same format afaik, and it works just fine (albeit somewhat slow to unpack the x-coord)

the serialization we have to is tracking multiple public_keys, integers, etc

monero has used a custom implementation that is strictly ordered, such that no "tag" or "value" information is serialization - all fields must be in a specific order

so ASN.1, msgpack, and all kinds of binary formats are "less acceptable" because they bloat the binary size a bit more with field type/name information

Im not sure of a comparable standard serialization format off the top of my head, although I will search for one to help with seraphis/ukoehb

but we'll probaly just continue with custom, because a strictly ordered format breaks golang, javascript, and a whole bunch of other languages that assume unordered fields in their framework

vtnerd: Oh, thank you for the detailed explanation.

vtnerd: by crypto serialization I mean whether a point can be decoded from 32 bytes or if a scalar is in reduced form (sc_check())

ok, so the plan was to re-use `cryptonote::serialization` for the fields? because I thought that was part of the discussion as well

there's nothing wrong with using it, just that so few other projects can unpack because its so linked into monero atm

no I'm only talking about how crypto stuff is represented in 32-byte buffers, nothing to do with saving to file or whatever

there are invalid 32-byte sequences which we want to detect

ok, I vastly misunderstood then. you think a custom serialization for this will be better? if it could unpack/detect without doing the full mult_by_group_size ?

I think you're the expert and should decide lol

all I do is add checks like this github.com/UkoeHB/monero/blob/24ef2…src/seraphis/tx_validators.cpp#L192

Storing points multiplied by 1/8 does th trick.

yes all the proof points and scalars are validated in the proof verifiers

oh then you do a mul8 immediately after unpack? which also does a clear

e.g. github.com/UkoeHB/monero/blob/24ef2…aphis/sp_composition_proof.cpp#L275

vtnerd: no you do mul8 in the verification code so the verifier knows it is valid

Voilà :)

like here: github.com/UkoeHB/monero/blob/24ef2…aphis/sp_composition_proof.cpp#L303

and here: github.com/UkoeHB/monero/blob/24ef2…04ba7/src/seraphis/grootle.cpp#L200

and here: github.com/UkoeHB/monero/blob/24ef2…eraphis/bulletproofs_plus2.cpp#L953

<moneromooo> "Storing points multiplied by 1/8..." <- I just want to make sure that it is true :) I mean, all the points stored are really multiplied by 1/8 and the miners reject the blocks otherwise.

And if it is a scalar, then it has to be canonical

Using these sc_checks

I hope it's true, we do that in, er, BP I think. sarang approved, so it's got to be right.

As for sc_check, it's very fast IIRC, so a check on load is fine.

well ok, I see what you want. I will dig into ristretto and decaf for some ideas ..

vtnerd: this stuff doesn't impose a requirement on the wire format

maybe I am misusing the word serialization (or causing confusion)

ristretto still requires that you point-deserialize (ge_frombytes_vartime() equivalent) to check if a point is a valid representation

There is a big difference between writing it in the code and verifying what is actually in the blockchain. Everyone can modify the software and create custom transactions with whatever he wants inside.

or at least I think I see

if there is a compressed canonical form, then the issue is handled immediately ... but damn lots of stress on that code being correct

I just think it is a really bad idea to allow that.

* There is a big difference between writing it in the generation code and then verify what is actually in the blockchain. Everyone can modify the software and create custom transactions with whatever he wants inside.

I'll once again comment ristretto is faster to decode than a decompression mul8 :p

So if our comment becomes to save everything inv 8, we can optimize proving AND verifying by ristretto

ristretto is just for the compression/decompresion stage ... ? or did I misunderstand how that works

because I thought it was ed25519_group->ristretto_group->compressed_format

and then the reverse. but there wasn't necessarily anything being done in the compression group

kayabanerve[m] ^

btw, you need a strictly ordered serialization format for transactions at least for hashing purposes

ah yeah good point, cryptonote solved both issues

Encode/decode

Where encode decode includes compression

yeah, we should likely switch to that format then because there probably isn't a penalty

There is compared to raw. There just isn't compared to torsion safe raw

And be making it default...

I put all the hashing of tx content in-code instead of hashing the serialized version

in order to use the transcript system

We get the safety and ease of use of not dealing with this

so it does an in-place hash instead of serialize then hash?

hmmm probably faster I guess

I have nits with xmr serialization but overall it's fine, and not something I'll suggest changing

no github.com/UkoeHB/monero/blob/24ef2…eraphis/txtype_squashed_v1.cpp#L185

it's faster if you already have a parsed transaction

If we did one thing, I'd like to remove duplicated lengths and replace some variants with fixed bytes

s/variants/varints/

Relating to tevador's comment, protobuf isn't ordered which is stupid

er maybe yes if that's what you meant by in-place

this kind of hashing is probably fine for a Fiat-Shamir transform, but not for getting the TXID

tevador: Care to clarify?

if we had a non-ordered serialization format, then e.g. a transaction with 10 fields can have 10! ~ 3 million different representations