<ack-j:matrix.org> Claude review of the paper:

<ack-j:matrix.org> paste.debian.net/hidden/dde67217

<jpk68:matrix.org> Have people come to some consensus on what should be done with regards to findings detailed in the previous paper?

<jpk68:matrix.org> The one by the same authors (I think) that had to do with Tor

They did report their findings on HackerOne in advance, but when asked to test a patch in their simulation they never followed up.

<gingeropolous> is the patch already in the code?

not yet

it's not PRed yet

<gingeropolous> ah ok. i was gonna instruct my claude code to go through the paper and then reproduce the sim in monerosim why not

<gingeropolous> well still gonna do that. but could also test the patch

<gingeropolous> dangit got flagged. opus 4.8 it is

<ofrnxmr> @gingeropolous: U tried fable and expected it not to be? 🙃

<ofrnxmr> Fable will flag if you ask it if a chefs knife should be put in the dishwasher

<slowbeardigger:matrix.org> Grok build is more useful than Claude for security

<isaac_newton:matrix.org> Chat GPT is not bad recently

AI in general are not good

<isaac_newton:matrix.org> if you have that attitude then you probably arent testing the latest models or have much of an understanding if thats true or not

AI is subject to random errors, which it makes a substantial quantity of

<isaac_newton:matrix.org> Yes we all no those humans are error free.

<slowbeardigger:matrix.org> it hallucinate a lot, even today

<slowbeardigger:matrix.org> but you can’t just blindly trust AI

Sadly, many do

<isaac_newton:matrix.org> @slowbeardigger:matrix.org: Apparently you can!

<slowbeardigger:matrix.org> That’s why you must review, read, check, double check etc, and know when is best to use the AI, just a tool after all

<slowbeardigger:matrix.org> @isaac_newton:matrix.org: well yeah but

<slowbeardigger:matrix.org> If you got a head, use it xD

<isaac_newton:matrix.org> If youre using ai with the thought that "this is a perfect tool which makes no mistakes and I wont have to use any effort" you arent having a realistic outlook on what AI is or how to use it

<slowbeardigger:matrix.org> I was one of the ones who blindly trusted AI spit time ago

<slowbeardigger:matrix.org> @slowbeardigger:matrix.org: Then well, reality + experience + not being fucking stupid

<isaac_newton:matrix.org> I could read 100 coding books, how long would that take me? I can start working in the same night with those 100 books with AI.

<slowbeardigger:matrix.org> :p you can do decent stuff with clankers

<isaac_newton:matrix.org> And AI is the only way to do that.

<slowbeardigger:matrix.org> @isaac_newton:matrix.org: Rigthio

Yeah, good luck doing that with any codebase of significant size and complexity

<isaac_newton:matrix.org> For some people, their only chance of becoming a coder or researcher, might be with assistance. I see AI more as a butler or organizer.

<isaac_newton:matrix.org> tusko: Thats where learning how to use AI comes in handy, which if you have a hatred for it you are starting off late.

People who use AI to write their code have very little chance of achieving that goal

<isaac_newton:matrix.org> tusko: Is that from the CIA factbook?

You tell me misses officer

<slowbeardigger:matrix.org> tusko: that’s why you need to know when is best to use a clanker

<isaac_newton:matrix.org> do I give off feminine energy?

<slowbeardigger:matrix.org> lmao

<isaac_newton:matrix.org> i dont think there are any bees in this chat, dont worry.

<isaac_newton:matrix.org> we should probably be using the longue for this type of conversation

<slowbeardigger:matrix.org> the thing is, AI is ok to use, if you dont use that, also fine.

<slowbeardigger:matrix.org> a fucking tool

<slowbeardigger:matrix.org> there was a time that VSC had no autocomplete and basic shit

<slowbeardigger:matrix.org> There was a time when using VSC was retarded

<isaac_newton:matrix.org> its like calculators. Remember how much they used to seethe about how we wouldn't always have a calculator available. Now its pretty acceptable.

<isaac_newton:matrix.org> One of my co workers reminisces about how you used to get your fingers hit with a ruler if you counted on them. Sadism is only for the enjoyment of the abuser, it doesn't benefit the victim at all.

<slowbeardigger:matrix.org> Just don’t stop using your brain.

<isaac_newton:matrix.org> @slowbeardigger:matrix.org: Trust me that wont happen, but I think some people would prefer if I stopped using my brain.

Its usually very obvious after a bit if someone heavily relies on AI for their code

<isaac_newton:matrix.org> research often requires testing. Sometimes reality testing. :)

<slowbeardigger:matrix.org> tusko: true

Human brain is good at detecting the types of inconsistencies that prevail in such a regime

<isaac_newton:matrix.org> tusko: What if the AI is being instructed to act with best practice based on actual coding books

<slowbeardigger:matrix.org> tusko: also true

<slowbeardigger:matrix.org> @isaac_newton:matrix.org: you will notice

Still it will fail. I could give you an example.

<slowbeardigger:matrix.org> brain automatically says this shit is not human generated

<slowbeardigger:matrix.org> Like random articles or shit with em dashes

Or they don't follow common software development practices

And then you ask "would someone who wrote something like this really make a mistake like that?"

<slowbeardigger:matrix.org> tusko: there is code that sometimes is difficult to understand or read, not optimal for “human” just to say it

<isaac_newton:matrix.org> I dont think it matters what the code looks like and in the future people wont care if you use AI to code, it will probably become expected.

<isaac_newton:matrix.org> Like trying to build a house with your bare hands is considered foolish.

<isaac_newton:matrix.org> Yeah, you can really tell when industrial materials are used to build homes. You're right.

<isaac_newton:matrix.org> The thing is, no one will care.

<slowbeardigger:matrix.org> yeah and the tool will get good enough

I think quite the opposite might occur. AI makes random errors. Other AI train on material containing random errors and makes other random errors. ... compunding errors

<slowbeardigger:matrix.org> not to blindly trust it but the quality gonna be good

<isaac_newton:matrix.org> ive been using AI for a number of years a bit too much, trying to use it to code, and I can say that its currently in a state where it is good enough.

We will likely have semantic errors all throughout codebases before too long

<slowbeardigger:matrix.org> We learn it learns

<isaac_newton:matrix.org> @slowbeardigger:matrix.org: obviously blind trust never works, the AI needs more and more context only a human can give

AI doesn't learn

learning is a human activity

<isaac_newton:matrix.org> you're not going to build a massive application by sending one intern to go get coffee, thats the equivalent of thinking one prompt will build your app

<isaac_newton:matrix.org> tusko: objectively false

<slowbeardigger:matrix.org> Even if saying today that AI doesn’t learn is somehow valid

<slowbeardigger:matrix.org> just a matter of short time

<isaac_newton:matrix.org> I guess you can get really pedantic, AI condenses information. and it needs input to do so. So whatever it is, resembles learning and grows like learning does

I just fundamentally disagree but we're in some kind of philosophical question now

I can cede that you can make philosophical arguments about "what is thinking" "what is learning" etc and thats a deep rabbit hole

<isaac_newton:matrix.org> I could make a number of philosophical arguments, ill try to avoid that here.

please take it to lounge, not a lab conversation