hey i was redirected here from the dev room, i have an idea and background to add a feature to the mining algorithm

i was asked to post here from reddit as I spoke with rucknium regarding my proposal

my background is in control theory and applied math. never done this before so im not sure if i should just put my proposal out on the chat to get feedback or wait for more people?

If its something you can share in a few messages, just post it here and give people some time to respond. If its a longer idea, maybe share a link to your proposal?

These rooms aren't too formal, but can be slow on a Sunday evening.

I'm dying to hear your idea though

i can describe it concisely: basically, it should be theoretically possible to automatically select and adjust thread utilization based on CPU temperature. For example, I first started mining on a workstation using a Xeon processor, which quickly rose the temps to 170F alone, but I found there was no good way to really pick the thread count that prevents both excessive temperature rise and fluctuation

my idea is simple: adaptively switch thread utilization based on either CPU temperature measurements (say 150F for example), or alternative sensors such as voltage+current utilized by the power supply or CPU

it would technically be a type of modified bang-bang control, but has some difficult nuances

whats the goal?

1) we need to both regulate temperature and minimize a phenomina called chattering, which causes rapid fluctuations in control actionn or in this cause temperaturechanges 2) it has to work on a lot of different operating systems and hardware 3) there is no good math model available to use "readily available" methods you might find in a textbook

I think this fits into MRL open research question "Increase mining decentralization"

If I understand correctly, your proposal is about an improved control system for mining software, not a change to the actual PoW algo itself, right?

the goal would be 1) minimize hardware degredation 2) potentially increase energy efficiency

yes, correct

i dont have a background in cryptography

Mining is a zero sum game, yes, but auto-tuning rigs for hobbyist miners increase the incentive for hobbyists to mine. Professional miners already have well-tuned rigs, I imagine.

yes, you can think of it as another tool in your tool box

for example, you can use a cheapo tempterature regulation control architecture for your oven, but surely you want a robust solution for a lab grade kiln

It may also make mining feasible on a wider variety of hardware, again increasing decentralization.

This would be a good proposal to improve Xmrig

right now im not even sure if something like this would be wanted

If it would improve efficiency at all, you bet miners will want it.

ill be upfront and say it will take me about a year to work on alone. its not going to be easy, the problem statement is simple but the work under the hood wont be

If funding is an issue, then you could always open a CCS proposal to request funding. You might benefit from talking to the Xmrig devs (maybe open an issue for your proposal)

There are several people here that can help with most of the software/system stuff. Not sure if youre suggesting the control system maths alone would take a year, or if you mean the entire software implementation would take a year.

<Rucknium[m]> "It may also make mining feasible..." <- We should make it extremely easy for a grandpa or a small kid to run monero miner on their devices. It should be made resistant to no just asic or gpus but any high end rigs to ensure every device is compensated proportionate to time and not hardware.

My understanding is that degradation is going to be more of a function of the general CPU temperature, rather than the gradients between cores, etc. Do CPU dies develop a significant enough temperature gradient to warrant this sort of control? If you want to limit degradation I would think you'd just use a better CPU cooler.

I would think that you would want RandomX going all out, and not throttling any resource.

degradation is due to thermal stressess, which caused by both high temperatures and temperature changes in time

for example, putting tempered glass container in the oven then taking it out and running cool water over it

the deformation in the glass causes cracks, which lead to failure

but, that is just one consideration, you can also look at energy usage if you have a means to measure it

you can even, theoretically throttle higher end CPUs to make a 51% attack more difficult using high end machines

you can think of this as an artifical damping

I static OC and undervolt my ryzen cpus so they run cool as a cucumber

<Halver[m]> "Even if we imagine hiding the..." <- Even block order can be shuffled in a cryptographically preserving way. When a new block is mined it is submitted to a secure node , the node will shuffle all the block request preserving the cryptography such that no external observer can decode the correct order of the blocks

and no choosing which CPUs to throttle is not an idea

control theory comes down to manipulating systems such that you obtain a desired behavior. the questions such as your objective, your actuators, states you can observe, and controllable states, dictate what you can do

you would only need to add a damping term that is a function of hash rate, no knowledge of a specific cpu is needed

its just an idea. im not saying it should or shouldnt be done

ok

Hardware degradation probably is not much concern to most miners. (I too run several ryzens, and temps are no concern for me)

Efficiency would get a lot of attention

there are several unknowns with my proposal. i can tell you i work on R&D programs for a living and manage two. I know how to approach big unknown problems

to do an efficiency based algorithm first we need knowledge of what can be measured

for example, are we limited to CPU temps, how strong of a correlation is there between this variable and energy usage, can we measure input power to the rig in real time?

maybe that alone is a good starting trade study

maxwellsdemon: MRL has meetings every Wednesday at 17:00 UTC. We can put you on the agenda if you'd like:

i think i can manage that

is it on this?

Yes. It's just text chat here in Matrix/IRC in this channel. Meetings usually last about an hour.

that should be feasible

only issue i might have is im temporarily taking care of a sick family member, so unless something unexpected happens with them i should be good to go

Ok great. How should I describe your idea/proposal as an agenda item?

adaptive cpu regulation for improved mining performance

thank you

i just had an interesting idea that the adaptive algorithm can be used to resist ASCI development.

ill save it for then

Ok it's on the agenda 😎

bitchin

Tari/XMR atomic swap discussion and request for input 👆

crypto_grampy[m]: Do you have any relation to this?

Nadda. Just saw it mentioned in another channel.

I was going to comment it seems to have some misunderstandings, along with inaccuracies. I was trying to even find out what Tari *is* when all I found out was that info is by no means easily available.

It's a fluffypony production 😅

Anyways. I'll try to leave a GH comment about the issues when I have a moment, and hopefully it's just late at night and I'm misreading this. They propose two methods, one I don't endorse and one I do, yet say the first is akin to Farcaster/COMIT while the second is their own? When the first is akin to noot's work and the second is F/C.

Yeah, when I was trying to see what it was I saw they had IRC logs with a few large Monero names...

I still have 0 idea though. Literally

Like I get it's a private by default chain merge mined with Monero which apparently has some level of scripting. I have no idea how its protocol is designed or what specific decisions it made regarding its tech.

Private by default digital asset thingamahoo

Every single link seems to be a mountain of real time communications, a basic tutorial, or an invite to join their community with no actual specifications or dev docs.

Like their dev docs link literally just talk about running a node. Nothing about scripts/the RPC.

It's not live yet... I'm guessing the docs will come after things are more concrete

The problem with noot-esque designs, despite being more than fine for PoCs AND actually more efficient as they remove a ZK proof, is that they shift responsibility to chains (fine for Tari, not for ETH) and they aren't scalable in the slightest

... flexible. They aren't flexible in the slightest.

Tari is discussing using three separate curves of op codes. What happens when I want to swap shielded ZEC which uses Jubjub and is soon moving to Pallas/Vesta. I just can't?

And they do acknowledge the DlEq proof but it absolutely doesn't appear to be the one designed by the MRL. I meant to ask about that as it's not being formatted as intended so I may just not be able to read whatever shorthand this is properly...

Anyways. Conversation for there, not for here :p It is good to see works on further adoption of swap tech though

Tari is based on MimbleWimble as far as I know, kayabaNerve

What are the requirements for a chain to be able to use farcaster's system with monero ? ie, must support A and B...

Thanks for the heads up

moneromooo: The swap protocol (theory) or Farcaster (impl)?

Hmm... Both I guess.

Theoretically, HTLCs and custom scripting accordingly.

Like, if you have enough Script to impl a HTLC, you're good.

Farcaster is working on a variant of the protocol enabled by Taproot though, which replaces the HTLC with a PTLC thanks to the usage of Schnorr signatures.

If you wanted to implement Farcaster for an arbitrary chain, there should be a set of Rust traits you'd have to implement which could then be routed into a UI selecting building blocks all under that common interface.

See github.com/farcaster-project/farcaster-core

I can't really comment further, sorry, but hope that gives you the info you need. I'm sure h4sh3d or lederstrumpf[m] could comment more.

That's enough for my curiosity, thanks :)

I’d say that we already use PTLC even with ECDSA, that’s why it’s is complicated to have a proper, strong, production impl for non-taproot. Otherwise, after skimming through the blog post (not RFC yet) I’m wondering if the TariScript described disable the first output path when moving to the next one (e.g. refund)

If not then I have a problem: you can race the refund and the buy, both can be in the mempool, leading to both partial keys leaked

That’s why we have an intermediary tx on btc (that is not needed for eth) to invalidate the buy state on-chain

And I’m not sure to understand if they have some game theory to force the refund on xtr

But I have to read more carefully to be sure that my comments are accurate

Nice to see some work on that tho

maxwellsdemon: my opinion is that your stated goal would not be very useful work. There are multiple reasons why:

For starters, CPUs don't really "wear out" significantly over their expected lifespan (~10 years or so) - and even then, this marginal wear is caused by the heat expansion/contraction due to large temperature swings. Mining rigs tipically run in roughly constant conditions (ie temperature) with minimal fluctuations, so technically they experience *less* wear than an "common" cpu performing work in short, high-intensity bursts

The only significant threat posed by temperature is when the chip reaches temps that actually risk melting the circuits (typically ~100°C or so, depending on the manufacturing process and generation) - but if you're reaching those temperatures then your cooling is severely inadequate, and you should not be mining in the first place until you fix that (or doing anything else cpu-intensive, for that matter)

Also, cpus have had overtemperature protections for over a decade at this point, I think. Ie. if they actually get to 100°C they instantly shut down the system to protect themselves from this kind of heat damage

As for temperature vs performance, this is already handled by the cpu/OS as well: if your cpu starts reaching "high" temperatures (typically around ~80°C, iirc), the CPU will start "thermal throttling", ie preemptively reducing its performance to reduce the power consumption (and therefore heat generation). A good mining setup requires adequate cooling (and perhaps some system tuning) to keep the temperatures around 70-75°C *maximum* - which is

actually not that hard to do

If your cooling system is capable of satisfying this temperature constraint, then it will typically have no problem maintaining it under constant load - so the temperatures won't really swing up and down by more than 1-2°C at most unless something in the environment changes (and even then - they will easily compensate for this by adjusting fan speeds)

As for lower temperatures than the ones I mentioned - the heat/temperature difference that we're talking about becomes so small (relative to typical base/idle loads) that I doubt it poses a significant problem

Furthermore, all miners already allow you to (manually) configure the number of cpu cores you want to use for mining (and in the case of xmrig, you even have the option to change this value on the fly by editing the config file without even stopping the miner - the software will automatically detect the change and apply it right away)

So if someone wants to reduce their cpu load, they can already do that manually pretty easily with just a little trial and error

Note that mining is typically bottlenecked by the hardware capabilities of the cpu itself (for example, the amout of cpu L3 cache available on the die) - if you have temperature problems, that just means that you have bad cooling (which, strictly speaking, is an independent issue from the mining itself)

Regulating mining load automatically based on a target temperature might be a "nice to have" feature for people who want to mine on hardware with poor cooling (eg laptops - but those should be avoided for mining due to other additional issues), but it wouldn't really increase mining decentralization per se by a significant margin (imo - let me know if I'm wrong). It also wouldn't help very-low-end-hardware that can't currently mine, because the

real limitation is not the cooling but the actual cpu hardware itself

Sidenote: I don't think your example of the Xeon server is very representative of a typical mining setup for multiple reasons:... (full message at libera.ems.host/_matrix/media/r0/do…d7a410f38659e7e23c5075ff27af40bbccf)

(Sorry for the wall of text, but I wanted to be as thorough as possible with my explanations - hope it gives you a clearer picture of how mining works, from the perspective of cooling/temperature)

A more interesting application of your knowledge of control theory would be looking into the difficulty adjustment algorithm used by the network to regulate mining difficulty for the next block. Our current algorithm uses a simple moving average of the last blocks - which works fine for the most part, but doesn't react optimally in some scenarios (such as sudden, large hashrate decreases) and suffers from (small, at least in our case) oscillations

I believe this should be right up your alley

For a quick primer, have a look at zawy12/difficulty-algorithms #50 and zawy12/difficulty-algorithms #48

(That whole repository is a collection of issues related to mining difficulty adjustments, you might find other interesting stuff/ideas in the other issues)

monero.stackexchange.com/a/7981 - this is our current difficulty adjustment algorithm

The c++ implementation is in github.com/monero-project/monero/bl…src/cryptonote_basic/difficulty.cpp

I doubt there's any perfect solution here. if you try to overdamp the oscillations you'll react poorly to longer term hashrate changes

whatever time interval you tune the adjustments to, an attacker can abuse the algorithm by making changes outside that window

The goal would not be using a SMA, rather than tuning it

*would be not using SMA

regardless

you also lack timely and reliable information

a miner can turn on a mining farm, quickly churn out a few blocks while the difficulty is low, then shut them off again when the difficulty rises

causing blocks to lag until the difficulty drops back down again. then lather rinse repeat for as long as it amuses them.

Issue #50 shows some interesting stuff actually (such as `D = D0 * e^((N*T-timestamp)/M)`, tough it seems it might not be applicable to CryptoNote-based coins?)

and there is no way you can defeat that sort of abuse because you have no way to accurately count the number of active miners nor their relative power

You're still thinking of an algo that doesn't take into account the change in the rate of block production

Derivatives exist for a reason

it could just be random luck

so you can't take a meaningful slope without defining a relatively wide time window

so you can't react instantly.

and the game I just described can be repeated ad nauseum

PID controllers used by drones to stay up would like to disagree with you

You're just listing design goals of the control system

Which are the whole point of control theory

drones are hardly a comparable system

unless you think they can cope with attackers messing with their airflow\

Aka wind

wind is more uniform than that

And yes they can

lmfao

turbulence would like to have a word with you

all the turbulence in the world can still be plotted as sine waves

whereas the games miners can play would be plotted as square waves

good luck with your derivatives for infinite slope

I guess you've just proven that fourier transforms of square waves don't exist, and that digital control systems don't know how to handle them

( /s, in case it's not clear)

lol

well in that case I guess the problem's solved. next topic.

Dude

You're the one who keeps listing design parameters of the control system as if they are impossible things that nobody ever could solve

if it were so simple it would have been done by now

Possible != simple

theoretically possible also != feasible in practice

And since we have someone proficient in control theory in the room, I would like to hear their opinion

particularly with imperfect information

essentially what you want is a low-pass filter, to damp down high frequency oscillations. in terms of an fft, your data samples are individual blocks.

so again, you need a relatively large window to have enough samples to do a meanngful computation

and we don't even have reliable clocks

It's as if you've never heard of the concept of "control robustness"

come back when you figure out how large the sample window needs to be...

sigh

You'll need to introduce game theory into the mix in some way -- which is feasible, since game theory has been thoroughly studied.

Hi, kayabaNerve_ Here is a link to the scripting in Tari: rfc.tari.com/RFC-0201_TariScript.html

Most of our current stuff is the RFCs.

Might I enquire about the misunderstandings and inaccuracies, we would love to fix them.

what is the question, specifically?

<merope> "monero.stackexchange.com..." <- If you could implement a difficulty adjustment algorithm better than the current one ^, which doesn't handle fast changes in network hashrate (particularly drops) very well and is prone to oscillations. The difficulty adjustment algorithm is what ensures that miners produce new blocks at a roughly constant rate (target time: 2 minutes) as the hashrate of the network increases or decreases over time

(or stays stable)

In other words: the goal is that new blocks show up every 2 minutes regardless of how many people are mining (whether it's 1 MH/s or 100 GH/s or whatever). The catch is that we have no way of knowing what's the real hashrate that's actually mining - we can only make inferences by looking at the timestamps of the previous blocks found

(which are not always strictly accurate, and could be potentially manipulated by an attacker trying to mess with the system)

Right now, when the hashrate drops by a significant amout all of a sudden, blocks times become longer than 2 minutes for a while (because the difficulty can't adjust yet, because no blocks are being found, because difficulty hasn't adjusted...)

(not they they aren't being found, but they aren't being found fast enough for the adjustment to kick in)

thats something i can look into, yes. My first question to you would be what type of behavior is needed/what are the performance requirements

consider it wishlist, not requirements

nominally, blocks should be found every 2 minutes

that cannot be assured in the face of sudden hashrate drops, but it would be nice to recover more quickly than currently

the mining difficulty is calculated based on the preceding blocks timing

a miner gets this difficulty value and then goes to work on trying to solve a block meeting that difficulty

if the actual network hashrate suddenly drops, then there won't be enough miners for the difficulty value that was handed out to them

so the next block will take much longer than the nominal 2 minutes

to actually fix this requires the network/nodes to recognize that the next block is taking "too long" and compute an easier difficulty value

and requires the miners to also recognize "too long" time has passed, and query the network/nodes for a new difficulty value that's more reflective of the current hash power

^ that's the ideal solution. not feasible in reality.

wait, are you talking about adjusting the difficulty before the next block is found? (ie. "nobody has found a block in 10 minutes, so let's reduce difficulty for everyone"?)

that would be the ideal solution, yes. but as I said, not possible.

It is theoretically possible I think.

<merope> "Issue #50 shows some interesting..." <- I mean, technically there *is* this solution mentioned in issue #50 that I linked - which only requires the timestamp of the origin block and the current timestamp

You could maybe allow blocks with difficulty 50% of the target one, which would only give, say, 20% of the block reward. And possibly have some rule which gets them orphaned easier than "real" blocks.

So if blocks are found faster than expected, difficulty rises exponentially, and if no blocks are found for a while then it decreases exponentially

But more generally speaking, it's simpler to implement a system based on the timestamps of the previous blocks found

That way, if the last N blocks are consistently taking longer than average, you might consider reducing the difficulty accordingly

with mooo's idea we could just implement a smooth decay. at 2 minutes + N time, difficulty is decreased by N%

(Keeping in mind that individual blocks times follow a Poisson distribution)

(in the absence of adversaries)

timestamps can be fake

yeah I wouldn't base it on block timestamps, but on node receipt time

Pretty sure that's a dead end with the existing consensus model. You need SCP or a similar model based on trust networks to consense on timestamps.

Node receipt time can't be used to determine new diff since they're jiggly (and even worse for historical sync).

hm good point

The "low diff block" thing is "kinda" that: a timestamp that nodes do agree with, that comes before the "real" full diff block.

50% being of course a placeholder amount.

if the real hashrate has dropped tho, the "real" full diff block may be hours away

what youre talking about is essentially an exponentially waitin FIR filter

its definitely doabbe, stability will be a big concern. you can implement a recursive filter which will have a more smooth performance, but it can "blow up"

its certainly an interesting problem. a good first start would be getting access todata sets that emulate the hash rate charactaristics

Note BCH's ASERT difficulty adjustment algorithm. It was implemented in BCH in Nov 2020:

another approach would be to use a least square filter, which is very stable, but can be slow if you have crappy hardware

i would recommend taking an iterative approach - start with a simple solution and refine through design cycles

Slow if you have poor hardware? How?

I suppose you can use block timestamps from late 2017-mid 2018, since we had massive hashrate fluctuations around then due to ASICs

This is as much of an economics problem as an engineering problem, by the way.

you have to compute a least squares solution eachbtime step

requires matrix inversion, it can be slow

it can also be poorly conditioned, which means you have a large difference in eigenvalues of your system. the result is a solution that might numerically divirge

what's the problem trying to be solved i.e. what do we gain from a s more predictable block time? I would imagine throughput is one advantage resulting from fewer empty blocks

Rucknium[m]: it is I agree. essentially we are making decisions based on filtered data, which by its nature means a loss of information

see hyc's reply to me

i think this might be a good thing for the community to have: hash rate data as a time series, maybe other variables too

good for devs anyway

maxwellsdemon: Aren't there very efficient methods of OLS [ordinary least squares]? QR factorization, etc? And we are not talking about very many observations here

the monero-blockchain-stats utility will give you the historical difficulty

though perhaps not as fine grained as you'd want

there are many choices

Rucknium[m]: if you can avoid matrix inversion that is preferred. lots of ways to approach the problem

If we need difficulty of every block, then neptune can probably provide it.

i would look at the rawest data possible, the discrete events of block arrival times, if enough trusted community members have the data logged

and compute the distributions by convolving that to some half a gaussian kernel, or such kind of functions, that the future doesnt contaminate the past

Historical data is going to be of limited use. Change the system and you change the behavior. Still useful, of course.

<maxwellsdemon[m]> "another approach would be to use..." <- How slow are we talking? A few milliseconds, or several seconds?

Keeping in mind that the base process is: compute difficulty from the last block, start mining, someone finds a new block, everyone recomputes difficulty, start mining again, and so on

if historical block timestamps are too unreliable then you just have to run a testnet with real miners

<hyc> "I suppose you can use block..." <- Also the two big hashrate spikes earlier this year

Everyone who syncs the blockchain with monerod will have to verify that the difficulty of each block was calculated correctly, though, right? So speed could be a concern even if it was very fast.

yes

how can you make a reliable estimate of a poisson process without having a large window of observation?

you can't. ... will always need a large sample window ...

Shrinkage estimator.

And now we are on to statistics! My favorite 😍

With a good shrinkage estimator you could do it.

hyc: you can use recursive methods that only dependson the output of the filter, requiring little passed information. A numerical integrator is the most simple example. Again, they can be unstable, which i think is very important for this use case

indeed

im also not using autocorrect on my phone, which is why my typing looks like shit

i dont trust it to not send data

A shrinkage estimator is not unstable. In fact, one could argue that it is the opposite.

A shrinkage estimator will in general reduce variance at the cost of increased (finite-sample) bias.

im not familiar with a shrinkage estimator

what does it do

At a guess, it estimates the age of shrinks ?

Shrinkage estimators are a broad class of estimators. Bayesian estimators can be thought of as a type of shrinkage estimator, for example.

Basically, it "shrinks" the estimate toward some value that is not contained in the observational data itself. If the data is uninformative, the estimate will be close to that value that is outside of the data. If the data is informative, the estimate will be close to what a more standard "frequentist" estimator would yield.

Rucknium[m]: how much data do you need for these real time estimates?

The Wikipedia page is fine:

how many events?

u guys over estimate how shit reality is haha

zkao: What do you mean by this comment? I work with empirical data almost every day.

zkao: I don't know for now. It depends on myriad factors.

the block rate is very low, Rucknium[m]

Compared to bitcoin it is five times faster.