Original title: ARE MINING POOLS BECOMING A PROBLEM?
Original author: Bitcoin Mechanic
Original compilation: Luccy, BlockBeats
Editor’s note: Mining is one of Satoshi Nakamoto’s designs that is both solid and elegant, and one of the most compelling aspects of Bitcoin. However, mining is not just about hashing.
Bitcoin advocate and educator Bitcoin Mechanic delves into the various issues and challenges that exist in Bitcoin mining, with a particular focus on the dynamics between mining pools and miners. He proposed solutions to existing problems, focusing on improving miners’ autonomy and the overall decentralization trend.
Among them, the discussion of the Stratum V2 protocol and the future development of the mining ecosystem is full of unique insights. In addition, the article also points out that recent developments in ASIC manufacturing and mining pool infrastructure provide new possibilities for decentralization in the mining field. Through a critical analysis of existing mining models and an outlook on possible future improvements.
Bitcoin miners provide an extremely important service to the entire ecosystem. They receive rewards from the network as part of ensuring the security of the network, which is one of the most solid and elegant designs designed by Satoshi Nakamoto and is one of the most striking aspects of Bitcoin.
However, more and more people seem to be overlooking that mining is not just about hashing.
Those involved in the entire process must run a node to get the latest state of the blockchain and then start building a new block. This includes verifying the validity of the previous block, discovering unconfirmed transactions and generally selecting the most profitable of them. Pay yourself in generating transactions, build multiple Merkle trees of those transactions, and finally hash to actually solve the block. The transactions in the block template will constantly change as new transactions are broadcast to the network. When someone else finds a new block, miners must instead build on it and dump all the transactions already in the blockchain to populate a new template.
bifurcated activation
As you can see, doing the hashing to actually solve the block is only part of the process. Bitcoin mining ASICs can also only perform hash operations. In the current environment, all other aspects of mining are typically delegated to mining pools. This has led to some confusion. For example, in any discussion of activating a soft fork via version bit flipping within a block template, people will always mention that this process is a MASF, or miner activated soft fork, and someone will always come out and say, This is the sole responsibility of the mining pool, and the mining pool is not the miner. They may also point out that miners are still ultimately responsible since they can simply switch pools if they wish to upgrade but the pool they are in does not. For the sake of clarity, in the rest of this article I will only refer to those people who are only involved in hashing, leaving all other aspects of mining to mining pools as “hashers”.
Going back to soft forks, in the current environment where more than 99% of blocks are built by the same organization, it might be more accurate to call these pool-activated soft forks, although no one is currently calling it that. This leads to a dangerous illusion: mining can be considered decentralized simply because of the distribution of computing power. This argument becomes less credible when all computing power is limited to a few mining pools, so the content of the Bitcoin blockchain will ultimately not include anything that these few entities deem unacceptable, and a Series of other questions.
By merely hashing blocks constructed by mining pools, Bitcoin miners largely give up a key component of their role. That this is not just possible, but the smoothest path, suggests we have a systemic problem.
Mining pools and block space markets
Just doing the hashing and letting mining pools do everything else has implications far beyond soft fork activation. For example, currently miners have absolutely no idea what the resolved block will look like, meaning miners work under the blind trust that the block only contains the expected transactions. But you see a blatant violation of that trust in blocks like this, the famous block that started the “ordinal” craze. Note that the miners working on this block actually only enjoyed about $200 in Bitcoin transaction fees, while the blocks on either side of it had average transaction fees of about $5,000 in Bitcoin.
Block space has value, which is part of the reason why Bitcoin works over the long term. But in a world where only a few players can make the templates they build end up on the blockchain, these entities have almost exclusive rights to sell the space and get outsized compensation for doing so. Do they have a responsibility, maybe even confess to their miners that they are doing this? Of course, in this case, the intention was to surprise everyone. In the future, will they forward the payments they receive for selling out-of-band block space to their hashers?
In short, while the incentives of mining pools and their hashers are generally aligned with the goal of maximizing profits, it is possible for mining pools to sell block space for something other than regular Bitcoin transactions, and miners income More limited, unless the pool chooses to be transparent and agree to share the revenue. Even if they did, verification would require permission from the mining pool, not verification of funds earned from subsidies and transaction fees (also tricky when using FPPS pools, more on that later).
The further influence of mining pools as builders of Bitcoins centralized block template stems from a more basic fact - at a more basic level, there are twelve super nodes with their own super memory pools.
This leads to people dealing directly with mining pools and completely ignoring mempools. Some argue that mempools are doomed to fail anyway. The current state of centralized template building only accelerates this process, but is not advisable in any case, and in a world where truly decentralized template building is considered feasible, this assumption may be too pessimistic to make. Then, if the person who purchased the block space wants it included in the blockchain within the same time frame, the out-of-band payment must be passed on to a larger group of people. This could be more transparent and similar to how things currently work. Instead, supernodes are expected to be broken into smaller parts and therefore no longer able to provide the same guarantees.
To stray away from this mining aspect, let’s shift the focus to how current payment methods are handled.
Mining pool payment model
Almost all mining pools pay their hashers via FPPS (Full Payment Per Share) or similar. The only exception is that ViaBTC offers PPLNS (Payment in Last N Shares) in addition to FPPS. Antpool also offers PPLNS, but the hashers must give up all transaction fee income, which illustrates the point I am about to try to make. Basically, FPPS is not a model that works well in a world focused on transaction fee revenue rather than subsidies. It is worth mentioning that Braiins mining pool (formerly Slushpool) uses a system called score, which is actually very similar to PPLNS.
Why is there such a strong preference for FPPS? From the hashers perspective, they get paid no matter what happens on the blockchain. This is consistent with the purpose of pool mining and the income is more consistent. FPPS provides more consistent payments because pools pay based on projected revenue and settle independently of the blockchain.
This makes life very easy for miners who want to minimize problems due to disruptions in cash flow, but of course there are disadvantages - important ones that I wish to highlight here.
FPPS first requires mining pools to become custodians of all newly mined Bitcoins. These Bitcoins cannot be forwarded to miners for a minimum of 100 blocks after being mined, so newly mined Bitcoins are not available until then. In fact, mined Bitcoins are the same as the miners ultimately receive when withdrawing from the mining pool. Bitcoin has nothing to do with it. The risks of escrow should be obvious to almost anyone reading this article, so I will skip that and move on to other issues with FPPS.
The next concern arises from the more general issue that FPPS mining pools are an important intermediary between hashers and the network itself. We have established that hasheres have no way of knowing in advance what the block they are working on will ultimately look like until it is resolved. FPPS means they now dont even care if the block is discovered, its the mining pools problem. Ignoring the increased predictability of payouts (which would never happen if a mining pool decided to cheat hashers), we have to acknowledge the trade-offs of doing so.
Miners who are paid directly by Bitcoin itself - which is possible in alternatives like PPLNS or of course independent mining - can expect to be fully rewarded, including transaction fees. An FPPS pool can only calculate this after the fact because there is simply no way to predict what the fees will be when determining how much fees a hasher actually receives per share. Mining pools cannot simply assume that the fee will be some value greater than 0 and include it in the miners income when mining, because if the fee is lower than this value, they will simply pay the miner out of their own pocket. They must regularly allocate fees and attribute them to the miners who actually hold custody of the pool.
From a hashers perspective, complete trust in the mining pool is required because without complete transparency and cooperation from the mining pool, verification is nearly impossible. As mentioned above, this wasnt a big issue before because most mining revenue came from subsidies and transaction fees were just a fraction of the satoshis sprinkled in, but this is not, and cannot be, the growing demand for Bitcoin mining. The future of mining. In the future, miners will earn revenue primarily through transaction fees, which are harder to predict and monitor than subsidies when using mining pools.
Compared to payment schemes like PPLNS, where hashers accept greater changes (and the luck of the mining pool becomes the hasher’s luck), we see that the mining ecosystem is extremely inclined to prioritize payment consistency over Not the ability to verify what is received. To add to the twist, some hashers actually prefer this approach - wanting to present themselves to government agencies as a hash service that is completely separate from Bitcoin - some are even proud of it. This is because FPPS is so far from the ideal miner/mining pool dynamic that its difficult to describe what exactly hashers are doing, even Bitcoin mining.
In reality, the FPPS mining pool is a large independent miner that pays hashers to solve its blocks. Afterwards, they have an internal and opaque process by which the amount paid to the hasher is determined. To really drive home the point, hashers could even (in some not-too-hard-to-imagine scenarios) pay their fees with something other than Bitcoin.
why not? If you dont care if any blocks are found, let alone what they look like before being built, then why not just get an independent miner paying you fiat to point your ASIC to them in whatever currency is most convenient? Bitcoin is not always the most frictionless option, but even if it is, it is reasonable to imagine continuing down the next path, where the hashing can be performed by as many entities as you like, but all on behalf of a small group of miners. Pool is done, and the entire network needs their permission to actually record anything on the blockchain.
Who is doing the hashing anyway?
Lets look at this issue in a broader context. We have already mentioned that some of the larger players want to stay as far away from Bitcoin as possible and are therefore willing to delegate as much Bitcoin-related activity as possible to their mining pools. These mining pools are open to regulation and their large amounts of computing power are perfectly happy with it.
This again introduces economic irrationality from the perspective of the network itself, manifested in the behavior of mining blocks that meet some arbitrary criteria. When this has happened in the past, it hasnt lasted long due to community backlash and the absurdity of trying to actively please a jurisdictions ever-changing regulatory regime without being required to do so. But the fact that this is a chosen option exposes the risks of having a centralized block template build. Will miners in one jurisdiction try to ban or refuse to process transactions from another jurisdiction? Are miners simply an extension of bad behavior from governments or influencers? There are concrete examples of some mining pools rejecting transaction fees to profit illegally, sometimes simply to comply with regulatory pressure. From a network perspective, this again seems economically irrational.
The most extreme recent example was a 19 BTC transaction fee paid on a transaction that was ultimately discovered by F2 Pool as an ostensibly error. As an FPPS mining pool, they became the custodian of these 19 BTC mining fees and chose to return them to the person who made the mistake. This perfectly illustrates the cost of placing too large an intermediary between miners and the Bitcoin network. In the PPLNS mining pool, this is less likely to happen. Not because PPLNS mining pools are necessarily trustless or non-custodial, but because fee income can be monitored and verified at the exact moment a block comes in, which may be harder for a mining pool to attempt as it may already be internally reporting to miners accounts deposited their share of the mining funds earned, causing an even greater backlash. Although in principle there isnt any difference, until you contrast what would happen if a mining pool paid its miners in coinbase/generated transactions. In that case, the money is already in the hands of the miner and the mining pool cannot intercept the fee income. So, in this example, the mining pool trying to appear generous or fair ended up costing its miners $500,000 in fee revenue, a decision it should not have been in.
Next question: 51% attacks and other attacks
This should be easy to explain: everyone knows what a 51% attack is by now. However, what is far less understood (until networks bypass it) is that 51% is a requirement for this type of attack to be guaranteed and consistently successful rather than just destructive.
Realistically, any entity with more than 20% of the network can cause problems through a number of attack vectors, some of which are executed in the wild but rarely discussed, and Ill go into more detail later. But before that, we can see that there are only two entities in this network whose total hashrate reliably exceeds 51%. To make matters worse, one of the largest mining pools did not carefully conceal the additional 10% of blocks it discovered through another large mining pool that maintains a strategic relationship with its parent company. The fact that this farce is still going on is unconvincing.
There are two common responses to this. First, it was pointed out that miners could vote simply by changing the mining pool if they banded together to carry out a 51% attack. Secondly, it would be crazy for any mining pool to try to do this, for the simple reason that disrupting Bitcoin will cause the price to fall, and no one investing in the ecosystem wants that to happen. The second argument ignores human history and further assumes that people are never forced to act destructively, causing destruction simply for the sake of destruction or for other nefarious purposes. It also fails to take into account that the market is not always a good indicator of problems with Bitcoin, see 2017’s hard fork wars.
However, the first argument is stronger and assumes that in the event that a pool does become too large, miners will always switch to other pools. In fact, if a mining pool tried to do this, reality would step in and we would realize that despite building 99% of the block templates, the mining pool is not actually a miner. One famous case study is Ghash.io, which suffered a fatal downward spiral due to over 40% of its hashrate.
So, so weve proven that this isnt actually an issue and miners can be trusted to switch to other mining pools. Realistically, this assumption is less reliable if large mining operations are tied up in red tape, but lets at least assume were fairly confident that such an attack is unlikely to happen.
Unfortunately, the realization that any mining pool’s hash power will migrate elsewhere once it crosses a terrible threshold causes them to self-regulate. But this approach doesn’t help because they don’t have to actually maintain computing power below a threshold, they just need to create that impression. This basically amounts to accepting all the hash power they can get while forwarding it to other mining pools to avoid reminding the world of the chaos they are capable of causing.
In this way, we have an unknown picture of what is going on with the network. 30% of the blocks can be blatantly found by the largest mining pool and accepted by everyone, while the other 10% of the total network hashpower is still pointed to that mining pool, just secretly transferred to one or more smaller ones Mining pool. The miners responsible for this 10% may not be aware that it is used this way, and it is much harder to detect with stratum V2. I will explain it in detail later.
This already undesirable situation becomes even worse when you consider that this redirected computing power can be exploited to the detriment of smaller mining pools through block mining attacks.
Specifically, the attacker mainly participated in the mining process as a normal user of the victim mining pool. As a result, they receive a portion of the reward from any block found by the mining pool as expected. The reward eventually flows to the attacker, who can then pay the actual miners without losing any funds. The only damage done so far is a false impression of mining pool hashpower, making it appear smaller than it actually is, but smaller pools remain unscathed.
Now, if the attacker decides not to tell the victim pool when a block is found, damage can occur. This can make the victim pool look unlucky. They just seem to find fewer blocks than they should and pay out rewards to more participants who are actually mining honestly, i.e. inevitable losses, assuming they dont make up for those losses in other ways.
If FPPS mining pools were attacked in this way, they would have to burn revenue and pay miners out of pocket to make up the difference. If its PPLNS, miners will wonder why they didnt get what they should have. Either way, block mining attacks are anti-competitive and could destroy the victimized mining pool by giving it a bad reputation.
From the attacking pools perspective, lets assume they account for 5% of the victim pools computing power. This means that they still receive 95% of expected revenue, while the mining pool looks to be 5% less lucky than expected. This is enough to easily destroy a mining pool, whereas in a larger mining pool a 5% loss in redirected hashrate will be far less significant. If it represents only 1% of the total hashrate of the larger mining pool, then the attacker has only lost 5% - 0.05% of 1% of the expected reward. This is a clear advantage for any malicious, sizable mining pool prepared to act unethically.
The smaller the mining pools, the more vulnerable they are to this attack. The bigger the mining pools are, the more likely they are to block competing smaller pools. This risk increases as large mining pools approach a level where their total hash power starts to cause panic in the community, which further motivates them to at least hide their hash power in smaller pools, even if they wont actually use it for attacks or attacks. is not frequent enough that the problem ultimately boils down to variance. Larger mining pools have enjoyed reduced variability as the network offers more consistent payouts, meaning they are able to operate within tighter margins and thus be able to charge their miners less. From the perspective of each unattacked miner/pool, this attack means they will enjoy lower difficulty as the Bitcoin network adjusts to fewer total blocks.
Is block holding merely theoretical? Absolutely not. Even in early 2015, several mining pools were attacked in this way. Preventing this attack is very difficult because mining pools have to monitor all miners and make a thoughtful decision whether to kick them out of the pool and/or refuse to pay them if their luck reaches the point where it is statistically impossible for a miner to The pool could reasonably believe they were acting maliciously. Such attacks also incentivize mining pools to want to “know their miners” and regulate payments, which of course makes life more difficult for those who wish to mine permissionless.
Anyway, the overall effect of all this is that people prefer to mine with larger pools for another reason.
We have heard large miners publicly stating that they are abandoning smaller pools because they are not being paid what they expected.
This is highly undesirable because larger mining pools and the larger miners who use them are more susceptible to regulatory burdens and therefore more susceptible to engaging in behavior that harms Bitcoin, even beyond the centralization and manipulation of block templates. Temporary escrow of all block rewards.
Mining pools effectively become agents, imposing bureaucratic nonsense on behalf of their miners. Currently, the two largest mining pools require users to jump through a bunch of tedious steps, including those that reveal their identities, which should not and must not be something people mining Bitcoin have to go through outside of solo mining.
One final point about block withholding, aside from the threat of making life more difficult for smaller mining pools and those who wish to mine with them, I say to those who may still be tempted to dismiss this as pure theory, even if it Having obviously happened in the past, do we think that mining pools will naturally be able to maintain a consistent and apparently tolerable size? This means that new hashrate coming online is always distributed somewhat evenly. We have to believe that a mining pool can suddenly appear, grow rapidly, and then stop right there around the threshold required before people get upset. Do we see mining pools asking people to stop mining with them, or simply limiting account creation and kicking out miners who exceed the allowed hashrate within their existing accounts? of course not.
The two more likely scenarios are that miners are collectively regulating themselves. But this is unlikely, as it is known that mining with smaller pools now means miners earn less Bitcoin, even if the reasons I presented in this article do not fully explain why. Not to mention that every instance of mass migration from one mining pool to another is very high profile, or that the mining pool is simply misleading people about the hashrate they are pointing to.
In addition to this, smaller pools have another problem: they may not find a block for several days in a row, while larger pools may not find a block for more than a few hours. This is a resolution issue, the higher your computing power, the closer you are to expectations in the short term. Unfortunately, this leads to a lower limit below which mining pools cannot hope to recoup their losses during the doom, at which point it becomes impossible to compete.
The two-week period between difficulty epochs means that enough blocks must be found within those two weeks so that any bad luck has a chance of being balanced out by subsequent good luck. If not, for example, if a mining pools expected block rate is 1 block every 13 days and no blocks are found before the difficulty adjustment upwards causes them to drop to a projection of 1 block every 15 days, then the time before The window will be closed forever. In the case of the PPLNS mining pool, miners earn less than they might otherwise. In the case of FPPS pools, the pool will burn through a lot of cash and/or go bankrupt.
This means that there are only so many mining pools that can exist, at least the way mining pools operate today. Simply put, there cant be hundreds, as many of them may crash during adverse times, and since they hold less than 1% of the networks hashrate, they may not even be able to reliably find a block every day, encounters that may not happen for weeks. The period of the block. This is a limitation that Bitcoin itself imposes on us.
How do miners and mining pools communicate?
The communication protocol between miners and mining pools is Stratum (which is gradually being replaced by Stratum V2). Stratum V1 is both ancient and deeply flawed. First, all communications are transmitted in clear text. This means that ISPs not only know that you are mining, but also the scale of your mining. Anyone who can eavesdrop on your network traffic can perform a man-in-the-middle attack, causing your computer and computing power to be used for someone elses mining. This has been abused before by unknown attackers in order to steal hash power away from the intended mining pool.
Besides some inefficiencies, Stratum V1 failed to provide a practical way for miners to build their own block templates and still mine in a mining pool. All of these issues were addressed in the very attractive Stratum V2 (originally called GBT and then Better Hash), which we will return to later.
Hardware/firmware
Before we discuss the pool/miner dynamics, we’re going to deviate a bit, as this article would be incomplete if we didn’t mention that there are only two companies making ASICs on any meaningful scale, namely Bitmain and MicroBT. There are other companies, but virtually all hashing happens on machines built by these two companies.
This is obviously bad, and the fundamental reason is that chip manufacturing is very difficult and therefore too centralized.
It is beyond the scope of this article to discuss solutions here, but there are people working on making home mining more practical (in North America the main issues are the need for 220-240 volts and dealing with the harsh noise). Those working on the pleb-mining project argue that if enough ordinary Bitcoin users can do this, they could start to account for a significant proportion of the networks total hash power, which would be necessary for most mining operations operating at the scale are preferable because they are more susceptible to regulatory intervention.
The difficulty of this task is that the firmware is closed source. Even custom firmware that can jailbreak an ASIC is often closed source to ensure that the user pays the development fee, that is, the cost of the market firmware you use represents the mining team.
Stock firmware on ASICs, especially Bitmains, is a good indication of how confident they have become in their market dominance. Besides being closed source, its clearly malicious. When launching Antminer you are forced to mine for them, although the miner can at least prevent this from happening by blocking connections, or installing market firmware, and then paying development fees, but these fees cannot be prevented, otherwise the miner will refuse to mine. Bitmain has been caught adding malicious backdoors to the firmware of its miners on several occasions (see Antbleed) and actively works to block firmware developers from the market.
The fact that factory firmware does this is truly shocking and clearly highlights the desperate need for competition in ASIC manufacturing.
Would anyone feel comfortable if network rules were enforced by closed-source Bitcoin nodes? Also imagine if we all knew that these nodes would cause users to lose Bitcoins to the developers of the software, would anyone be okay with that? When it comes to mining, almost no one cares about the sovereignty of the participants. Of course, node software and ASIC firmware are not of equal importance, and we certainly focus more on the former, as we should, but the latter is not irrelevant and is clearly overly neglected.
Having said all that, let’s move on to some solutions, specifically focusing on expanding the range of miner possibilities and improving existing models.
P2P OOL
There’s not much to say about this except that it basically decentralizes every aspect of pool mining. While this does a lot of ideal things on a small scale, it requires each user to download, verify and track every other users share, and prove to each other that they calculated everything correctly in the template. Achieving this in a confrontational environment of any scale is essentially an impossible task. Due to the basic nature of pooled mining, the resources required are significantly more than those required to run a Bitcoin full node, not to mention making the miners job more complicated.
For these reasons, it is ignored by most people and only used by more technical users or idealists who understandably cannot bring themselves to use other alternatives to mining.
STRATUM V2
This is undoubtedly the easiest problem to solve, and it provides practical solutions to many of the problems mentioned in this article.
First, by allowing encrypted communication between mining pools and miners, ISPs and any other entities with access to your network traffic will no longer easily realize that you are mining, or to what extent you are mining. Therefore, MITMing you to hash on the attackers behalf becomes impossible, or less easy.
Secondly, and perhaps most importantly, it will also allow miners to build their own block templates, so while mining pools will still be trusted coordinators of reward distribution and may still be custodians of block rewards, this will Representing power from mining pools to miners is undoubtedly a good thing.
In a world where Stratum V2 is the standard, miners are keen to build their own templates. Ideally, mining pools will provide incentives for miners to build their own templates, which will have a more powerful Bitcoin.
The community is largely united in its efforts to upgrade the mining ecosystem to Stratum V2, but historically miners have generally avoided these solutions due to the extra effort (albeit negligible compared to p2p ool) and lack of incentives .
With or without Stratum V2, there is a lot of room for improvement. What is needed is a mining pool that provides miners with the ability to directly host their coins while mining. This requires a mining pool (or its miners) to build block templates in which miners rewards are paid directly into the coinbase/generation transactions included in each block. Under the FPPS system, this is impractical in practice, meaning that any mining pool that implements this approach will face some miner reluctance, but those who convert will enjoy greater transparency, as Bitcoin itself Payments will be made directly to them above a certain threshold, with easily verifiable revenue allocations for subsidies and fees. This could be used in conjunction with mining pools, even before Stratum V2, to at least give miners an idea of the block template being built for them before solving a block, and after Stratum V2 just verify that all miners builds accurately reflect the reward distribution template without all miners having to do this continuously.
Mining pools can also address miners reluctance to do so by providing incentives for miners to make their own block templates, such as by charging them lower fees. It appears that this additional incentive may be necessary if miners are still unwilling to shoulder the burden once it becomes practical again.
The above suggestions will greatly improve the situation.
Many new plans and announcements are emerging regarding ASIC manufacturing and mining pool infrastructure, which should be a welcome development for anyone looking to ensure that mining moves toward greater decentralization.
