Protocol Pandemonium
Three Things #122: June 2, 2024
A lot of thought and hard work went into the design of the Spacemesh protocol. This effort was necessary to achieve something unprecedented: a blockchain mined collectively, collaboratively, and sustainably by thousands of ordinary people around the world running the software on their home computers. Achieving this required designing and building some entirely novel software, including the way Spacemesh miners establish eligibility to produce blocks, which we call proof of spacetime.
This eligibility mechanism, known as a Sybil-resistance mechanism, is crucial for ensuring that only legitimate miners can submit blocks to the Spacemesh network. Misbehaving miners are punished by canceling their identity and preventing them from creating new blocks or earning more rewards. To guarantee that every miner gets at least one slot and one reward per epoch, we require every miner to generate and gossip an eligibility proof every epoch.
Initially, we designed Spacemesh to handle up to about 10,000 miners and 10,000 such proofs per epoch, thinking this would suffice for the first year. However, the growth of the Spacemesh network has been breakneck almost since genesis, far exceeding our expectations. While our forecast for the total amount of space committed to the network was roughly accurate, we underestimated the number of miner identities due to how miners manage their nodes and storage space. Today, there are millions of miner identities, and the number keeps growing.
In a sense it’s a good problem to have, and we’ve managed to keep the network alive thus far through heroic engineering. But as the number of miners continues to climb rapidly it’s beginning to put a serious strain on the network. Everything has a limit; the number can’t continue to grow forever. So, what can we do about it?
Here are some strategies we might take to fix this problem once and for all.
Please note that these ideas represent my own personal exploration of the possibility space and possible future paths for Spacemesh. Most of these ideas are not currently on the roadmap and most never will be; I’ll specifically point out the things that are. For our official stance on the issue and our immediate plans to address it, please see Reducing Network Load.
Many of these ideas would require significant changes to the Spacemesh protocol, a great deal of additional exploration and discussion, and many months if not years to design and implement. Please also note that there’s an important distinction between a “miner” (as in a person or other entity) and a “miner identity” (which refers to a single instance of the Spacemesh node software managing a single miner public key). For the purposes of this article and in order to keep the writing concise, however, I will simply refer to “miners” rather than “miner identities.”
Thing #1: Reduce the Number of Miners ✂️
The most obvious way to address the issue of too many miners is to reduce their number, but this isn’t a trivial thing to achieve in a permissionless network. While we can’t stop new miners from joining, there are a few strategies we can explore.
Currently, many large miners in Spacemesh split their storage into numerous small identities for various reasons. Some of these reasons stem from the current design of the Spacemesh protocol, where proving space is more effective with many small plots rather than one large one. Others arise from limitations in the current implementation, such as the need for parallel proving and handling disk failures.
We’re working on important protocol improvements to remove most, if not all, incentives for splitting a large identity. If we also offer incentives to merge identities, we expect a significant reduction in the number of miners as large miners consolidate their identities into one. This is an ongoing effort. However, exploring additional ideas is worthwhile for defense-in-depth engineering, in case we encounter unforeseen issues with the proposed design, or need quicker workarounds.
Another approach is to raise the minimum amount of space a miner needs to commit to join the network. This is currently 256 GiB, as it has been since genesis. Raising this 10x would drastically reduce the number of miner identities. Ideally, we’d also enable miners to merge multiple identities into a single, larger identity. The downside is that small home miners would no longer qualify. Since supporting small home miners is a core value, we’d need to ensure they can still participate by joining a pool to pool their storage with others, or by mining at L2 (more on this later). Unfortunately, we can’t grandfather in existing miners due to the protocol’s inability to determine a miner’s start date. To do so would encourage new miners to pretend to be old miners.
Alternatively, we could drastically increase the difficulty of creating new storage, making it, for example, 10x harder (so a plot that takes one day to generate today would take ten days). An in-protocol difficulty adjustment mechanism could automatically increase or decrease the difficulty to target a specific network size. The challenge here is once again dealing with existing miners. Requiring everyone to regenerate their storage would be extremely unpopular. Grandfathering existing storage plots poses similar challenges as mentioned earlier.
A fairer approach might be to allow only a subset of miners to be eligible each epoch. For example, odd-numbered identities could be eligible in odd-numbered epochs, and even-numbered identities in even-numbered epochs. This wouldn’t reduce the total number of identities but would significantly reduce network traffic, including proposals and ballots. This could be implemented deterministically or probabilistically, ensuring miners are eligible in one out of two epochs on average over time.
We could also introduce an entry queue, similar to the Ethereum Beacon chain. The protocol could gate the number of new miners allowed to join the active set each epoch, processing the queue faster or slower depending on the current size of the active set. The queue size could be dynamic, set by the protocol based on parameters like current layer size or number of proposals.
Note that many of these ideas are orthogonal, are not mutually exclusive, and could potentially work well together.
Thing #2: Make Rewards Probabilistic 🎲
Spacemesh is designed around the principle that home miners don’t need to join a pool to mine effectively. In fact, joining a pool can economically disadvantage them. This is achieved by guaranteeing every miner at least one reward slot per epoch (every two weeks). Even the smallest miner, with only 256 GiB of committed space, has always received a small reward each epoch, and still does if they keep their nodes online and in sync. The deterministic nature of these rewards is what makes it possible for small miners to operate independently of pools. (While many miners still choose to join pools, this is more due to implementation issues than protocol design. Running a node today is more challenging than it should be, but this will get easier over time, reducing the incentive to join pools.)
This system was originally designed to handle a few thousand miner identities. Miners are invited to participate in block production in a deterministic, pseudorandom fashion, with more weight (i.e., more committed storage) translating to more eligibility and/or greater weight of their eligibilities. However, if the network has more miner identities than eligibility slots per epoch, the smallest miners would never be selected. To address this, the protocol ensures that each miner gets at least one slot per epoch. As a result, the size of each layer increases to accommodate these extra miners. Spacemesh layers started at 50 proposals each, but has already grown by around 20x and the size continues to grow with the number of miners.
I’m proud that the smallest miners still receive regular, guaranteed rewards, but there are limits to how many miners the protocol can handle, and we’re nearing that limit today. We can’t continue to grow the number of proposals per layer indefinitely. Doubling or even quadrupling the epoch length would result in more eligibility slots per epoch, but it would mean miners have to wait much longer for their first reward and cause small miners to receive rewards less frequently.
A more straightforward approach would be to introduce some degree of randomness to rewards. Small miners would be eligible for rewards less frequently, but when they are eligible, their rewards would be larger so that over time, the total rewards would remain the same. They’d continue to receive rewards proportional to their share of the total network size. This could be implemented using proof of work or another construction.
This approach has the advantage of scaling to any number of miners. This is precisely what makes proof of work, Chia, the Ethereum Beacon Chain, and other networks that use similar methods so scalable and efficient.
While this doesn’t theoretically impact economics, in practice, it disadvantages small miners by making their rewards less frequent. While a factor of two (i.e., one reward per month rather than every two weeks) might be manageable, going much higher would strongly incentivize small miners to join pools, which is less than ideal. Nevertheless, there is no perfect solution, and I believe we should consider this approach, especially alongside some of the other ideas discussed here.
Thing #3: Think Outside the Box 🧃
When conventional solutions fall short, sometimes the best approach is to think radically different. In other words, we may eventually need to deviate significantly from the existing Spacemesh protocol design. When faced with unpalatable options, it can be beneficial to step back and revisit basic assumptions. The downside to this approach is that a more outside-the-box idea, a change that significantly deviates from the current Spacemesh protocol, would take a long time to design and build. It would be much harder to implement on the existing Spacemesh network (not impossible, since nothing in software is impossible, but more difficult). Such changes might in fact be more likely to land in a testnet, an L2 chain, or a new chain based on Spacemesh rather than the existing Spacemesh L1 chain. Nevertheless, it’s absolutely worth considering some of these more innovative ideas.
One idea that comes to mind, and the least radical of these, is a better proof construction: one that’s smaller and faster, both to generate and to verify. The current issue with miner identities not only their large number but also that the proof of eligibility each miner generates each epoch is relatively large and expensive to validate (it involves costly hash operations). Additionally, these proofs need to be stored forever, and all full nodes must validate all historical proofs (at least until we find a way to compress them using succinct proofs). If we could develop a different proof construction that achieves the same goal but results in proofs that are smaller and much faster to validate, the issue of handling millions of proofs each epoch might be mitigated. The details of what such a proof might look like are beyond the scope of this discussion, and I’m not entirely sure it’s possible, but it might involve using a more efficient hash function or data structure for generating and validating the PoST proof.
A more radical idea is to couple block production and proof generation. As mentioned earlier, in proof of work, miners don’t need to generate, gossip, and validate proofs of eligibility—the proof of eligibility is the successfully mined block itself! We could implement something similar in Spacemesh. Currently, Spacemesh miners submit proofs of eligibility each epoch and then receive eligibility slots for participating in block production and earning rewards. If we’re comfortable with making rewards probabilistic, as described earlier, we could potentially eliminate the need for eligibility proofs entirely and have miners compete to generate blocks. We could design the system so that a miner’s chance of winning any given proposal slot is proportional to their relative amount of network storage. While this would sacrifice the race-free nature of the current Spacemesh protocol, there might be clever ways to allow miners to pre-generate these proofs and only gossip them when they know they’re going to win a block. Chia uses a similar approach.
An even more radical idea is to shard the Spacemesh network into a hierarchy of many networks rather than a single, monolithic network. Imagine a version of Spacemesh with, say, four tiers. Running a miner for the bottom tier, which is the most secure, would require a huge amount of space, on the order of PiB. Smaller miners could profitably mine on higher tiers, which require less storage space and provide proportionally smaller rewards and less security. This concept is similar to the Ethereum quadratic rollup design (“rollups within rollups”), and it makes sense technically (hierarchical structures scale, monolithic ones do not). While this means breaking the existing, integrated Spacemesh namespace and losing some degree of synchronous composability, this change is bound to happen anyway with Athena and L2s, and we can learn from the Ethereum ecosystem on how to handle this.
Considering our plans for Athena, this design isn’t as radical as it initially sounds. We haven’t worked out all the details yet, but once we have a faster Spacemesh L2 built on top of Athena, it might make sense to explore having smaller Spacemesh L1 miners transition to the L2. In the long term, I envision this evolving to look more like the IPC design with many tiers of consensus for various workloads.
Of course, these ideas, like many others I’ve shared here in the past, are purely speculative and exploratory. Spacemesh isn’t a static protocol. It continues to evolve day by day, and it’s still early in its journey. Even harebrained ideas like these shouldn’t be completely dismissed. If you have other crazy ideas on how to address the challenges Spacemesh is facing today, I’d love to hear them.