
Along with the rest of the team, I gave my heart and soul to the Spacemesh project full time for nearly six years. When I first learned about Spacemesh in 2018, I was already tired and jaded by the incessant profiteering I saw in the crypto industry, by the relentless focus on monetization and financialization. I saw the industry making many of the same mistakes as traditional finance, and simply recreating the same unequal power structures on chain. Spacemesh felt refreshing, like an antidote to all of this, as I wrote when I joined the project in 2019. Spacemesh led with a clear set of values, something that was (and, sadly, still is) extremely rare in this industry. It was an attempt to get the design of blockchain correct from first principles, including not only tech but also the human side of the story.
Fast forward six years and, unfortunately, things didn’t quite go according to plan. We got a lot of things right at Spacemesh, including, in my opinion, the economics and the VM, but we also made a lot of mistakes. Over time, those mistakes caught up with the project. I left Spacemesh at the end of last year, and just a few days ago, the Spacemesh operating company declared bankruptcy. It turns out that being motivated by a clear set of values isn’t enough in and of itself.
Of course the project isn’t dead yet—blockchains don’t work that way. As long as there’s an active community, and a few people running node software, the blockchain can never die. But most of the network metrics are falling, most of the previous team is now working on other things, and it’s a very critical time for the project.
As we contemplate what comes next for Spacemesh, it makes sense to reflect on what we did wrong.
Thing #1: Idealism 🌌
If you read my recent series on Ethereum, you’ll recall that I wrote that Ethereum is too idealistic, and too focused on values like decentralization and censorship resistance, rather than on product and on the use cases that are already working. The same is unfortunately true of Spacemesh to an even greater extent.
Ideals matter enormously. All successful cults have a very clear, simple differentiator. Christians are Christians because they believe in Jesus Christ. Bitcoin is Bitcoin because the network will never have more than 21M coins. Ethereum is Ethereum because it puts a Turing complete VM on top of Bitcoin (already a little bit less clear but still basically true). The thing that makes Spacemesh Spacemesh is a relentless focus on doing things right by the little guy, i.e., leveling the playing field. This is why Spacemesh was founded. This is why I joined. This is why we developed a novel consensus mechanism rather than taking something “off the shelf” like Proof of Work or Proof of Stake, etc.
A relentless focus on the ideals is both a strength and a weakness. It united the team and kept us together through thick and thin. It’s the cornerstone of the community. The Spacemesh team and community are unlike any other in the industry, uniquely strong, uniquely resilient, for this reason. It motivated literally everything that we did at Spacemesh. Spacemesh’s goal, to level the playing field, is a good goal, but it’s also sort of an impossible goal, and we chased it relentlessly, tirelessly, to our demise. Like Ahab, we monomaniacally chased the white whale until it became our end.
We sacrificed many things in pursuit of this goal. We made it super easy to join the network, too easy, resulting in a massive influx of identities that nearly crashed the network on multiple occasions. Supporting millions of identities was always a massive burden. It made the software extremely complex, even more than it already was, and it made it very difficult to run a node. We didn’t want to make initialization too difficult for home users, and we wanted to keep the minimum commitment size small, which further increased the problems. We put a lot of effort into building Smapp, our simple frontend for home miners, rather than putting resources into making life easier for “industrial” miners, but Smapp was never very good. I could go on.
We didn’t see the writing on the wall until it was too late. We could’ve stepped on the brakes when millions of identities joined the network, we could’ve hit pause or reset, but we kept pushing ahead. We didn’t stop even when a single mining pool took over most of the network. If nothing else, that should’ve been a warning sign that the protocol needed to change, and that we needed to relax our absolutist stance with respect to values like decentralization and the level playing field. The great irony here is that, in single minded pursuit of this goal, due to the lack of peripheral vision, we ended up completely failing to achieve the goal.
If I were redesigning Spacemesh today, I’d tone down the idealism a little bit and be much more pragmatic. I’d use a proven, off the shelf consensus mechanism rather than trying to reinvent the wheel. I’d recognize that, as frustrating as it may be, economies of scale are a hard economic reality that you cannot win against in the long run, and that plutocracy, and inequality more generally, aren’t going away anytime soon. I’d work within those constraints, carve out a niche for Spacemesh that’s true to its values, and then build an amazing, delightful product that caters to that niche.
Thing #2: Boiling the Ocean 🫕
I like to think of a startup is as a bundle of assumptions. Those assumptions are what set you apart from the competition. They’re both your strength and your potential weakness (it depends whether your assumptions are correct or incorrect). The process of developing a startup involves systematically testing those assumptions, one by one, in order from biggest/riskiest to smallest/least risky.
Spacemesh is no different. It’s also a bundle of assumptions, including that the tech would work as advertised and that the value proposition would appeal to people. Some of these assumptions proved true and some false. But the biggest problem was that Spacemesh simply had too many assumptions to test. In other words, we were trying to do way too many things: we were trying to boil the ocean.
First and foremost there’s technical risk. This wasn’t limited to one protocol or one technology. The Spacemesh software is a bundle of ten or twelve different protocols and subcomponents, including PoST initialization, proof generation and verification, P2P (gossip and direct messaging), the economics library, the VM, the mempool, distributed block creation and validation, PoET, etc. Each one of these is complicated and contains substantial technical risk. On top of these core protocols and subprotocols there’s all of the other software and tooling we wrote and maintained: Smapp, PoST init tools, checkpointing tools, the PoET servers, economics tools, the Athena VM, the dashboard, the explorer, the node API, etc. Literally none of this was “off the shelf”: it was all designed, built, and maintained by Spacemesh. We definitely had a case of Not Invented Here Syndrome.
A team of 100 experienced engineers would’ve had trouble maintaining all of this technology, and Spacemesh never had anywhere near 100 engineers. In fact, it’s remarkable that it all worked as well as it did, and that we made it as far as we did, all things considered. It’s testament to the talent and dedication of the team.
But there were other risks, too, of a less technical nature. There’s economic risk: that the economic model proved to be viable and sound, that investors had the right incentives, etc. There’s treasury risk: that the company treasury was well managed, diversified, invested well, etc. There’s market risk: that the market for the coin was being managed well, that there’s enough liquidity, etc. There’s social risk: that people buy the Spacemesh story, that they share our values, etc. There’s product risk: that we understood our customers and users well enough, and were able to build a product that solves real problems for them. There’s R&D risk: that these abstract, Ivory Tower ideas could actually be built into a product and a platform that works in practice given the real technical constraints on the network, the hardware, etc. In short, I just think we bit off more than we could chew.
If I’ve seen one pattern among successful startups, it’s that the most successful ones are almost always the ones best able to focus. That means focusing on one or two problems, products, and customer segments at a time, especially in the beginning. I know how hard it can be as a founder and builder to “fire” a product or customer segment. It’s doubly hard in the case of a network like a blockchain that by definition has many different critical classes of customers whose competing needs must all be balanced. But we were never able to home in on that first killer use case for that first critical user demographic.
Most blockchains are for developers, at least early on. This is also true of Spacemesh. But because we decided not to use an off the shelf VM and decided to design and build our own VM, because we had several false starts and didn’t get that process started until very late, we never actually had anything meaningful for developers to play with. We had lots of rough ideas of who we were building Spacemesh for, but they were all quite abstract, fuzzy, pie in the sky visions. We never had a really concrete user persona in mind.
If I were redesigning or relaunching Spacemesh, I’d start with product basics: who’s the essential, pioneer Spacemesh user? What does their life look like, and how does Spacemesh fit in? What problem is it solving for them? How can we make sure that it solves a real problem in a very compelling way, such that the user would shout and scream if we took it away? That’s really the secret to startup success in a nutshell, and we were too busy chasing dragons and white whales to focus on the simplest, most important question.
And I’d probably severely reduce the amount of novel tech we’re working with, by about 80%. Most of the blockchain technology stack is commoditized and open source today, so it makes sense to take more off the shelf. It’s the right place to start, and you can always iterate from there. And, most importantly, it allows you to focus on the one or two key, unique ideas that you bring to the table.
Thing #3: Research 🛒🐎
I also wrote about the issue that Ethereum has with research a few weeks ago: in the case of Ethereum, I called it a research fetish, and I explained how in Ethereum research trumps all, including products, apps, and use cases. Unfortunately, the same was true of Spacemesh to an even greater extent.
Spacemesh was founded on the basis of the research of the project’s two chief scientists, around ideas like proofs of spacetime and proofs of sequential work. Indeed, the Spacemesh vision wouldn’t have been possible without these novel ideas. So it makes sense that research was always central to the Spacemesh story, and that getting the research right was always a priority. The chief scientists were instrumental not only in the design of these particular primitives, but indeed in the design of the entire stack of technologies and protocols, top to bottom. I dearly love both of them, working with them was a privilege, and I learned a great deal from it.
Having said that, I learned that it’s a mistake to let academics design your protocol. As smart and knowledgeable as they are, they don’t have product sense. They have a lot of shiny, exciting ideas, but many of those ideas simply didn’t work in practice. As they say, in theory, theory and practice are the same. In practice, they’re not. We ran into all sorts of cold, hard realities attempting to implement and maintain these ideas. Hash functions ran orders of magnitude more slowly than we expected. The core PoST protocol had severe design flaws and wasn’t incentive compatible in practice. The protocols were orders of magnitude more gossipy than we expected: i.e., running a node required an extraordinary amount of bandwidth, far more than most people could sustain from an ordinary home internet connection. There are lots of other examples. And implementing them all was fiendishly complicated.
There’s never a single reason for this, but for some reason it’s a universal truth of software development: ideas that seem pretty simple and straightforward in theory and in the lab end up being far more complex in the wild. This is doubly true of open, distributed systems like blockchains because you have to account for lots of edge conditions, adversarial scenarios, and potential vulnerabilities that just aren’t present in the theory. I didn’t previously appreciate just how difficult P2P networks are, and how unreliable most home Internet connections here.
In addition to the difficulty of implementing and maintaining the code itself, complexity is bad for another reason. By definition it’s difficult to get your head around, difficult to understand, and difficult to explain. By the end there was no one person who could fit the entire Spacemesh project, all of the protocols, the design of the client, etc. into their head, i.e., into working memory, to say nothing of the degree of understanding on the part of the broader community. I was in a full time R&D role and even I had days when I couldn’t remember how half of the thing worked and struggled to communicate about it, because there were simply so many moving parts and everything was constantly in flux!
This eventually happens to all sufficiently complex technological projects, but there are ways to manage complexity with a larger team and ample resources, neither of which Spacemesh had. Things got too complex too soon, and research was a big part of the reason why. Every time we discovered a flaw, every time something broke, the reflex was to add something to address it, whereas all along we should’ve been removing things and progressively simplifying the protocol. Spacemesh was simply too complex for its own good.
While our chief scientists were as incentivized as any of us to see the blockchain launch and be successful, they also had other, competing incentives. One, unsurprisingly, was to publish. A lot of time and effort was spent on their part writing, and rewriting, an academic paper presenting the Spacemesh protocol. A big part of this paper was formal proofs that all of the theory was correct. In retrospect, I don’t think any of this mattered to the success of the Spacemesh blockchain (or lack thereof), and it wasn’t a good use of the project’s scarce resources. This was also an important lesson for me.
In short, Spacemesh put the research cart before the product horse. Product must always lead research, not the other way around. If we had spent less time in the Ivory Tower, less time perfecting the formal proofs, and had instead spent that time and those resources on the ground, in the field, meeting the Spacemesh community and the potential users where they’re at, understanding their needs and how they see the world, and subsequently building a protocol and a product that worked really well for them, then the outcome might’ve been different.
These lessons will inform my future work at NEAR and elsewhere. If we think of Spacemesh less as a science experiment and more as a product, who are its users? What’s its unique value proposition? In my opinion these questions should be the starting point of any conversation about rebooting the Spacemesh project.