Note: this week’s post was delayed due to personal reasons.
I published an article a few months ago called Spacemesh Postmortem, where I laid out some of the reasons for the unfortunate demise of the Spacemesh project earlier this year. Actually, I initially wrote a private list with over a dozen reasons, and I was only able to touch upon three of them in the article.
I’ve been reflecting further upon the unfortunate outcome for Spacemesh recently, since the project is well and truly over at this point: the coin has been delisted from exchanges, the infrastructure went down, and the network itself is no longer running, as far as I can tell. As mentioned in the previous article, the Spacemesh development company declared bankruptcy earlier this year.
I think I have a fairly unique perspective on the things that went wrong at Spacemesh, since my work touched almost every area of the project, I worked closely with management, and I worked on it for over five years. Upon reflection, I also think that it’s important to share more about what went wrong at Spacemesh for the benefit of posterity, and so that other teams and communities, including NEAR, can avoid the same outcome for the same reasons.
Here are three more reasons Spacemesh failed when and how it did, as I see it.
Thing #4: Contrarian 🙃
I’m a contrarian. I don’t like to play by the rules, at least not simply for the sake of doing so. I like to question everything, and chart my own course. I’m not good at following instructions or rules, and I tend to chafe under someone else’s management. In these respects, I fit well on the Spacemesh team, and it’s a big reason I stayed as long as I did. It’s the most contrarian team I’ve ever been a part of. Heck, the company building Spacemesh was called “Unruly” for a reason.
Contrarianism is a good thing. The world needs contrarians. It needs unreasonable people. In a sense, it’s the only way progress is made. If everyone always followed all of the rules all the time, society would be stuck and would eventually end up in a death spiral, unable to keep up with change. (In a sense, this is what’s happening today in places like Korea and Taiwan, where the birth rate has fallen drastically below the replacement level, but that’s a topic for another day.)
Contrarianism is also useful because it’s a way of asserting agency over the world around us. I’m a contrarian simply because a lot of rules, norms, and customs I see are stupid, ineffective, outdated, and wasteful, and I’ve taken the time and made the effort to see this for myself. Of course, not all rules are stupid. In my opinion it’s totally okay to follow a rule once you’ve taken the time to understand it and why it exists, and if you agree with its existence. Actually, this is true of most rules, and I do happily follow most rules, but some dumb rules exist to be broken.
Spacemesh followed very few rules. We didn’t break the law, nothing like that, but we deviated very far from the norms of the industry. In fact, the list of examples is so long, I don’t even know where to begin. The most obvious one is probably the fact that we decided to develop our own, novel consensus mechanism, with a stack of ideas and technologies and protocols to support it, rather than using something tried and tested such as proof of work or proof of stake. We wanted to have our cake and eat it too—in the sense that we wanted a hybrid mechanism with the best properties of both proof of work and proof of stake—but this led to boiling the ocean. We bit off more than we could chew.
In spite of these ambitious goals, we raised substantially less money than the other L1s we were competing with, which put us at a disadvantage. (And we didn’t really want to raise more, because we thought we had enough. We were wrong.) Our economic model was radically different than other L1s: we insisted on giving well over 90% of the coins to miners in the community, rather than keeping them for ourselves or unilaterally endowing them to “ecosystem funds.” (For the record, this is one of the decisions I’m the most proud of, and if I had it all to do over again, I might do the same thing again here.) The fact that, unlike basically every other project, we had no premine meant that our inflation appeared to be very high, something that the market never took the time to understand.
Another example: we emphasized mining from home, even when early indications were that this would be difficult: difficult for home miners, and difficult for us to support. We doubled down on this, both with the protocol design, and with a frontend app, which severely distracted us from working on the protocol, i.e., the only thing most L1 projects focus on for good reason. We were laser focused on becoming the “most decentralized L1,” which in practice meant millions of miner identities, which both made the network slow and also meant that running a node required an enormous amount of compute power and bandwidth. Our block time was five minutes, in an era when most new blockchains were targeting subsecond block times. We initially didn’t pursue things like exchange listings, and then only belatedly did it later with tier-2 and tier-3 exchanges.
I could go on and on, but I’ll cite just one additional example for now: we didn’t invest enough in marketing, insisting instead on leading with code and letting the product speak for itself. Where we did marketing, it was ineffectual. (More on this in a moment.)
My biggest lesson here is that, when being contrarian, you need to be selective about the things you’re acting contrary to. You can’t be contrarian about everything, all at once. In other words, choose your battles.
Thing #5: Narrative 📢
In my previous postmortem, I wrote that some of Spacemesh’s failures, including its idealism and fetishization of research, were also issues for Ethereum. Here’s another example of how Spacemesh’s failures in some way resembled those of Ethereum: as I wrote previously, Ethereum has struggled quite a bit over the years to refine and define its narrative. So did Spacemesh, for a few reasons.
The core Spacemesh narrative, which of course also evolved over time, was that Spacemesh was the world’s most decentralized L1 blockchain. The target was to have 1M mining nodes on the network, and in a sense, we met and exceeded that goal. To put things into perspective, the most decentralized production blockchain platforms today, such as Bitcoin and Ethereum, have on the order of 10,000 nodes, and probably an order of magnitude or two less miners. (I didn’t understand at the time, but I understand now, that there’s a reason for this number. Networks cannot realistically support more participants than this!)
The other core Spacemesh narrative was that, due to the proof of spacetime consensus mechanism and the ease of mining from home, Spacemesh would level the playing field and make it easy for ordinary people everywhere to fully participate in, and profit from, the blockchain economy. In this respect, Spacemesh was “the people’s chain,” its coin was “the people’s coin,” and it was intended to be a radical departure from basically every other layer one blockchain, which all feature a tiny set of whales who control the lion’s share of wealth and power. We leaned very heavily into these narratives and this branding; this was a major reason our economic model was so radically different from other blockchains.
There are two problems with these narratives.
The first is that decentralization and fairness aren’t product features, they’re values—and they’re contentious values, at that (compared to, say, “kindness,” which is clear and unambiguously positive). Incidentally, the same thing is true of privacy—it’s a value, not a feature—which is the reason that privacy projects have struggled so much to market themselves. There are, of course, plenty of people in the world who care about things like decentralization, fairness, and privacy, but in relative terms these groups are tiny, and in my experience these abstract values simply don’t appeal to 99% of people in the world.
The pattern was always the same. I’d mention Spacemesh. They’d ask, what makes it special? I’d respond, we’re more decentralized. Then, their eyes would glaze over. They’d ask, So what? What’s that good for? And I never had a really compelling answer.
In my experience the narratives that make the most sense to the market, and to the world at large, are product narratives: We care a great deal about such-and-such a problem for such-and-such a reason, so we built an incredible product to solve that problem for people like us. The problem with blockchains is that by and large they’re not products. They’re infrastructure. Infrastructure is a product to a developer that’s building something on top of it, sort of, but it’s a tough story to sell to non-developers.
In fact, it’s even worse than that. It’s not simply that these values aren’t product features; in fact, they’re directly at odds with a functional product! Decentralization simply makes products and systems unusable (this is one of the core tensions of the blockchain space, and the crypto products that have succeeded are almost all centralized in meaningful ways). Spacemesh was no exception. The disk and bandwidth requirements were extraordinarily high. The block time was embarrassingly slow. And the desire for fairness exacerbated all of these problems: it meant even more miners, which slowed things down further.
The second problem with this narratives is that, as nice as they sounded, in practice they simply weren’t true.
It’s true that we had a few million miner identities on the Spacemesh network, at its peak. But, as I’ve written before, decentralization is complex, hard to measure, and multifaceted. On Spacemesh, one miner identity didn’t map to a single miner in any meaningful sense, as in, a single person or entity operating a miner; due to limitations and flaws in the protocol, large miners were strongly incentivized to split their mining power into tens or even hundreds of identities. Which means the actual number of distinct miners on the network was orders of magnitude less than this (we don’t know the exact number but we can make an educated guess).
We also had the issue of mining pools, and of one pool in particular that became especially dominant. While it may be true that the underlying storage space represented by the pool was contributed by hundreds or thousands of distinct miners, in practice, the singular pool operator exerted an inordinate amount of control and power in the Spacemesh network: by, for instance, deciding whether and when to install upgrades. This is not the hallmark of a healthy, decentralized network, and the fact that one pool represented over 90% of the mining power on the network at its peak means that Spacemesh was, in simple point of fact, one of the least decentralized production blockchains. (There are aspects of the protocol, and certain flaws, that led to this scenario, but I don’t have room to get into them here.)
Finally, with respect to the second narrative, that Spacemesh was the people’s coin and the people’s network, I’ve already pointed out several ways in which this wasn’t true in practice. It was, actually, very difficult to run a miner from home, in spite of our best efforts to the contrary, and it was getting harder over time, not easier, due to the rapidly accumulating block data. What’s worse, one of the core guarantees we made to the Spacemesh community was that, unlike in other blockchains, there would be no economies of scale in Spacemesh mining: a miner with twice the space would earn precisely twice the rewards. This simply wasn’t true, again for multiple reasons. There were protocol flaws that mean that, in fact, there were benefits to being a larger miner, and moreover, there were plenty of extraprotocol economies of scale. It turns out that running a big data center, or a big mining pool, will always be much easier and cheaper on a per unit basis than mining from home.
So we failed to deliver on even the most core narratives, and we didn’t update these narratives over time or find a new narrative that was sustainable, authentic, and relatable.
Thing #6: High Ego Friends 🗿
I joined Spacemesh in mid 2019. Over the following 3-4 years, the project experienced extremely high turnover. When I left the project late last year there was only one other person left from the time I joined, not counting the founders. Out of around 30 people, that’s around 3% remaining, which means 97% turnover. In one year alone, there was at least 50% turnover or attrition. These aren’t healthy numbers.
It took me a while to figure out what had happened, why so many people had left, and why the culture had shifted so dramatically. It finally made sense to me when another founder explained the problem of hiring “high ego friends.” This is both the group of people that’s most readily available to you when starting a project, and it’s also the very last people you should hire.
Let’s start with the first point. The kind of person who’s going to reach out to join you on a startup is almost always a high ego friend. The friend part is obvious; the high ego part isn’t. I can’t quite put my finger on it, but there’s something audacious and brash about a new startup project, especially one as experimental and risky as Spacemesh. You have to be kind of crazy to work on something like this, and you have to be irrationally self confident to feel that it could actually work. I think there’s also something about small, early stage startup life that attracts high ego people because it lets them stand out, and become “head” of this or that department, rather than being forced to climb the greasy pole at a big company.
Why are these the worst people to hire? For one thing, because their expectations are too high. They expect a high salary, a title, and lots of equity. What’s worse, they don’t expect to work too hard. After all, they’re just lending a hand on their friend’s project. You won’t get mad at them if they don’t want to work 100 hour weeks, right?
Once I understood all of this, I immediately began to see the Spacemesh history in a new light. I got the strong sense that the founding team was, indeed, the set of high ego friends around the founders (and some friends of friends). Not to downplay the work that went into building Spacemesh—it was a herculean effort, and many of the early team members did a lot of the heavy lifting—but what I personally witnessed was high ego folks who had been there since the beginning who were getting paid pretty well to do very little work. They came in late and left early. They wanted to party, not work nights and weekends. (We never worked nights and weekends, which in and of itself is a bad sign.)
This is precisely the sort of person who’s the most dangerous at a startup: they work, but not hard enough, and their expectations are out of line with the realities of startup life. These people must never be hired, and when discovered they must be let go immediately. They belong at more mature companies, where one person’s contributions matter less in the grand scheme of things. But, at a startup, their presence can be an existential risk, as it was for Spacemesh.
Eventually they were let go, or they parted ways, usually with understandable claims of burnout. Startup life is much less fun after four or five years, especially without revenue, without many users, and as the money begins to dwindle. The high ego folks move on.
Eventually, somehow, in spite of its constraints and its other failures, Spacemesh managed to hire a much stronger team of hard-working, low ego contributors. Spacemesh did, eventually, end up with an excellent team. But the project was already too complex, had already spent too much money, had too much technical debt, and had many other issues besides, as I’ve written about here. It was too little, too late to save the project. The damage had already been done.