Everywhere, All At Once
Three Things #118: May 5, 2024
If it isn’t clear already I spend a lot of time thinking big picture about things like human-computer interaction and developer experience, operating systems and VMs, and application architecture. My main motivation for this, other than the fact that I just find it endlessly fascinating, is that I think we can do better on all these points.
At some point it became obvious to me that things work a certain way today but that the way things work today isn’t necessarily the only or even the best way things could or should work. In fact if you really think about it, it becomes obvious that the likelihood that we more or less randomly stumbled upon the one, true, final and perfect design is vanishingly small. This is true of most things, but I understand software best so that’s what I’m focused on.
Incidentally this is what excites me so much about projects like Urbit. It’s not that Urbit is perfect or optimal but it represents a very different vision for what a computer could and should be, for how we should think about applications—how we architect, run, and manage them—and that’s a very valuable experiment. It’s sort of a “weird wacky parallel universe” version of computing, like the place we might’ve ended up if not for some small historical quirk.
Blockchains and their smart contract engines represent a true paradigm shift in computing, one as big as previous paradigm shifts like mainframe, desktop, mobile, cloud, etc. Each of these paradigm shifts had an impact on those same things—how we architect, build, run, and manage apps, and the sort of applications it became possible to build. (And to the extent that you believe that software is eating the world they also represent paradigm shifts in human society at large.)
Years ago when I was working on Ewasm a teammate shared this video. It blew my mind—I still think it’s the best tech talk I’ve ever seen—and it was my primary motivation for working on Ewasm and on the VM stack more generally. Wasm (which didn’t yet exist when this was recorded, but which it presciently forecast) seemed like the missing piece of the puzzle, and my mind spun with the possibilities of a platform-independent bytecode format that could run as well on a web browser as it could on a toaster oven or a satellite.
To be honest I had completely forgotten about the video but rediscovered it recently along with a talk I gave six years ago on my wacky, ambitious vision for where Wasm, the web, and general-purpose VMs like Ethereum might one day take us. It’s hard to believe six years have passed since then and a lot of things have changed in the interim. I’ll try to paint a picture of that vision here, updated to the present.
Thing #1: Where We Started 🖥️
“640 KB of memory ought to be enough for anybody.” - Bill Gates in 1981
One of the long trends in computing is that the place we run applications has bounced back and forth between “my computer” and “someone else’s computer.”
In the beginning there was the mainframe, and you ran applications locally on the mainframe because that’s all there was. Later dumb terminals and time sharing were invented and you could work with the mainframe remotely, so that you were effectively running your application on a primitive form of “the cloud”—i.e., a mainframe across campus. Over time microcomputers and personal computers emerged and got better and eventually you began to run more interesting programs on them locally. Then the web, the cloud, and mobile took off and we switched again to running our most interesting applications remotely. This is still how most web2 applications work today, although the trend has begun to reverse again. The silver lining on the cloud has faded, cloud computing has become prohibitively expensive for many startups, and we’ve begun to care more about things like privacy, so there’s a shift back towards running applications on your own hardware.
Both the “where” and the “how” of writing and running programs has changed. We started with programs that were very primitive by today’s standards written on punch cards or entered via toggle switches. From there we got assembly code, then low-level languages like C, and finally the interpreted, high-level languages that are so popular today like Python and Javascript. In all of these cases, these programs are compiled down to native machine code and run on a processor, a process that’s managed by an operating system.
One important thing that hasn’t changed is that applications run on only one system at a time. An application running on an operating system is called a “process.” A process can have many threads which can exchange data and pass messages to each other, and multiple processes on the same machine can do the same. But a running application means a process and a process runs on one machine.
It’s possible to split an app into multiple components that run on multiple machines, most typically a frontend and a backend using a client-server architecture. It’s possible to scale an app horizontally by making it stateless and storing all state in a database that’s accessed simultaneously by many parallel instances of the app. You can go a step further and shard the database so that this, too, scales horizontally. This is a bit reductionist but it’s more or less how all scalable apps work today—and the point remains that, from the user perspective, the application (as in, the state the user sees and interacts with) is still running on one machine.
There are a few relatively obscure techniques and bits of infrastructure that allow you to migrate running processes, containers, or entire VMs from one physical system to another without much downtime but I’m not aware of any previous, successful attempt to run a single process in multiple places at the same time. The key thing to understand about the application model is that, while many clients may access the same backend at the same time, each instance of the application, each process, is independent, isolated, and ignorant of all the others. Each instance may be doing something totally different.
The first successful exception was Bitcoin. Bitcoin changed the existing model in two ways. One, by turning the application architecture into a protocol so that many implementations can all run in parallel and speak to one another. And two, by introducing a state machine and consensus so that, unlike horizontally scaled applications, each instance of Bitcoin is by definition doing the same thing at the same time (if it isn’t, by definition it isn’t Bitcoin!). In other words, you can think of Bitcoin not as thousands of independent nodes running separate applications but indeed as a single, coordinated “super app” running in many places at the same time. If that sounds like sci-fi, it should. (Pretty much all apocalyptic AI stories start this way don’t they?)
I’d argue that the world’s first successful “super app” was Bitcoin and that it changed the application architecture game completely, but also that this transformation has only just begun and we’ve only just begun to understand and take advantage of what this paradigm shift offers. But we’re already getting ahead of ourselves.
Thing #2: Where We are Now 📱
“The point of building virtual machines is to run the code we have now with all of its flaws, while allowing a better language to run on the same platform.” - The Birth and Death of JavaScript by Gary Bernhardt (as quoted by ChatGPT)
The biggest breakthrough of the past couple of decades in computing is virtualization (and a related concept, containerization), which powers most modern, scalable computing. As described above applications used to mostly run directly on hardware (“bare metal”). They still do, of course, but an increasing number of them are run with one or more layers of indirection between the application and the hardware. The simplest example of this is an interpreted language like Python or Javascript. While a program written in C and compiled to machine code runs directly on the hardware (as mediated by the OS), a Python script isn’t compiled to machine code at all (usually). Instead, the program running on the hardware is the Python interpreter. Your script runs as a program inside of that program.
There are advantages to adding layers of indirection. Abstraction is the best tool we have to deal with complexity so it allows programmers to write much more complex programs than they could otherwise. The main disadvantage is performance since a program running inside another program will never be as fast as a program that’s optimized and compiled directly to native machine code. But the performance of interpreted languages, containers, and virtualization has gotten better and better over time, which has enabled more and more complexity.
The stack of technologies that underlie modern applications is staggering. To put things in context, I’m writing this in a web app running in Javascript inside a modern browser that’s probably written in C++ that’s running in a modern operating system that’s probably written in C++ that’s compiled to machine code that’s running on hardware. The backend is even more impressive. This web app is exchanging data with a backend application hosted in a distant datacenter written in a language like Go that’s compiled to machine code that’s running on a guest operating system (probably Linux) that’s written mostly in C inside a hypervisor that’s itself written in a language like C++ or Go that’s compiled to native code that’s running on hardware. That’s to say nothing of the thousands of peripheral libraries and drivers involved, the infrastructure involved in moving the data back and forth between client and server, etc.. And things can get much deeper, weirder, and more recursive than this. Virtualization, containerization, and abstraction, including the ability to run applications inside of a web browser, makes all of this possible.
There are two additional, recent breakthroughs in the application stack that are also highly relevant. The first is blockchains like Bitcoin and Ethereum. As mentioned before, Bitcoin was the first-ever “super app” that allows thousands of computers around the world run by independent actors that don’t trust one another to coordinate and run a single program together. It’s the world’s first successful, decentralized, massively large scale distributed system. The Bitcoin network and bitcoin the asset are emergent properties of this “Bitcoin program”: without it, they wouldn’t exist, and the “Bitcoin program” is exactly these things and nothing more.
Just as Python the interpreter runs Python scripts, Bitcoin the network runs Bitcoin programs, also called scripts. Of course Bitcoin is quite limited in the sort of programs it can run, by design. By contrast, Ethereum introduced a Turing complete VM which means that it can run any application at all. With the introduction of Ethereum the world had its first “super app” that was capable of running any other application at all. If Bitcoin was like a “super app” spreadsheet, Ethereum is like a “super app” version of a Python interpreter that can run any program at all.
In other words Ethereum was the first time thousands of computers around the world could coordinate to run arbitrarily complex programs, which has already given rise to innovations from fungible tokens to NFTs to stable coins to prediction markets to a massive DeFi ecosystem, and this is just the tip of the iceberg. It’s interesting to note that this characterization of Ethereum, the “decentralized world computer,” was one of the ways it was originally described, although it’s fallen out of favor recently relative to other narratives.
The second recent breakthrough is platform-independent ISAs like Wasm and RISC-V. Wasm was the first serious attempt to fully specify a program that could run on any system regardless of underlying hardware or software. Of course we’ve always had source code that could, in theory, compile and run on many different systems, but in practice it’s nigh on impossible to write a program in a language like C that just works everywhere due to the differences in architectures, processors, operating systems, etc. (and books on this subject run to 600 pages!). Interpreted languages like Python and Javascript help but even these run into issues with things like talking to hardware, the file system, networking, etc.
By contrast Wasm is fully platform- and machine-independent and is designed to run at near-native speed anywhere. RISC-V is a more recent attempt to do something very similar. The main difference is that Wasm, which as the name suggests grew out of the web and browser ecosystem, is more complex and higher-level, whereas RISC-V mimics an actual hardware instruction set and is thus lower-level and much simpler (“RISC” stands for “reduced instruction set computer”). Importantly, both Wasm and RISC-V plug into the LLVM compiler pipeline which means that programs written in dozens of common languages including C, Rust, and flavors of Go and Typescript can target them, and additionally that they benefit from the rich, mature tooling in the LLVM ecosystem. Both technologies are relatively new but have already begun to see uptake in applications like cloud computing, blockchain and, even more recently, zero knowledge proofs and verifiable computing.
All of this may sound like nerd sniping and may not feel exciting or relevant, but trust me, it’s a big deal. Blockchains and platform-independent ISAs represent the next step in the ongoing evolution of the virtualization of computing. If you thought things were already weird thanks to virtualization wait until you see where we go from here. Virtualization has already enabled incredible leaps forward in software and systems architecture and these new paradigm shifts will have an impact that’s at least as large.
Thing #3: Where We’re Going 🥽
“Whatever I might want to know is in an individual mind somewhere, maybe in many of them. If it is very fundamental, such as the meaning of the word ‘chair,’ it is in every mind. But even if it is something esoteric that is in only one small portion of Gaia’s mind, I can call it up if I need it, though such recall may take a bit longer than if the memory is more widespread… if you want to know something that isn’t in your mind, you look at some appropriate book-film, or make use of a computer’s data banks. I scan Gaia’s total mind.” - Bliss, Foundation's Edge, Isaac Asimov
Today we have super applications running on thousands of computers on blockchains like Ethereum. We have efficient, platform-independent instruction sets like Wasm and RISC-V. We have the ability to succinctly prove correct execution of programs (and, where necessary, to preserve privacy) using zero knowledge proofs, including the ability to prove ordinary programs of arbitrary complexity in modern, mature languages like Rust. There are more mind bending unlocks just around the corner through cutting edge technologies like MPC and FHE. Where does all of this take us?
In short, we need to evolve the way we think about applications. An application used to be a monolithic bit of code running locally on one computer. Over time that definition expanded to include code running on a remote system, to a client-server model where the application includes both a frontend and a backend, to distributed systems and a microarchitecture model where an application is composed of dozens of microservices, to a blockchain world where thousands of computers work together to create a singleton instance of an app and a shared database. All of these are applications; none is more or less an application than any other.
Today we make an important distinction between apps that run locally (on premise or on device) and those that run “in the cloud.” Think for instance of an AI model powering a ChatGPT-like application. It’s much easier to run these applications in the cloud on extraordinarily powerful infrastructure because inference is very compute intensive, the models are very large, etc. (to say nothing of the challenge of training a model in the first place). But there are severe privacy implications. There are many kinds of questions you wouldn’t want to ask an AI agent owned and operated by a third party company, and you probably don’t want a “cloud” AI tool to index all of your sensitive personal data. That’s why it’s so important that you be able to run AI models and do inference locally, which is difficult for the reasons I just mentioned.
But if we take the idea of blockchain and the super app seriously then maybe we don’t have to make this distinction. Maybe we don’t have to choose; maybe it’s a false choice. Maybe the application can run everywhere at once. Maybe a local node or instantiation of an app (we need better terminology) can be both a local instance and part of a larger instance at the same time, like one member of a hive mind. And maybe we can use this application both in “online” and “offline” mode as necessary.
To achieve this we need to radically change the way we think about software and applications. We need to change the way we architect applications. We need to change the language we use to describe applications and their architecture. And we need to change the tools we use to build applications, up to and including the programming languages and frameworks. We simply don’t have the language to talk about applications that run everywhere all at once and we don’t have the tools to design or build such applications. Every instruction in a modern application’s source code is designed to run locally on one system. What would a programming language look like if it were instead designed to orchestrate an application running across many systems? We can get a hint from a tool like Kubernetes which does exactly this, albeit for a very specific domain (devops).
Maybe someday when we begin using a new device we won’t have to install and log into a hundred apps again from scratch. Maybe those apps will already be running “out there,” and our device will just need to join a “sync group” with our other devices to get up to speed on our personal data and saved state. I use four or five devices on a daily basis and I’m continually frustrated by how my data, apps, and services are out of sync across the devices. It’s a little better within, say, the Apple ecosystem, and the very best designed and engineered apps do a pretty good job of providing a seamless experience across all of those devices. Trello, Slack, Telegram, and Github are good examples of such apps, while there’s a special place reserved in hell for apps like Zoom, Whatsapp, Wechat, and Signal that limit the number of devices where you can log in simultaneously.
But even with the best-designed apps there’s still room for improvement; logging in to a third party server shouldn’t be necessary in the first place! It could be way easier for app developers to build this sort of experience. My dream is that one day when I get a new device there’s no setup or onboarding. Just like running Bitcoin for the first time on a new system, it can more or less immediately get up to speed with the rest of the network, then continue to sync and verify historical information in the background as necessary.
From the perspective of advancing application architecture blockchains have achieved only about 1% of their potential so far. My goal is to push the envelope on this as much and as fast as possible. Athena will be a platform that enables this sort of radical innovation.