Roots and Branches: Centralization and the Role of Software Startups
Is it not the loftiness of thy vineyard—have not the branches thereof overcome the roots which are good? And because the branches have overcome the roots thereof, behold they grew faster than the strength of the roots, taking strength unto themselves. Behold, I say, is not this the cause that the trees of thy vineyard have become corrupted?
For most of the past 10 years I’ve had a fairly romanticized view of startups, with the core belief being roughly “If you’re ambitious and you like building software, then being a startup founder is probably a great career path.” I started questioning this sometime last year, and I was delighted recently to find a Roots of Progress essay which reinforces and clarifies some of the new thoughts I’ve been having:
Broadly speaking, there are three domains of activity important to technological progress: science, invention, and business. Science discovers new knowledge; invention creates useful machines, chemicals, processes, or other products; and business produces and distributes these products in a scalable, self-sustaining way. …
My hypothesis is that while science and business have functioning career paths, invention today does not. …
The bottom line is that if a young person wants to focus their career on invention—as distinct from scientific research, corporate engineering or entrepreneurship—the support structure doesn’t exist. There isn’t a straightforward way to get started, there isn’t an institution of any kind that will hire you into this role, and there isn’t a community that values what you are focused on and will reward you with prestige and further opportunities based on your success. In short, there is no career path.
That essay also links to Ben Reinhardt’s PARPA document which was on Hacker News the other day. It has several interesting sections on the limitations of research and entrepreneurship, and overall the document lays out a plan for filling the gap between the two.
My area of interest isn’t quite the same as those authors: PARPA isn’t focused on software. Fortunately, spending time on invention is easier for software developers than for people who work in the physical world. We still need to pay living costs somehow, but at least we don’t need expensive equipment. Meaningful work can be done by individuals (or groups working independently over the internet), and there are already communities for open-source developers where you can share your work and build connections. My favorite example of software invention is Rich Hickey creating Clojure, and I think it even works as a definition (emphasis mine):
I started working on Clojure in 2005, during a sabbatical I funded out of retirement savings. The purpose of the sabbatical was to give myself the opportunity to work on whatever I found interesting, without regard to outcome, commercial viability or the opinions of others.
Although there’s overlap, software invention doesn’t have to be open-source. You might build an application and retain the source in case you want to monetize it at some point. In the latter case, invention differs from independent businesses and would-be startups because making money is just a secondary objective. Though if an invention grows fast or makes enough money, you might decide to turn it into a business.
If you wanted to spend your whole career on software invention, it’s at least within the realm of possibility. I am myself transitioning to part-time consulting so I can continue my work on The Sample and some other projects without entrepreneurial pressure. In general I think part-time freelancing is a good path, transitioning to consulting as you gain expertise in some area (which hopefully will be expedited by all the time you spend on invention). Other funding sources can then be pursued opportunistically. This is the career advice I wish I could give 20-year-old me, especially since clients don’t care if you have a college degree.
Since institutional support isn’t absolutely necessary, the main impediment for software invention careers might just be lack of mindshare. If more people tried it and wrote about their experience, maybe invention would become as popular as grad school. And this isn’t some radical new thing anyway; freelancing is a common occupation for open-source developers. Calling it “software invention” is mainly about branding—forming a tighter community around a specific set of values.
If I ended the essay here, I would’ve called it “A Career Path for Software Inventors.” But this is about more than just helping some programmers move up a notch on Maslow’s hierarchy (though I do care about that): society needs more invention.
Nadia Eghbal’s report Roads and Bridges describes open-source software as “digital infrastructure” and advocates for greater institutional support. (The alliteration between “roads and bridges” and “roots and branches” is completely coincidental and brings me much joy). I’d like to make a similar argument for open protocols like email, RSS, etc.
As with open-source, standards and protocols provide a foundation upon which businesses are built. For example, thanks to the HTTP protocol, you can create web applications that are available to anyone. Your business doesn’t have to be at the mercy of a platform unless you decide the trade-offs are worth it. Protocols also facilitate interoperability and innovation. You can build applications that do one thing and do it well instead of doing many things with mediocrity. If you want to improve part of a system, you don’t have to overcome a lack of network effects first (the dreaded chicken-and-egg problem). These are all important elements of a healthy software ecosystem.
More generally, this is how market economies work: most decisions are decentralized instead of being dictated by a central authority. But decentralization has downsides, so within individual businesses, decisions are centralized. The optimal system is usually a hybrid rather than being decentralized all the way down. Zach Tellman gives an excellent description of the philosophy behind this in Elements of Clojure. From the section “Systems of Modules” (emphasis in original):
We can build a principled system, which has predictable relationships between its modules. Alternately, we can build an adaptable system, which has sparse and flexible relationships between its modules.
A principled system minimizes internal indirection and is usually structured as a hierarchy. The implementation of each component is guided by the central design principles. These principles, applied from the top down, allow each component to make broader assumptions. This makes each component smaller and often faster, since there are consistent methods used throughout. …
An adaptable system has a high degree of internal indirection and is usually structured as a graph. Each component is purposefully blind to the internals of adjacent components, which leads to redundancies. This makes each component larger, and often less efficient.
Principled components allow us to explore within a uniform context without any need to reorient ourselves. This uniformity, however, makes them fragile. They are constructed towards a fixed purpose and cannot be easily reoriented. ...
And so we do not want a system that is wholly principled. We want a collection of principled components, built to be discarded, separated by interfaces that are built to last.
So efficiency is one reason that businesses tend towards centralization. Apple is a nice case study: the Apple ecosystem is highly principled, and that is one of its main benefits. They provide a seamless, polished experience. The other impetus for centralization is value capture; for example, Apple’s tight control allows them to charge the 30% app store fee. Together, these forces mean that businesses are engines of centralization, all operating on top of a shared layer of protocols (interfaces).
The problem is when massive growth in business outpaces growth in its foundation—when the branches outgrow the roots. I am convinced that this is happening and that it is the root cause (no pun intended) of various problems in tech right now. For example, Protocols, Not Platforms applies this argument specifically to free speech: social media moderation might be more tractable if it were decentralized.
Thus I think the tech industry needs to focus more on “nourishing the roots”: driving greater adoption of existing protocols and creating new ones as needed. And I think the best way to do this is to build a career path for software invention. The intrinsic motivations of inventors/open-source developers, as opposed to those of businesses, are more naturally aligned with root nourishing. Inventors are also incentivised to build adaptable systems because they are small entities—decentralization gives them more freedom since they’re not large enough to push centralization in their own favored direction.
So we need lots of inventors. The grassroots freelancing approach I described above is one possibility. Greater institutional support would be swell, too. I don’t have any bright ideas here (I’ve mainly been thinking about things I can do, and I don’t have much institutional sway), so I’ll just refer again to Roads and Bridges.
However it happens, I hope that a career in software invention becomes as exciting to ambitious programmers as the idea of being a startup founder was to me when I first discovered Paul Graham’s essays as a teenager.
Behold, the branches of the wild tree have taken hold of the moisture of the root thereof, that the root thereof hath brought forth much strength; and because of the much strength of the root thereof the wild branches have brought forth tame fruit.
Published 11 May 2021