Power & Transcendence in the Intelligence Age

Will today’s technocapitalism eventually give way to a new economic and religious order?

Two intertwined threads are central to human culture. The first braid is materialism, which today takes the form of technocapitalism. The second braid is transcendence, which in prior ages was embodied in religion, and today is embodied in the pursuit of authentic self actualization, or pejoratively, therapy culture.

During the age of Christendom, materialism was embodied in the feudal system, and transcendence in the church. The monarch represented the culmination of both.

The industrial revolution changed both of these things. Before the industrial revolution, the most effective way to attain power was to field soldiers in battle and conquer land. After the industrial revolution, as Adam Smith chronicles in The Wealth of Nations, a) Capitalism increased the power of the bourgeoisie, who allied with minor lords to reduce the power of the king, b) Enclosure gave way to private ownership of land, increasing agricultural productivity and freeing up industrial labor, and c) The industrial economy created a new class of consumer goods, leading to a transfer of wealth from aristocrats (who wanted these goods) to capitalists (who made them). As the king began increasingly to rely on capitalists to finance war and colonial expansion, power shifted further from aristocrats to capitalists.

The industrial revolution kicked off a wave of secularization. By the late 19th century, Freud observed that religion had lost its therapeutic effect. Whether by reason of science or otherwise, believers found it increasingly difficult not to question God, depriving them of a stable set of rules by which to live their lives. Freud saw this impotent religiosity as leading to neurosis, and he invented analysis to help alleviate the suffering this caused. While the rise of secularity had many causes, the most direct catalyst was economic: Humans had learned to convert energy into labor power, and this reshaped daily life, authority, and belief.

Today, a similarly powerful revolution is underway: we have discovered how to turn energy into intelligence. A number of debates are swirling on how this transformation will play out, and ultimately, how it will reshape society. These debates often focus on winner vs. loser dynamics, such as: 1) Which corporations will win, 2) Which segments of the population will win (e.g., job loss), and 3) How society will compensate for such radical changes in economic structure (e.g., universal basic income). Outside of this mainstream, a more extreme group of people, especially in Silicon Valley, are worried about human extinction and other such existential risks.

I wonder if the changes that will actually arrive in the coming intelligence age may end up being closer in some sense to the evolution that we saw during the industrial revolution: Namely, a new materialistic system of control (feudalism → capitalism → ?) and a new system of transcendence (religion → therapy → ?). This is simultaneously less extreme than some of the doomerism that has become so popular in Silicon Valley, but also more subtle than the some of the abstract AGI debates, which so quickly converge on an all-out consumerist expansion and a world of plenty where no one needs to work.

Feudalism → capitalism → ?

Capitalism has greatly expanded in the post-war period. If we weren’t actively living through this historic transition, I think it would surprise us a little bit, given how ardently capitalism was critiqued in the early 1900s. The growth in capitalism, I think, can be linked to rising globalization — many of the former laborers, for whom the industrial critique was made, ultimately became part of the capitalist economy, as knowledge workers or in the service economy, where inelastic labor supply expanded wages. Very few Americans still work in manufacturing, as has been noted from many observers.

If knowledge work really does become obsolete, or at least devalued, there is a question of where all this labor will go. Some observers, such as the recent Citrini research piece, have speculated that this labor will shift into the service economy, but this is incoherent in my view. Lower employment should lead to less demand for services, which should shrink the services sector as well. Many AI luminaries have therefore focused on redistribution of wealth by the government as a necessary part of the post-AGI economy. But redistribution is inherently a tool of state capitalism. What if instead, we get an entirely new materialistic quest?

To pose an extreme thought experiment: If the ability to field soldiers was the definition of power for most of human history, and the ability to field capital was the definition of power in our industrial age, then shouldn’t the ability to field the AI be the ultimate power in a coming intelligence age? Could it be possible that we will develop some type of new cultural quest, some new mechanism to determine who controls this AI? And could there be some new definition of value in that system, that is non-monetary?

Religion → therapy → ?

Within each of us, there is some balance that we find between the materialistic quest and the transcendence quest. For most of human history, one could have very little influence on one’s future material well-being: Societal roles were relatively fixed, optionality low, and sustenance the goal. Thus, a thriving religious culture, commemorated now by incredible artifacts that our ancestors left behind: Churches and pyramids and great works of literature and art.

With economic upside unleashed in recent centuries, it should be no surprise that religiosity has declined. The materialistic quest has gained ground over the spiritual one. With what was left of its old vigor, the transcendence quest has evolved into a new form that is more compatible with materialism. This is the rise of the therapeutic. Out are interdicts telling us what we can and cannot do. In is a quest for authentic self-fulfillment, one that each of us must define for ourselves.

I think that this new spiritual understanding is failing a lot of people. It turns out that it’s really hard to figure out how to satisfy oneself from scratch. This new transcendence quest is too open-ended. How many of us know people this is really working for? How many of us know people who are persisting nobly on this quest, but it is not bearing fruit for them?

The rationalist and EA communities in SF are examples of an attempt to construct new ideologies of transcendence. I do not think these are final destinations, but they are forerunners of what is to come. They are interdictory, for example — they have clear rules for what is allowed and is not allowed. Philip Reiff wrote of therapy culture as being a remissive, transitory cultural phase. All remissive ideologies ultimately give way to new interdictory ones, because humans thrive when given structure. Might the new AI age — with its economic and social upheavals — bring with it a new interdictory regime that governs human behavior?

In the extreme, will humans worship AI? What would such worship even look like? Whenever I try to imagine the future of the transcendent, I find myself coming up short, just as a person living in the 17th century probably could never have imagined the therapeutic couch replacing the church pew. Science fiction that imagines everyone on drugs and consuming pleasure all day, such as The Culture, fundamentally misunderstands human motivations and our need for a structure outside of the interiority of the self.

Conclusion

One of my favorite writers, Alasdair Nairn, has pointed out that the AI revolution could simultaneously be of a) An extreme high magnitude, b) Could take place over many decades or even centuries, and c) Could include many peaks and troughs in technological progress and investor enthusiasm. As Nairn points out, the steam engine was not the end of the industrial revolution, but rather a precipitant of many innovations that followed. Likewise, modern LLMs are likely to be a precipitant of wondrous future innovations.

If this pattern is indeed true, then the subtler ways that AI affects our culture — namely, how it permanently alters the quest for power (materialism) and the quest for meaning (transcendence) is likely in its most nascent seed stage. Through the coming decades, these quests will warp and evolve, as all cultural phenomena do, in our minds.

All revolutionary changes begin in the dialogue of ideas. We are in a golden age of ideas right now, with the Internet a hotbed of fascinating discussion. I wonder, who will be the next Adam Smith or Karl Marx of our era? Who will be the Moses or Freud? It is not just a new ideology that we are searching for, but prophets of a new promised land.