The Ghost of Hari Seldon: Asimov's Foundation Revisited in the Light of Complexity Science

yuval bloch

July 14, 2025

The Ghost of Hari Seldon: Asimov’s Foundation Revisited in the Light of Complexity Science

As a child, no book shaped my worldview more than Isaac Asimov’s Foundation. While others dreamed of studying at Hogwarts or exploring Narnia, I longed to shape the future of the galaxy as part of the First or Second Foundation. Years later, with a computer science degree in hand, I embarked on a search for my next intellectual challenge. That search led me to Stefano Allesina’s paper, “Going Big.”

As I read it, one thought echoed in my mind: “This is Hari Seldon’s psychohistory.” The paper’s central idea—shifting focus from the unpredictability of individual elements to the emergent patterns visible at massive scales—felt like a real-world version of Asimov’s vision. The notion that systems chaotic at the micro level can reveal stable, predictable patterns when viewed in aggregate immediately brought me back to my childhood dream. I’ve encountered this idea repeatedly in my journey through complexity science—whether in scaling laws for cities, network theory, or ecological dynamics.

Of course, complexity theory isn’t psychohistory. Many of Asimov’s imagined predictions are mathematically impossible, and some real-world discoveries would have been unimaginable in the 1950s. Yet, there’s something prophetic in how Foundation anticipated the shift toward understanding large-scale systems, collective behavior, and emergent order.

In this post, I want to explore some of Asimov’s most visionary ideas—those that complexity science has since affirmed, those that remain beyond our reach forever, and those that might still be developed in a future Galactic Empire.

As I’ve been writing this, an intriguing idea has emerged: Revisiting Asimov’s universe through the lens of modern complexity theory could be achieved through fanfiction. Imagine a Foundation-inspired story that leverages our existing knowledge of complex systems. It would be an ambitious project—a narrative vehicle to explore scientific ideas through fiction. I’ll only dive into it if I know there’s a curious crowd out there.

If this concept excites you, please respond to the link that led you here or send me an email. I’d love to hear your thoughts.

And now, let’s begin.

Unveiling Hidden Laws:

The core principle of psychohistory is the existence of predictable, hidden laws and patterns in mass behavior. While Asimov imagined these laws they yet to be found in social systems, but since then we have made significant progress.

Urban Scaling Laws

One of the most remarkable findings in this field concerns the relationship between key properties in cities, such as GDP, innovation, and even crime. These elements demonstrate a superlinear increase with a city’s size. This means that simply adding more people to a city doesn’t just proportionally boost these properties; it enhances them more than their share, following a precise mathematical formula. This phenomenon is primarily driven by the exponential increase in interactions within dense societies, fostering opportunities for both innovation and, unfortunately, certain types of crime. Network theory provides a robust framework for understanding how this precise mathematical relationship emerges.

What’s even more surprising is the underlying driver of a significant portion of this super-linearity. A pivotal 2023 paper, “Urban scaling laws arise from within-city inequalities,” reveals that much of this accelerated growth is attributed to the “long tail” of the distribution. This refers to the small number of individuals or entities disproportionately account for a large share of the total. Specifically, as a city grows:

The wealthiest few become even richer, at a rate that outpaces the overall growth of the population.
The most effective innovators tend to innovate more frequently and have a greater impact.

While these highly successful individuals might attribute their achievements solely to personal decisions and a small circle of partners, the mathematical reality of urban scaling demonstrates that their level of wealth and innovative output is deeply intertwined with the city’s total population and the billions of interactions within the urban fabric that they may not even be consciously aware of. Their success, in part, is an emergent property of the vast, interconnected system of the city itself.

This is just one example of many mathematical rules of society that have been discovered in the last few decades, and there is no reason to believe we will not find more in the future.

Now, let’s challenge the concept of psychohistory.

Predictability and Population Size

Another core principle of Asimov’s psychohistory is that the more people you have, the easier the prediction becomes. This implies that individual deviations average out in larger populations, resulting in more stable and predictable macro-level trends. But sometimes even small groups exhibit surprising easy-to-predict behavior

Consider, for example, the classic Karate Club Network, which I’ve already described in my post on conflicts Empathy in the Emergent Storm This real-world event involved a university karate club with just 34 members. A conflict arose between the instructor and the club’s administrator, leading to a dramatic split of the entire club into two distinct factions.

A researcher, Wayne Zachary, meticulously documented the social interactions between all members before the split. The fascinating part is that, based purely on the network structure of friendships and interactions, he was able to predict with remarkable accuracy (all but one member) which faction each person would join after the conflict.

This example of predictability on a small scale presents a subtle challenge to the “more people, easier prediction” axiom and highlights a key distinction between complexity science and psychohistory. Complex systems, due to their fractal or self-similar nature, often exhibit scale invariance. This means the underlying rules governing structure at one level can re-manifest across different scales—for instance, the organization of a house within a block, a block within a neighborhood, or a neighborhood within a city. Consequently, if a rule holds for a large population, it often applies to smaller ones as well, allowing insights from micro-scale interactions to inform macro-level understanding and vice versa.

Zachary's Karate Club network — The Zachary Karate Club network is split using network modularity (dashed lines) and based on the actual conflict-driven division (colors). The network structure accurately predicts the real-world split of the club.

The Edge of Predictability

Now, after this start, you might think we’ll soon predict everything, even beyond Asimov’s imagination. Not so fast. There are fundamental mathematical limits to prediction.

One limit, as Asimov noted, is that a prediction’s subject can’t know the prediction. If revealed, their behavior changes, rendering the prediction false. This implies the predictor needs a sustained knowledge advantage.

Consider stock prices. Everyone wants to know which stock will rise tomorrow, but you can’t. This is due to the Efficient Market Hypothesis (EMH). While each agent focuses on their own profit, financial markets naturally become complex, decentralized computers, continually optimizing stock prices to represent a company’s prospects based on all publicly available information. This means any prediction derived from existing data is already “priced in.” Future price changes occur only with new information, making them inherently unpredictable.

This illustrates a core limitation: agents in complex systems (like markets) rapidly adapt to new information or predictive methods (e.g., AI). To make consistent predictions, you must possess an informational or computational edge that the rest of the agents lack.

This applies beyond markets. Many complex systems thrive on natural decentralization, allow them to solve challenging computational problems and often outperforming centralized approaches. Think of slime mold solving logistics better than city planners (see in life on the edge in section Memory, Computation, and Becoming), or how organically grown cities outperform planned ones, natural system tend to become an efficient computer to solve specific problem.

Simply put, you cannot predict a system that is efficiently solving complex computational problems unless you can solve those problems yourself – a task often impossible even for the largest supercomputers (e.g., NP-hard problems).

The Ultimate Boundary: Chaos

While complex systems often thrive at the “edge of chaos”—a state of dynamic equilibrium between order and disorder—they generally don’t fully cross that line. However, some systems, or certain phases within them, are truly chaotic.

When a system is genuinely chaotic, it means that even the tiniest, most imperceptible change in its current state will dramatically and fundamentally alter its behavior over time. Though deterministic (meaning they follow underlying rules), chaotic systems are, by their very nature, inherently unpredictable in the long term. This “butterfly effect” represents the ultimate boundary to any form of psychohistory, as even perfect knowledge of the present state would not guarantee accurate foresight into a distant future. Asimov, on some level, accounted for this with the creation of the Second Foundation, implying that some systems cannot be predicted, and thus their impact on the rest cannot be entirely foreseen. Therefore, a living, adapting system had to account for those events and direct the plan based on their results.

Phase diagram of the Lorenz equations — The Lorenz equations provide a classic example of chaos in a simple deterministic system. Tiny differences in initial conditions can lead to drastically different outcomes over time.

The Enduring Legacy of Isaac Asimov

In conclusion, it’s truly remarkable how many of Isaac Asimov’s visionary ideas, conceived 70 years ago in Foundation, have proven to be prescient. The enduring relevance of his work is evident not only in contemporary science fiction but also in real scientific discourse and the perspectives of influential thinkers today. Foundation has undeniably shaped modern ideas about complex systems, large-scale behavior, and the very nature of prediction.

While some of Asimov’s specific predictions may not have materialized, the true power of science fiction isn’t about being perfectly accurate. It’s about conceiving grand concepts that remain relevant, stimulate thought, and inspire generations to explore the profound questions of humanity and its future. Asimov’s “dead hand” continues to guide and challenge our understanding of the universe.