A Dive Into Complexity

Throw yourself into the unknown with a pace and fury defiant

Part 1 - Hard Times and Good Policy

I have previously argued that our human pattern matching faculties make any popular saying seem true. On that topic, I was watching a video about World War I and it reminded me of the saying:

Hard times make hard men, hard men make good times, good times make soft men, and soft men make hard times.

Or:

Wealth does not last beyond three generations.

Which is why I find Bryan Caplan's essay "The Idea Trap" so insightful. He argues precisely the opposite, that good ideas cause good policies, good policies cause good growth, and finally good growth cause good ideas. It follows then, he argues, that bad growth causes bad ideas.

Is it not the case that most poor countries have not caught up to rich ones, as would be predicted by the theory of catch-up growth? Is it not the case that Communism was most popular internationally in the 1930s, when millions starved under Soviet rule? Did not the Soviet Union and Mao's Communism persist until the 1970s and 80s, decades after the Holodomor and the Great Leap Forward?

You cannot think straight in the midst of hyper-inflation. The society becomes unglued.

Our country is in crisis, unemployment and inflation is sky high, war is on the horizon and there are riots on the streets. Will we elect a stooge of the ruling power, or will we elect the revolutionary, who promises to overturn the failing system? But the reverse of stupidity is not intelligence. And pointing out the flaws of a system does not make one-tenth of the understanding needed to fix it.

When the revolutionaries have rode into power on the back of a bad idea, what else can they do but spur it harder once it falls? As socialism was rising in popularity, many economists comforted themselves that as it was implemented, its own failures would act as a braking mechanism. This was not so.

On the topic of the ruling power, I am reminded of another profound essay, "Rudyard Kipling", by George Orwell. In it, Orwell attempts to explain why a middling, moralising, imperialist poet in Kipling has persisted in his popularity, while generations of liberal poets who derided him have faded into obscurity. It is this passage by Orwell which affected me the most:

He identified himself with the ruling power, and not with the opposition. In a gifted writer, this seems strange to us, or even disgusting, but it did have the advantage of giving Kipling a certain grip on reality. The ruling power is always faced with the question, "In such and such circumstances, what would you do?", whereas the opposition is not obliged to take responsibility or make any real decisions.

Thus I view with horror the accelerationists, whose eyes light up with glee at the arson of what they call "capitalism", who they accord all of society's ills. Better to destroy a corrupt system and rebuild it from anew, they argue. At best, they are callous spectators of our modern Hunger Games, rooting for, from a safe distance, their favoured storyline, be it the riotous David versus the mercantile Goliath, or the Opioid Crisis as a post-industrial morality play. At worst, they are petulant children who, in playing a game of chess, find they can make no winning moves; thus, they overturn the board. And in kicking down Chesterton's Fence, the sheep escape the pen, the wolves eat the sheep, and the homestead starves.

Sidebar 1 - The Banality of Revolution

Politics is inherently suffused with the drama of life, which calls us against all our apathy towards it, yet blinds us to the ordinarity of its operations. Thus to vaccinate ourselves against the smallpox of heroic delusion, we must expose ourselves to the delusion in a different, more banal context.

I thus refer you to "Things you should never do, Part I", by Joel Spolsky, on why you should never rewrite an app from scratch. A few choice quotes:

There’s a subtle reason that programmers always want to throw away the code and start over. The reason is that they think the old code is a mess. And here is the interesting observation: they are probably wrong. The reason that they think the old code is a mess is because of a cardinal, fundamental law of programming:

It’s harder to read code than to write it.

The idea that new code is better than old is patently absurd. Old code has been used. It has been tested. Lots of bugs have been found, and they’ve been fixed. There’s nothing wrong with it. It doesn’t acquire bugs just by sitting around on your hard drive. Au contraire, baby! Is software supposed to be like an old Dodge Dart, that rusts just sitting in the garage? Is software like a teddy bear that’s kind of gross if it’s not made out of all new material?

It’s important to remember that when you start from scratch there is absolutely no reason to believe that you are going to do a better job than you did the first time. First of all, you probably don’t even have the same programming team that worked on version one, so you don’t actually have “more experience”. You’re just going to make most of the old mistakes again, and introduce some new problems that weren’t in the original version.

Part 2 - The Chessboard

The chessboard is not a carelessly chosen metaphor - there are many multitudes more possible moves than there are correct moves. So it is multiple orders of magnitude easier to point out an incorrect move than to make a correct one. I have heard it said many times that people prefer simplicity over truth (by both sides of the political aisle to accuse the other of ignorance, so you know the saying is a bit rubbish - if a word can used to describe everything, it does not really describe anything at all), and in so hearing we imagine a lazy people, incapable of or apathetic to thought.

It is a part of the story, but it is not the whole. Mathematicians pursue simplicity. Simplicity is parsimony is purity is truth. If you ask someone whether a car will go forward or go back when you press on the gas pedal, and they shrug and answer it could be either, you suspect they do not know anything about cars. 

Simplicity is the engine of reason, and if we view complexity as the height of intellectual pursuit, such that we can state with utter seriousness "technological advancement is both good and bad", we will never be wrong, and we will never learn anything. So the partisan ideologue who at least puts himself in the arena is in some sense superior to the mediocre intellectual who languishes in timid neutrality.

But how can I at once decry the over-simplistic communist (the word "socialist" has been diluted beyond all reason, and if I use it here you would have no idea what I meant, and neither would I), while at the same time praising simplicity? At the heart of the confusion is the fact that there are two types of complexity (Don't you just love the English language?)

First, there is the complexity of a thing as it is observed. Whatever we observe is being acted upon by many things, like a pollen grain jiggling in water. This means something we all know - that correlation is not causation. But it means something even more profound - that causation is not even correlation. This complexity we must unravel as much as we can to its simplest parts, to identify causes and effects.

Second, there is the complexity of a thing as it interacts with other things. We understand precisely the physics of a pendulum, but it is exceedingly difficult to predict the exact movement of a double pendulum in a vacuum, which is extremely sensitive to starting conditions. This complexity we cannot avoid.

A sensible person might argue that these two types of complexity are the same thing. Unravel the knot of cause and effect far enough, and the we describe all things in the universe. The keyword is far enough. 

The Shannon Number is an estimate of the complexity of a typical game of chess i.e. the number of possible board states after 40 moves by each player. This is given as 10^120. There are an estimated 10^80 atoms in the observable universe. It is tempting to give a credulous ~wow~ to this statement and immediately forget about it (I have, many times.) But for all it's intellectual pretensions, there is nothing special about chess. You could conjure a similarly large number by combining silly things like the total number of books we could read, tv shows and movies we could watch, varieties of food and drinks we could consume etc. with the number of days in an average human's life to create a fake-profound measure of the the vastness of human experience. Complexity is normal. It is simplicity which is abnormal.

Thus in practice we only unravel the infinite knot of causality here and there, and only as much as we need. We live our day-to-day lives on islands of relative predictability, in an ocean of great unknowns, and we are every now and then battered by the ocean's squall. How to overcome the terror of the open ocean is beyond my current understanding, but in the interest of not giving a problem without at least giving an attempt at a solution, I will say this:

When a system cannot be measured fully, we naturally regress to proxy measures. When a politician commits an error we excoriate him, without understanding if it necessarily benefits the functioning of society. Similarly, we demand greater accountability and transparency, without accounting for all causes and effects. (If it seems obvious we should do so, consider instead if a teenager commits an error - is it obvious that the solution is to excoriate him for his foolishness, and demand he hand over his passwords?)

This is perfectly reasonable. We cannot possibly account for all things. Our unthinking reactions are themselves part of the system, and they, through the miracle of natural selection, work on some basic level. Yet, while a prodigious 5 year-old who describes things as being made of discrete atoms should not be discouraged for being wrong, he should also not be allowed to work as a particle physicist until he learns why he is wrong. Just so, if we are serious about asking ourselves "In such and such circumstances, what would I do?" we must hold ourselves to a higher standard. Not for everything, everywhere, all the time, but we shouldn't give up on unravelling the knot of causality just because it's long, and we especially should not flip the board. With time and practice, we develop better proxy measures, or a greater sense of when to use which measure. Most importantly, we must have a goal, which is the measure of measures. Without a goal, we are adrift, pushed here, reacting there, a piece of flotsam, not a captain of the high seas.

Sidebar 2 - Ideal Equilibrium

I should caution that the existence of this goal does not imply the existence of a system-wide optimum. In the first place, even in a made up political economy with only three or four cardinal values (honesty, compassion, achievement, etc., you can make up your own) that can be directly voted for, the mere fact different people have different preferences destroys any conception of an ideal equilibrium - ideal for whom? Now throw into the pot many, many more cardinal values, all mutually contradictory; which emerge, not from discrete institutions or acts of congress, but from different parts of different institutions, interacting not only with myriad people of different character and circumstance, but also with the myriad overlapping communities these people form; and also these interactions change their participants, so that changes are constantly cascading, interfering, and building up throughout the multi-million or -billion node mesh.

Perhaps if we only had three or four things interacting with each other, we could've calculated the system-wide optimum for some arbitrary value system, even if the value system we chose didn't reflect a meaningful aggregation of voter preferences. With a billion different things to dynamically optimise all hope is lost, and we must come to accept that whatever the differences between between the United States and Finland, or any other wealthy, developed country, are not merely the result of bad preferences, nor a bad copy of the Ideal Solution begrudgingly accepted as a concession to reality. These different societies are, in some sense, equally good. 

Part 3 - The Ghost in the Machine

There is another piece of confusion I want to dissect, that cuts deeper into the question of how we should structure society, that is best captured by the quote "Man is born free, but is everywhere in chains." (Note: I have not actually read Rousseau, so I do not know the proper context of the quote, but the meaning seems clear on the face of it.)

That is, within each of us there exists a free spirit, full of life and vigour, which drives us to become our authentic selves. But society chains this free spirit with its rules and expectations, the cold iron of its shackles draining the spirit of its liveliness. Thus we must overthrow society, and shape it to suit our will. Only then can we truly be free.

Yet.

Where does this spirit come from?

How can there exist a ghost in the machine?

The man who is free from society does not stand astride the world like Colossus, but floats alone in the deep darkness of space, coming from nowhere, going nowhere. The great chain of humanity does not shackle our wrists and ankles, but passes through us. To the extent we are driven, it is because we are being pulled by this chain. (For added complexity, imagine multiple chains, pulling in multiple directions, ouch.)

This is not an unfamiliar idea. Every cell in our body comes from another cell in our body, and the first cell in our body comes from our parents, whose cells came from their parents, up the branches of the tree ancestry, down the trunk of the tree of life. Caged birds never learn how to sing from their parents. Captive orcas never learn how to hunt. Knowledge must be transmitted from generation to generation, or, like a remote village in Appalachia, whose villagers came from the great building culture of England, forget how to even build stone churches.

Something cannot come from nothing, and everything which exists must have a cause. When we cannot predict what we think before we think it, we do not have free will, we merely do not know the outcome of our calculation before we have calculated it. When we want to do well in school, it is because our parents drive us, who were in turn driven by their parents. When we want to earn more money, we are driven by the status of our friends and the images of our idols, who were in turn driven by the status of their friends and their idols. Sooner or later, we tire of being driven and declare henceforth we shall be our own person. Yet, the wave which was passed onto us and which we've passed onto others sooner or later circles back and we are again caught in its pull. And it is when wave builds upon wave that we reach the heights of achievement.

I ask you, when you languish in ennui, lamenting all the things you could do if you just had the will, what thing outside of yourself is driving you? If you fail, what thing outside of yourself drives you to try again? If we dare only to paddle in our own backwash, should we be surprised that the riptide never brings us out to sea?

Part 4 - Mere Will

For people with certain beliefs, there is nothing scarier than to depend on others. To the person who distrusts those outside his circle of family and friends, it is an invitation to be at best hindered and at worst robbed. To the person who struggles with social relations, it is yet another yoke on their neck,  holding them back from all they wish to accomplish. To the person who is born far from those places where strong currents lead to lands of opportunity instead of rocky shore, it dashes the dream that they can from nothing, through mere will, become something. It is natural to be averse to this last point - we as modern people truly and fully believe that anyone can accomplish anything.

First, we should remind ourselves that these people already depend on society, whether they make the most of this realisation or not. Pretending otherwise doesn't make it so. And removing mere will from the equation of success does not add any more unfairness than already existed and which we already recognised.

Second, just because I am opposed to magical thinking, does not mean I think the poor man is therefore doomed to eternal failure. On the contrary, far better that the poor man recognises his lack of drive as situational, knowing that as he, by incremental steps, navigates himself out of his quagmire so too will his drive improve, rather than falsely convincing himself he has a personality defect just because he cannot move things with his mind.

I have also argued elsewhere that the poor man is not as behind as he thinks, so I will not repeat my arguments. A summary of those arguments is that the effectiveness of a person so strongly depends on shared community knowledge that so long as we can hop our way to those communities, to those specific islands of relative certainty, then after a while we are just as effective as those who have been there for much longer. 

What I really want to do in this section is to give one specific piece of advice from one island-hopper to another, that on the surface seems to contradict everything I said earlier in this essay.

Be delusional.

Not about the inclement weather, or the condition of the ship, or the safety of the passage - ignoring the truth of practical matters is equivalent to charting a one-way course to the bottom of the ocean. But be delusional about the ultimate odds of success of a multi-year voyage. This is not so much a deliberate miscalculation as a deliberate throwing out of our intuitive calculations, which tend to be straight line projections from the present to the future. These work relatively well for well-defined tasks and short time periods, and not at all for the far future, which tends to be very different to the present. Because the most important part of a successful voyage is starting, because the second most important part is not stopping, and because the world is so complex we do not truly know the odds of our success, it is by being delusional here we ultimately maximise the odds of our success.