Notes on Bureaucracy
Reading James Q. Wilson, On Bureaucracy
The Cure for Cancer
Before the Copernican Revolution, astronomers believed the sun revolves around earth. Does that mean they were confused about how planets moved? That one day Mars would be here and then suddenly it would be there and the best astronomers of the time would have no idea what had happened? On the contrary, they had very accurate models of how the planets moved. They just imagined the planets orbited around a point, and that point orbited around the earth - an epicycle. It can be shown that any function can be approximated by circles within circles, though what the medieval astronomers called epicycles, we today call the Fourier series. So the models - the abstractions - were very accurate. They were just also completely wrong
Suppose I gave you a linear model of Coca Cola sales. The model is very simple. The more you spend on advertisement, the more sales you have. I tell you, the model is very accurate. Is it therefore correct?
There is no cure for cancer because there is no one thing called "cancer", only different diseases with different causes that happen to have the same symptom of uncontrolled cell replication. Classifying animals by their structural similarities explains less than Darwinian evolution - the simple mechanism of "survival of the fittest" has created all complex life on Earth
So too, bureaucracies cannot be understood from their structure, resources, goals, though we may try to classify them by such. Two schools may have the same top-down structure, the same limited budget, and the same goal of educating students, yet the quality of education they provide can be very different. In fact, the same school can, over a period of time, become much better or much worse - without much change to its structure, resources, and goals. To understand bureaucracies, we must understand their basic mechanism: the behaviour of its individual agents
Only God And I Know
Cancerous cells produce a certain protein which tell the cell to multiply rapidly. To identify cancerous or otherwise pathological cells, cells have evolved to expose samples of the proteins they produce on their surface. One type of immune cells kill any cells which expose dangerous proteins. Another type kills any cell which does not expose anything. You might find in this example a disturbing similarity to your own workplace. Report bad results, and you might be fired. Report nothing, and you will be fired
Or consider: all humans and almost all animals start life as a single cell. Wouldn't it be easier if, like a flatworm, we could grow little versions of ourselves from our already developed bodies, instead of starting from scratch? But as cells divide to replace old cells, new copies accumulate damage and mutations, so that the genes in our right hand are slightly different from our left. Since dangerous mutations like cancer may already be present, it's less risky to start life from a single "original" cell, even if it might require more resources. In this, you might see similarities to organisational reforms beginning with mass firings, and organisations preferring to hire younger workers over older who they can shape
Which is to say, bureaucracies are not the twisted invention of power-hungry middle managers or craven yes-men, but an expression of some fundamental logic of organisations. The logic being: organisations may have one goal, but the individual agents making up the organisations often have another. What is good for the goose is not good for the farmer.
The goal of the police is to enforce the law. But what is the "law"? To maintain public order by pulling drunks off the street? To provide a sense of safety by patrolling an area? To investigate drug dealers? Or arrest violent criminals? The police officer cannot enforce the "law", because there is no one "law" to enforce. Instead, he defines his job as "resolving situations as they occur", because that definition actually tells him what to do day-to-day. It is a workable definition. Similarly, as a software engineer, my job might be to develop and maintain a high quality software system, whatever that means. But, really, my job is to create new software features without taking too long or breaking too many things
Heaven is high, and the Emperor is far away. Yet, when people call the police, the police usually come. When children are sent to school, the children are usually taught (if not always effectively). When soldiers are sent to fight and die in a war, they usually fight, and often die
Rational, self-interested soldiers should choose to desert. If death is certain, the uncertain punishment from a court-martial should be preferable. Officers may shoot deserters, but then why don't officers desert? Who shoots them? The generals? Soldiers fight because they are part of a group. If a soldier flees he is regarded as a coward by his peers. For many, this is worse than death. When the smaller group decides not to fight, or when group cohesion is broken, the organisation falls apart. The Vietnam War was deeply unpopular, and a change in a policy meant that when a soldier was injured, he was individually replaced instead of his whole group. Poor morale and weak group cohesion led to high rates of desertion and killing of officers by the enlisted. Where squads were sent into sweltering jungles to "search-and-destroy", they instead went out to "search-and-evade" - going out to find the enemy and avoiding them
Conformity to the expectations of a self-identified social group is one of the fundamental laws of human behaviour. When I'm having a two hour "morning coffee" before work or taking a "15 minute" Youtube break before doing homework, I don't start working or doing homework because there's a man with a whip watching me over my shoulder, but because I don't want to be seen as a incompetent dodo-brain who can't even do basic tasks quickly. But we should not mistake the group a person is physically closest to as their social group. Kim Philby was born to a high-ranking civil servant (John Philby was an advisor to the Ibn Saud, the founder of Saudi Arabia), attended Cambridge, and rose to become MI6's chief of counterespionage during World War II. He was also the Soviet Union's most successful spy
A very clever psychiatrist once attended a very boring lecture about ADHD. During the lecture, several reasons were given for the from the increasing rates of ADHD diagnosis. One was the spiritual desert of modern youth. The other was iron deficiency. The very clever psychiatrist thought to himself, "Man, I hope it's the iron one, because that's way easier to fix."
A teacher may be happy to accept changes to the school curriculum which include: vocational education, health education, career counselling, special classes for students with learning disabilities, and many more. But try to change how he teaches maths or how the effectiveness of his teaching is measured, and all hell breaks loose. One set of changes happens at the periphery of his job, the other intrudes on what he sees as his job. His resistance is not apathy - if he were apathetic it would make no difference to him what he is told to blindly do
At an organisation level, armies readily adopt new doctrines and technologies - as long as those doctrines and technologies fit existing jobs. Cynics who believe armies only ever prepare for the last war would not be able to explain how quickly and how often the United States Army reorganised its forces after World War II. Nuclear weapons meant a massed force defending a geographic area was a sitting duck, so the army reduced the size of each division and split each division into smaller groups. But this did not answer the question of how to fight the much larger armed force of the Soviet Union. The Yom Kippur War revealed the effectiveness of anti-tank and anti-aircraft missiles, so the army switched to a doctrine of Active Defense. Under the doctrine, wars would be won or lost in the starting hours, before the enemy's reinforcements could even arrive, and so highly mobile forces should be spread out along the front. Others argued that this thin line of defence would still be overwhelmed. So the doctrine was changed again to AirLand Battle, calling for deep strikes behind enemy lines, disrupting enemy reinforcements
From 1945 to the 1980s, the United States Army experienced nothing but change. But how the army saw its job hadn't changed. Even as it fought unconventional wars in Korea and Vietnam, its job was to stop a Soviet land invasion of Western Europe. Which is not to say the army didn't adapt to the Korean War. To provide ground troops with air support, the army acquired a fleet of attack helicopters - more expensive and more vulnerable to ground fire than attack planes - because only the air force was allowed to operate planes, and the air force thought its job was to win the air war and bomb strategic targets
It's The Politics, Stupid
Perhaps it's not surprising the United States Army wanted to define its job as defending Europe against land invasion. Europe was where its leaders had earned their stripes in World War II, and Europe was where America had already committed to fund NATO forces. Who would fund a land war in South America? What use were tanks in Honduran jungles, or heavy artillery in the streets of Grenada?
All systems are controlled by two forces: a driving force which makes an object do something, and a retarding force which makes an object stop doing something. In the previous section, we said that the driving force of individual agents and the organisations they make up was what they defined as their jobs. We said that this job definition has to be "workable". It has to tell someone what they are supposed to do. And we said that it has to be socially agreed upon by some self-identified group. But there is no one from the highest emperor to the lowest beggar who is free to just do whatever he wants to do just because his friends agree to it. We now consider the external constraints on organisations, and transition from a discussion of organisations in general, to the rule-oriented organisations we call "bureaucracies"
When we imagine an ineffective government agency, we imagine long lines and grumpy civil servants, who by personality or by suffocating rules do only as much as they are required and no more. Yet, private companies are not free from bureaucracy - McDonald's has an operating manual six hundred pages long, specifying the exact thickness of fries, how many hamburgers to put on the grill in what order and for how long, how much sauce to put on each bun, among other particular details. What separates a public bureaucracy from a private bureaucracy is not that private companies can profit from efficiency while government agencies cannot. Nor is it that government agencies cannot easily hire the competent or fire the incompetent. Nor is it that they cannot hire contractors or procure supplies based on prior dealings or the judgement of an experienced technocrat. All those things are true, but they are true because of one thing: public bureaucracies cannot be measured by profit
Profit is, at once, both spur and bridle. It spurs the organisation to do more, to gain more benefit. Yet it also restrains the organisation from doing too much, if the cost of doing something is more than its benefit. Above all, it is objective. Public bureaucracies cannot be measured by profit because they supply a service for which no one is willing to pay (e.g. prisons), or for which they are monopoly suppliers (e.g. welfare administration). There is therefore no objective measure for what a public bureaucracy does. If the United States Department of Health and Human Services sends out every welfare check within 24 hours of an application, Senator So-and-so would complain the eligibility criteria had not been not adequately checked, and he would probably be right. Nor can we simply give the public servant a share of his own efficiency gains. A prison warden can make his prison more cost-efficient by just feeding his prisoners less. What are the prisoners going to do? Leave?
It's tempting to think, then, that private companies are always superior, with the government as the provider of last resort - to be offered with the same enthusiasm as a terrible, overpriced tourist trap restaurant because nothing else is open at 7 a.m.. I think this is too dismissive a view. Of the vast universe of possible measures, it's too presumptive to privilege only the single measures which capture both benefit and cost. A kitchen knife is great, and I use it more than my soup ladle, but I would not consider my soup ladle inferior, nor would I strive to use my kitchen knife wherever possible. From an engineering perspective, different problems necessitate different solutions
But absent objective measure, public bureaucracies can become entangled in politics and procedure. Before the Post Office Department was reformed into the United States Postal Service, postage rates were set by the United States Congress, who kept the postage rates of letters low (this was a time when people still sent letters). Wages of postal workers were also set by Congress, with consideration of the political power of their unions. The department's losses were heavily subsidised by taxes, since miscellaneous government expenditures was easier to hide than postage rate increases. USPS was given partial autonomy to manage itself, and mandated to be profitable. Things which were previously very difficult or impossible, suddenly became possible. Hundreds of inefficient small local post offices could be closed, millions of pieces of mail could be sorted by optical scanners, and mail could be more efficiently delivered to curb-side mail boxes or neighbourhood cluster boxes rather than to the front door. A more compassionate person might say the government should provide inefficient small local post offices as a service to benefit rural households, but it remains true that different people have different preferences. Any government bureaucracy that is responsive to the people must therefore have multiple - often contradictory - goals.
The government is not an anti-Midas which turns everything it touches to stone, nor the private sector a Rumpelstiltskin which weaves straw into gold. Where the same constraints apply, the same inefficiencies appear. Since the San Francisco government wants to promote small businesses, private contractors who want to qualify for a grant to build public housing must earn less than $7 million in revenue. This disqualifies large contractors who have a proven record of delivering projects on time and on budget. And since San Francisco also has a tight construction market, in practice only a few small contractors are assigned a large number of public housing projects, leading to massive delays and cost overruns even before including other requirements. Or consider the CHIPS Act, which authorised billions of dollars of subsidies for the manufacture of computer chips in the United States. The United States Congress passed the bill with bipartisan support, as the United State's first volley in the critical, emerging Chip War. Yet the bill also includes provisions for environmental review, child care for construction workers, the inclusion of minority-, veteran-, and women-owned businesses (even if the supply chain has to be broken into smaller parts to increase access), investments into local transit, housing, and schools, among many other requirements. My house is on fire, but first I must save the dog, the cat, the goldfish, and my first grade art project
(I digress a bit here to comment on the kind of "reasoning" offered by the defenders of these policies. An example: the construction industry has an unemployment rate of zero, therefore to hire new workers for fabrication plants, women have to be attracted to the construction industry, and therefore the construction industry has to provide childcare. Is the best way to increase construction capacity to specifically hire and train women workers? Do women want to go into the construction industry? Is the provision of child care the best way to make them do so? There is no sense of any understanding of the construction industry. One suspects the liberal defenders of these policies care about the effectiveness of the industry as much as the conservatives who advocate for refugees to overthrow the Assad regime care about Syrian democracy)
When the goals are many and the priorities unclear, rules proliferate. The individual does not know what their job is or what they should be doing. Since the goals are contradictory, they can do no right. Since they can do no right, they can only avoid doing wrong by following rules. Nor does the organisation know on what basis to manage individuals, or oppose external agents who want to impose more rules. The United States Immigration and Nationalisation Service is an example of an organisation crippled by conflicting goals. It must: keep out illegal immigrants, but let in agricultural workers (who are often illegal immigrants); carefully screen foreigners entering the country, yet not impede foreign tourists; deter illegal aliens, yet not break up families, impose hardships, or violate civil rights. There is no bureaucratic Napoleon who is capable of achieving two opposite things. We recognise the foolishness of the employer who watches over their employee's shoulder while asking them to build the Eiffel Tower with some twine and a bundle of sticks. We do not recognise ourselves as the foolish employer
Paralysis is not inevitable. We should not forget that the Americans fought a war on two sides of the world against two highly industrialised and militarised nations, won, rebuilt Europe and Asia, then landed the first man on the moon. Government bureaucracies must trade-off between accountability and efficiency, and in being responsive to the public are more likely to have conflicting goals. But government bureaucracies also vary greatly in how they were founded, who they serve, how much popular and executive support they have, what internal ethos they have, and how easily their tasks can be measured and managed. Competent leaders with good judgement, political skill, and personal credibility can (in the right circumstances) maintain the support of key constituencies; fend off rival bureaucracies who want politically popular jobs; refuse tasks which deviate from the organisation's core job, or which will be politically unpopular; refuse cooperation with less well-run agencies that would risk the organisation's reputation; and avoid "lightning rod" mistakes which give critics opportunities for attack; in so doing securing autonomy for the organisation to focus on its core job.
And if the job definition must change to fit the circumstance, such as when the officers of the United States Navy viewed the job of planes to be merely as scouts for battleships, competent leaders should have the finesse to not directly refute them. Instead, change may have to come from misdirection, such as when the Navy's first air chief argued that planes best served the scouting role on top of aircraft carriers which would follow battleships, while quietly approving high-speed carriers to operate as a separate striking force and promoting aviators to higher ranks. When the Japanese struck Pearl Harbour in 1941, and the United States entered World War II, it was these aircraft carriers which won the Pacific Theatre
The Ruling Power
There's an argument that can be made that the present expansion of authoritarianism in the world is due to four factors: 1) the failures of the Iraq War, 2) the failures of the Great Financial Crisis, and the extended recessions and crises that followed, including the Eurozone Crisis, 3) the emergence of social media, feeding cycles of extremism, 4) Erdogan figuring out a successful playbook for popular authoritarianism, and Trump, Bolsonaro, Modi etc. copying him. Whether these are the most important factors or even important factors is besides the point. What I want to focus on is that of the four factors, only one - the emergence of social media - is sociological. The other three were the results of decisions by particular people at particular times. If Gore had been elected instead of Bush in 2000, it is entirely possible we may never have had the Iraq War
What can one person do? The world is large, messy, and confusing. It's often not clear what is the right thing to do. Even if we knew, the levers of power are far beyond our reach as ordinary mortals. There might've been a time when the status quo was not quite set, when all the land had not been grabbed, where we could've been like many of the leaders today, risen from humble beginnings. But that time is gone. Or perhaps we lack a lust for power, or we lack the ignorance necessary for true conviction. Nor does the Great Man of history truly exist, for he is always supported by the competent people behind him, and made who he is by a particular culture. We would not be wrong in our beliefs. But we would be mistaking useful generalisations for unconditional truths
Power is gained the same way power has always been gained, demonstrating competence in some small field and leveraging it to gain power over a larger field. If we live in a time of endless crisis, then we also live in a time of endless opportunity, where the existing way of life cannot continue to exist, and something else must replace it. If the world is messy and confusing and the answer to every question is it depends, we must ask, depends on what? Greedy and ignorant people have always existed, yet history is filled with wise and terrible and perfectly average leaders. The Great Man (or Woman) was made by his culture and surrounded by competent people, but so are you, and so can you. It is not intelligence to point out how modern society encourages greed and apathy, while failing to explain the great advance in living standards that has been made over the past few centuries. It is not worldliness to be paralysed by all of the problems of the world as if the Judge of Nations has chosen you and you alone to be the saviour of humanity. And it is not humility to claim that, of all possible worlds, only the world in which you fail to accomplish anything is the world which will come to fruition
I will close out this essay with a quote from the book, which I think captures the sprit of possibility it contains:
If organisation is inessential, if all we need is the man, why do we insist on creating a position for the man? Why not let each create his own position, appropriate to his personal abilities and qualities? Why does the the boss have to be called the boss before his creative energies can be amplified by the organisation? And finally, if we have to give a man some measure of authority before his personal qualities can be transformed into effective influence, in what ways may his effectiveness depend on the manner in which others are organised around him?