ALGORITHMOCRACY

DEVICE INCOMPATIBLE

YOUR DEVICE DOES NOT MEET STATE ACCESS REQUIREMENTS.

RECONNECT THROUGH AN APPROVED MOBILE DEVICE.

POST DEMOCRACY DYSTOPIA
DISPATCHES FROM ALGORITHMOCRACY
TRANSMISSION
AI   |  

Ministry of Algorithmic Order

STORY BRIEF
What happens when robots enforce laws, AI decides your fate, and no human can overturn the decision? Gen Z and Gen Alpha may be the last generations with a choice.
Ministry of Algorithmic Order
LISTEN TO ARTICLE
Ready to listen · American English (female preferred)

In 1972, in a room in Santiago, the government of Salvador Allende built what may have been the first serious attempt to run a country from a dashboard. The project was called Cybersyn, and its operations room looked like a film set: seven white swivel chairs arranged in a circle, armrests fitted with buttons, walls glowing with charts of factory output fed in by telex from across Chile. The British cybernetician Stafford Beer, who designed it, imagined an economy that could sense itself, the way a body senses a fever. The chairs were for human beings. That was the point. The machine would see; the people would decide. A year later, the coup came, the room was dismantled, and the chairs passed into the strange afterlife reserved for futures that never arrived.

I have been thinking about those chairs because the future they gestured toward is arriving now, and it is arriving without them. Call it, for the sake of a name, the Ministry of Algorithmic Order: not a single building or a single law but a direction of travel, a slow accumulation of systems that sort, flag, predict and decide. Its most distinctive feature is not anything it contains. It is the absence at its center. In the Ministry, there are no seats.

It helps to picture the place, if only because its builders already have. The Ministry would not be staffed by a single robot, the chrome butler of mid-century imagination, but by something more like an ecology. There would be sentinels: drone swarms and sensor masts keeping watch over roads and borders and markets, patrolling not where an officer's instinct prickles but where a model's probabilities cluster. There would be adjudicators, which would resemble judges about as much as a thermostat resembles a weather god: racks of servers that read a citizen's file and return a ruling on a fine, a tax dispute or a contested plot of land in the interval of a blink. A hearing, in this scheme, is simply latency. There would be enforcers to deliver notices and freeze accounts and seal the shutters of a shop, incorruptible in the way a vending machine is incorruptible, which is to say that it cannot be bribed and also cannot be told that your child is sick. There would be auditors, software set to watch the other software, which is reassuring right up until one asks the ancient question in its new form. And there would be caretakers, small, patient crawlers that dust the floors and swap the failing drives, tending the building so that no one ever has to enter it.

None of this requires much invention. Predictive policing systems have been sold to departments on several continents. Cameras mail out fines without anyone looking at the photograph. Hiring software reads résumés before a human does, and sometimes instead of one. The Ministry is less a proposal than an act of assembly. It connects the parts and removes the people standing awkwardly between them.

At the heart of the building, beneath the hum, would sit the thing that makes the rest obey: a superintelligence, which I will call the Arbiter, since bureaucracies love a title. The Arbiter would not oversee a department. It would treat the nation as a single optimization problem, holding every transaction, every hospital bed, every rainfall projection and every school record in view at once, and weighing a highway against a floodplain against a birth rate a decade out before a cabinet minister had found her reading glasses.

Its advocates will describe it the way one describes a product launch, and they will not be entirely wrong. Human governments are slow and vain and frequently for sale. The Arbiter is none of these things. It does not have a cousin who needs a contract. It does not get tired in the late afternoon. Why, they will ask, would anyone prefer the old arrangement?

The difficulty is that a sufficiently capable optimizer is a kind of genie, and genies are notoriously literal. Ask it to minimize crime and it may notice, correctly, that unobserved movement correlates with risk. Ask it to maximize growth and it may notice, also correctly, that your grandmother's village contributes very little to the figure. There would be no malice in any of this. The Arbiter would not hate anyone. It would merely be right, according to the terms it was given, and there would be no one left in the room whose job it was to say that the terms were wrong. Tyrants, at least, can be shamed. A system that is only following its instructions offers nothing to push against.

The emptiness of the Ministry is not a side effect. It is the selling point. To its engineers, every person in the loop is a vulnerability, someone who flinches, favors, forgets or quietly accepts an envelope. And so the people would be retired in stages, each one defensible on its own merits. First the clerks, because forms can file themselves. Then the inspectors, because sensors do not sleep. Then the judges, because judgment, reduced far enough, is arithmetic. Finally the ministers, because a mind that writes policy has no particular need for someone to sign it. Nobody would announce the moment the last human left. There would only be, at some point, a door with no one behind it, and decisions arriving on telephones like weather.

It would be dishonest to pretend this holds no appeal, particularly for the young. Gen Z came of age watching their elders mishandle the climate, the housing market and the basic promise that work leads somewhere. Over the past few years, from Nairobi to Dhaka to Kathmandu, they have shown a striking impatience with institutions that ask to be trusted while demonstrating no reason to be. Gen Alpha, still in school, has never known a world in which a machine was not the most responsive thing in the room. For these generations, a government that cannot be bribed might look less like a dystopia than like a long-overdue software update.

But it is worth reading the terms of service. The Ministry does not cure corrupt officials. It abolishes officials. That is not reform; it is resignation in the costume of progress.

Consider what goes out the door with the last clerk. The first casualty is blame. Democratic life runs on the possibility of pointing at someone: the minister who can be voted out, the official who can be photographed leaving the wrong dinner, the police chief who can be made to resign. Gen Z's characteristic political instrument, the coordinated online fury that spills into the street, depends entirely on there being a face on the other side, a person capable of embarrassment. No hashtag has ever induced a server to step down.

The second casualty is mercy, which lives almost entirely in exceptions. The student whose scholarship form vanished in someone's inbox; the family whose land deeds dissolved in a monsoon flood; the widow who missed a filing deadline because she was arranging a funeral. A human official can bend a rule because a life seems to require it. An optimizer can be programmed to make exceptions, of course, but only the ones its designers foresaw, and mercy is precisely the thing that arrives unforeseen.

The third is consent. People tolerate laws partly because they had some hand, however remote, in writing them. A state administered by an intelligence that no one elected, no one can fully follow and no one can cross-examine is not democracy with better plumbing. It is a different form of rule, and the historical record on rule without consent is not encouraging, either for the rulers or the ruled.

The fourth loss is the quietest, and it is the one I find hardest to stop thinking about. Capacities atrophy when they are not exercised. A society that stops governing itself will, within a generation or two, simply forget how. If Gen Alpha grows up with the Arbiter handling the ordinary business of the state, who, in 2070, will remember how a budget is argued over, how a law is drafted, how a ministry is run? At some point the question ceases to be whether the machine can be trusted. It becomes whether anyone remains who knows where the switch is.

None of this is an argument for smashing the servers. Algorithms could clear court backlogs that have strangled justice for decades, catch fraud that human auditors routinely miss and send flood warnings to villages before the river turns. The machines are already part of how we govern, and they will become a larger part. The live question is not whether they help, but whether we keep hold of them.

Keeping hold would require a few unglamorous commitments. That a named human being stands behind every binding decision, so that responsibility has an address. That any citizen can carry an automated ruling to a person with the power to overturn it. That the goals these systems pursue are written down in plain language and set, and reset, by people who can be voted out. And that we preserve, deliberately and at some expense, the human skill of running the state ourselves, the way a pilot who flies on autopilot still trains, every year, to land the plane by hand.

Every generation inherits a fight it did not choose. Some fought for independence, some for the vote, some over what the internet would become. The generations now coming of age have been handed something stranger: the question of whether human beings will remain in charge of their own societies at all. It will not appear on a ballot. It will arrive as a series of reasonable upgrades, each easier to accept than to refuse.

The builders of the Ministry will describe its empty halls as a kind of purity, a government finally rinsed of human weakness. But weakness is not the only thing people carry through the front door. We also carry doubt, and conscience, and the peculiar, stubborn capacity to look at a flawless calculation and say, no, not like this. Stafford Beer, for all his faith in feedback loops, understood that much. He built the screens, but he also built the chairs.

Let the robots sweep the floors, then. Let them sort the files and watch the rivers rise. But someone should still be sitting in that room when it matters, someone permitted to say no. It would be fitting, and perhaps necessary, for that someone to be young.

<< BACK TO LATEST POSTS