Human Out of the Loop, Asimov and Gradual Disempowerment

The doom scroll rules our lives. Classifiers and recommenders have understood the mechanisms of attention to the point that we are all addicted to technology (in Portuguese) and don’t recognize it. Society is reacting: Meta reached a settlement with the American states in the trial over teen addiction. The European Union wants to ban social media accounts for children under 15.

The machine, the algorithm, knows what you will like. And even what you won’t. It is well known how the people who design technology hijack our minds like a slot machine.

The same loop that rules your attention at night is the one running your Claude Code during the day. In any agentic tool, we are entering more intense work loops, with shorter cycles, without being given time to be critical of our own work. Some tweets blow up and show this climate of lost agency, besides a possible AI witch hunt. We are taking the human out of the loop:

Highlights mine; see the original tweet.

“If we were at least given the time to check out the code” can be read as the nostalgia of the extinct typist, but it is a plea to keep deciding. It is Terence Tao’s helicopter (in Portuguese): you drop straight onto the waterfall without understanding the path.

How do we resist this predefined path? The conservative Society for Humanity appears in the last story of I, Robot:

If they had time on their hands, they would oppose mathematics or literature. The reactionaries who make up the Society for Humanity claim that the Machine robs man of his soul. Yet there is still a shortage of truly capable men in our society; we still need men intelligent enough to find the right questions to put to the Machine.

I read “The Evitable Conflict” in 2008 and reread it this week. In the story, artificial intelligence plans and governs the whole of the human economy, leaving nothing short and nothing in excess. Production and consumption so aligned that it no longer makes sense to argue about Adam Smith or Marx (in Asimov’s words). At some point, planet Earth starts having small problems… delays in construction here, a small food shortage there… How, if the Machine is perfect? Could people be trying to sabotage the “algorithm”?

Then listen. Every action taken by a director who does not follow exactly the instructions of the Machine he works with becomes part of the data presented to the Machine in the next problem. As a result, the Machine knows that this director has a certain tendency to disobey. The Machine can incorporate that tendency into the data, even quantitatively, that is, calculating exactly when and in what direction the disobedience will occur. Its answers would be skewed just enough that, when the director disobeyed, he would automatically correct the answers, bringing them to the optimal direction. The Machine knows, Stephen!

Given the laws of robotics, the machine will find a way to help and protect us. Even if that means paying whatever cost it deems necessary. A “well-intentioned” paperclip maximizer that, instead of consuming human blood, takes in data and produces a sufficient minimum of ultra-processed culture:

Perhaps an agrarian or pastoral civilization, with fewer people and less culture, would be better. If so, the Machines must move in that direction, preferably without telling us, since, because of our ignorant prejudices, we only consider good what we are used to, and we would fight the change. Or perhaps the answer is total urbanization, or a caste-ridden society, or complete anarchy. We don’t know. Only the Machines know; they are going there and taking us with them.

The Asimov passages above are translated back from the Brazilian edition I read; the original is in English.

Total urbanization or pastoral society? Only the Machines know. When Agent Smith interrogates Morpheus, he says the first Matrix was a perfect world. It was a disaster: we couldn’t handle such a universe, and the Machine added suffering and misery to keep us busy.

Play

p(doom), the probability of a disaster happening through artificial intelligence, doesn’t need armed robots or biological disasters. It is enough to keep dominating the human psyche. Choosing for us. Taking the human out of the loop, slowly, produces gradual disempowerment: we have only a false sense of agency, but it has already been handed over to the machines, to the networks, to the stories and short videos… ultra-processed culture.

We delegated route planning and we obey Waze. We outsourced finding restaurants to apps. We are about to put our conversations and emotions at the disposal of the chatbots’ echo chambers. The film Her is the past.

Literature has drawn this scenario for even longer. In 1958, in Asimov’s “The Feeling of Power”, an unimportant technician shows his “incredible ability” to multiply nine by seven without a computer, to the amazement of the New Pentagon. In 2005, Ted Chiang, of the film Arrival, wrote a piece in Nature, “What’s expected of us”, where the Predictor toys manage to turn on a light one second before you press them… with a devastating effect on their users:

Like a legion of Bartleby the Scriveners, they no longer engage in spontaneous action. Eventually, a third of those who play with a Predictor must be hospitalized because they won’t feed themselves. […] The ability to move remains, but the motivation is gone.

Resisting may make you look like a reactionary, like the Society for Humanity. And the Machine already knows you will resist, it knows where you will click like… it’s in the data! Ted Chiang ends his story with a recommendation: pretend you have free will. Act as if your decision matters, because it does. Stay in the loop. Debate, criticism and regulation are collective actions too.