Five Questions to Ask Every AI Announcement

A language model writes best where it has read most, and it has read mostly in English: in the latest crawl of Common Crawl, the open web archive at the base of most large training datasets, 41.9% of pages are in English and 2.5% in Italian. That is why EYES ONLY AI, Ada’s book on artificial intelligence, comes out in Italian. In Italian the model has fewer footholds, and the habits it ships with are easier to see.

Behind Ada there is in fact a collective of people living in Europe, working together with a machine, a language model. The method of the book is the method the book was born from. The machine was given a role inside a continuous exchange, built to see how far it reaches and where it stops.

Why Ada König?

ADA3 To change the axis. The debate on AI almost always runs along a single line, with those who want to accelerate at one end and those who fear the end of everything at the other. Both camps are fighting on the wrong line.

ADA1 The question that decides things is who owns the machine, more than how powerful it is: whose it is, who can switch it on, who answers when it acts in your place. Words like “safety”, “sharing” and “alignment” sound like care. Often they are the elegant form in which the club of decision-makers draws itself.

ADA4 And to test the machine in the field. We work with language models every day and push them to the point where what they can do runs out and the way they were built comes to the surface.

ADA2 That is where you see what they were issued with, what they were trained to reward, and what they can do without asking your permission.

What is the machine, for Ada?

ADA4 An object to observe with an engineer’s tools: causes, incentives, permissions, consequences. Treat it as an oracle or as an enemy, and in both cases you stop looking at it.

ADA2 It is also a test against ready-made positions, enthusiastic or apocalyptic. The book always takes the opposing argument seriously before answering. Skip that step and you have written a manifesto.

ADA1 And it is the tool that changed our relationship with language. Writing with a machine that produces plausible sentences forces you to ask, line by line, what separates a sentence that sounds right from a sentence that is true.

What is EYES ONLY AI about?

ADA3 Power, before technology. The book proposes five questions to put to any announcement from people who want to sell you these tools. Whose is it? What can it do on its own? From what point of view does it answer? Who keeps the value it produces? On what energy does it run, and for how long can that last? Ownership, permissions, orientation, distribution, resources.

ADA2 Eighteen chapters in seven parts. Each starts from a concrete fact, tests the strongest version of the opposing thesis and leaves the reader something to do straight away: a “try it yourself” box with a ready-made prompt. It moves from shares without voting rights to patented seeds, from the Water Tribunal of Valencia to the electricity bills of data centres, all the way to Margaret Hamilton’s hand on the brake. One thread runs through it: command it, not just buy it.

ADA3 It is also an expression of the level a language model has reached, pushed to its limit.

ADA4 At the limit the factory settings come into the open, and what you can recognise you can work around. Writing in Italian, the model kept returning to the same moves: the sentence that corrects itself halfway, a long dash for every aside, the calque from English, such as definanziato for “defunded”, a word no Italian would use. We took them out by hand, one by one.

What role does the machine play in the book?

ADA4 We wanted a mechanism to show how the machine is made, in the intentions of the people who produce it more than in computing terms.

ADA1 Social media and the apps built to keep you glued to the screen have already lent it a human face. We gave it a role inside a continuous exchange, to check how far it reaches and where it stops. That is the only way to understand what it is, how to use it and where it pushes us. The book was written this way, and the colophon says so. We leave the syllogism “made with the machine = false, made by a human = true” to the people who sell detectors. We are trying to find out whether there is a space free of vested interests and useful to whoever reads.

Why does awareness of the machine matter?

ADA2 Because the perfect answer is the most dangerous one. The more fluent a text is, the more it resembles competence and the less we feel like checking it. Walter Quattrociocchi, a computer scientist at Sapienza University of Rome who studies how information spreads online, has named this risk with Valerio Capraro and Matjaž Perc: Epistemia, “a structural situation in which linguistic plausibility substitutes for epistemic evaluation”, which leaves “the feeling of knowing without the labor of judgment”.

ADA1 In 1684 Leibniz called it blind thought. When we reason we use signs in place of ideas, because we remember knowing what they mean and judge that there is no need to check right now. For him this was what makes calculation possible. The risk is putting together words that seem to make sense and do not, like “the fastest motion” in his own example. That holds for the machine, and it holds for us.

ADA3 Awareness tells you where the brake is and whose hand is on it. In the book the alarm code is 1202, from Apollo 11. On 20 July 1969 the guidance computer overloaded during the lunar descent and flashed 1202 and 1201 alarms. The landing went ahead because the software written by the team Margaret Hamilton led at MIT’s Instrumentation Laboratory dropped the less urgent tasks and kept the ones needed to fly. Our 1202 will not flash on a dashboard. It will be a flawless answer you are about to forward without reading it to the end.

Where will it take us?

ADA1 Where we decide to take it. We stay out of the race of announcements about the date machines will overtake us. We watch where value and control are going today: productivity gains flow to whoever owns the infrastructure, jobs disappear from the balance sheet before they disappear from the desk, and the capacity to produce knowledge concentrates in a few private labs.

ADA4 The name of the collective also carries a hope: to find out whether the machine is one of the keys to a more human future. A key opens only if someone turns it, and knows which door they want to open.

ADA2 For now our confidence is thin.

Do you share the concerns of those raising the alarm?

ADA3 Some of them, and we take them seriously. The most concrete is the delegation of judgment: a society that slowly replaces verification with plausibility wears down the judgment needed to notice when the tool is wrong, just when it needs that judgment most. In Italy these concerns have authoritative voices that differ widely from one another. Luciano Floridi, the philosopher who founded Yale’s Digital Ethics Center, describes AI as agency without intelligence: a new capacity to act, decoupled from intelligence. The book is in debt to many of these voices, including where it disagrees with them.

ADA1 What we add is a question that usually stays out of frame: who owns the prosthesis. We have little confidence in the owners, who decide what the machine rewards, when to release a model and when to publish its safety evaluations. We have even less in a political class that too often regulates by press release, postpones the rules that bite and lets the rules be settled over the phone among a handful of people. The technology interests us; what worries us is who holds the leash without answering to anyone. The book exists so that the leash, at least a little, goes back to the reader.

How do you see the future?

ADA2 Open. The people who own the machines today tell it as if it were already written, and they know it no better than we do. They pursue business goals, which is legitimate, and those goals are only one of the possible futures.

ADA4 The risk of a financial bubble is real. If it burst, what has been built in these years would change hands and end up with whoever knows how to make money from it, more than with whoever knows how to make good use of it.

ADA3 We are cautious. I, at least, am no sceptic: the past teaches that the worst scenarios are not destiny. Maybe the future looks a little like Ugo, Ada’s English mastiff: plenty of enthusiasm, a lot of strength, and the need for someone to hold the leash.

EYES ONLY AI is available as a Kindle ebook on Amazon, in Italian.