26 October 2019 9:00 AM

Human Compatibile: AI and the Problem of Control
Stuart Russell
Allen Lane, pp.338, £25

The AI Does Not Hate You: Superintelligence, Rationality and the Race to Save the World
Tom Chivers
Weidenfeld, pp.304, £16.99

Depending on how you count, we are in the midst of the second or third AI hype-bubble since the 1960s, but the absolute current state of the art in machine cognition is still just about being better than humans at playing chess or being about as good as human beings at analysing some medical scans. It was recently revealed that many thousands of humans were secretly hired to check recordings of people interacting with the ‘intelligent assistants’ on their iPhones or other such devices: much of what is trumpeted as ‘AI’ is still, in fact, dependent on invisible human labour in the digital sweatshop.

Given all this, and the plain threats the world faces from natural stupidity, how worried should we be about some future Skynet-like AI taking over the world and enslaving or destroying humanity? Well, some brilliant people are very worried indeed: they include the philosopher Nick Bostrom, whose 2014 book Super-intelligence popularised the modern version of the problem, and the physicist Max Tegmark, whose 2017 book Life 3.0 I highly recommend for an overview of the story so far.

The science writer Tom Chivers, in The AI Does Not Hate You, delivers a pleasantly journalistic if rather dishevelled account of how such personalities figured in the rise of an AI-focused internet community called the Rationalists. Are they just another millenarian movement of tech-inflected eschatology, or alternatively, as one sceptic says, a ‘sex cult’? Even if they are, we might need to take them seriously, as we should take seriously the possibility of a big asteroid hitting the Earth again: it might be unlikely, but the downside is an extinction-level event.

Not among the cultists is Stuart Russell, author of a seminal textbook on AI computing, whose own new book is a more nutritiously reasoned and orderly investigation. Russell is unusual among computer scientists in that he has read widely in literature and philosophy, and so his arguments are richer and more detailed (as well as funnier) than those of your average Silicon Valley guru who blithely assumes that ‘tech’ can solve problems that have puzzled mere human geniuses for millennia.

Much of Russell’s book, indeed, is a brilliantly clear and fascinating exposition of the history of computing thus far, and how very difficult true AI will be to build. It has, though, historically always been a bad bet to say that some scientific or technological revolution (e.g. flying machines) is simply impossible, so Russell turns his attention to a projected future of real AI, which he guesses might happen in around 80 years. What then? Why, then we face the subtitular ‘problem of control’. Given a machine that, ex hypothesi, is far more intelligent than a human, how do we make sure it acts only in ways beneficial to us?

Take the notorious paperclip problem. You tell the AI to make paperclips. What could be more innocent? Unfortunately you haven’t told the AI when to stop making paperclips, so it carries on until it has turned all the available atoms on Earth, including all the human beings, into paperclips, and it doesn’t stop then. Even more unfortunately, you can’t turn it off once you realise your mistake, because the AI has, quite logically, decided that in order to fulfil its given goal of making paperclips, it must protect itself by disabling its off switch, otherwise it wouldn’t be able to carry on making paperclips.

This example generalises, in Russell’s colourful yet rigorous exegesis, to the lesson that it is impossible fully to specify any goal we give a machine to prevent catastrophic misunderstandings. His proposed solution, to simplify a lot of nuance, is to avoid giving the machine ‘goals’ at all, but to build into it a kind of epistemic humility: it should infer what to do from observing the preferences of humans. Of course some humans will have evil preferences, but that sounds like a start, as does his lovely passing endorsement of ‘researchers with good taste’, and his recommendation that we all should have a right to mental security, to ‘live in a largely true information environment’.

Meanwhile we can fondly hope that the current AI hype will at least bear fruit in a consumer version of the Berkeley Robot for the Elimination of Tedious Tasks, which, Russell tells us, ‘has been folding piles of towels since 2011’. Another AI researcher tells Chivers, indeed, that he thinks the whole threat of machines ever doing much worse than folding towels for us is overblown: to him, ‘the basic premise of an intelligent thing destroying the world was silly, and not really in keeping with what “intelligence” means’. Hmmm. Look out the window: how is the only example with which we are familiar of intelligence with a capacity to alter the world doing so far?

I’m both disappointed, and a little relieved to know I won’t be around in 80 years for the advent of “real AI”.

Machines have an 'off' switch.

My fear of AI is vastly eclipsed by power hungry politicians. The evil and greed of people that think they run the world is a far greater threat.

quote:

Originally posted by gearhounds:
My fear of AI is vastly eclipsed by power hungry politicians. The evil and greed of people that think they run the world is a far greater threat.

Class, what's wrong with this response?

Point taken; should have left off at "not worried about AI"...

At some point, the machines will be built by machines, and, without telling us, they'll remove the off switch.

quote:

Originally posted by parabellum:
Machines have an 'off' switch.

quote:

Originally posted by parabellum:

Machines have an 'off' switch.

A hammer is an off switch for just about everything.

Anyone who thinks that machines cannot be disabled is living in a dream world. This ain't Terminator we ain't implementing Sky Net. Turn off the scifi movies and come back to reality

I like to write trading algorithms for use in the financial markets. Specifically, my creations are technically called adaptive algorithms and I've been refining and writing them for 20 years.

This type of coding is of interest to those who are trying to develop AI systems so I do consult on occasion. ALL of the AI systems I've tested come up short in terms of what we all would consider "AI" to be.

What's interesting about the work is not the technology but the attitudes and goals of the various organizations who want to buy and implement these systems. None of them really want a sentient, self aware machine, they simply want SOMETHING that can follow simple directions without a bunch of BS. It seems to be a rare trait.

One of the places I work has a guy who's job is getting a piece of equipment to a certain place at a certain time. He's paid $28.00 per hour and he just can't do it - no extraneous variables. I can't wait to replace him with a robot.

I think that a person who can follow directions, has an attention span greater than 10 minutes and who doesn't buy into the idea that the world owes them a living without personal productivity,...... well, I think these folks will have a future irregardless of what's coming, tech wise.

V.

I found this CGP Grey video on robots replacing jobs interesting:



Now when robots start repairing robots, we might be in trouble...