& cplSiteName &

To Stop AI From Going Rogue, Make It Neurotic

Mitch Wagner
6/9/2017
50%
50%

To reduce risk of AI getting out of control, machines should be made to doubt themselves, according to researchers at the University of California Berkeley.

In other words, make AI neurotic.

In a paper, researchers Dylan Hadfield-Menell, Anca Dragan, Pieter Abbeel and Stuart Russell say that one of the best ways to prevent AIs from misbehaving is to make sure we can turn them off. Even an AI that doesn't care about its own life will want to remain active -- or alive -- to make sure it can continue to be useful. So the solution is to make the AI doubt whether it's useful.

It is clear that one of the primary tools we can use to mitigate the potential risk from a misbehaving AI system is the ability to turn the system off. As the capabilities of AI systems improve, it is important to ensure that such systems do not adopt subgoals that prevent a human from switching them off. This is a challenge because many formulations of rational agents create strong incentives for self-preservation. This is not caused by a built-in instinct, but because a rational agent will maximize expected utility and cannot achieve whatever objective it has been given if it is dead. Our goal is to study the incentives an agent has to allow itself to be switched off.

It's not enough that the robots will take our jobs. They'll also be neurotic, constantly requiring reassurance.

My colleague Jamie Davis at Telecoms.com has some other ideas for personality types that can be programmed into AI to keep them in line:

Conceited Carl: As an AI application, Carl is no danger. Carl spends too long staring into the mirror to fix his hair, down the gym pumping iron or on ASOS trying to figure out what Jon from Love Island was wearing last night. This AI is so concerned about how it appears there is very little risk of world domination. Just tell it that its algorithm looks a little bit flabby and it will be on the virtual cross-trainer in no time.

Also, Hypochondriac Henrietta, Boring Benjamin, and more. Read it here: "How do you stop AI from taking over the world? Make it neurotic of course."

I fear I may be Boring Benjamin. Would you like to discuss the relative merits of Mac to-do list software?

Related posts:

— Mitch Wagner Follow me on Twitter Visit my LinkedIn profile Visit my blog Friend me on Facebook Editor, Enterprise Cloud News


Keep up with the latest enterprise cloud news and insights. Sign up for the weekly Enterprise Cloud News newsletter.


(9)  | 
Comment  | 
Print  | 
Newest First  |  Oldest First  |  Threaded View        ADD A COMMENT
More Blogs from Wagner’s Ring
SD-WAN is about more than saving money – it also provides application delivery, insights and reliability. Find out more in this podcast sponsored by Citrix.
Platform is designed to enable enterprises to build big data analytics apps that move easily between public and private clouds.
Buying Evident.io extends Palo Alto's portfolio with API-based security capabilities and compliance automation.
Google wants to win the hearts of enterprise IT for Chrome OS on the desktop, but it has a long way to go.
IBM Cloud gets a security and Kubernetes performance boost.
Featured Video
From The Founder
Light Reading founder Steve Saunders talks with VMware's Shekar Ayyar, who explains why cloud architectures are becoming more distributed, what that means for workloads, and why telcos can still be significant cloud services players.
Flash Poll
Upcoming Live Events
May 14-16, 2018, Austin Convention Center
May 14, 2018, Brazos Hall, Austin, Texas
September 24-26, 2018, Westin Westminster, Denver
October 9, 2018, The Westin Times Square, New York
October 23, 2018, Georgia World Congress Centre, Atlanta, GA
November 7-8, 2018, London, United Kingdom
November 8, 2018, The Montcalm by Marble Arch, London
November 15, 2018, The Westin Times Square, New York
December 4-6, 2018, Lisbon, Portugal
All Upcoming Live Events
Hot Topics
I'm Back for the Future of Communications
Phil Harvey, US News Editor, 4/20/2018
Verizon: Lack of Interoperability, Consistency Slows Automation
Carol Wilson, Editor-at-large, 4/18/2018
AT&T Exec Dishes That He's Not So Hot on Rival-Partner Comcast
Mari Silbey, Senior Editor, Cable/Video, 4/19/2018
Facebook Hearings Were the TIP of the Data Iceberg
Dan Jones, Mobile Editor, 4/20/2018
Pay-for-Play Is a Sticking Point in Congress
Mari Silbey, Senior Editor, Cable/Video, 4/18/2018
Live Digital Audio

A CSP's digital transformation involves so much more than technology. Crucial – and often most challenging – is the cultural transformation that goes along with it. As Sigma's Chief Technology Officer, Catherine Michel has extensive experience with technology as she leads the company's entire product portfolio and strategy. But she's also no stranger to merging technology and culture, having taken a company — Tribold — from inception to acquisition (by Sigma in 2013), and she continues to advise service providers on how to drive their own transformations. This impressive female leader and vocal advocate for other women in the industry will join Women in Comms for a live radio show to discuss all things digital transformation, including the cultural transformation that goes along with it.

Like Us on Facebook
Twitter Feed