Find Us


Voice mimicry and mind control: Big Tech slams the ethics brakes on new Artificial Intelligence

New AI could be susceptible to hacking and manipulation of thoughts.
New AI could be susceptible to hacking and manipulation of thoughts. Copyright Canva
Copyright Canva
By Reuters
Published on
Share this articleComments
Share this articleClose Button

IBM told Reuters its AI Ethics Board has begun discussing how to police an emerging frontier: implants and wearables that wire computers to brains.


In September last year, Google's cloud unit looked into using artificial intelligence to help a financial firm decide whom to lend money to.

It turned down the client's idea after weeks of internal discussions, deeming the project too ethically dicey because the AI technology could perpetuate biases like those around race and gender.

Since early last year, Google has also blocked new AI features analysing emotions, fearing cultural insensitivity, while Microsoft restricted software mimicking voices and IBM rejected a client request for an advanced facial-recognition system.

All these technologies were curbed by panels of executives or other leaders, according to interviews with AI ethics chiefs at the three US technology giants.

Reuters reported for the first time their vetoes and the deliberations that led to them reflect a nascent industry-wide drive to balance the pursuit of lucrative AI systems with a greater consideration of social responsibility.

"There are opportunities and harms, and our job is to maximize opportunities and minimize harms," said Tracy Pizzo Frey, who sits on two ethics committees at Google Cloud as its managing director for Responsible AI.

Regulating AI

Microsoft, for instance, had to balance the benefit of using its voice mimicry tech to restore impaired people's speech against risks such as enabling political deepfakes, said Natasha Crampton, the company's chief responsible AI officer.

Rights activists say decisions with potentially broad consequences for society should not be made internally alone. They argue ethics committees cannot be truly independent and their public transparency is limited by competitive pressures.

Jascha Galaski, advocacy officer at Civil Liberties Union for Europe, views external oversight as the way forward, and US and European authorities are indeed drawing rules for the fledgling area.

If companies' AI ethics committees "really become transparent and independent – and this is all very utopist – then this could be even better than any other solution, but I don't think it's realistic," Galaski said.

The companies said they would welcome clear regulation on the use of AI, and that this was essential both for customer and public confidence, akin to car safety rules. They said it was also in their financial interests to act responsibly.

They are keen, though, for any rules to be flexible enough to keep up with innovation and the new dilemmas it creates.

Among complex considerations to come, IBM told Reuters its AI Ethics Board has begun discussing how to police an emerging frontier: implants and wearables that wire computers to brains.

Such neurotechnologies could help impaired people control movement but raise concerns such as the prospect of hackers manipulating thoughts, said IBM Chief Privacy Officer Christina Montgomery.

Noel Celis/AFP
Since early last year, Microsoft has restricted software mimicking voices.Noel Celis/AFP

Ethics committees

Tech companies acknowledge that just five years ago they were launching AI services such as chatbots and photo-tagging with few ethical safeguards, and tackling misuse or biased results with subsequent updates.

But as political and public scrutiny of AI failings grew, Microsoft in 2017 and Google and IBM in 2018 established ethics committees to review new services from the start.

Google said it was presented with its money-lending quandary last September when a financial services company figured AI could assess people's creditworthiness better than other methods.

The project appeared well-suited for Google Cloud, whose expertise in developing AI tools that help in areas such as detecting abnormal transactions has attracted clients like Deutsche Bank, HSBC and BNY Mellon.

There may be areas which are so sensitive that you will see tech firms staying out deliberately until there are clear rules of road.
Bill Foster
US Congressman

Google's unit anticipated AI-based credit scoring could become a market worth billions of dollars a year and wanted a foothold.

However, its ethics committee of about 20 managers, social scientists and engineers who review potential deals unanimously voted against the project at an October meeting, Pizzo Frey said.

The AI system would need to learn from past data and patterns, the committee concluded, and thus risked repeating discriminatory practices from around the world against people of colour and other marginalized groups.

What's more the committee, internally known as "Lemonaid," enacted a policy to skip all financial services deals related to creditworthiness until such concerns could be resolved.

Google also said its second Cloud ethics committee, known as Iced Tea, this year placed under review a service released in 2015 for categorising photos of people by four expressions: joy, sorrow, anger and surprise.


The move followed a ruling last year by Google's company-wide ethics panel, the Advanced Technology Review Council (ATRC), holding back new services related to reading emotion.

The ATRC - over a dozen top executives and engineers - determined that inferring emotions could be insensitive because facial cues are associated differently with feelings across cultures, among other reasons, said Jen Gennai, founder and lead of Google's Responsible Innovation team.

Reproducing voices and faces

Microsoft, meanwhile, developed software that could reproduce someone's voice from a short sample, but the company's Sensitive Uses panel then spent more than two years debating the ethics around its use and consulted company President Brad Smith, senior AI officer Crampton told Reuters.

She said the panel - specialists in fields such as human rights, data science and engineering - eventually gave the green light for Custom Neural Voice to be fully released in February this year. But it placed restrictions on its use, including that subjects' consent is verified and a team with "Responsible AI Champs" trained on corporate policy approve purchases.

IBM's AI board, comprising about 20 department leaders, wrestled with its own dilemma when early in the COVID-19 pandemic it examined a client request to customize facial-recognition technology to spot fevers and face coverings.


Montgomery said the board, which she co-chairs, declined the invitation, concluding that manual checks would suffice with less intrusion on privacy because photos would not be retained for any AI database.

Six months later, IBM announced it was discontinuing its face-recognition service.

What is the EU saying about it?

In an attempt to protect privacy and other freedoms, lawmakers in the European Union and the United States are pursuing far-reaching controls on AI systems.

The EU's Artificial Intelligence Act, on track to be passed next year, would bar real-time face recognition in public spaces and require tech companies to vet high-risk applications, such as those used in hiring, credit scoring and law enforcement.

US Congressman Bill Foster, who has held hearings on how algorithms carry forward discrimination in financial services and housing, said new laws to govern AI would ensure an even field for vendors.


"When you ask a company to take a hit in profits to accomplish societal goals, they say, 'What about our shareholders and our competitors?' That's why you need sophisticated regulation," the Democrat from Illinois said.

"There may be areas which are so sensitive that you will see tech firms staying out deliberately until there are clear rules of road".

Indeed some AI advances may simply be on hold until companies can counter ethical risks without dedicating enormous engineering resources.

After Google Cloud turned down the request for custom financial AI last October, the Lemonaid committee told the sales team that the unit aims to start developing credit-related applications someday.

First, research into combating unfair biases must catch up with Google Cloud's ambitions to increase financial inclusion through the "highly sensitive" technology, it said in the policy circulated to staff.


"Until that time, we are not in a position to deploy solutions".

Share this articleComments

You might also like