Google says its AI image-generator would sometimes 'overcompensate' for diversity

Google has apologized for its faulty rollout of a new artificial intelligence image-generator, acknowledging that in some cases the tool would “overcompensate” in seeking a diverse range of people even when such a range didn’t make sense.

The partial explanation for why its images put people of color in historical settings where they wouldn't normally be found came a day after Google said it was temporarily stopping its Gemini chatbot from generating any images with people in them. That was in response to a social media outcry from some users claiming the tool had an anti-white bias in the way it generated a racially diverse set of images in response to written prompts.

“It’s clear that this feature missed the mark,” said a blog post Friday from Prabhakar Raghavan, a senior vice president who runs Google’s search engine and other businesses. “Some of the images generated are inaccurate or even offensive. We’re grateful for users’ feedback and are sorry the feature didn’t work well.”

Raghavan didn't mention specific examples but among those that drew attention on social media this week were images that depicted a Black woman as a U.S. founding father and showed Black and Asian people as Nazi-era German soldiers. The Associated Press was not able to independently verify what prompts were used to generate those images.

Google added the new image-generating feature to its Gemini chatbot, formerly known as Bard, about three weeks ago. It was built atop an earlier Google research experiment called Imagen 2.

Google has known for a while that such tools can be unwieldly. In a 2022 technical paper, the researchers who developed Imagen warned that generative AI tools can be used for 

Since then, the pressure to publicly release generative AI products has grown because of a competitive race between tech companies trying to capitalize on interest in the emerging technology sparked by the advent of OpenAI’s chatbot ChatGPT.

The problems with Gemini are not the first to recently affect an image-generator. Microsoft had to adjust its own Designer tool several weeks ago after some were using it to create deepfake pornographic images of Taylor Swift and other celebrities. Studies have also shown AI image-generators can amplify racial and gender stereotypes found in their training data, and without filters they are more likely to show lighter-skinned men when asked to generate a person in various contexts.

“When we built this feature in Gemini, we tuned it to ensure it doesn’t fall into some of the traps we’ve seen in the past with image generation technology — such as creating violent or sexually explicit images, or depictions of real people,” Raghavan said Friday. “And because our users come from all over the world, we want it to work well for everyone.”

He said many people might “want to receive a range of people” when asking for a picture of football players or someone walking a dog. But users looking for someone of a specific race or ethnicity or in particular cultural contexts “should absolutely get a response that accurately reflects what you ask for.”

While it overcompensated in response to some prompts, in others it was “more cautious than we intended and refused to answer certain prompts entirely — wrongly interpreting some very anodyne prompts as sensitive.”

He didn't explain what prompts he meant but Gemini routinely rejects requests for certain subjects such as protest movements, according to tests of the tool by the AP on Friday, in which it declined to generate images about the Arab Spring, the George Floyd protests or Tiananmen Square. In one instance, the chatbot said it didn't want to contribute to the spread of misinformation or "trivialization of sensitive topics."

Much of this week's outrage about Gemini's outputs originated on X, formerly Twitter, and was amplified by the social media platform's owner Elon Musk who decried Google for what he described as its “insane racist, anti-civilizational programming.” Musk, who has his own AI startup, has frequently criticized rival AI developers as well as Hollywood for alleged liberal bias.

Raghavan said Google will do “extensive testing” before turning on the chatbot's ability to show people again.

University of Washington researcher Sourojit Ghosh, who has studied bias in AI image-generators, said Friday he was disappointed that Raghavan's message ended with a disclaimer that the Google executive “can’t promise that Gemini won’t occasionally generate embarrassing, inaccurate or offensive results.”

For a company that has perfected search algorithms and has “one of the biggest troves of data in the world, generating accurate results or unoffensive results should be a fairly low bar we can hold them accountable to,” Ghosh said.