San Francisco Frontiers - Ghost in the machine: Deepfake tools warp India election

Political parties are harnessing powerful artificial intelligence tools to make deepfakes, reproducing famous faces and voices in ways that often appear authentic.

Both the government and campaigners have warned that the spread of such tools is a dangerous and growing threat to the integrity of elections in India.

With a marathon six-week general election starting on April 19, so-called "ghost appearances" -- the use of dead leaders in videos -- have become a popular mode of campaigning in the southern Tamil Nadu state.

Actress turned politician J. Jayalalithaa died in 2016, but she has been featured in a voice message deeply critical of the state's current governing party, once led by arch-rival M. Karunanidhi.

"We have a corrupt and useless state government," her digital avatar says. "Stand by me... we are for the people."

Karunanidhi died in 2018 but has appeared in AI-generated videos -- clad in his trademark black sunglasses -- showering praise on his son M. K. Stalin, the state's current chief minister.

Recycling "very charismatic" speakers offered a novel way to grab attention, said Senthil Nayagam, founder of Chennai-based firm Muonium, which made the AI video purporting to be Karunanidhi.

Resurrecting dead leaders is also a cost-effective way of campaigning compared to traditional rallies, which are time-consuming to organise and expensive to stage for voters accustomed to a grand spectacle.

"Bringing crowds is a difficult thing," Nayagam told AFP. "And how many times can you do a laser or drone show?"

- 'Very thin line' -

Prime Minister Narendra Modi's Bharatiya Janata Party (BJP) has been an eager early adopter of technology in election campaigning.

In 2014, the year he swept to power, the party expanded Modi's campaign reach by using 3D projections of the leader to make him appear virtually at rallies.

But harnessing technology that can clone a politician's voice, and create videos so seemingly real that voters struggle to decipher reality from fiction, has naturally sparked concern.

Ashwini Vaishnaw, the communications minister, said in November that deepfakes were "a serious threat to democracy and social institutions".

AI creator Divyendra Jadoun said he had received a "huge surge" of requests for content from his company, The Indian Deepfaker.

"There is a huge risk in this coming election, and I am pretty damn sure many people are using it for unethical activities", the 30-year-old said.

Jadoun's repertoire includes voice cloning, chatbots and mass dissemination of finished products through WhatsApp messaging, sharing content instantly with up to 400,000 people for 100,000 rupees ($1,200).

He insisted that he turned down offers that he disagreed with, but said it was a "very thin line" to determine whether or not a request for his services was unethical.

"Sometimes even we get confused," he added.

Jadoun said the rapidly advancing technology was little understood by a "big part of the country", and AI products were taken by many to be true.

"We only tend to fact-check videos which don't align with our preconceived notions," he warned.

- 'Threat to democracy' -

Most AI-generated campaign material has so far been used to lampoon rivals, especially through song.

This week a leader of the BJP's youth wing posted an AI-generated video of Arvind Kejriwal, a leading opponent of Modi arrested last month in a graft probe.

It shows him sitting behind bars, strumming a guitar and singing a verse from a popular Bollywood song: "Forget me, for you have to live without me now."

Elsewhere, digitally altered videos purport to show lawmaker Asaduddin Owaisi, one of India's most prominent Muslim politicians, singing devotional Hindu songs.

A caption alongside the video on Facebook jokes that "anything is possible" if Modi's party, known for its Hindu-nationalist politics and accused of discriminating against India's Muslim minority, wins again.

Joyojeet Pal, an expert in the role of technology in democracy from the University of Michigan, said that ridiculing a political opponent was a more effective campaigning tool than "calling them a thug or a crook".

Mocking opponents in political cartoons is a centuries-old tactic, but Pal warned that AI-generated images can easily be misinterpreted as real.

"It is a threat to what we can and cannot believe," he said. "It is a threat to democracy as a whole."

A.Smith--SFF