San Francisco Frontiers - Why AI makes computer worms harder to stop

A computer worm is malicious software that spreads between devices on its own.

In June, Nicolas Papernot, an associate professor of electrical and computer engineering at the University of Toronto, led a team that built a prototype AI worm using freely available AI models.

Papernot, who has signed an open letter with recommendations for policymakers, spoke to AFP about why AI makes these attacks harder to stop.

The interview has been edited for length and clarity.

Q: How is AI changing these types of attacks?

A: The reality is that a lot of these "IoT" (Internet of Things or connected home) devices have very poor cybersecurity hygiene, and so they are very easy targets for bad actors to compromise.

Historically, a worm would exploit a fixed set of vulnerabilities, and that meant you could fix these vulnerabilities and prevent the worm from propagating any further to devices that had been updated.

Now, with AI, an attacker can design a worm that tailors the attack strategy to that device's specifics and these vulnerabilities can be different.

This makes it significantly harder to interrupt the propagation of an AI-driven computer worm, because you have to anticipate all the possible vulnerabilities the devices might present.

Some of these vulnerabilities are not software bugs.

They can also be misconfigurations -- for instance, if someone has used the same password for multiple services, or hasn't configured a database properly.

What AI-driven malware does is accelerate the pace at which these vulnerabilities are exploited and the scale.

In the past, your corner coffee shop, or the laptop in your basement that you use very rarely, might not be up to date, but it wouldn't be targeted by a human attacker, because they don't have the resources and the time to do that.

Now, the autonomy gained by using AI means that all these devices that are less immediate targets become interesting targets as a launchpad to attack more interesting targets.

But we have to keep in mind that a human is launching an agent.

Once launched, the agent can replicate, as we showed in our work, but it is launched by a human.

Q: What can we do to mitigate the risks?

A: We won't bring the risk to zero and we have to realize that, despite the fact that some of these vulnerability fixes are available, people don't update their devices or don't apply the fixes in a timely manner.

Despite cybersecurity experts calling for increased cybersecurity for decades, we still live in a world where a lot more can be invested in improving the security of our computer systems.

There are really three things we recommend in the very short term.

The first is to change development and testing practices to ensure that AI models are initially disconnected from the internet, as a precaution, to understand their behavior better before we expose critical infrastructure on the internet [to them].

The second recommendation is to create an international body to standardize both how we train and how we contain AI models. Both are important. We want to train models that are safer.

The last part is about making reporting of security incidents mandatory. Right now, we are only getting information as a function of what AI developers are willing to release, and that's a big problem.

In particular, the academic community needs a lot more information about incidents to be able to develop countermeasures much more quickly.

Otherwise, we have to do a lot of research on understanding the threat first, before we can develop the countermeasures. Otherwise we won't see what bad actors are doing until an attack is launched.

T.Martin--SFF