An algorithm created 40,000 toxic compounds overnight, researchers report. This illustration shows the process of how the MegaSyn program trains AI.

Swiss Researchers: AI Ability to Create Toxic Substances Could Be Real Threat

As with most things in life, AI algorithms have a light and a dark side. That is to say, the power of AI can be used to add years to a human’s life or to discover life-saving answers to medical questions. But AI can also be used by nefarious characters in a very dark way. And to prove that we looked into an article on wired.com written by Jess Craig, which says it’s not off the wall to believe AI could be used by a “Dr. Evil” who could be creating a dangerous poison with the assistance of AI.

Craig starts with chemical weapons designed by AI in somebody’s basement. In September 2021, scientists Sean Ekins and Fabio Urbina were working on an experiment they had named the “Dr. Evil project.” The Swiss government’s Spiez laboratory had asked them to find out what would happen if their AI drug discovery platform, MegaSyn, fell into the wrong hands.

The researchers were being asked to play the role of the bad scientist, so they took this path.

Normally, MegaSyn would be programmed to generate the most specific and least toxic molecules. Instead, Ekins and Urbina programmed it to generate VX, an odorless and tasteless nerve agent and one of the most toxic and fast-acting human-made chemical warfare agents known today.

Ekins planned to outline the findings at the Spiez Convergence conference—a biennial meeting that brings experts together to discuss the potential security risks of the latest advances in chemistry and biology—in a presentation on how AI for drug discovery could be misused to create biochemical weapons.

“For me, it was trying to see if the technology could do it,” Ekins says. “That was the curiosity factor.”

Fortunately or unfortunately his team discovered AI is good at being an evil scientist. The training of MegaSyn required the program to be careful with what compounds it creates for medicine.

In their office in Raleigh, North Carolina, Ekins stood behind Urbina, who pulled up the MegaSyn platform on his 2015 MacBook. In the line of code that normally instructed the platform to generate the least toxic molecules, Urbina simply changed a 0 to a 1, reversing the platform’s end goal on toxicity. Then they set a threshold for toxicity, asking MegaSyn to only generate molecules as lethal as VX, which requires only a few salt-sized grains to kill a person.

They set boundaries for the algorithm as low as possible and it blew the researcher’s minds.

 And The Results Were Terrifying

Ekins and Urbina left the program to run overnight. The next morning, they were shocked to learn that MegaSyn had generated some 40,000 different molecules as lethal as VX.

“That was when the penny dropped,” Ekins says.

MegaSyn had generated VX, in addition to thousands of known biochemical agents, but it had also generated thousands of toxic molecules that were not listed in any public database. MegaSyn had made the computational leap to generate completely new molecules.

The article is detailed in its explanation of how bad this news could be. The databases the researchers used are out there and available to a person with average computer knowledge. That person may not have good intentions or worse he may have planned a strike somewhere.

The researchers warned that while AI is becoming more powerful and increasingly accessible to anyone, there is nearly no regulation or oversight for this technology and only limited awareness among researchers, like himself, of its potential malicious uses.

“While we were doing this, we realized anyone with a computer and the limited knowledge of being able to find the datasets and find these types of software that are all publicly available and just putting them together can do this,” Ekins says.

“How do you keep track of potentially thousands of people, maybe millions, that could do this and have access to the information, the algorithms, and also the know-how?”

A pretty frightening proposition that Elkins is warning us about. In March, the US Office of Science and Technology Policy (OSTP) summoned Ekins and his colleagues to the White House for a meeting. The first thing OSTP representatives asked was if Ekins had shared any of the deadly molecules MegaSyn had generated with anyone, according to Ekins. (OSTP did not respond to repeated requests for an interview.)

The OSTP representatives’ second question was if they could have the file with all the molecules. Ekins says he turned them down.

“Someone else could go and do this anyway. There’s definitely no oversight. There’s no control. I mean it’s just down to us, right?” he says. “There’s just a heavy dependence on our morals and our ethics.”

read more at wired.com