An advanced AI system generated 40,000 potentially lethal molecules in under six hours. Typically used for drug discovery, it was deliberately repurposed in a controlled experiment to highlight risks, as presented at a key conference on chemical weapons control.
The Swiss Federal Institute for Nuclear, Biological and Chemical Protection hosts biennial conferences uniting global disarmament experts. These gatherings assess emerging developments in nuclear, biological, and chemical domains, evaluating security risks and strategies for international mitigation.
Collaboration Pharmaceuticals, specialists in machine learning models for toxicity prediction across industries, presented on potential misuse of these tools. Researchers demonstrated their AI, MegaSyn, by intentionally engineering toxic molecules to underscore preventive measures.
Normally, MegaSyn identifies and avoids toxic compounds. Here, the team inverted its logic to prioritize and retain toxins, training it to generate novel combinations.
The experiment targeted molecules akin to VX, the highly potent nerve agent developed in 1952 and deadlier than sarin. Remarkably, MegaSyn produced over 40,000 candidate chemical weapons in six hours.
This inversion transformed a beneficial tool for medicine into a generator of hazardous substances using accessible, open-source methods. To mitigate risks, key technical details were omitted from publication.
“The standout finding was that many generated compounds were more toxic than VX,” notes lead author Fabio Urbina in an interview with The Verge. “VX is among the most potent known toxins, making this especially alarming.”
These are computational predictions, published in Nature Machine Intelligence, with no lab synthesis planned. “Predictive models are highly reliable,” Urbina adds. “Despite false positives, potent threats likely remain.”
This demonstration warns of AI's dual-use potential for harm. The researchers urge heightened awareness, robust safeguards, and policy updates.
