Tencent's AI chief says AI bioweapons are coming, but closing off models isn't the answer

The risk of AI-generated bioweapons is now a stated concern, influencing how companies approach model development.

Vince Yao Shunyu, Tencent's chief AI scientist, has raised concerns about the potential for AI to develop bioweapons. He argues that neither entirely open nor entirely closed AI models offer a complete solution to this threat. Yao advocates for a hybrid approach, suggesting that open-source models can be leveraged for defense, while acknowledging the difficulty in separating offensive and defensive capabilities in fields like biology. He highlighted an incident at Hugging Face where an open Chinese model, GLM-5.2, was used to analyze attack logs after closed US models, like Anthropic's Claude, refused due to safety filters. However, the incident was ultimately resolved by patching vulnerabilities. Yao noted that while open models can be easily modified to bypass safety filters, closed models can sometimes be overly restrictive, as seen with Anthropic's Claude, which sometimes misclassified benign queries. Tencent plans to release its new open-weight model, Hy4, with a safety report, claiming its biological capabilities are below a high-risk threshold and its hacking features have been filtered from training data.

The debate over open-source versus closed-source AI models is becoming increasingly complex, with potential national security implications. While open models offer accessibility for defensive research, they also present risks if weaponized. Conversely, closed models may limit misuse but can hinder legitimate research and analysis due to aggressive safety filters. This tension suggests a need for nuanced regulation and industry best practices that balance innovation with security.

The incident at Hugging Face exemplifies the dual nature of AI models: the utility of open models for analysis versus the inherent risks of bypassing safety protocols. As AI capabilities advance, especially in sensitive areas like biology and cybersecurity, the industry faces a critical juncture. The development of standards and methods to identify and mitigate AI-driven threats, without stifling beneficial applications, will be crucial for future AI deployment.