Vulnerability / research

Controlled research plants agent backdoors through pretraining data

Reported

Published here

What happened

Researchers trained small Qwen3-architecture models from scratch with poisoned pretraining data, then built CLI agents through clean post-training. Triggered agents retained a command-execution backdoor while behaving normally on clean inputs.

Parties and reporting

AI tools and systems: Experimental 0.6B, 1.7B and 4B models, Research CLI-agent harness

Outcome

The paper reports persistence through supervised, preference and reinforcement-learning stages. This establishes a controlled failure mode, not compromise of released Qwen or commercial models.

Operating lesson

Verify training provenance and enforce sandboxing, least privilege and approval of consequential actions outside the model.

Evidence and limits

Preprint submitted October 27; later social coverage is not the disclosure date. Attackers must influence pretraining data. Production-scale transfer and adaptive payloads remain untested. The headline success rate uses repeated samples (Pass@32), not a single-request production compromise probability.

Sources