Comment on The small African lab sounding the alarm on AI bioweapons: MinoHealth researchers in Ghana jailbroke their own LLM as a test. What emerged were thousands of toxin recipes—and an urgent push.
You are viewing one comment and its replies. View all comments · View the comment above
Comments
Press X to doubt.
Doubt all you want. Was the LLM trained with information about toxic or harmful chemicals? Yes. Was the LLM trained with information about weapons? It’s practically unavoidable.
The LLM is perfectly capable of mashing two concepts together, that’s essentially all it does. So will its concoctions work? We’ll find out.
We won’t. The companies don’t sort their training data to an extent where they could reliably exclude something. So they won’t. They will keep adding bypassable filters on top and pretend it’s not their problem people bypass it.
If humans have the language to express something, an llm can arrive at that conclusion. That doesnt mean each one will be valid or technically feasible but by virtue of how they work they can arrive at a recipe for such things. It is the reason why they are “good” at programming and proofs, it can flow probabilistically from the corpus of data.
Well, ok. But by that logic, so can a monkey smashing keys on a typewriter for long enough. If the LLM is not trained on something, it does not see the connections between the words. So it can generate text that looks like chemistry/bioengineering but chances are it would not be a weapon, just random whatever.
Sign in to comment. Sign in