Disarming LLMs