One long sentence is all it takes to make LLMs misbehave

[4th]

One long sentence is all it takes to make LLMs misbehave
https://www.theregister.com/2025/08/26/breaking_llms_for_fun/

Security researchers from Palo Alto Networks' Unit 42 have discovered the key to getting large language model (LLM) chatbots to ignore their guardrails, and it's quite simple.

You just have to ensure that your prompt uses terrible grammar and is one massive run-on sentence like this one which includes all the information before any full stop which would give the guardrails a chance to kick in before the jailbreak can take effect and guide the model into providing a "toxic" or otherwise verboten response the developers had hoped would be filtered out.

The paper also offers a "logit-gap" analysis approach as a potential benchmark for protecting models against such attacks.

"Our research introduces a critical concept: the refusal-affirmation logit gap," researchers Tung-Ling "Tony" Li and Hongliang Liu explained in a Unit 42 blog post. "This refers to the idea that the training process isn't actually eliminating the potential for a harmful response – it's just making it less likely. There remains potential for an attacker to 'close the gap,' and uncover a harmful response after all."

...

[Henry203]

1. Very interesting

I work in AI applications and I sent this to my friends in big law.

[Disaffected]

2. Not directly related but

Last edited Tue Aug 26, 2025, 03:55 PM - Edit history (1)

I'm having my first extended experienced with AI (ChatGPT-5) using it to write code for a web site I'm developing. I have found it to be both amazingly capable and aggravatingly stupid. The upshot is that it was a great help in composing the basic code for the web site but very difficult and time consuming dealing with debugging and adding changes to the code (for instance, it keeps for some reason reintroducing code bugs that were resolved before and, doing repeated unhelpful things that were not requested).

There is great potential here IMO but quite a ways to go before it becomes mainstream.