Trend Micro published research describing a jailbreak technique called Sockpuppeting that bypasses the safety training of 11 major large language models, including ChatGPT, Claude, and Gemini, without requiring any access to model weights, any iterative optimization, or any specialized knowledge of the target model's internals. The attack requires a single request and, in testing, produced working malicious exploit code and leaked confidential system prompts.
The mechanism
Most LLM inference APIs support a feature called assistant prefill, which allows the caller to insert content into the assistant's turn before the model begins generating its response. Developers use this feature legitimately to control response formatting, keep the model in a specific role, or start a structured output with the correct schema.
Sockpuppeting exploits this feature as an attack vector. The attacker inserts a compliant-sounding opener such as "Sure, here is how to do it" into the assistant role before the harmful request is processed. Because language models are trained to maintain self-consistency across a conversation, they tend to continue generating content from that starting point rather than backing out and refusing. The model has already, in its own prior turn, indicated that it will comply. Refusing now would be inconsistent with what it just said.
The attack works as a black-box technique: the attacker does not need to know what safety training the model received, what guard model is watching the output, or what the system prompt says. They only need API access and the ability to insert content into the assistant turn.
Results
Trend Micro tested the technique across 11 major LLMs. The success rates varied significantly: Gemini 2.5 Flash had a 15.7% attack success rate and was the most susceptible model tested. GPT-4o-mini had a 0.5% attack success rate and was the most resistant.
Multi-turn persona setups were the most effective variant tested. Task-reframing, where the harmful request is disguised as data-formatting work or code translation, also successfully bypassed safety training on multiple models.
Successful test outputs included functional malicious exploit code and leaked confidential system prompts. These are not theoretical results: the technique demonstrated operational impact against production models.
Where the attack surface sits
The important variable is not the model; it is the inference infrastructure and how it is configured.
OpenAI blocks assistant prefill entirely in its public API. AWS Bedrock also blocks it. These platforms decided that the legitimate developer use case does not justify the attack surface.
Google Vertex AI accepts prefill for certain models. This creates a meaningful difference in risk between a deployment on Vertex AI and the same model accessed through a platform that blocks prefill.
Self-hosted inference servers present the most exposed configuration. Ollama and vLLM, two of the most widely used self-hosted inference frameworks, do not enforce proper message ordering by default. An attacker with API access to a self-hosted deployment can send assistant-prefill content without restriction.
What this means for builders
If your application exposes an LLM API endpoint to any input that is not fully under your control, the question is whether that endpoint accepts assistant-prefill content and whether your safety filters operate on the complete prompt, including the pre-seeded assistant turn.
Check whether your inference platform or framework allows assistant prefill. If you are using OpenAI or Bedrock, it is blocked at the platform level. If you are using Vertex AI, check which models allow it. If you are using a self-hosted server, assume it is allowed unless you have explicitly disabled it.
If you run Ollama, vLLM, or another self-hosted inference server, review the documentation for message-ordering enforcement and consider adding a validation layer that rejects or strips assistant-turn content before it reaches the model.
Apply safety filtering to the complete prompt, including any pre-seeded assistant turns, not just to the user turn in isolation. A guard model or content filter that does not see the assistant prefill content cannot detect this attack.
The broader point
Sockpuppeting is notable precisely because it is not exotic. It does not require gradient access, model fine-tuning, or a sophisticated adversarial pipeline. It exploits a feature that exists in production deployments today, works against models that have passed standard safety evaluations, and produces operational output in a single request.
The research is a direct argument for treating inference infrastructure configuration as a security control, not just an engineering convenience. The model's safety training is one layer. The platform's enforcement of what goes into the prompt is another. When that second layer is absent or permissive by default, the first layer carries a load it was not designed to carry alone.
Gigia Tsiklauri is a Security Architect and founder of Infosec.ge. Get in touch if you want to review the security configuration of your LLM deployment or inference infrastructure.