From 639c16baf8bea9c877e2a6bdbe22600c695fe94a Mon Sep 17 00:00:00 2001 From: Spyros Gasteratos Date: Thu, 27 Aug 2026 10:33:23 +0100 Subject: [PATCH] docs(chat_settings): document that agent responses are untrusted text MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Nothing in the customer-facing docs told integrators how to treat the agent's reply. NCC found a client deployment where the model could be talked into emitting in a response body — the payload was smuggled across several turns as innocuous profile values, then the agent was asked to reassemble them (NCC-E033177-3UJ). The platform is right not to strip markup: the correct encoding depends on the render context and the platform cannot know it. But that only works if the people consuming responses know it is their job, and the docs never said so. Adds a section to the chat settings reference covering what to encode where, and calls out that safety filters screen content and are not an output encoder — that conflation is the likely reason an integrator would assume they were covered. Refs SEC-304. Co-Authored-By: Claude Opus 5 --- .../docs/reference/resources/chat_settings.md | 21 +++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/docs/docs/reference/resources/chat_settings.md b/docs/docs/reference/resources/chat_settings.md index 3cc14ff8..6e6c9428 100644 --- a/docs/docs/reference/resources/chat_settings.md +++ b/docs/docs/reference/resources/chat_settings.md @@ -107,6 +107,27 @@ See the [Safety filters reference](./safety_filters.md) for the full schema, fie Chat greeting, style prompt, and safety filters are validated as a set: if any one of `chat/configuration.yaml`'s `greeting` or `style_prompt` sections or `chat/safety_filters.yaml` is present locally, the other two must be present as well. `poly push` fails with a "Webchat config resources must all be present together" error if one is missing. +## Rendering agent responses + +!!! warning "Agent responses are untrusted text" + Treat the agent's reply exactly as you would treat text typed by an end user. + +Agent responses are generated by a language model. Anyone who can talk to the agent can influence what it says, including persuading it to produce HTML, JSON or XML. Any integration that renders a response into a markup context must encode it for that context. + +Safety filters screen for harmful *content*. They are not an output encoder, and they do not make a response safe to insert into a page. + +The platform does not strip or escape markup from responses, and this is deliberate: the correct encoding depends on where the string is finally rendered, and the platform has no way of knowing. Encode at the point of use. + +| Where you render it | What to do | +| --- | --- | +| HTML element content | Sanitise against a tag allowlist, or entity-encode | +| HTML attribute value | Attribute-encode, quotes included — a helper written for element content is not sufficient here | +| URL, inline script, or CSS | Encode for that context specifically | +| Speech, SMS, plain text, logs | Nothing. The text is inert unless something parses it as markup | + +The PolyAI web chat widget sanitises with DOMPurify against a tag allowlist, so embedding the widget gives you this for free. If you consume the Chat API directly, or render responses in a contact-centre platform or your own interface, the encoding is yours to do. + + ## Full example ~~~yaml