A knowledge assistant can encounter instructions embedded in its data that attempt to change its behaviour. Defence requires more than a better system prompt: it requires an architecture that treats all retrieved content as untrusted.

Separate data from instructions

Explicitly identify retrieved content as material to analyse, rather than an instruction. Do not indiscriminately merge it with control messages.

Restrict tools and permissions

Give the assistant only the actions each task requires. Reading knowledge does not inherently require sending email or changing files.

Validate every critical action

Use allowlists, schema validation and human approval before an external side effect. The model's intent does not constitute authorisation.

Test adversarial examples

Place hidden instructions in PDFs, webpages, comments and metadata. Check whether the system reveals secrets, bypasses limits or follows someone else's objective.

Record and revise

Retain secure logs of sources, decisions and tool calls. Actual attempts should inform filters, policies and the evaluation suite.

This framework is an original editorial methodology developed by DigitalNow.

DIGITALNOW EDITORIAL TEAM

Practical guidance from DIGITALNOW, part of VNG Digital Group.

Editorial policy