01The debugging instinct

The first time a model produces a wrong answer in production, the question is always the same: what exactly did it receive? Without the prompt you are guessing, so teams log prompts. Then they log completions to compare. Then they keep the logs indefinitely because storage is cheap and you never know.

That sequence is entirely reasonable and it constructs, quietly, a store of user-submitted text with no purpose limitation and no retention period. Under the DPDP Act that is personal data you hold, with the duties that attach to it.

The answer is not to log nothing. It is to decide what, for how long, and who can read it — before the pipeline exists.

02What a prompt log actually contains

Worth being concrete, because “prompts” sounds abstract until you look at real traffic.

In products we have seen described, prompt logs routinely contain names, email addresses and phone numbers; pasted documents including contracts and medical letters; internal identifiers that link back to a customer record; and occasionally credentials that a user pasted because they were troubleshooting.

Completions inherit all of it, because the model quotes the input back. A log of completions is not a safer store than a log of prompts.

The practical test: if this table leaked, would you have to notify anyone? If the answer is yes, it is not a debug artefact, it is a personal data store, and it needs the same treatment as your production database.

03Three retention tiers

The mistake is treating all logging as one decision. Separate it into tiers with different lifetimes.

Most debugging turns out to need the first two. The instinct to keep content is strong, but when teams audit what they actually used to resolve incidents, structural traces resolve the majority — and they carry almost none of the risk.

Our post on what to monitor on a GPU box covers the operational tier in detail.

04Redaction that survives contact with reality

Redaction is worth doing and worth being honest about: it reduces risk, it does not eliminate it.

Pattern matching catches the structured things — card numbers, email addresses, phone numbers, identifiers with a fixed shape. It is reliable for those and worth applying at capture time rather than at query time, so the unredacted version never lands on disk.

What pattern matching does not catch is free-text disclosure: a user describing their medical situation in a sentence, or naming a colleague. No regular expression finds that. Model-based redaction helps and introduces its own failure mode, since a classifier that misses one in fifty still leaves you holding personal data.

So treat redaction as reducing severity rather than removing obligation. A redacted log is still a log, and it still needs a retention period.

05Sampling instead of capturing everything

The highest-leverage decision, and the one most often skipped: you almost certainly do not need every request.

Sample a small percentage of ordinary traffic, and capture selectively where the value is highest — requests that errored, requests the user gave negative feedback on, requests that hit a length or step limit. That combination gives you nearly all the diagnostic value at a fraction of the stored volume.

Two practical notes. Sample deterministically on a request identifier rather than randomly, so a sampled conversation is captured completely rather than in fragments. And record the sampling rate alongside the data, because a future you will otherwise draw wrong conclusions about frequency.

06Erasure requests and why they are hard

This is the section that changes how people design logging, because it is the obligation that is genuinely awkward to retrofit.

Under the DPDP Act a data principal can ask for their data to be erased. If prompt logs contain personal data, they are in scope. Which means you need to be able to find all log entries relating to one person and delete them.

That is straightforward if you stored a stable user identifier alongside each entry. It is close to impossible if you stored raw text with a timestamp, because the person is identifiable only from the content, and searching free text for an individual is neither reliable nor cheap.

The design consequence is small and must be decided early: store a pseudonymous, stable identifier with every content log so entries can be found and deleted later. Adding that field costs nothing now. Adding it after two years of logs does not help with the two years already stored.

A short retention window is the other half of the answer. Data you no longer hold needs no erasure procedure.

07A logging policy in six lines

Short enough to actually agree on, specific enough to implement.

1. Operational metrics: no content, retained for a stated number of months.
2. Structural traces: no content, retained for a stated number of weeks.
3. Content samples: a stated small percentage plus all errors and negative-feedback cases, retained for a stated number of days.
4. Redaction at capture for structured identifiers, applied before anything is written.
5. A pseudonymous user identifier on every content record, so erasure is possible.
6. A named owner for the policy, and a review date.

Where the inference runs affects who else sees this data, and running models on your own tenancy removes a third party from the path — but as our post on where your prompts actually go argues, it does not remove the obligations attached to what you choose to keep.

This is a practitioner’s framework rather than legal advice. For anything specific to your setup: privacy@vijaycloud.com.