The masking rules find a value by what is around it: DB_PASSWORD= before
a password, a bound an email may not touch (so an email that ends a
sentence isn't one). The vendor writes about a placeholder in words of
its own ("your password is {{SECRET_…}}", "I will write to {{EMAIL_…}}."),
the agent reads the value there (Restore), and on the next turn that text
is in the history, where no rule matches it, so it went out as written.
Now every value a placeholder was made for (redact's in-memory values,
bounded by maxValues, never written anywhere) is also matched as an exact
string in each string Mask and MaskJSON look at, and goes as the same
placeholder it had (same kind, same keyed hash), so history and vendor
caches stay stable:
- only from 8 characters (the assignment rule's minimum): the 3-character
passwords the user:password@ rule takes, and the user's own words
(which Options.Words matches everywhere already), are not;
- personal values only while personal masking is on, secrets only while
secret masking is on;
- not as a piece of a longer value: a phone or card number with a digit
beside it, an email inside a longer address (a full stop after it may
end the sentence), a secret with a letter or digit against it;
- where a rule finds the same span, the rule's kind still wins, so
nothing masked before changes placeholder;
- in what a vendor signed (7137feb4), the values masked elsewhere in the
request go back as their placeholders as before, and so do known
secrets, for a turn whose other copies were compacted away; known
personal values don't, since a number the vendor wrote of its own
(13800138000) may be one masked in another conversation, and masking
it would break the signature.
A prefix index and a 4-byte bitset keep the lookup cheap: a ~1 MB body
masked in ~79 ms with 0, 1k or 200k known values alike (rules dominate).
Every route goes through redact.MaskJSON / Mask (chat, Responses,
Anthropic, Gemini, count_tokens, embeddings/rerank, the Codex backend,
image and video prompts), so each gets it; Scrub, the archive's, gets the
known secrets too.
Tests: TestEchoedValueMaskedNextTurn (gateway: two chat turns where the
fake vendor echoes the placeholders in other words; the second request
must not hold the password or the email, and keeps the first turn's
placeholders; then the echoed text alone through /v1/responses,
/v1/messages, count_tokens, Gemini generateContent, embeddings and the
Codex backend) fails on origin/main with the second turn sending
"…password is Mv4kestrelPass28, and I will write to
tomas.ng@quietfield.dev.". TestKnownValueMaskedInOtherWords and
TestKnownInSigned fail there too; TestShortValueNotMaskedEverywhere and
TestKnownReindexed pin the bounds and the eviction rebuild.
Run: gofmt -l (nothing); go vet -tags nogui ./... on darwin,
GOOS=linux and GOOS=windows; go build -tags nogui for darwin, linux,
windows; go test -tags nogui ./... under a temp HOME with pinned caches
(all ok); internal/redact -count=3 and -race. Not tried against a real
vendor.
yetone
3c7473eb3f