lc asked in #feedback (message 1558300896142696549) for magpie's masking
to cover more of what identifies a person or a machine. Under Settings >
Privacy > Mask personal data, each kind now has its own Off/On
(settings.redactKinds, redact.Options.Kinds). What each one masks:
- US SSNs: dashed numbers, except ones never issued (000, 666 and 9xx
areas) and well-known examples. Undashed ones only next to "SSN" or
"social security".
- Passport numbers, next to passport, Reisepass, 护照 or パスポート.
- IBANs: compact or in groups of four, with the country's length and the
mod-97 check.
- Birthdays: a date only next to birthday, DOB, born, 生日 and the like.
A date on its own stays.
- MAC addresses, but not documentation examples or all-decimal groups.
- Serial numbers, next to S/N, serial or 序列号.
- The user name in /Users/x, /home/x and C:\Users\x. The name is masked
and the path is kept.
- User and host in a shell prompt or an ssh/scp/rsync target (user@host
goes out as two placeholders, not as an email address), and on
hostname/user lines whose value ends the line. Code such as
`hostname = socket.gethostname()` is left alone.
- Public IPv4 and IPv6 addresses. Private, loopback, link-local, CGNAT,
documentation, multicast and well-known resolver addresses stay, and so
do version numbers.
- Cloud bucket names in s3://, gs://, oss://, cos://, abfss:// and
storage URLs. The object path is kept.
The last four are off by default. Agents send their working folder, the
prompt and log lines in nearly every request, and the model then has to
copy the placeholder exactly inside a path or URL. The first six are
rare and sensitive, so they are on. Each placeholder is restored in
replies, in streamed text and in tool-call JSON. Context rules run only
in a window around their words (rule.matches), so masking with the
defaults on 400KB of app.js takes 136ms (112ms before; 222ms with every
kind on).
Tests, run at this head:
- gofmt -l is clean. go vet passes on darwin, GOOS=linux and
GOOS=windows (-tags nogui) for internal/redact, gateway, gui and
settings.
- go test -tags nogui ./... under a temp HOME with pinned caches passed
everywhere except internal/proc's TestNoCommandBypassesProc. That
failure came from 2da8fbe4, and b934d64a fixed it on main. After the
rebase, the changed packages and internal/proc were run again and
passed. New tests:
- internal/redact TestMoreKinds, TestMoreKindsLeaveAlone (versions,
timestamps, hashes, UUIDs, model ids, code, ordinary dates, URLs in
code), TestCategoryDefaults, TestMoreKindsKnown, TestMoreKindsJSON.
- internal/gateway TestRedactedKinds.
- internal/gui TestRedactKindsAreRedacts.
- I broke the IBAN checksum by hand and TestMoreKindsLeaveAlone went
red.
- GUI tests pass in Chromium and WebKit:
- the new redact-kinds.test.cjs (en, zh, zh-TW, ja and de, at 1100
and 640 px);
- i18n-keys, gui-de, gui-ja, gui-zh-tw, redact-rule-row,
privacy-hides-accounts, privacy-usage-stats, settings-groups,
settings-sections and mobile-web;
- 29 more files that open the Privacy page (252 tests).
privacy-usage-stats now scrolls with the wheel to reach Count me as a
user, which the new rows push below its 760px window. WebKit doesn't
let a script scroll Settings.
- Real gateway check: a `magpie serve` built from this tree, under a
sandbox HOME, on 127.0.0.1:3492, with a fake OpenAI upstream on 3491.
An Anthropic /v1/messages request went to the upstream with each kind
as a placeholder. Turning SSNs off and home folders on changed what
was sent the same way. The streamed reply came back with every value
restored.
yetone
d7a1b02032