Public share and form pages only added a robots meta tag from JavaScript,
so the HTML for /share/* and /form/* was served without any indexing or
caching directive, and the form page had none at all. Error responses
from /api/public/* also lacked X-Robots-Tag.
Send one policy everywhere a token-bearing link is served:
X-Robots-Tag: noindex, nofollow, noarchive, nosnippet, noimageindex
Cache-Control: private, no-store
Referrer-Policy: no-referrer
- /api/public/*: shared middleware in the application, so it applies on
Cloudflare and Docker, including 404/403 responses. Routes that already
set Cache-Control keep it.
- /share/* and /form/* HTML: _headers rules for Cloudflare static assets
and the self-hosted static server.
- Referrer-Policy keeps the share token out of Referer headers sent to
linked sites.
robots.txt is deliberately not used to block these paths: a disallowed
URL can still be indexed from external links, because crawlers never see
the noindex header.
Co-authored-by: tianma-if <yingwaizhiying@gmail.com>
