Release v0.1.9

This commit is contained in:
adminroot committed 2026-08-25 17:26:49 +00:00
1 parent c5fdbdf963
commit 1ade118e69
15 files changed
+511 -224

No files matched your search

+5 -2
View File
@@ -188,9 +188,12 @@ jobs:
if: forgejo.event_name == 'push' && startsWith(forgejo.ref, 'refs/tags/v')
run: |
set -eu
token="${REGISTRY_TOKEN:-${FORGEJO_TOKEN:-${GITHUB_TOKEN:-}}}"
token="${REGISTRY_TOKEN:-}"
username="${REGISTRY_USERNAME:-${FORGEJO_ACTOR:-${GITHUB_ACTOR:-}}}"
test -n "$token"
if [ -z "$token" ]; then
echo 'REGISTRY_TOKEN must contain a Forgejo token with write:package access' >&2
exit 1
fi
test -n "$username"
printf '%s' "$token" \
| crane auth login "$REGISTRY" --username "$username" --password-stdin
Generated
+1 -2
View File
@@ -618,13 +618,12 @@ checksum = "a89322df9ebe1c1578d689c92318e070967d1042b512afbe49518723f4e6d5cd"
[[package]]
name = "pixhelf"
version = "0.1.8"
version = "0.1.9"
dependencies = [
"anyhow",
"axum",
"blake3",
"image",
"mime_guess",
"natord",
"serde",
"serde_json",
+1 -2
View File
@@ -1,6 +1,6 @@
[package]
name = "pixhelf"
version = "0.1.8"
version = "0.1.9"
edition = "2024"
build = "build.rs"
@@ -9,7 +9,6 @@ anyhow = "1.0.104"
axum = { version = "0.8.9", features = ["json", "query", "tokio"] }
blake3 = "1.8.7"
image = { version = "0.25.10", default-features = false, features = ["jpeg", "png", "webp"] }
mime_guess = "2.0.5"
natord = "1.0.9"
serde = { version = "1.0.229", features = ["derive"] }
serde_json = "1.0.149"
+40 -53
View File
@@ -1,77 +1,64 @@
# Pixhelf
基于 Axum 和 Preact 的自托管图片画廊。Preact 生产资源会嵌入 Rust 可执行文件,运行时只需要一个程序文件和外部图库目录。
Pixhelf 是一个给自己的照片目录准备的轻量画廊。把图片放进文件夹,启动服务,
就能在浏览器里按相册浏览、搜索和排序;手机和桌面都能舒服地看图。
## 运行
它支持 JPEG、PNG 和 WebP,会自动留意图库变化,并在后台生成 WebP 缩略图。
原图始终只读,缓存单独存放。Preact 前端会直接嵌进 Rust 程序里,部署时不用
再带一套静态文件。
默认从当前工作目录的 `./pic` 读取图片,并监听 `0.0.0.0:3002`:
## 用 Docker 跑起来
```bash
./target/release/pixhelf
```
HTTP 服务会立即开放,并优先生成首批 60 张缩略图。每张图片只生成一套最长边 720px、质量 82 的高质量 WebP 缩略图;其余缩略图由后台任务持续处理。新增、修改或删除图片会在定期扫描后自动同步,图库为空时也可正常启动并等待后续图片。
首屏图库数据与压缩后的关键样式会随 HTML 一并下发,首张图片会提前加载。JS 使用内容版本化的长期缓存,API 使用 ETag 协商缓存,并按浏览器能力启用 Brotli 或 gzip 压缩。
缩略图缓存在 `./.pixhelf-cache/thumbnails` 下的单一版本化目录,原图不会被修改。查看器拥有独立于瀑布流可见性的资源队列:已经解码的显示资源会直接复用,不再回退到缩略图;桌面端使用原图,移动端在原图像素不超过屏幕目标时也直接使用原图,大图则在当前图片稳定后从中性色缩略图升级到与视口及 DPR 匹配的高质量位图。全分辨率大图只在放大时按需加载,复位后回到更适合屏幕采样的视口位图,连续切图会取消过期升级任务。缩略图不做额外亮度或饱和度处理,升级时由高质量图层覆盖淡入,避免透明度叠加造成画面先暗后亮。只有超过安全边界的极端大图才绘制到 GPU 安全画布。查看器支持键盘方向键、鼠标前进/后退侧键和滚轮缩放,以及移动端滑动、双击和双指缩放;在适应屏幕状态下继续向下滚动或上滑,可从图片首屏连续进入详情第二屏。关闭查看器时会在瀑布流布局稳定后把当前图片动画送回对应卡片,即使期间发生续载或窗口尺寸变化也会重新锁定落点。窗口宽度引发瀑布流重排时,会按图片 ID 保持当前焦点或视口中心图片的相对位置。缓存目录必须位于图库目录之外。
## 构建
需要当前稳定版 Rust、Node.js 20+ 和 npm:
```bash
cd frontend
npm ci
cd ..
cargo build --release
```
`cargo build` 会自动执行 React 生产构建,并将结果嵌入 `target/release/pixhelf`。
发布工作流使用 musl 静态链接,并在同一台 AMD64 构建机上交叉编译两个版本:
- `pixhelf-amd64-linux`:`x86_64-unknown-linux-musl`
- `pixhelf-arm64-linux`:`aarch64-unknown-linux-musl`
两个 Release 产物都是无扩展名的独立可执行文件,下载后需要执行
`chmod +x pixhelf-amd64-linux` 或 `chmod +x pixhelf-arm64-linux`。
## Docker
每个版本同时发布 `linux/amd64` 和 `linux/arm64` 镜像。Docker 会自动选择与宿主机匹配的架构:
镜像放在 [eureka6688/pixhelf](https://hub.docker.com/r/eureka6688/pixhelf)。
在项目目录里执行:
```bash
mkdir -p pic
docker compose up -d
```
默认读取当前目录的 `./pic`,监听宿主机的 `3002` 端口,并使用命名卷保存缩略图缓存。
可以通过环境变量指定版本、图库目录和端口:
把图片放进 `./pic`,然后打开 <http://localhost:3002>。
`compose.yml` 直接使用 `eureka6688/pixhelf:latest`,图库以只读方式挂载,
生成的缩略图则保存在 `pixhelf-cache` 命名卷里。
## 自己构建
想从源码构建,需要稳定版 Rust、Node.js 20+ 和 npm:
```bash
PIXHELF_TAG=0.1.8 \
PIXHELF_GALLERY=/srv/photos \
PIXHELF_PORT=8080 \
docker compose up -d
cd frontend
npm ci
cd ..
cargo build --release --locked
```
镜像地址为 `git.pixhelf.com/adminroot/pixhelf`。发布工作流优先使用 Actions Secret
`REGISTRY_TOKEN` 和 `REGISTRY_USERNAME` 登录 Forgejo Container Registry;未配置时会尝试使用
当前工作流的临时令牌和触发用户。
这一步会先构建 Preact 前端,再把资源嵌入 `target/release/pixhelf`。
最终运行这一个文件就够了。
推送 `master` 或在 Forgejo Actions 页面手动运行 `Build and release` 工作流时,会完整构建两种
架构并验证多架构 OCI 镜像,但不会推送镜像或创建 Release;只有推送 `v*` 标签才会正式发布。
发布版本也提供两个静态可执行文件,按机器架构选择即可:
## 参数
- `pixhelf-amd64-linux`
- `pixhelf-arm64-linux`
下载后记得添加执行权限:
```bash
chmod +x pixhelf-amd64-linux
```
## 可调参数
大多数时候默认值就够用。需要更换目录、端口或扫描频率时,可以使用:
```text
--gallery-dir PATH 图库目录,默认 ./pic
--cache-dir PATH 缩略图缓存,默认 ./.pixhelf-cache/thumbnails
--listen HOST:PORT 监听地址,默认 0.0.0.0:3002
--initial-batch N 启动前生成数量,默认 60
--workers N 后台压缩并发数,默认按 CPU 自动选择 2-4
--scan-interval SEC 图库重新扫描间隔,默认 10 秒
--initial-batch N 启动时优先生成数量,默认 60
--workers N 后台任务数,默认按 CPU 自动选择 2-4
--scan-interval SEC 图库扫描周期,默认 10 秒
-h, --help 显示帮助
```
支持 JPEG、PNG 和 WebP 文件。其他文件不会进入图库索引。
启动前请确认图库目录已经存在,也别把缓存目录放进图库里。
+6 -1
View File
@@ -9,6 +9,7 @@ fn main() {
"frontend/package.json",
"frontend/package-lock.json",
"frontend/tsconfig.json",
"frontend/tsconfig.app.json",
"frontend/vite.config.ts",
"frontend/src",
] {
@@ -34,7 +35,11 @@ fn main() {
fn asset_version(frontend_dir: &std::path::Path) -> String {
let mut hash = 0xcbf29ce484222325u64;
for relative in ["dist/assets/app.js", "dist/assets/app.css"] {
for relative in [
"dist/index.html",
"dist/assets/app.js",
"dist/assets/app.css",
] {
let path = frontend_dir.join(relative);
let content = fs::read(&path)
.unwrap_or_else(|error| panic!("cannot read built asset {}: {error}", path.display()));
+3 -3
View File
@@ -1,11 +1,11 @@
services:
pixhelf:
image: git.pixhelf.com/adminroot/pixhelf:${PIXHELF_TAG:-latest}
image: eureka6688/pixhelf:latest
restart: unless-stopped
ports:
- "${PIXHELF_PORT:-3002}:3002"
- "3002:3002"
volumes:
- "${PIXHELF_GALLERY:-./pic}:/data/pic:ro"
- "./pic:/data/pic:ro"
- pixhelf-cache:/data/.pixhelf-cache
volumes:
+2 -2
View File
@@ -1,12 +1,12 @@
{
"name": "pixhelf-frontend",
"version": "0.1.8",
"version": "0.1.9",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "pixhelf-frontend",
"version": "0.1.8",
"version": "0.1.9",
"dependencies": {
"preact": "^10.27.1"
},
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "pixhelf-frontend",
"private": true,
"version": "0.1.8",
"version": "0.1.9",
"type": "module",
"scripts": {
"dev": "vite --host 0.0.0.0",
+18 -2
View File
@@ -33,6 +33,7 @@ import {
getDecodedViewerOriginal,
preloadOriginalImage,
prepareViewerImages,
viewerThumbnailUrl,
} from "./viewerAssets";
const PAGE_SIZE = 60;
@@ -46,6 +47,7 @@ const MOBILE_NAV_EXIT_MS = 240;
const VIEWER_RETURN_SETTLE_MS = 32;
const VIEWER_RETURN_TIMEOUT_MS = 480;
const VIEWER_RETURN_ANIMATION_MS = 260;
const READY_THUMBNAIL_CACHE_LIMIT = 2048;
const COUNT_FORMATTER = new Intl.NumberFormat("zh-CN");
const INITIAL_BOOTSTRAP = takeInitialBootstrap();
@@ -446,6 +448,15 @@ const CARD_LOAD_CALLBACKS = new WeakMap<Element, () => void>();
const READY_THUMBNAIL_IDS = new Set<string>();
let cardLoadObserver: IntersectionObserver | null = null;
function rememberReadyThumbnail(id: string): void {
READY_THUMBNAIL_IDS.delete(id);
READY_THUMBNAIL_IDS.add(id);
if (READY_THUMBNAIL_IDS.size > READY_THUMBNAIL_CACHE_LIMIT) {
const oldest = READY_THUMBNAIL_IDS.values().next().value;
if (oldest !== undefined) READY_THUMBNAIL_IDS.delete(oldest);
}
}
function observeCardLoad(element: Element, load: () => void): () => void {
if (!("IntersectionObserver" in window)) {
load();
@@ -1575,13 +1586,18 @@ const MasonryGallery = memo(function MasonryGallery({
useLayoutEffect(() => {
const anchor = resizeAnchorRef.current;
const masonry = ref.current;
let correctionFrame = 0;
resizeAnchorRef.current = null;
if (anchor && masonry && preserveViewport) {
restoreMasonryViewportAnchor(masonry, anchor);
correctionFrame = window.requestAnimationFrame(() => {
restoreMasonryViewportAnchor(masonry, anchor);
});
}
stableViewportAnchorRef.current = preserveViewport && masonry
? captureMasonryViewportAnchor(masonry)
: null;
return () => window.cancelAnimationFrame(correctionFrame);
}, [columnCount, gap, images, preserveViewport, width]);
useEffect(() => {
@@ -1722,13 +1738,13 @@ const ImageCard = memo(function ImageCard({
return observeCardLoad(card, () => setLoadRequested(true));
}, [eager, loadRequested]);
const thumbnailUrl = `/api/images/${encodeURIComponent(image.id)}/thumbnail`;
const thumbnailUrl = viewerThumbnailUrl(image);
const retryQuery = attempt ? `retry=${attempt}` : "";
const imageUrl = retryQuery ? `${thumbnailUrl}?${retryQuery}` : thumbnailUrl;
const markLoaded = useCallback(() => {
window.clearTimeout(retryTimerRef.current);
READY_THUMBNAIL_IDS.add(image.id);
rememberReadyThumbnail(image.id);
setLoaded(true);
setFailed(false);
setRetrying(false);
+82 -6
View File
@@ -1,5 +1,7 @@
import type {
Album,
BootstrapData,
GalleryImage,
GallerySummary,
ImagesPage,
SortMode,
@@ -8,7 +10,77 @@ import type {
const REQUEST_TIMEOUT_MS = 15_000;
async function getJson<T>(url: string, signal?: AbortSignal): Promise<T> {
type JsonObject = Record<string, unknown>;
type JsonValidator<T> = (value: unknown) => value is T;
function isObject(value: unknown): value is JsonObject {
return typeof value === "object" && value !== null;
}
function isNonNegativeInteger(value: unknown): value is number {
return typeof value === "number" && Number.isSafeInteger(value) && value >= 0;
}
function isPositiveInteger(value: unknown): value is number {
return isNonNegativeInteger(value) && value > 0;
}
function isAlbum(value: unknown): value is Album {
return isObject(value)
&& typeof value.path === "string"
&& typeof value.name === "string"
&& isNonNegativeInteger(value.count);
}
function isGalleryImage(value: unknown): value is GalleryImage {
return isObject(value)
&& typeof value.id === "string"
&& typeof value.name === "string"
&& isPositiveInteger(value.width)
&& isPositiveInteger(value.height);
}
function isGallerySummary(value: unknown): value is GallerySummary {
return isObject(value)
&& isNonNegativeInteger(value.total)
&& Array.isArray(value.albums)
&& value.albums.every(isAlbum)
&& typeof value.revision === "string";
}
function isImagesPage(value: unknown): value is ImagesPage {
return isObject(value)
&& Array.isArray(value.items)
&& value.items.every(isGalleryImage)
&& isNonNegativeInteger(value.total)
&& isNonNegativeInteger(value.offset)
&& isPositiveInteger(value.limit)
&& (value.nextOffset === null || isNonNegativeInteger(value.nextOffset));
}
function isThumbnailStatus(value: unknown): value is ThumbnailStatus {
return isObject(value)
&& isNonNegativeInteger(value.total)
&& isNonNegativeInteger(value.ready)
&& isNonNegativeInteger(value.queued)
&& isNonNegativeInteger(value.processing)
&& isNonNegativeInteger(value.failed)
&& typeof value.initialBatchReady === "boolean"
&& typeof value.backgroundComplete === "boolean";
}
function isBootstrapData(value: unknown): value is BootstrapData {
return isObject(value)
&& isGallerySummary(value.summary)
&& isThumbnailStatus(value.status)
&& isImagesPage(value.images);
}
async function getJson<T>(
url: string,
validate: JsonValidator<T>,
signal?: AbortSignal,
): Promise<T> {
const controller = new AbortController();
let timedOut = false;
const abortFromCaller = () => controller.abort(signal?.reason);
@@ -35,12 +107,15 @@ async function getJson<T>(url: string, signal?: AbortSignal): Promise<T> {
if (!response.headers.get("content-type")?.includes("application/json")) {
throw new Error("服务器返回了无效数据");
}
let payload: unknown;
try {
return (await response.json()) as T;
payload = await response.json();
} catch (error) {
if (controller.signal.aborted) throw error;
throw new Error("服务器返回了无效数据", { cause: error });
}
if (!validate(payload)) throw new Error("服务器返回了无效数据");
return payload;
} catch (error) {
if (timedOut) throw new Error("请求超时,请稍后重试", { cause: error });
throw error;
@@ -57,7 +132,8 @@ export function takeInitialBootstrap(): BootstrapData | null {
try {
const source = element.textContent?.trim();
if (!source || source.startsWith("__PIXHELF_")) return null;
return JSON.parse(source) as BootstrapData;
const payload: unknown = JSON.parse(source);
return isBootstrapData(payload) ? payload : null;
} catch {
return null;
} finally {
@@ -66,11 +142,11 @@ export function takeInitialBootstrap(): BootstrapData | null {
}
export function getGallery(signal?: AbortSignal): Promise<GallerySummary> {
return getJson<GallerySummary>("/api/gallery", signal);
return getJson("/api/gallery", isGallerySummary, signal);
}
export function getStatus(signal?: AbortSignal): Promise<ThumbnailStatus> {
return getJson<ThumbnailStatus>("/api/status", signal);
return getJson("/api/status", isThumbnailStatus, signal);
}
export function getImages(
@@ -92,5 +168,5 @@ export function getImages(
if (options.album) params.set("album", options.album);
if (options.search) params.set("search", options.search);
if (options.seed) params.set("seed", options.seed);
return getJson<ImagesPage>(`/api/images?${params}`, signal);
return getJson(`/api/images?${params}`, isImagesPage, signal);
}
+13
View File
@@ -5,6 +5,7 @@ const ORIGINAL_CACHE_LIMIT = 4;
const VIEWPORT_RENDER_CACHE_LIMIT = 4;
const THUMBNAIL_PRELOAD_DISTANCE = 10;
const ORIGINAL_PRELOAD_DISTANCE = 2;
const ASSET_LOAD_TIMEOUT_MS = 30_000;
type AssetPriority = "high" | "low";
@@ -117,9 +118,11 @@ function loadImageAsset(
lastUsed: ++assetClock,
};
let settled = false;
let timeout = 0;
const settle = (status: Exclude<ViewerAssetStatus, "loading">) => {
if (settled) return;
settled = true;
window.clearTimeout(timeout);
asset.status = status;
asset.lastUsed = ++assetClock;
resolveAsset();
@@ -136,6 +139,11 @@ function loadImageAsset(
element.addEventListener("error", () => settle("failed"), { once: true });
cache.set(url, asset);
element.src = url;
timeout = window.setTimeout(() => {
if (settled) return;
element.removeAttribute("src");
settle("failed");
}, ASSET_LOAD_TIMEOUT_MS);
pruneCache(cache, limit, url);
return asset;
}
@@ -237,6 +245,7 @@ function startNextViewportRenderJob(): void {
activeViewportRenderJob = job;
void (async () => {
const { asset, attempt, controller, image, key, resolve } = job;
const timeout = window.setTimeout(() => controller.abort(), ASSET_LOAD_TIMEOUT_MS);
try {
const requestOptions: RequestInit & { priority: "high" | "low" } = {
cache: attempt ? "reload" : "force-cache",
@@ -246,6 +255,9 @@ function startNextViewportRenderJob(): void {
};
const response = await fetch(viewerOriginalUrl(image, attempt), requestOptions);
if (!response.ok) throw new Error(`original request failed: ${response.status}`);
if (!response.headers.get("content-type")?.startsWith("image/")) {
throw new Error("original response is not an image");
}
const blob = await response.blob();
if (!blob.size) throw new Error("original response is empty");
const bitmap = await createImageBitmap(blob, {
@@ -263,6 +275,7 @@ function startNextViewportRenderJob(): void {
} catch {
asset.status = "failed";
} finally {
window.clearTimeout(timeout);
if (job.cancelled) {
asset.bitmap?.close();
asset.bitmap = null;
+78 -17
View File
@@ -6,7 +6,7 @@ use std::{
time::UNIX_EPOCH,
};
use anyhow::{Context, Result, anyhow};
use anyhow::{Context, Result, anyhow, bail};
use image::{ImageDecoder, ImageReader, metadata::Orientation};
use serde::Serialize;
use tracing::warn;
@@ -27,6 +27,46 @@ pub struct ImageRecord {
pub(crate) modified_ns: u128,
}
impl ImageRecord {
pub(crate) fn matches_metadata(&self, metadata: &fs::Metadata) -> bool {
let fingerprint = FileFingerprint::from_metadata(metadata);
metadata.is_file()
&& self.size == fingerprint.size
&& self.modified_ns == fingerprint.modified_ns
}
pub(crate) fn ensure_source_is_current(&self) -> Result<()> {
let metadata = fs::metadata(&self.path)
.with_context(|| format!("cannot inspect {}", self.path.display()))?;
if !self.matches_metadata(&metadata) {
bail!("source image changed since the gallery scan");
}
Ok(())
}
}
#[derive(Clone, Copy)]
struct FileFingerprint {
size: u64,
modified_ms: u64,
modified_ns: u128,
}
impl FileFingerprint {
fn from_metadata(metadata: &fs::Metadata) -> Self {
let modified = metadata
.modified()
.unwrap_or(UNIX_EPOCH)
.duration_since(UNIX_EPOCH)
.unwrap_or_default();
Self {
size: metadata.len(),
modified_ms: modified.as_millis().min(u128::from(u64::MAX)) as u64,
modified_ns: modified.as_nanos(),
}
}
}
#[derive(Clone, Debug, Serialize)]
#[serde(rename_all = "camelCase")]
pub struct Album {
@@ -35,16 +75,20 @@ pub struct Album {
pub count: usize,
}
#[derive(Clone, Default)]
#[derive(Default)]
pub struct GalleryIndex {
pub images: Vec<Arc<ImageRecord>>,
pub albums: Vec<Album>,
pub revision: String,
images_by_id: Vec<Arc<ImageRecord>>,
}
impl GalleryIndex {
pub fn same_revision(&self, other: &Self) -> bool {
self.revision == other.revision
pub fn image(&self, id: &str) -> Option<&Arc<ImageRecord>> {
self.images_by_id
.binary_search_by(|image| image.id.as_str().cmp(id))
.ok()
.map(|index| &self.images_by_id[index])
}
}
@@ -94,17 +138,11 @@ pub fn scan_gallery(root: &Path, previous: Option<&GalleryIndex>) -> Result<Gall
continue;
}
};
let modified = metadata
.modified()
.unwrap_or(UNIX_EPOCH)
.duration_since(UNIX_EPOCH)
.unwrap_or_default();
let modified_ns = modified.as_nanos();
let size = metadata.len();
let fingerprint = FileFingerprint::from_metadata(&metadata);
if let Some(existing) = existing
&& existing.size == size
&& existing.modified_ns == modified_ns
&& existing.size == fingerprint.size
&& existing.modified_ns == fingerprint.modified_ns
{
images.push(Arc::clone(existing));
continue;
@@ -130,7 +168,7 @@ pub fn scan_gallery(root: &Path, previous: Option<&GalleryIndex>) -> Result<Gall
.and_then(|name| name.to_str())
.unwrap_or(&relative_path)
.to_owned();
let id = image_id(&relative_path, size, modified_ns);
let id = image_id(&relative_path, fingerprint.size, fingerprint.modified_ns);
let search_key = relative_path.to_lowercase();
images.push(Arc::new(ImageRecord {
@@ -142,9 +180,9 @@ pub fn scan_gallery(root: &Path, previous: Option<&GalleryIndex>) -> Result<Gall
album,
width,
height,
size,
modified_ms: modified.as_millis().min(u64::MAX as u128) as u64,
modified_ns,
size: fingerprint.size,
modified_ms: fingerprint.modified_ms,
modified_ns: fingerprint.modified_ns,
}));
}
@@ -170,10 +208,13 @@ pub fn scan_gallery(root: &Path, previous: Option<&GalleryIndex>) -> Result<Gall
count,
})
.collect();
let mut images_by_id = images.clone();
images_by_id.sort_unstable_by(|left, right| left.id.cmp(&right.id));
Ok(GalleryIndex {
images,
albums,
revision,
images_by_id,
})
}
@@ -268,6 +309,26 @@ mod tests {
assert_eq!(index.images[0].album, "album");
assert_eq!((index.images[0].width, index.images[0].height), (20, 10));
assert_eq!(index.albums[0].count, 1);
assert_eq!(
index
.image(&index.images[0].id)
.map(|image| image.name.as_str()),
Some("image.png")
);
assert!(index.image("missing").is_none());
}
#[test]
fn detects_sources_changed_after_the_scan() {
let temp = tempfile::tempdir().unwrap();
let path = temp.path().join("image.png");
image::RgbImage::new(20, 10).save(&path).unwrap();
let index = scan_gallery(temp.path(), None).unwrap();
let record = &index.images[0];
assert!(record.ensure_source_is_current().is_ok());
fs::write(&path, "replacement with a different size").unwrap();
assert!(record.ensure_source_is_current().is_err());
}
#[test]
+23 -24
View File
@@ -3,11 +3,11 @@ mod gallery;
mod thumbs;
mod web;
use std::sync::Arc;
use std::{path::PathBuf, sync::Arc};
use anyhow::{Context, Result};
use config::Config;
use gallery::scan_gallery;
use gallery::{GalleryIndex, scan_gallery};
use thumbs::ThumbnailManager;
use tokio::{net::TcpListener, sync::RwLock};
use tracing::{error, info, warn};
@@ -26,10 +26,7 @@ async fn main() -> Result<()> {
let config = Config::from_env_and_args()?;
info!(gallery = %config.gallery_dir.display(), "scanning gallery");
let gallery_dir = config.gallery_dir.clone();
let index = tokio::task::spawn_blocking(move || scan_gallery(&gallery_dir, None))
.await
.context("gallery scan task stopped")??;
let index = scan_gallery_async(config.gallery_dir.clone(), None).await?;
if index.images.is_empty() {
warn!(
gallery = %config.gallery_dir.display(),
@@ -48,7 +45,7 @@ async fn main() -> Result<()> {
.take(config.initial_batch)
.map(|record| record.id.clone())
.collect();
let index = Arc::new(RwLock::new(index));
let index = Arc::new(RwLock::new(Arc::new(index)));
let thumbnails = ThumbnailManager::new(config.cache_dir.clone())?;
{
let current = index.read().await;
@@ -86,38 +83,40 @@ async fn main() -> Result<()> {
async fn rescan_gallery(state: AppState, config: Config) {
loop {
tokio::time::sleep(config.scan_interval).await;
let previous = state.index.read().await.clone();
let previous = {
let index = state.index.read().await;
Arc::clone(&index)
};
let old_count = previous.images.len();
let root = config.gallery_dir.clone();
let scan = tokio::task::spawn_blocking(move || {
let updated = scan_gallery(&root, Some(&previous));
(previous, updated)
})
.await;
let (previous, updated) = match scan {
Ok((previous, Ok(updated))) => (previous, updated),
Ok((_, Err(error))) => {
let updated = match scan_gallery_async(root, Some(Arc::clone(&previous))).await {
Ok(updated) => updated,
Err(error) => {
error!(%error, "gallery rescan failed");
continue;
}
Err(error) => {
error!(%error, "gallery rescan task stopped");
continue;
}
};
if previous.same_revision(&updated) {
if previous.revision == updated.revision {
continue;
}
let old_count = previous.images.len();
let new_count = updated.images.len();
state.thumbnails.reconcile(&updated.images);
*state.index.write().await = updated;
*state.index.write().await = Arc::new(updated);
state.thumbnails.cleanup_stale().await;
info!(old_count, new_count, "gallery changes indexed");
}
}
async fn scan_gallery_async(
root: PathBuf,
previous: Option<Arc<GalleryIndex>>,
) -> Result<GalleryIndex> {
tokio::task::spawn_blocking(move || scan_gallery(&root, previous.as_deref()))
.await
.context("gallery scan task stopped")?
}
async fn shutdown_signal() {
#[cfg(unix)]
{
+23 -27
View File
@@ -27,7 +27,6 @@ use crate::gallery::ImageRecord;
const THUMBNAIL_EDGE: u32 = 720;
const WEBP_QUALITY: f32 = 82.0;
const CACHE_VERSION: &str = "720-webp-q82-v1";
const THUMBNAIL_FILTER: FilterType = FilterType::Triangle;
const LEGACY_VIEWER_CACHE_VERSION: &str = "viewer-3200-webp-q88-fast-v1";
const MAX_ATTEMPTS: u8 = 3;
@@ -501,24 +500,7 @@ fn shard_path(root: &Path, id: &str) -> PathBuf {
}
fn generate_thumbnail(record: &ImageRecord, output: &Path) -> Result<()> {
generate_image_cache(
record,
output,
THUMBNAIL_EDGE,
THUMBNAIL_FILTER,
WEBP_QUALITY,
"thumbnail",
)
}
fn generate_image_cache(
record: &ImageRecord,
output: &Path,
max_edge: u32,
filter: FilterType,
quality: f32,
label: &str,
) -> Result<()> {
record.ensure_source_is_current()?;
let reader = ImageReader::open(&record.path)
.with_context(|| format!("cannot open {}", record.path.display()))?
.with_guessed_format()?;
@@ -527,13 +509,22 @@ fn generate_image_cache(
let mut image = DynamicImage::from_decoder(decoder)?;
image.apply_orientation(orientation);
let (width, height) = image.dimensions();
let image = if width.max(height) > max_edge {
image.resize(max_edge, max_edge, filter)
let image = if width.max(height) > THUMBNAIL_EDGE {
image.resize(THUMBNAIL_EDGE, THUMBNAIL_EDGE, FilterType::Triangle)
} else {
image
};
write_webp(&image, output, quality)
.with_context(|| format!("cannot store {label}: {}", output.display()))
write_webp(&image, output, WEBP_QUALITY)
.with_context(|| format!("cannot store thumbnail: {}", output.display()))?;
if let Err(error) = record.ensure_source_is_current() {
if let Err(remove_error) = fs::remove_file(output)
&& remove_error.kind() != std::io::ErrorKind::NotFound
{
warn!(path = %output.display(), %remove_error, "cannot remove stale thumbnail");
}
return Err(error);
}
Ok(())
}
fn write_webp(image: &DynamicImage, output: &Path, quality: f32) -> Result<()> {
@@ -712,7 +703,7 @@ mod tests {
}
#[tokio::test]
async fn failed_thumbnail_can_be_retried() {
async fn changed_source_requires_a_fresh_gallery_record() {
let temp = tempfile::tempdir().unwrap();
let gallery = temp.path().join("gallery");
fs::create_dir(&gallery).unwrap();
@@ -724,14 +715,19 @@ mod tests {
let manager = ThumbnailManager::new(temp.path().join("cache")).unwrap();
manager.reconcile(&index.images);
manager.start_workers(1);
let id = &index.images[0].id;
let old_id = index.images[0].id.clone();
assert!(matches!(
manager.ensure_ready(id).await,
manager.ensure_ready(&old_id).await,
Err(ThumbnailError::Generation(_))
));
image::RgbImage::new(20, 10).save(&image_path).unwrap();
let thumbnail = manager.ensure_ready(id).await.unwrap();
let updated = scan_gallery(&gallery, Some(&index)).unwrap();
let new_id = updated.images[0].id.clone();
assert_ne!(new_id, old_id);
manager.reconcile(&updated.images);
let thumbnail = manager.ensure_ready(&new_id).await.unwrap();
assert!(valid_cache_file(&thumbnail));
}
+215 -82
View File
@@ -14,6 +14,7 @@ use tower::ServiceExt;
use tower_http::{
compression::{CompressionLayer, CompressionLevel},
services::ServeFile,
set_header::SetResponseHeaderLayer,
trace::TraceLayer,
};
use tracing::error;
@@ -36,11 +37,13 @@ const APP_CSS: &[u8] = include_bytes!(concat!(
"/frontend/dist/assets/app.css"
));
const ASSET_VERSION: &str = env!("PIXHELF_ASSET_VERSION");
const BOOTSTRAP_PAGE_SIZE: usize = 60;
const DEFAULT_PAGE_SIZE: usize = 60;
const MAX_PAGE_SIZE: usize = 200;
const MAX_QUERY_VALUE_BYTES: usize = 4096;
#[derive(Clone)]
pub struct AppState {
pub index: Arc<RwLock<GalleryIndex>>,
pub index: Arc<RwLock<Arc<GalleryIndex>>>,
pub thumbnails: Arc<ThumbnailManager>,
}
@@ -56,7 +59,60 @@ struct ImagesQuery {
limit: Option<usize>,
}
#[derive(Debug)]
struct ImageRequest {
album: Option<String>,
search: Option<String>,
sort: ImageSort,
seed: String,
offset: usize,
limit: usize,
}
impl ImagesQuery {
fn normalize(self) -> Result<ImageRequest, StatusCode> {
if [&self.album, &self.search, &self.seed]
.into_iter()
.flatten()
.any(|value| value.len() > MAX_QUERY_VALUE_BYTES)
{
return Err(StatusCode::BAD_REQUEST);
}
Ok(ImageRequest {
album: self.album.filter(|album| !album.is_empty()),
search: self
.search
.as_deref()
.map(str::trim)
.filter(|search| !search.is_empty())
.map(str::to_lowercase),
sort: self.sort,
seed: self.seed.unwrap_or_default(),
offset: self.offset.unwrap_or(0),
limit: self
.limit
.unwrap_or(DEFAULT_PAGE_SIZE)
.clamp(1, MAX_PAGE_SIZE),
})
}
}
impl Default for ImageRequest {
fn default() -> Self {
Self {
album: None,
search: None,
sort: ImageSort::default(),
seed: String::new(),
offset: 0,
limit: DEFAULT_PAGE_SIZE,
}
}
}
#[derive(Clone, Copy, Debug, Default, Deserialize)]
#[repr(u8)]
#[serde(rename_all = "kebab-case")]
enum ImageSort {
#[default]
@@ -123,6 +179,18 @@ pub fn router(state: AppState) -> Router {
.route("/assets/{version}/app.css", get(app_css))
.fallback(frontend)
.layer(CompressionLayer::new().quality(CompressionLevel::Precise(5)))
.layer(SetResponseHeaderLayer::if_not_present(
header::X_CONTENT_TYPE_OPTIONS,
HeaderValue::from_static("nosniff"),
))
.layer(SetResponseHeaderLayer::if_not_present(
header::X_FRAME_OPTIONS,
HeaderValue::from_static("DENY"),
))
.layer(SetResponseHeaderLayer::if_not_present(
header::REFERRER_POLICY,
HeaderValue::from_static("no-referrer"),
))
.layer(TraceLayer::new_for_http())
.with_state(state)
}
@@ -138,9 +206,11 @@ async fn gallery_summary(State(state): State<AppState>, headers: HeaderMap) -> R
return not_modified(&etag, "private, no-cache");
}
let mut response = Json(summary_from_index(&index)).into_response();
let _ = add_cache_headers(&mut response, &etag, "private, no-cache");
response
with_cache_headers(
Json(summary_from_index(&index)).into_response(),
&etag,
"private, no-cache",
)
}
fn summary_from_index(index: &GalleryIndex) -> GallerySummary {
@@ -156,27 +226,28 @@ async fn images(
Query(query): Query<ImagesQuery>,
headers: HeaderMap,
) -> Response {
let query = match query.normalize() {
Ok(query) => query,
Err(status) => return status.into_response(),
};
let index = state.index.read().await;
let etag = images_etag(&index.revision, &query);
if is_not_modified(&headers, &etag) {
return not_modified(&etag, "private, no-cache");
}
let mut response = Json(image_page(&index, &query)).into_response();
let _ = add_cache_headers(&mut response, &etag, "private, no-cache");
response
with_cache_headers(
Json(image_page(&index, &query)).into_response(),
&etag,
"private, no-cache",
)
}
fn image_page(index: &GalleryIndex, query: &ImagesQuery) -> ImagesResponse {
let search = query
.search
.as_deref()
.map(str::trim)
.filter(|search| !search.is_empty())
.map(str::to_lowercase);
let album = query.album.as_deref().filter(|album| !album.is_empty());
let requested_offset = query.offset.unwrap_or(0);
let limit = query.limit.unwrap_or(60).clamp(1, 200);
fn image_page(index: &GalleryIndex, query: &ImageRequest) -> ImagesResponse {
let album = query.album.as_deref();
let search = query.search.as_deref();
let requested_offset = query.offset;
let limit = query.limit;
match query.sort {
ImageSort::NameAsc if album.is_none() && search.is_none() => {
@@ -188,14 +259,14 @@ fn image_page(index: &GalleryIndex, query: &ImagesQuery) -> ImagesResponse {
ImageSort::NameAsc => page_from_ordered_records(
index.images.iter().map(Arc::as_ref),
album,
search.as_deref(),
search,
requested_offset,
limit,
),
ImageSort::NameDesc => page_from_ordered_records(
index.images.iter().rev().map(Arc::as_ref),
album,
search.as_deref(),
search,
requested_offset,
limit,
),
@@ -204,15 +275,13 @@ fn image_page(index: &GalleryIndex, query: &ImagesQuery) -> ImagesResponse {
.images
.iter()
.map(Arc::as_ref)
.filter(|record| record_matches(record, album, search.as_deref()))
.filter(|record| record_matches(record, album, search))
.collect();
match query.sort {
ImageSort::Newest => matches.sort_by_key(|record| Reverse(record.modified_ms)),
ImageSort::Explore => {
let seed_key =
*blake3::hash(query.seed.as_deref().unwrap_or_default().as_bytes())
.as_bytes();
let seed_key = *blake3::hash(query.seed.as_bytes()).as_bytes();
matches.sort_by_cached_key(|record| exploration_rank(&seed_key, &record.id));
}
ImageSort::NameAsc | ImageSort::NameDesc => unreachable!(),
@@ -336,20 +405,27 @@ async fn original(
request: Request,
) -> Result<Response, StatusCode> {
// Resolve the opaque id through the index instead of accepting a filesystem path
// from the request. Cloning the path also avoids holding the read lock while the
// file is streamed to a slow client.
let path = {
// from the request. Cloning the record releases the read lock before filesystem I/O.
let image = {
let index = state.index.read().await;
index
.images
.iter()
.find(|image| image.id == id)
.map(|image| image.path.clone())
.ok_or(StatusCode::NOT_FOUND)?
index.image(&id).cloned().ok_or(StatusCode::NOT_FOUND)?
};
let metadata = match tokio::fs::metadata(&image.path).await {
Ok(metadata) => metadata,
Err(error) if error.kind() == std::io::ErrorKind::NotFound => {
return Err(StatusCode::NOT_FOUND);
}
Err(error) => {
error!(path = %image.path.display(), %error, "cannot inspect original image");
return Err(StatusCode::INTERNAL_SERVER_ERROR);
}
};
if !image.matches_metadata(&metadata) {
return Err(StatusCode::NOT_FOUND);
}
serve_file(
path,
image.path.clone(),
request,
&format!("\"original-{id}\""),
"public, max-age=31536000, immutable",
@@ -363,15 +439,8 @@ async fn serve_file(
etag: &str,
cache_control: &'static str,
) -> Result<Response, StatusCode> {
if request
.headers()
.get(header::IF_NONE_MATCH)
.and_then(|value| value.to_str().ok())
== Some(etag)
{
let mut response = StatusCode::NOT_MODIFIED.into_response();
add_cache_headers(&mut response, etag, cache_control)?;
return Ok(response);
if is_not_modified(request.headers(), etag) {
return Ok(not_modified(etag, cache_control));
}
let response = ServeFile::new(path)
@@ -379,36 +448,34 @@ async fn serve_file(
.await
.map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?;
let (parts, body) = response.into_parts();
let mut response = Response::from_parts(parts, Body::new(body));
add_cache_headers(&mut response, etag, cache_control)?;
Ok(response)
Ok(with_cache_headers(
Response::from_parts(parts, Body::new(body)),
etag,
cache_control,
))
}
fn add_cache_headers(
response: &mut Response,
etag: &str,
cache_control: &'static str,
) -> Result<(), StatusCode> {
fn with_cache_headers(mut response: Response, etag: &str, cache_control: &'static str) -> Response {
let etag = match HeaderValue::from_str(etag) {
Ok(etag) => etag,
Err(error) => {
error!(%error, "cannot create ETag response header");
return StatusCode::INTERNAL_SERVER_ERROR.into_response();
}
};
response.headers_mut().insert(
header::CACHE_CONTROL,
HeaderValue::from_static(cache_control),
);
response.headers_mut().insert(
header::ETAG,
HeaderValue::from_str(etag).map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?,
);
response.headers_mut().insert(
header::X_CONTENT_TYPE_OPTIONS,
HeaderValue::from_static("nosniff"),
);
Ok(())
response.headers_mut().insert(header::ETAG, etag);
response
}
fn revision_etag(kind: &str, revision: &str) -> String {
format!("\"{kind}-{revision}\"")
}
fn images_etag(revision: &str, query: &ImagesQuery) -> String {
fn images_etag(revision: &str, query: &ImageRequest) -> String {
let mut hasher = blake3::Hasher::new();
hasher.update(b"pixhelf-images-query-v1\0");
hasher.update(query.album.as_deref().unwrap_or_default().as_bytes());
@@ -416,9 +483,9 @@ fn images_etag(revision: &str, query: &ImagesQuery) -> String {
hasher.update(query.search.as_deref().unwrap_or_default().as_bytes());
hasher.update(b"\0");
hasher.update(&[query.sort as u8]);
hasher.update(query.seed.as_deref().unwrap_or_default().as_bytes());
hasher.update(&query.offset.unwrap_or_default().to_le_bytes());
hasher.update(&query.limit.unwrap_or(60).to_le_bytes());
hasher.update(query.seed.as_bytes());
hasher.update(&query.offset.to_le_bytes());
hasher.update(&query.limit.to_le_bytes());
let query_hash = hasher.finalize().to_hex();
format!("\"images-{revision}-{}\"", &query_hash[..16])
}
@@ -431,14 +498,16 @@ fn is_not_modified(headers: &HeaderMap, etag: &str) -> bool {
values
.split(',')
.map(str::trim)
.any(|value| value == etag || value == "*")
.any(|value| value == "*" || value.strip_prefix("W/").unwrap_or(value) == etag)
})
}
fn not_modified(etag: &str, cache_control: &'static str) -> Response {
let mut response = StatusCode::NOT_MODIFIED.into_response();
let _ = add_cache_headers(&mut response, etag, cache_control);
response
with_cache_headers(
StatusCode::NOT_MODIFIED.into_response(),
etag,
cache_control,
)
}
async fn app_js(Path(version): Path<String>, headers: HeaderMap) -> Response {
@@ -480,8 +549,7 @@ fn embedded_asset(
response
.headers_mut()
.insert(header::CONTENT_TYPE, HeaderValue::from_static(content_type));
let _ = add_cache_headers(&mut response, &etag, "public, max-age=31536000, immutable");
response
with_cache_headers(response, &etag, "public, max-age=31536000, immutable")
}
async fn frontend(State(state): State<AppState>, request: Request) -> Response {
@@ -501,13 +569,7 @@ async fn frontend(State(state): State<AppState>, request: Request) -> Response {
return not_modified(&etag, "private, no-cache");
}
let summary = summary_from_index(&index);
let images = image_page(
&index,
&ImagesQuery {
limit: Some(BOOTSTRAP_PAGE_SIZE),
..ImagesQuery::default()
},
);
let images = image_page(&index, &ImageRequest::default());
let preload = images.items.first().map_or_else(String::new, |image| {
let url = format!("/api/images/{}/thumbnail", image.id);
@@ -554,8 +616,7 @@ async fn frontend(State(state): State<AppState>, request: Request) -> Response {
header::CONTENT_TYPE,
HeaderValue::from_static("text/html; charset=utf-8"),
);
let _ = add_cache_headers(&mut response, &etag, "private, no-cache");
response
with_cache_headers(response, &etag, "private, no-cache")
}
fn escape_script_json(json: String) -> String {
@@ -574,7 +635,7 @@ mod tests {
let temp = tempfile::tempdir().unwrap();
let thumbnails = ThumbnailManager::new(temp.path().to_owned()).unwrap();
router(AppState {
index: Arc::new(RwLock::new(GalleryIndex::default())),
index: Arc::new(RwLock::new(Arc::new(GalleryIndex::default()))),
thumbnails,
})
}
@@ -628,6 +689,30 @@ mod tests {
);
}
#[test]
fn normalizes_image_queries_once() {
let query = ImagesQuery {
album: Some("album".into()),
search: Some(" PHOTO ".into()),
offset: Some(usize::MAX),
limit: Some(usize::MAX),
..ImagesQuery::default()
}
.normalize()
.unwrap();
assert_eq!(query.album.as_deref(), Some("album"));
assert_eq!(query.search.as_deref(), Some("photo"));
assert_eq!(query.offset, usize::MAX);
assert_eq!(query.limit, MAX_PAGE_SIZE);
let oversized = ImagesQuery {
search: Some("x".repeat(MAX_QUERY_VALUE_BYTES + 1)),
..ImagesQuery::default()
};
assert_eq!(oversized.normalize().unwrap_err(), StatusCode::BAD_REQUEST);
}
#[tokio::test]
async fn rejects_unknown_sort_modes() {
assert_eq!(
@@ -658,7 +743,14 @@ mod tests {
request("/api/images/missing/original").await.status(),
StatusCode::NOT_FOUND
);
assert_eq!(request("/missing").await.status(), StatusCode::NOT_FOUND);
let response = request("/missing").await;
assert_eq!(response.status(), StatusCode::NOT_FOUND);
assert_eq!(
response.headers()[header::X_CONTENT_TYPE_OPTIONS],
"nosniff"
);
assert_eq!(response.headers()[header::X_FRAME_OPTIONS], "DENY");
assert_eq!(response.headers()[header::REFERRER_POLICY], "no-referrer");
}
#[tokio::test]
@@ -674,7 +766,7 @@ mod tests {
let thumbnails = ThumbnailManager::new(temp.path().join("cache")).unwrap();
thumbnails.reconcile(&index.images);
let app = router(AppState {
index: Arc::new(RwLock::new(index)),
index: Arc::new(RwLock::new(Arc::new(index))),
thumbnails,
});
@@ -699,6 +791,35 @@ mod tests {
assert!(!body.is_empty());
}
#[tokio::test]
async fn stale_image_ids_do_not_serve_replaced_files() {
let temp = tempfile::tempdir().unwrap();
let gallery = temp.path().join("gallery");
std::fs::create_dir(&gallery).unwrap();
let image_path = gallery.join("image.png");
image::RgbImage::new(20, 10).save(&image_path).unwrap();
let index = crate::gallery::scan_gallery(&gallery, None).unwrap();
let id = index.images[0].id.clone();
let thumbnails = ThumbnailManager::new(temp.path().join("cache")).unwrap();
thumbnails.reconcile(&index.images);
let app = router(AppState {
index: Arc::new(RwLock::new(Arc::new(index))),
thumbnails,
});
std::fs::write(&image_path, "replacement with a different size").unwrap();
let response = app
.oneshot(
Request::builder()
.uri(format!("/api/images/{id}/original"))
.body(Body::empty())
.unwrap(),
)
.await
.unwrap();
assert_eq!(response.status(), StatusCode::NOT_FOUND);
}
#[tokio::test]
async fn versioned_assets_are_immutable() {
let response = request(&format!("/assets/{ASSET_VERSION}/app.js")).await;
@@ -752,5 +873,17 @@ mod tests {
response.headers()[header::CACHE_CONTROL],
"private, no-cache"
);
let weak_response = empty_router()
.oneshot(
Request::builder()
.uri("/api/gallery")
.header(header::IF_NONE_MATCH, "W/\"gallery-\"")
.body(Body::empty())
.unwrap(),
)
.await
.unwrap();
assert_eq!(weak_response.status(), StatusCode::NOT_MODIFIED);
}
}