davsync, gui: a server file that isn't a backup is said as what it is, and can be replaced by this computer's setup (JasonLeeForOnly on Discord)

JasonLeeForOnly's WebDAV sync failed with only "not a magpie backup" for
several versions, and nothing on the page got past it. A WebDAV GET of the
backup can come back as a body that isn't the file in several ways; each is
now reproduced with an httptest server that answers the way the real one
does:

Unpacked (WebDAV and S3): gzip with or without Content-Encoding (a proxy
that drops the header), x-gzip, zstd with or without the header, deflate
(zlib or raw), and a header that says gzip over a plain body. Bodies are
told by magic number; a backup is JSON, so either magic is never the file.
Brotli (br) can't be unpacked and is said as itself, with what to turn off.

Said, with no upload offered: an HTML page answered with 200 (Nextcloud
login after a redirect, Cloudflare Access, an AList/OpenList web app, an
rclone directory listing, a captive portal), with its <title> and the
host/path a redirect ended at (query, fragment and user dropped), and
where the WebDAV address is for 坚果云 and Nextcloud; an XML answer, by its
root element.

Said, with Upload as the way out: an empty (0-byte) file, a backup cut
short or damaged (the automatic rebuild is unchanged and still goes first
when its rules allow), another app's file ("format": "…"), other JSON,
and any other file by its content type. Messages carry the size, kind and a
page title only, never the file's contents.

backup.Open now tells ErrNotBackup (doesn't begin like a backup) from
ErrCorrupt (begins like one, cut short). The status carries the kind as
serverFile; Settings › Sync and backup says it in the reader's language,
wraps it whole, and shows "Upload this computer's setup" only when the file
may be replaced. davsync.Upload (POST /api/davsync/upload, `magpie webdav
upload`, `magpie s3 upload`) asks first in the GUI, re-reads the file,
refuses a backup this computer opens, one sealed with another passphrase,
a newer magpie's, a page and an XML answer, keeps the server's copy in the
sync folder (0600, newest 5) and writes nothing if that fails, then writes
with If-Match on the version read. New strings in en, zh, zh-TW, ja, de.

Verified: internal/davsync/not_backup_test.go (TestReadCompressedBackup,
TestReadBrotliSaid, TestServerAnswerSaid, TestCutBackupSaidAndUploaded,
TestS3ReadCompressedObject; Upload-only parts stripped to compile) fails
on e4a90aa6 for every compressed case but "gzip said", Brotli, every
server-answer case, the cut case and S3 gzip, and passes here. Dropping
the Replace check in Upload turns the page/XML cases red ("Upload wrote
over a page"). gofmt -l clean; go vet darwin, GOOS=linux and GOOS=windows
-tags nogui on ., davsync, backup, gui; full go test -tags nogui ./... ok;
-race on davsync and backup ok. GUI: sync-not-backup.test.cjs (en, zh,
zh-TW, ja, de; 900 and 440px), sync-*, s3-sync, gui-de, gui-ja, gui-zh-tw,
click-scroll in Chromium and WebKit, 81 pass. Not tried against a real
WebDAV server or on the Windows box; the reporter's server is unknown.

Note: 2018882e on main carries this subject by mistake (another session read a shared message file); it holds only GTK DPI code. This commit is the actual WebDAV change.
This commit is contained in:
yetone committed 2026-10-09 12:14:26 +08:00
1 parent 28c31a365e
commit be2037285a
12 files changed
+1007 -29

No files matched your search

+7 -2
View File
@@ -109,6 +109,11 @@ var ErrPassphrase = errors.New("wrong passphrase, or the file was changed")
// the damaged one from this computer rather than fail on it forever.
var ErrCorrupt = errors.New("not a magpie backup")
// ErrNotBackup is a file that doesn't begin like a backup at all: an empty
// one, a page a server answered with, another app's file. It reads as
// ErrCorrupt does, but is never rebuilt over by a sync on its own.
var ErrNotBackup = errors.New("not a magpie backup")
// BundleVersion is the bundle's structure: 2 carries the per-part
// credential markers (ProvidersKeys, SettingsKeys, LibraryKeys) as their
// own three-state flags; 1 and before had the whole-bundle Keys alone.
@@ -278,10 +283,10 @@ func Open(data []byte, pass string) (Bundle, error) {
if isBackupJSON(data) {
return Bundle{}, ErrCorrupt
}
return Bundle{}, errors.New("not a magpie backup")
return Bundle{}, ErrNotBackup
}
if e.Format != format {
return Bundle{}, errors.New("not a magpie backup")
return Bundle{}, ErrNotBackup
}
if e.Version != 1 || e.KDF != "pbkdf2-sha256" {
return Bundle{}, errors.New("this backup was made by a newer magpie; update magpie to open it")
+19 -13
View File
@@ -2,7 +2,6 @@ package davsync
import (
"bytes"
"compress/gzip"
"context"
"encoding/xml"
"errors"
@@ -14,6 +13,8 @@ import (
"strconv"
"strings"
"time"
"github.com/yetone/magpie/internal/backup"
)
// folder and file are where the backup lives under the address given.
@@ -279,21 +280,26 @@ func (d *dav) get(ctx context.Context, have version) (data []byte, v version, er
case res.StatusCode != http.StatusOK:
return nil, version{}, fmt.Errorf("reading %s from the WebDAV server: HTTP %d", file, res.StatusCode)
}
body := io.Reader(res.Body)
// asked for as it is, the file still came compressed: Go unpacks it
// only when it asked for gzip itself
if strings.EqualFold(strings.TrimSpace(res.Header.Get("Content-Encoding")), "gzip") {
zr, err := gzip.NewReader(res.Body)
if err != nil {
return nil, version{}, fmt.Errorf("reading %s from the WebDAV server: %w", file, err)
}
defer zr.Close()
body = zr
}
data, err = io.ReadAll(io.LimitReader(body, 64<<20))
data, err = io.ReadAll(io.LimitReader(res.Body, 64<<20))
if err != nil {
return nil, version{}, err
}
// asked for as it is, the file still came compressed: Go unpacks it
// only when it asked for gzip itself
if data, err = unpacked("WebDAV", data, res.Header.Get("Content-Encoding")); err != nil {
return nil, version{}, fmt.Errorf("reading %s from the WebDAV server: %w", file, err)
}
// a redirect that ended at something other than the file — a web
// app's sign-in or files page — is said with where it went: the
// address is the thing to fix
if res.Request != nil && res.Request.Response != nil && !backupHead(bytes.TrimSpace(data)) {
if f := whatIs(data); !f.Replace {
to := *res.Request.URL
to.RawQuery, to.Fragment, to.User = "", "", nil
f.Moved = to.Host + to.Path
return nil, version{}, &notBackup{f: f, where: "the file " + folder + "/" + file + " on the WebDAV server", cmd: "webdav", err: backup.ErrNotBackup}
}
}
return data, versionOf(res.Header), nil
}
+30 -11
View File
@@ -170,6 +170,9 @@ type state struct {
// Undo is the file the last Restore kept this computer's setup in, from
// just before it: Undo brings it back
Undo string `json:"undo,omitempty"`
// File is what the server held where the backup should be, when the
// last sync couldn't read it as one
File *ServerFile `json:"file,omitempty"`
}
func path(name string) string { return filepath.Join(settings.Dir(), name) }
@@ -377,6 +380,9 @@ type View struct {
Auto int `json:"auto"`
// Undo is whether the last Restore can be undone
Undo bool `json:"undo,omitempty"`
// File is what the server holds instead of a backup, when Error is
// that it isn't one; File.Replace offers Upload
File *ServerFile `json:"serverFile,omitempty"`
}
// OtherView is the other kind's server as the Settings page shows it.
@@ -404,6 +410,9 @@ func Status() View {
if st.Key == stateKey(c) {
v.Last = st.Last
v.Undo = st.Undo != "" && isFile(st.Undo)
if st.Error != "" {
v.File = st.File
}
if c.Usage && st.Usage != nil {
v.UsageError = st.Usage.Error
}
@@ -480,21 +489,27 @@ func cached(want string) []byte {
// that always answers the same unreadable way) would otherwise pile up a
// copy each time. The name is sorted by time, so the oldest are the first
// to go.
func keepDamaged(data []byte) {
func keepDamaged(data []byte) { keepAside(data, "server-damaged", 3) }
// keepAside writes data under the sync folder as <time>-<tag>, keeping the
// newest n so tagged.
func keepAside(data []byte, tag string, n int) error {
dir := path("sync")
if os.MkdirAll(dir, 0o700) != nil {
return
if err := os.MkdirAll(dir, 0o700); err != nil {
return err
}
name := filepath.Join(dir, time.Now().Format("2006-01-02-150405")+"-server-damaged"+backup.Ext)
if edit.WriteAtomic(name, data) != nil {
return
name := filepath.Join(dir, time.Now().Format("2006-01-02-150405")+"-"+tag+backup.Ext)
if err := edit.WriteAtomic(name, data); err != nil {
return err
}
old, _ := filepath.Glob(filepath.Join(dir, "*-server-damaged"+backup.Ext))
os.Chmod(name, 0o600)
old, _ := filepath.Glob(filepath.Join(dir, "*-"+tag+backup.Ext))
slices.Sort(old) // names begin with the time, so oldest first
for len(old) > 3 {
for len(old) > n {
os.Remove(old[0])
old = old[1:]
}
return nil
}
func sum(b []byte) string {
@@ -540,7 +555,11 @@ func syncNow(ctx context.Context, force bool) error {
if errors.Is(err, errChanged) { // another computer got in between: again, over its version
err = syncOnce(ctx, c, &st)
}
st.Error = ""
st.Error, st.File = "", nil
var nb *notBackup
if errors.As(err, &nb) {
st.File = &nb.f
}
if err != nil {
st.Error = err.Error()
} else {
@@ -950,7 +969,7 @@ func syncOnce(ctx context.Context, c Config, st *state) error {
// still checked against the server and another computer's parts
// survive the way they would on any sync.
if !errors.Is(err, backup.ErrCorrupt) || st.Local == nil {
return err
return described(c, data, err)
}
good := cached(st.Sum) // the version this computer last read whole
// The damaged body must be that version cut short — a prefix of it,
@@ -970,7 +989,7 @@ func syncOnce(ctx context.Context, c Config, st *state) error {
// smallest — and a relay cuts a real write deep into the data, well
// past "data", so legit rebuilds still go.
if good == nil || !bytes.HasPrefix(good, data) || len(data) <= bytes.Index(good, []byte(`"data"`)) {
return err
return described(c, data, err)
}
remote, err = backup.Open(good, c.Passphrase)
if err != nil {
+368
View File
@@ -0,0 +1,368 @@
package davsync
import (
"bytes"
"compress/gzip"
"compress/zlib"
"context"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"strings"
"testing"
"github.com/klauspost/compress/zstd"
"github.com/yetone/magpie/internal/backup"
"github.com/yetone/magpie/internal/provider"
"github.com/yetone/magpie/internal/settings"
)
// "not a magpie backup" was all a sync said whatever the server sent where
// the backup should be (JasonLeeForOnly on Discord, over several versions),
// and nothing in magpie could get past it. Each case below is a server
// behaving as a real one does: one that compresses though asked not to, a
// web page in front of WebDAV, a file left empty or cut short, another
// app's file at the path. What magpie can read it reads; the rest is said
// as what it is, and a file that is really there and not a backup can be
// replaced from this computer, the server's copy kept first.
// served is a fake WebDAV whose reads of the backup file answer as get
// says, when it says; writes and everything else go to the fake.
type served struct {
fake *fakeDAV
get func(w http.ResponseWriter, r *http.Request) bool
}
func (s *served) ServeHTTP(w http.ResponseWriter, r *http.Request) {
if s.get != nil && r.Method == http.MethodGet && s.get(w, r) {
return
}
s.fake.ServeHTTP(w, r)
}
// syncedServer is a fake WebDAV that computer a has synced its setup to:
// the server's backup is a's, sealed with "horse".
func syncedServer(t *testing.T) (*served, *httptest.Server, Config, []byte) {
t.Helper()
fake := &fakeDAV{files: map[string][]byte{}, etags: map[string]string{}, dirs: map[string]bool{"/dav": true}}
s := &served{fake: fake}
srv := httptest.NewServer(s)
t.Cleanup(srv.Close)
cfg := Config{URL: srv.URL + "/dav/", User: "me", Password: "pw", Passphrase: "horse", Keys: true, Agents: true}
a := newComputer(t)
a.use(t)
provider.Save(provider.Provider{ID: "deepseek", Name: "DeepSeek", Chat: "https://api.deepseek.com/v1", Key: "k1"})
if err := Configure(cfg); err != nil {
t.Fatal(err)
}
if err := Now(context.Background()); err != nil {
t.Fatal(err)
}
return s, srv, cfg, onServer(fake, backupFile)
}
func gz(b []byte) []byte {
var buf bytes.Buffer
w := gzip.NewWriter(&buf)
w.Write(b)
w.Close()
return buf.Bytes()
}
// A server that compresses the file though asked for it as it is: with the
// header said, with it dropped by a proxy, in zstd and deflate, and one
// that says gzip and sends the file as it is. Each reads as the backup.
func TestReadCompressedBackup(t *testing.T) {
zs := func(b []byte) []byte {
e, _ := zstd.NewWriter(nil)
return e.EncodeAll(b, nil)
}
zl := func(b []byte) []byte {
var buf bytes.Buffer
w := zlib.NewWriter(&buf)
w.Write(b)
w.Close()
return buf.Bytes()
}
cases := map[string]struct {
enc string
pack func([]byte) []byte
}{
"gzip said": {"gzip", gz},
"gzip unsaid": {"", gz},
"x-gzip": {"x-gzip", gz},
"zstd": {"zstd", zs},
"zstd unsaid": {"", zs},
"deflate": {"deflate", zl},
"gzip said not done": {"gzip", func(b []byte) []byte { return b }},
}
for name, tc := range cases {
t.Run(name, func(t *testing.T) {
s, _, cfg, good := syncedServer(t)
s.get = func(w http.ResponseWriter, r *http.Request) bool {
if r.URL.Path != backupFile {
return false
}
if tc.enc != "" {
w.Header().Set("Content-Encoding", tc.enc)
}
w.Header().Set("ETag", `"x"`)
w.Write(tc.pack(good))
return true
}
b := newComputer(t)
b.use(t)
if err := Configure(cfg); err != nil {
t.Fatal(err)
}
if err := Now(context.Background()); err != nil {
t.Fatalf("a compressed backup didn't read: %v", err)
}
if got := ids(); len(got) != 1 || got[0] != "deepseek=k1" {
t.Fatalf("b didn't take a's setup: %v", got)
}
})
}
}
// Brotli, which magpie can't unpack, is said as itself.
func TestReadBrotliSaid(t *testing.T) {
s, _, _, _ := syncedServer(t)
s.get = func(w http.ResponseWriter, r *http.Request) bool {
w.Header().Set("Content-Encoding", "br")
w.Write([]byte{0x1b, 0x2a, 0x00, 0xf8, 0x8d, 0x94})
return true
}
err := Now(context.Background())
if err == nil || !strings.Contains(err.Error(), "Brotli") {
t.Fatalf("a Brotli body should be said as one: %v", err)
}
}
// What a server answers instead of the file, as real ones do: each is said
// as what it is, with what to do; a page is never offered for replacing,
// and nothing is written to the server for it.
func TestServerAnswerSaid(t *testing.T) {
page := func(title, body string) []byte {
return []byte("<!DOCTYPE html>\n<html lang=\"en\"><head><meta charset=\"utf-8\"><title>" + title + "</title></head><body>" + body + "</body></html>")
}
type answer struct {
status int
ctype string
body []byte
// to: the read is redirected there first
to string
}
cases := map[string]struct {
a answer
want []string
what string
replace bool
}{
// a Nextcloud's address given without /remote.php/dav: its web
// app sends the read to its sign-in page
"nextcloud login": {answer{to: "/login?redirect_url=/dav/magpie", ctype: "text/html; charset=UTF-8", body: page("Nextcloud", `<form method="post" name="login">`)},
[]string{"web page", `"Nextcloud"`, "redirect to 127.0.0.1", "/login", "WebDAV address"}, "page", false},
// Cloudflare Access in front of a tunnel
"cloudflare access": {answer{to: "/cdn-cgi/access/login/dav.example.com?kid=abc", ctype: "text/html", body: page("Sign in ・ Cloudflare Access", "")},
[]string{"Cloudflare Access", "redirect to"}, "page", false},
// Alist's and OpenList's web app answers any path that isn't
// /dav/ with its page
"alist web app": {answer{status: 200, ctype: "text/html", body: page("AList", `<div id="root"></div>`)},
[]string{`"AList"`, "WebDAV address"}, "page", false},
// rclone serve webdav (and a Synology's web station) list a
// folder as a page
"directory listing": {answer{status: 200, ctype: "text/html; charset=utf-8", body: []byte("<html><head><title>Directory listing of /magpie/magpie.magpie-backup/</title></head><body><a href=\"../\">../</a></body></html>")},
[]string{"Directory listing of /magpie/magpie.magpie-backup/"}, "page", false},
// a captive portal's page with no <html> to begin with
"captive portal": {answer{status: 200, ctype: "text/html", body: []byte("<head><title>Hotel WiFi – Sign in</title></head><body><h1>Welcome</h1></body>")},
[]string{"Hotel WiFi – Sign in"}, "page", false},
// a WebDAV error document answered 200 by a broken proxy
"xml": {answer{status: 200, ctype: "application/xml", body: []byte(`<?xml version="1.0" encoding="utf-8"?><d:error xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns"><s:exception>Sabre\DAV\Exception\NotAuthenticated</s:exception></d:error>`)},
[]string{"XML document", "<d:error>"}, "xml", false},
// a write that never landed left an empty file
"empty": {answer{status: 200, ctype: "application/octet-stream", body: []byte{}},
[]string{"is empty (0 bytes)", "magpie webdav upload"}, "empty", true},
// another app's file at the same path
"other app": {answer{status: 200, ctype: "application/octet-stream", body: []byte(`{"format":"cc-switch-backup","version":3,"data":"..."}`)},
[]string{`"cc-switch-backup" file`, "another app", "upload"}, "format", true},
"json": {answer{status: 200, ctype: "application/json", body: []byte(`[1,2,3]`)},
[]string{"JSON file that isn't a magpie backup (7 bytes)"}, "json", true},
"zip": {answer{status: 200, ctype: "application/octet-stream", body: append([]byte("PK\x03\x04"), make([]byte, 40)...)},
[]string{"application/zip (44 bytes)"}, "other", true},
}
for name, tc := range cases {
t.Run(name, func(t *testing.T) {
s, _, cfg, good := syncedServer(t)
// a file really at the path is the server's own; a page is
// answered in front of it
if tc.replace {
s.fake.mu.Lock()
s.fake.n++
s.fake.files[backupFile], s.fake.etags[backupFile] = tc.a.body, `"other"`
s.fake.mu.Unlock()
} else {
s.get = answering(tc.a.to, tc.a.ctype, tc.a.body)
}
// a fresh computer: nothing of its own to rebuild with
b := newComputer(t)
b.use(t)
if err := Configure(cfg); err != nil {
t.Fatal(err)
}
puts := s.fake.puts
err := Now(context.Background())
if err == nil {
t.Fatal("the sync went on over what isn't a backup")
}
t.Log(err)
for _, w := range tc.want {
if !strings.Contains(err.Error(), w) {
t.Errorf("the error doesn't say %q: %v", w, err)
}
}
if strings.Contains(err.Error(), "data\":") || strings.Contains(err.Error(), "<form") {
t.Errorf("the error carries the body: %v", err)
}
v := Status()
if v.File == nil || v.File.What != tc.what || v.File.Replace != tc.replace {
t.Fatalf("the page isn't told what the server holds: %+v", v.File)
}
if s.fake.puts != puts {
t.Fatal("a sync wrote over it")
}
// Upload writes over only what is a file there
err = Upload(context.Background())
if !tc.replace {
if err == nil || s.fake.puts != puts {
t.Fatalf("Upload wrote over a page the server answered with: %v", err)
}
if got := onServer(s.fake, backupFile); !bytes.Equal(got, good) {
t.Fatal("the server's backup changed")
}
return
}
if err != nil {
t.Fatalf("Upload: %v", err)
}
s.get = nil
if _, err := backup.Open(onServer(s.fake, backupFile), "horse"); err != nil {
t.Fatalf("the uploaded file doesn't open: %v", err)
}
if v := Status(); v.Error != "" || v.File != nil {
t.Fatalf("the error stayed after Upload: %+v", v)
}
kept, _ := filepath.Glob(filepath.Join(settings.Dir(), "sync", "*-server-replaced"+backup.Ext))
if len(tc.a.body) == 0 {
if len(kept) != 0 {
t.Fatalf("an empty file was kept: %v", kept)
}
} else if len(kept) != 1 {
t.Fatalf("the server's file wasn't kept before it was replaced: %v", kept)
} else if b, _ := os.ReadFile(kept[0]); !bytes.Equal(b, tc.a.body) {
t.Fatal("what was kept isn't what the server held")
}
if err := Now(context.Background()); err != nil {
t.Fatalf("the sync after Upload: %v", err)
}
})
}
}
// answering is a server that answers every read of the backup with body,
// after a redirect to to when there is one.
func answering(to, ctype string, body []byte) func(w http.ResponseWriter, r *http.Request) bool {
return func(w http.ResponseWriter, r *http.Request) bool {
if r.URL.Path == backupFile && to != "" {
http.Redirect(w, r, to, http.StatusFound)
return true
}
if r.URL.Path != backupFile && (to == "" || !strings.HasPrefix(to, r.URL.Path)) {
return false
}
w.Header().Set("Content-Type", ctype)
w.Write(body)
return true
}
}
// A backup cut short that this computer can't rebuild (it never read the
// version that broke) is said as cut short, and Upload replaces it.
func TestCutBackupSaidAndUploaded(t *testing.T) {
s, _, cfg, good := syncedServer(t)
cut(s.fake, backupFile, len(good)/2)
b := newComputer(t)
b.use(t)
if err := Configure(cfg); err != nil {
t.Fatal(err)
}
err := Now(context.Background())
if err == nil || !strings.Contains(err.Error(), "cut short") || !strings.Contains(err.Error(), "upload") {
t.Fatalf("a cut backup should be said as one: %v", err)
}
if _, rerr := Restore(context.Background()); rerr == nil || !strings.Contains(rerr.Error(), "cut short") {
t.Fatalf("Restore should say it too: %v", rerr)
}
if v := Status(); v.File == nil || v.File.What != "cut" || !v.File.Replace {
t.Fatalf("%+v", v.File)
}
if err := Upload(context.Background()); err != nil {
t.Fatal(err)
}
if _, err := backup.Open(onServer(s.fake, backupFile), "horse"); err != nil {
t.Fatalf("the uploaded file doesn't open: %v", err)
}
}
// Upload never writes over a backup: one this computer opens (a sync
// merges with it) or one sealed with another passphrase.
func TestUploadKeepsABackup(t *testing.T) {
s, _, cfg, good := syncedServer(t)
puts := s.fake.puts
if err := Upload(context.Background()); err == nil || !strings.Contains(err.Error(), "Sync now") {
t.Fatalf("Upload over a good backup: %v", err)
}
b := newComputer(t)
b.use(t)
cfg.Passphrase = "other"
if err := Configure(cfg); err != nil {
t.Fatal(err)
}
if err := Upload(context.Background()); err == nil || !strings.Contains(err.Error(), "passphrase") {
t.Fatalf("Upload over another passphrase's backup: %v", err)
}
if s.fake.puts != puts || !bytes.Equal(onServer(s.fake, backupFile), good) {
t.Fatal("a backup was written over")
}
}
// S3: an object stored compressed without saying so reads as the backup.
func TestS3ReadCompressedObject(t *testing.T) {
f, srv := newFakeS3(t)
cfg := f.config(srv)
const key = "team x+y/magpie/magpie.magpie-backup"
a := newComputer(t)
a.use(t)
provider.Save(provider.Provider{ID: "deepseek", Name: "DeepSeek", Chat: "https://api.deepseek.com/v1", Key: "k1"})
if err := Configure(cfg); err != nil {
t.Fatal(err)
}
if err := Now(context.Background()); err != nil {
t.Fatal(err)
}
f.mu.Lock()
f.store(key, gz(f.objects[key]))
f.mu.Unlock()
b := newComputer(t)
b.use(t)
if err := Configure(cfg); err != nil {
t.Fatal(err)
}
if err := Now(context.Background()); err != nil {
t.Fatalf("a gzipped object didn't read: %v", err)
}
if got := ids(); len(got) != 1 || got[0] != "deepseek=k1" {
t.Fatalf("b didn't take a's setup: %v", got)
}
}
+236
View File
@@ -0,0 +1,236 @@
package davsync
import (
"bytes"
"compress/flate"
"compress/gzip"
"compress/zlib"
"encoding/json"
"errors"
"fmt"
"html"
"io"
"mime"
"net/http"
"regexp"
"strings"
"unicode/utf8"
"github.com/klauspost/compress/zstd"
"github.com/yetone/magpie/internal/backup"
)
// ServerFile is what the server held, or answered with, where the backup
// should be, when it isn't one: what the Settings page says of it, and
// whether Upload may write this computer's setup over it. It never carries
// the file's contents — only its size, its kind, and a page's title.
type ServerFile struct {
// What is "empty", "cut" (a magpie backup cut short), "page" (a web
// page), "xml", "format" (another app's JSON file, named by its
// "format"), "json" or "other"
What string `json:"what"`
Size int `json:"size"`
Title string `json:"title,omitempty"` // a page's <title>
Format string `json:"format,omitempty"` // a JSON file's own "format"
Type string `json:"type,omitempty"` // the kind of file, as its first bytes tell
Moved string `json:"moved,omitempty"` // where a redirect took the read: host and path
// Replace is whether the file is one Upload may write over: a page or
// an XML answer is the server's (or a proxy's), not a file at that path
Replace bool `json:"replace,omitempty"`
}
// notBackup is the error for a server file that doesn't open as a backup:
// what it is and what to do, instead of only "not a magpie backup".
type notBackup struct {
f ServerFile
where string // the file and server, as messages name them
cmd string // the CLI command: "webdav" or "s3"
err error
}
func (e *notBackup) Unwrap() error { return e.err }
func (e *notBackup) Error() string {
f := e.f
upload := fmt.Sprintf("upload this computer's setup over it (Settings › Sync and backup, or magpie %s upload); the server's copy is kept in the sync folder first", e.cmd)
switch f.What {
case "empty":
return fmt.Sprintf("%s is empty (0 bytes): a write to it didn't finish. To start it again, %s", e.where, upload)
case "cut":
return fmt.Sprintf("%s is a magpie backup cut short or damaged (%d bytes): a write to it was interrupted. Sync on the computer that wrote it last to rebuild it, or %s", e.where, f.Size, upload)
case "page", "xml":
got := "a web page"
if f.Title != "" {
got += fmt.Sprintf(" (%q)", f.Title)
}
if f.What == "xml" {
got = "an XML document"
if f.Title != "" {
got += fmt.Sprintf(" (<%s>)", f.Title)
}
}
got += " instead of the file"
if f.Moved != "" {
got += ", after a redirect to " + f.Moved
}
if e.cmd == "s3" {
return fmt.Sprintf("reading %s, the server answered with %s: check the endpoint is the S3 API's, not the storage's website or console", e.where, got)
}
return fmt.Sprintf("reading %s, the server answered with %s: check the address is the server's WebDAV address (坚果云: https://dav.jianguoyun.com/dav/, Nextcloud: https://…/remote.php/dav/files/<user>/), not its web page, and that no sign-in page or proxy stands in front of it", e.where, got)
case "format":
return fmt.Sprintf("%s is a %q file, not a magpie backup: another app keeps a file there. Point sync at another folder, or %s", e.where, f.Format, upload)
case "json":
return fmt.Sprintf("%s is a JSON file that isn't a magpie backup (%d bytes). Point sync at another folder, or %s", e.where, f.Size, upload)
}
return fmt.Sprintf("%s isn't a magpie backup: it is %s (%d bytes). Point sync at another folder, or %s", e.where, f.Type, f.Size, upload)
}
// fileWhere names the backup file and the server in messages.
func fileWhere(c Config) string {
if c.S3() {
if s, err := newS3(c); err == nil {
return s.where()
}
return "the file in the S3 bucket"
}
return "the file " + folder + "/" + file + " on the WebDAV server"
}
func cmdOf(c Config) string {
if c.S3() {
return "s3"
}
return "webdav"
}
// described is err, from opening data as a backup, said as what data is
// when it isn't a backup at all; any other error stays as it is.
func described(c Config, data []byte, err error) error {
if !errors.Is(err, backup.ErrCorrupt) && !errors.Is(err, backup.ErrNotBackup) {
return err
}
var nb *notBackup
if errors.As(err, &nb) {
return err
}
return &notBackup{f: whatIs(data), where: fileWhere(c), cmd: cmdOf(c), err: err}
}
var titleRe = regexp.MustCompile(`(?is)<title[^>]*>(.*?)</title>`)
var rootRe = regexp.MustCompile(`<([A-Za-z_][\w.:-]*)`)
// whatIs tells what a body that isn't a backup is, from its bytes alone.
func whatIs(data []byte) ServerFile {
f := ServerFile{Size: len(data), Replace: true}
trim := bytes.TrimSpace(bytes.TrimPrefix(data, []byte("\xef\xbb\xbf")))
head := strings.ToLower(string(trim[:min(len(trim), 512)]))
switch {
case len(data) == 0:
f.What = "empty"
case backupHead(trim):
f.What = "cut"
case strings.HasPrefix(head, "<") && (strings.Contains(head, "<html") ||
strings.HasPrefix(http.DetectContentType(trim), "text/html")):
f.What, f.Replace = "page", false
if m := titleRe.FindSubmatch(trim[:min(len(trim), 64<<10)]); m != nil {
f.Title = clip(strings.Join(strings.Fields(html.UnescapeString(string(m[1]))), " "), 80)
}
case strings.HasPrefix(head, "<?xml") || strings.HasPrefix(head, "<"):
f.What, f.Replace = "xml", false
body := trim
if strings.HasPrefix(head, "<?xml") {
if i := bytes.Index(body, []byte("?>")); i >= 0 {
body = body[i+2:]
}
}
if m := rootRe.FindSubmatch(body); m != nil {
f.Title = clip(string(m[1]), 40)
}
case json.Valid(trim):
var top struct {
Format any `json:"format"`
}
if json.Unmarshal(trim, &top) == nil {
if s, ok := top.Format.(string); ok && s != "" {
f.What, f.Format = "format", clip(s, 40)
return f
}
}
f.What = "json"
default:
f.What = "other"
f.Type, _, _ = mime.ParseMediaType(http.DetectContentType(data))
if f.Type == "" {
f.Type = "application/octet-stream"
}
}
return f
}
// backupHead is whether b begins like a sealed backup, as Seal writes it.
func backupHead(b []byte) bool {
head := strings.Map(func(r rune) rune {
if r == ' ' || r == '\t' || r == '\n' || r == '\r' {
return -1
}
return r
}, string(b[:min(len(b), 64)]))
const want = `{"format":"magpie-backup"`
return strings.HasPrefix(head, want) || len(head) > 0 && strings.HasPrefix(want, head)
}
func clip(s string, n int) string {
if len(s) <= n {
return s
}
for n > 0 && !utf8.RuneStart(s[n]) {
n--
}
return s[:n] + "…"
}
// Magic numbers of the compressed forms a server may send the file in.
var (
gzipMagic = []byte{0x1f, 0x8b}
zstdMagic = []byte{0x28, 0xb5, 0x2f, 0xfd}
)
// unpacked is a read's body as the file is: the Content-Encoding a server
// sent it in though asked for it as it is — gzip, deflate, zstd — undone,
// and a gzip or zstd body sent without saying so (a proxy that dropped the
// header) undone too. A backup is JSON, so a body that begins with either's
// magic number is never the file as it is. A header that names an encoding
// the body isn't in (a server that says gzip and sends the file) is
// ignored; one magpie can't undo (br) is said plainly.
func unpacked(kind string, data []byte, encoding string) ([]byte, error) {
enc := strings.ToLower(strings.TrimSpace(encoding))
if i := strings.LastIndexByte(enc, ','); i >= 0 { // the last applied is undone first
enc = strings.TrimSpace(enc[i+1:])
}
switch {
case bytes.HasPrefix(data, gzipMagic):
return readAll(gzip.NewReader(bytes.NewReader(data)))
case bytes.HasPrefix(data, zstdMagic):
d, err := zstd.NewReader(bytes.NewReader(data))
if err != nil {
return nil, err
}
defer d.Close()
return readAll(d, nil)
case enc == "deflate" && len(data) > 0 && data[0] != '{':
if out, err := readAll(zlib.NewReader(bytes.NewReader(data))); err == nil {
return out, nil
}
return readAll(flate.NewReader(bytes.NewReader(data)), nil)
case enc == "br" && len(data) > 0 && data[0] != '{':
return nil, fmt.Errorf("the %s server sent the backup compressed with Brotli (Content-Encoding: br), though magpie asked for it as it is, and magpie can't unpack that: turn off Brotli compression for it on the server, or on the proxy in front of it", kind)
}
return data, nil
}
func readAll[R io.Reader](r R, err error) ([]byte, error) {
if err != nil {
return nil, err
}
return io.ReadAll(io.LimitReader(r, 64<<20))
}
+72 -1
View File
@@ -70,7 +70,7 @@ func Restore(ctx context.Context) ([]string, error) {
return errors.New("the passphrase doesn't open the file on the server: it was sealed with another one; use the passphrase set on your other computers")
}
if err != nil {
return err
return described(c, data, err)
}
local, err := collect(c)
if err != nil {
@@ -114,6 +114,77 @@ func Restore(ctx context.Context) ([]string, error) {
return brought, err
}
// Upload writes this computer's setup over the server's file when that
// file isn't a backup a sync can read — empty, cut short, another app's —
// so a sync stopped on it has a way on that the user chooses. A file that
// opens as a backup, one sealed with another passphrase, a newer magpie's,
// and a page the server answered with instead of a file are never written
// over. What the server held is kept in the sync folder first, and nothing
// is written when it can't be. The write is made over the version read, so
// a file another computer wrote meanwhile isn't.
func Upload(ctx context.Context) error {
return locked(func() error {
c, ok := Load()
if !ok {
return ErrOff
}
d, err := newRemote(c)
if err != nil {
return err
}
data, ver, err := d.get(ctx, version{})
if err != nil {
return err
}
if data != nil {
_, err := backup.Open(data, c.Passphrase)
switch {
case err == nil:
return errors.New("the file on the server is a magpie backup this computer can open: Sync now merges with it instead, and nothing was written over it")
case errors.Is(err, backup.ErrPassphrase):
return errors.New("the file on the server is a magpie backup sealed with another passphrase, so it wasn't written over: use the passphrase set on your other computers")
}
err = described(c, data, err)
var nb *notBackup
if !errors.As(err, &nb) || !nb.f.Replace {
return err
}
if len(data) > 0 {
if err := keepAside(data, "server-replaced", 5); err != nil {
return fmt.Errorf("keeping the server's file before writing over it, so nothing was written: %w", err)
}
}
}
local, err := collect(c)
if err != nil {
return err
}
b := local
b.Created, b.App = time.Now().UTC(), "magpie"
sealed, err := backup.Seal(b, c.Passphrase)
if err != nil {
return err
}
v, err := d.put(ctx, sealed, ver.ETag)
if errors.Is(err, errChanged) {
return errors.New("the file on the server changed while it was being replaced, so it wasn't: sync now to see what it is")
}
if err != nil {
return err
}
st := loadState()
if st.Key != stateKey(c) {
st = state{Key: stateKey(c)}
}
// as the first sync to an empty server leaves it
st.Local, st.Remote, st.Sum, st.Server = hashes(local), hashes(b), sum(sealed), v
remember(sealed)
st.Last, st.Error, st.File = time.Now(), "", nil
saveState(st)
return nil
})
}
// Undo puts back the setup the last Restore replaced. The next sync takes
// it to the server, as any change made here.
func Undo() ([]string, error) {
+5
View File
@@ -216,6 +216,11 @@ func (s *s3) get(ctx context.Context, have version) (data []byte, v version, err
if err != nil {
return nil, version{}, err
}
// Go unpacks a gzip it asked for; an object stored compressed under
// another encoding, or without saying so, is unpacked here
if data, err = unpacked("S3", data, res.Header.Get("Content-Encoding")); err != nil {
return nil, version{}, fmt.Errorf("reading %s: %w", s.where(), err)
}
return data, versionOf(res.Header), nil
}
+67 -2
View File
@@ -18610,7 +18610,7 @@ async function renderSync(v) {
const hostOf = (u) => { try { return new URL(u).host; } catch { return u; } };
// a bucket is named by its address, and the server it is on unless AWS
const host = s3 ? [v.url, v.endpoint && hostOf(v.endpoint.includes("://") ? v.endpoint : "https://" + v.endpoint)].filter(Boolean).join(" · ") : hostOf(v.url);
status = v.error ? t("Couldn't sync: {error}", { error: v.error })
status = v.error ? t("Couldn't sync: {error}", { error: v.serverFile ? serverFileSaid(v.serverFile) : v.error })
: v.last ? t("Synced {when} · {host}", { when: syncWhen(v.last), host }) : t("Not synced yet · {host}", { host });
// the other kind's server, kept from before sync moved here
if (v.other) status += " · " + t("{kind} settings kept", { kind: v.other.kind === "s3" ? "S3" : "WebDAV" });
@@ -18620,11 +18620,19 @@ async function renderSync(v) {
? [btn(t("Sync now"), async (e) => { e.target.classList.add("busy"); renderSync(await api("davsync/now", {}).catch((x) => ({ ...v, error: x.message }))); }),
btn(t(syncOpen === "dav" ? "Close" : "Edit"), toggle("dav"))]
: [btn(t(syncOpen === "dav" ? "Close" : "Set up"), toggle("dav"))]));
if (v.error) sub.classList.add("bad");
// a failure is shown whole: what the server sent and what to do don't
// fit one line
if (v.error) sub.classList.add("bad", "wraps");
// usage that couldn't be shared is a line of its own, in red: beside
// "Synced" in the same grey it read as part of a sync that went fine
// (#1259)
if (v.on && !v.error && v.usageError) sub.after(el("div", "sub bad wraps sync-usage-err", t("Couldn't share usage: {error}", { error: v.usageError })));
// the server's file isn't a backup a sync can read, and is a file: this
// computer's setup can go up in its place, the user choosing it
if (v.on && v.error && v.serverFile?.replace) {
row(t("Upload this computer's setup"), t("It replaces the file on the server. That file is kept in the sync folder first."),
btn(t("Upload…"), () => askUpload(v.serverFile)));
}
if (v.notice) {
const n = v.notice, r = el("div", "row pref sync-note");
const lines = [];
@@ -18706,6 +18714,63 @@ function askRestore(v, parts) {
cancel.focus({ preventScroll: true });
}
// serverFileSaid: what the server holds where the backup should be, in the
// reader's language, from what davsync told of it (never its contents).
function serverFileSaid(f) {
const size = fmtBytes(f.size || 0);
switch (f.what) {
case "empty": return t("the file on the server is empty (0 bytes)");
case "cut": return t("the file on the server is a magpie backup cut short ({size}): a write to it was interrupted", { size });
case "page": {
let s = f.title ? t("the server answered with a web page (“{title}”) instead of the file", { title: f.title }) : t("the server answered with a web page instead of the file");
if (f.moved) s += t(", after a redirect to {where}", { where: f.moved });
return s + t(": check the address is the server's WebDAV address, not its website");
}
case "xml": return t("the server answered with an XML document instead of the file: check the address is the server's WebDAV address");
case "format": return t("the file on the server is another app's (“{format}”), not a magpie backup", { format: f.format });
case "json": return t("the file on the server is a JSON file that isn't a magpie backup ({size})", { size });
}
return t("the file on the server isn't a magpie backup: it is {type} ({size})", { type: f.type || "?", size });
}
// askUpload asks before this computer's setup replaces a server file that
// isn't a backup; the server's file is kept in the sync folder first.
function askUpload(f) {
const ed = el("div", "editor restore-ask upload-ask");
const head = el("div", "ehead");
head.append(el("b", "", t("Replace the file on the server?")));
ed.append(head);
ed.append(el("p", "lib-confirm", t("This computer's setup goes up in place of the file on the server, and your other computers sync with it from then on.")));
if (f.what === "cut") ed.append(el("p", "lib-confirm", t("What another computer wrote into that file and has nowhere else is lost. If one of them synced it last, Sync now there may rebuild it instead.")));
ed.append(el("p", "lib-confirm", t("The file on the server is kept in the sync folder first.")));
const err = el("p", "editor-error", "");
ed.append(err);
const bar = el("div", "bar");
const go = el("button", "text primary danger-fill", t("Upload this computer's setup"));
go.onclick = async (e) => {
e.stopPropagation();
go.disabled = true;
go.classList.add("busy");
try {
const r = await api("davsync/upload", {});
closeConfirmAsk();
renderSync(r);
} catch (x) {
go.disabled = false;
go.classList.remove("busy");
err.textContent = x.message;
}
};
const cancel = el("button", "text", t("Cancel"));
cancel.onclick = (e) => { e.stopPropagation(); closeConfirmAsk(); };
bar.append(el("span", "grow"), cancel, go);
ed.append(bar);
confirmAsk = ed;
openModal(ed);
$("#modal").classList.add("lib");
cancel.focus({ preventScroll: true });
}
// refreshAfterSync: what a sync or an import brought in reaches the other
// pages, leaving this one (and what it says was done) as it is.
function refreshAfterSync() {
+68
View File
@@ -1848,6 +1848,23 @@ const I18N = {
"This computer's {parts} become the ones on the server, as your other computers last synced them. Whatever was changed here since is replaced.": "本机的{parts}将换成服务器上的(其他电脑最后同步的版本),本机之后的改动会被替换。",
"What is here now is sealed into the sync folder first, and Undo puts it back.": "当前配置会先加密备份到 sync 文件夹,可点「撤销」换回。",
"Restore setup": "恢复配置",
"the file on the server is empty (0 bytes)": "服务器上的文件是空的(0 字节)",
"the file on the server is a magpie backup cut short ({size}): a write to it was interrupted": "服务器上的文件是一个不完整的 magpie 备份({size}):上次写入被中断了",
"the server answered with a web page (“{title}”) instead of the file": "服务器返回的是一个网页(“{title}”),而不是备份文件",
"the server answered with a web page instead of the file": "服务器返回的是一个网页,而不是备份文件",
", after a redirect to {where}": ",重定向到了 {where}",
": check the address is the server's WebDAV address, not its website": ":请检查地址是服务器的 WebDAV 地址,而不是它的网站",
"the server answered with an XML document instead of the file: check the address is the server's WebDAV address": "服务器返回的是一份 XML 文档,而不是备份文件:请检查地址是服务器的 WebDAV 地址",
"the file on the server is another app's (“{format}”), not a magpie backup": "服务器上的文件是别的应用的(“{format}”),不是 magpie 备份",
"the file on the server is a JSON file that isn't a magpie backup ({size})": "服务器上的文件是一个 JSON 文件,但不是 magpie 备份({size})",
"the file on the server isn't a magpie backup: it is {type} ({size})": "服务器上的文件不是 magpie 备份,而是 {type}({size})",
"Upload this computer's setup": "上传本机的配置",
"It replaces the file on the server. That file is kept in the sync folder first.": "用本机配置替换服务器上的文件,替换前先把那个文件保存到 sync 文件夹。",
"Upload…": "上传…",
"Replace the file on the server?": "替换服务器上的文件?",
"This computer's setup goes up in place of the file on the server, and your other computers sync with it from then on.": "本机的配置会上传并替换服务器上的文件,之后其他电脑都和它同步。",
"What another computer wrote into that file and has nowhere else is lost. If one of them synced it last, Sync now there may rebuild it instead.": "另一台电脑写进那个文件、别处没有的内容会丢失。如果是其他某台电脑最后同步的,在那台电脑上点「立即同步」可能可以直接修复。",
"The file on the server is kept in the sync folder first.": "服务器上的文件会先保存到 sync 文件夹。",
"This computer's setup is already the server's: nothing to restore": "本机的配置已与服务器一致,无需恢复",
"OK": "知道了",
"Export": "导出", "Export…": "导出…",
@@ -5670,6 +5687,23 @@ const I18N = {
"This computer's {parts} become the ones on the server, as your other computers last synced them. Whatever was changed here since is replaced.": "本機的{parts}將換成伺服器上的(其他電腦最後同步的版本),本機之後的改動會被替換。",
"What is here now is sealed into the sync folder first, and Undo puts it back.": "目前設定會先加密備份到 sync 資料夾,可點「復原」換回。",
"Restore setup": "恢復設定",
"the file on the server is empty (0 bytes)": "伺服器上的檔案是空的(0 位元組)",
"the file on the server is a magpie backup cut short ({size}): a write to it was interrupted": "伺服器上的檔案是一個不完整的 magpie 備份({size}):上次寫入被中斷了",
"the server answered with a web page (“{title}”) instead of the file": "伺服器回傳的是一個網頁(「{title}」),而不是備份檔案",
"the server answered with a web page instead of the file": "伺服器回傳的是一個網頁,而不是備份檔案",
", after a redirect to {where}": ",重新導向到了 {where}",
": check the address is the server's WebDAV address, not its website": ":請檢查位址是伺服器的 WebDAV 位址,而不是它的網站",
"the server answered with an XML document instead of the file: check the address is the server's WebDAV address": "伺服器回傳的是一份 XML 文件,而不是備份檔案:請檢查位址是伺服器的 WebDAV 位址",
"the file on the server is another app's (“{format}”), not a magpie backup": "伺服器上的檔案是別的應用程式的(「{format}」),不是 magpie 備份",
"the file on the server is a JSON file that isn't a magpie backup ({size})": "伺服器上的檔案是一個 JSON 檔案,但不是 magpie 備份({size})",
"the file on the server isn't a magpie backup: it is {type} ({size})": "伺服器上的檔案不是 magpie 備份,而是 {type}({size})",
"Upload this computer's setup": "上傳本機的設定",
"It replaces the file on the server. That file is kept in the sync folder first.": "用本機設定取代伺服器上的檔案,取代前先把那個檔案儲存到 sync 資料夾。",
"Upload…": "上傳…",
"Replace the file on the server?": "取代伺服器上的檔案?",
"This computer's setup goes up in place of the file on the server, and your other computers sync with it from then on.": "本機的設定會上傳並取代伺服器上的檔案,之後其他電腦都和它同步。",
"What another computer wrote into that file and has nowhere else is lost. If one of them synced it last, Sync now there may rebuild it instead.": "另一台電腦寫進那個檔案、別處沒有的內容會遺失。如果是其他某台電腦最後同步的,在那台電腦上點「立即同步」可能可以直接修復。",
"The file on the server is kept in the sync folder first.": "伺服器上的檔案會先儲存到 sync 資料夾。",
"This computer's setup is already the server's: nothing to restore": "本機的設定已與伺服器一致,無需恢復",
"OK": "知道了",
"Export": "匯出",
@@ -9462,6 +9496,23 @@ const I18N = {
"This computer's {parts} become the ones on the server, as your other computers last synced them. Whatever was changed here since is replaced.": "このコンピューターの{parts}が、ほかのコンピューターが最後に同期したサーバーのものに置き換わります。ここでその後に変えたものは置き換えられます。",
"What is here now is sealed into the sync folder first, and Undo puts it back.": "今の設定は先に暗号化して同期フォルダーに保存され、「取り消す」で戻せます。",
"Restore setup": "設定を復元",
"the file on the server is empty (0 bytes)": "サーバー上のファイルが空です(0 バイト)",
"the file on the server is a magpie backup cut short ({size}): a write to it was interrupted": "サーバー上のファイルは途中で切れた magpie バックアップです({size}):書き込みが中断されました",
"the server answered with a web page (“{title}”) instead of the file": "サーバーがファイルではなく Web ページ(「{title}」)を返しました",
"the server answered with a web page instead of the file": "サーバーがファイルではなく Web ページを返しました",
", after a redirect to {where}": "({where} にリダイレクト)",
": check the address is the server's WebDAV address, not its website": ":アドレスが Web サイトではなく、サーバーの WebDAV アドレスか確認してください",
"the server answered with an XML document instead of the file: check the address is the server's WebDAV address": "サーバーがファイルではなく XML 文書を返しました:アドレスがサーバーの WebDAV アドレスか確認してください",
"the file on the server is another app's (“{format}”), not a magpie backup": "サーバー上のファイルは別のアプリのもの(「{format}」)で、magpie のバックアップではありません",
"the file on the server is a JSON file that isn't a magpie backup ({size})": "サーバー上のファイルは magpie のバックアップではない JSON ファイルです({size})",
"the file on the server isn't a magpie backup: it is {type} ({size})": "サーバー上のファイルは magpie のバックアップではありません:{type}({size})です",
"Upload this computer's setup": "このコンピューターの設定をアップロード",
"It replaces the file on the server. That file is kept in the sync folder first.": "サーバー上のファイルを置き換えます。そのファイルは先に同期フォルダーに保存されます。",
"Upload…": "アップロード…",
"Replace the file on the server?": "サーバー上のファイルを置き換えますか?",
"This computer's setup goes up in place of the file on the server, and your other computers sync with it from then on.": "このコンピューターの設定がサーバー上のファイルの代わりにアップロードされ、以後ほかのコンピューターはそれと同期します。",
"What another computer wrote into that file and has nowhere else is lost. If one of them synced it last, Sync now there may rebuild it instead.": "ほかのコンピューターがそのファイルに書き込み、ほかにはない内容は失われます。最後に同期したのがほかのコンピューターなら、そこで「今すぐ同期」すると修復できる場合があります。",
"The file on the server is kept in the sync folder first.": "サーバー上のファイルは先に同期フォルダーに保存されます。",
"This computer's setup is already the server's: nothing to restore": "このコンピューターの設定はすでにサーバーと同じです。復元するものはありません",
"OK": "OK",
"Export": "エクスポート",
@@ -13273,6 +13324,23 @@ const I18N = {
"This computer's {parts} become the ones on the server, as your other computers last synced them. Whatever was changed here since is replaced.": "{parts} dieses Computers werden zu denen auf dem Server, wie Ihre anderen Computer sie zuletzt synchronisiert haben. Was hier seitdem geändert wurde, wird ersetzt.",
"What is here now is sealed into the sync folder first, and Undo puts it back.": "Was jetzt hier ist, wird zuerst verschlüsselt im Sync-Ordner abgelegt, und „Rückgängig“ holt es zurück.",
"Restore setup": "Einrichtung wiederherstellen",
"the file on the server is empty (0 bytes)": "die Datei auf dem Server ist leer (0 Byte)",
"the file on the server is a magpie backup cut short ({size}): a write to it was interrupted": "die Datei auf dem Server ist ein abgeschnittenes magpie-Backup ({size}): ein Schreibvorgang wurde unterbrochen",
"the server answered with a web page (“{title}”) instead of the file": "der Server hat statt der Datei eine Webseite („{title}“) geliefert",
"the server answered with a web page instead of the file": "der Server hat statt der Datei eine Webseite geliefert",
", after a redirect to {where}": ", nach einer Weiterleitung zu {where}",
": check the address is the server's WebDAV address, not its website": ": Prüfe, ob die Adresse die WebDAV-Adresse des Servers ist und nicht seine Website",
"the server answered with an XML document instead of the file: check the address is the server's WebDAV address": "der Server hat statt der Datei ein XML-Dokument geliefert: Prüfe, ob die Adresse die WebDAV-Adresse des Servers ist",
"the file on the server is another app's (“{format}”), not a magpie backup": "die Datei auf dem Server gehört einer anderen App („{format}“) und ist kein magpie-Backup",
"the file on the server is a JSON file that isn't a magpie backup ({size})": "die Datei auf dem Server ist eine JSON-Datei, aber kein magpie-Backup ({size})",
"the file on the server isn't a magpie backup: it is {type} ({size})": "die Datei auf dem Server ist kein magpie-Backup, sondern {type} ({size})",
"Upload this computer's setup": "Einrichtung dieses Computers hochladen",
"It replaces the file on the server. That file is kept in the sync folder first.": "Ersetzt die Datei auf dem Server. Diese wird vorher im Sync-Ordner gesichert.",
"Upload…": "Hochladen…",
"Replace the file on the server?": "Datei auf dem Server ersetzen?",
"This computer's setup goes up in place of the file on the server, and your other computers sync with it from then on.": "Die Einrichtung dieses Computers ersetzt die Datei auf dem Server, und deine anderen Computer synchronisieren ab dann damit.",
"What another computer wrote into that file and has nowhere else is lost. If one of them synced it last, Sync now there may rebuild it instead.": "Was ein anderer Computer in diese Datei geschrieben hat und sonst nirgends liegt, geht verloren. Hat einer von ihnen zuletzt synchronisiert, kann „Jetzt synchronisieren“ dort sie stattdessen wiederherstellen.",
"The file on the server is kept in the sync folder first.": "Die Datei auf dem Server wird vorher im Sync-Ordner gesichert.",
"This computer's setup is already the server's: nothing to restore": "Die Einrichtung dieses Computers ist schon die des Servers: nichts wiederherzustellen",
"OK": "OK",
"Export": "Exportieren",
+2
View File
@@ -125,6 +125,8 @@ func backupRoutes(mux *http.ServeMux, w Windows) {
brought, err = davsync.Restore(ctx)
case "undo":
_, err = davsync.Undo()
case "upload": // this computer's setup over a server file that isn't a backup, kept first
err = davsync.Upload(ctx)
case "reveal":
err = w.OpenFolder(filepath.Join(settings.Dir(), "sync"))
default:
+120
View File
@@ -0,0 +1,120 @@
// Run with Node's test runner and Playwright on the module path; see README.md.
// JasonLeeForOnly on Discord: WebDAV sync said "not a magpie backup" for
// versions, and nothing on the page got past it. When the server holds an
// empty, cut-short or other app's file where the backup should be, the
// status says so in the reader's language and an Upload row offers this
// computer's setup in its place, asked first; a web page answered instead
// of the file is said with its title and redirect, and offers no upload.
// In English, Chinese, Traditional Chinese, Japanese and German, Chromium
// and WebKit, at 900 and 440px. The API is faked; nothing is synced.
const assert = require("node:assert/strict");
const fs = require("node:fs/promises");
const path = require("node:path");
const { test } = require("node:test");
const { chromium, webkit } = require("playwright");
const assets = path.resolve(__dirname, "../assets");
const EN = "the file magpie/magpie.magpie-backup on the WebDAV server is empty (0 bytes): a write to it didn't finish.";
function serve(lang, state) {
const settings = {
theme: "light", lang, tray: "panel", version: "test", dir: "/tmp/magpie",
gateway: "http://127.0.0.1:3425", visionModels: [], imageGenModels: [],
fx: { rate: 7.2, stale: false },
};
return async (route) => {
const request = route.request();
const pathname = new URL(request.url()).pathname;
const json = (data) => route.fulfill({ json: data });
if (pathname === "/boot.js") return route.fulfill({ contentType: "text/javascript", body: `window.bootPrefs = {lang:"${lang}",theme:"light",web:false};` });
if (pathname === "/wails/runtime.js") return route.fulfill({ contentType: "text/javascript", body: "export const Window = {};" });
if (pathname === "/api/state") return json({ agents: [], profiles: [], settings });
if (pathname === "/api/settings") return json(settings);
if (pathname === "/api/usage/quotas") return json([]);
if (pathname === "/api/groups") return json({ groups: [], models: [] });
if (pathname === "/api/plugins") return json({ plugins: [] });
if (pathname === "/api/davsync") return json(state.sync);
if (pathname === "/api/davsync/upload") {
state.uploads++;
state.sync = { ...base, last: new Date().toISOString() };
return json(state.sync);
}
if (pathname.startsWith("/api/")) return json({});
const file = path.join(assets, pathname === "/" ? "index.html" : pathname);
const contentType = { ".html": "text/html", ".js": "text/javascript", ".css": "text/css", ".svg": "image/svg+xml", ".png": "image/png" }[path.extname(file)];
return route.fulfill({ body: await fs.readFile(file), contentType });
};
}
const words = {
en: { empty: "the file on the server is empty (0 bytes)", cut: "cut short (5.4 KB)", upload: "Upload…", go: "Upload this computer's setup", page: "web page (“Nextcloud”)", synced: "Synced" },
zh: { empty: "服务器上的文件是空的(0 字节)", cut: "不完整的 magpie 备份(5.4 KB)", upload: "上传…", go: "上传本机的配置", page: "网页(“Nextcloud”)", synced: "已同步" },
"zh-TW": { empty: "伺服器上的檔案是空的(0 位元組)", cut: "不完整的 magpie 備份(5.4 KB)", upload: "上傳…", go: "上傳本機的設定", page: "網頁(「Nextcloud」)", synced: null },
ja: { empty: "サーバー上のファイルが空です(0 バイト)", cut: "途中で切れた magpie バックアップです(5.4 KB)", upload: "アップロード…", go: "このコンピューターの設定をアップロード", page: "Web ページ(「Nextcloud」)", synced: null },
de: { empty: "die Datei auf dem Server ist leer (0 Byte)", cut: "abgeschnittenes magpie-Backup (5.4 KB)", upload: "Hochladen…", go: "Einrichtung dieses Computers hochladen", page: "Webseite („Nextcloud“)", synced: null },
};
const base = { on: true, kind: "webdav", url: "https://dav.jianguoyun.com/dav/", user: "me", passwordSet: true, passphraseSet: true, keys: true, agents: true, library: true, auto: 3 };
for (const engine of (process.env.BROWSER ? [process.env.BROWSER] : ["chromium", "webkit"])) {
for (const lang of Object.keys(words)) {
for (const width of [900, 440]) {
const w = words[lang];
test(`${engine} ${lang} ${width}px: a server file that isn't a backup is said, and can be replaced`, async (t) => {
const browser = await (engine === "webkit" ? webkit.launch() : chromium.launch({ channel: "chromium" }));
t.after(() => browser.close());
for (const c of [
{ name: "empty", sync: { ...base, error: EN, serverFile: { what: "empty", size: 0, replace: true } }, said: [w.empty], upload: true },
{ name: "cut", sync: { ...base, error: EN, serverFile: { what: "cut", size: 5432, replace: true } }, said: [w.cut], upload: true },
{ name: "page", sync: { ...base, error: EN, serverFile: { what: "page", size: 900, title: "Nextcloud", moved: "cloud.example.com/login" } }, said: [w.page, "cloud.example.com/login"], upload: false },
{ name: "plain error", sync: { ...base, error: "the WebDAV server refused the user name or password (HTTP 401)" }, said: ["HTTP 401"], upload: false },
]) {
const state = { sync: c.sync, uploads: 0 };
const page = await (await browser.newContext({ viewport: { width, height: 800 }, reducedMotion: "reduce" })).newPage();
const errors = [];
page.on("pageerror", (e) => errors.push(e.message));
await page.route("**/*", serve(lang, state));
page.setDefaultTimeout(5000);
await page.goto("http://magpie.test/");
await page.locator("#prefs").click();
await page.locator("#setTab-sync").click();
const first = page.locator("#syncList .row.pref").first();
await first.waitFor({ state: "visible" });
const said = await first.locator(".who > .sub").first().textContent();
for (const s of c.said) assert.ok(said.includes(s), `${c.name}: ${JSON.stringify(s)} in ${said}`);
const whole = await first.locator(".who > .sub").first().evaluate((e) => ({ sw: e.scrollWidth, cw: e.clientWidth, right: e.getBoundingClientRect().right }));
assert.ok(whole.sw <= whole.cw + 1 && whole.right <= width, `${c.name}: the reason is shown whole: ${JSON.stringify(whole)}`);
if (c.sync.serverFile && lang !== "en") assert.ok(!said.includes("magpie.magpie-backup on the WebDAV"), `${c.name}: said in ${lang}, not the English error: ${said}`);
const up = page.locator("#syncList button", { hasText: w.upload });
assert.equal(await up.count(), c.upload ? 1 : 0, `${c.name}: upload offered`);
if (c.upload) {
const top = () => page.evaluate(() => document.querySelector("#view-settings").scrollTop);
const before = await top();
await up.click();
const ask = page.locator(".upload-ask");
await ask.waitFor({ state: "visible" });
assert.equal(state.uploads, 0, "nothing is uploaded before it is confirmed");
assert.equal(await ask.locator(".lib-confirm").count(), c.name === "cut" ? 3 : 2, `${c.name}: what a cut file loses is said only for it`);
const fit = await ask.evaluate((e) => e.getBoundingClientRect().right);
assert.ok(fit <= width, `the ask is inside the window: ${fit}`);
if (process.env.ARTIFACT_DIR) {
await fs.mkdir(process.env.ARTIFACT_DIR, { recursive: true });
await page.screenshot({ path: path.join(process.env.ARTIFACT_DIR, `${engine}-${lang}-${width}-${c.name}-upload-ask.png`) });
}
await ask.locator("button", { hasText: w.go }).click();
await ask.waitFor({ state: "detached" });
assert.equal(state.uploads, 1, "the upload was asked for");
const after = await first.locator(".who > .sub").first().textContent();
if (w.synced) assert.ok(after.includes(w.synced), `synced after: ${after}`);
assert.equal(await up.count(), 0, "no upload row once synced");
assert.equal(await top(), before, "nothing scrolled");
}
const over = await page.evaluate(() => document.documentElement.scrollWidth - document.documentElement.clientWidth);
assert.ok(over <= 0, `no sideways scroll: ${over}`);
assert.deepEqual(errors, []);
await page.close();
}
});
}
}
}
+13
View File
@@ -32,6 +32,8 @@ const webdavUsage = `usage:
magpie webdav auto off|3|15|30|60 how often the gateway syncs by itself, in minutes; off: only when asked
magpie webdav restore make this computer's setup the server's; what is here is kept first
magpie webdav undo put back what the last restore replaced
magpie webdav upload put this computer's setup over a server file that isn't a backup
(empty, cut short, another app's); the server's is kept first
magpie webdav dismiss clear what the last sync said it replaced
magpie webdav off turn it off; the file on the server stays
@@ -72,6 +74,8 @@ const s3Usage = `usage:
magpie s3 auto off|3|15|30|60 how often the gateway syncs by itself, in minutes; off: only when asked
magpie s3 restore make this computer's setup the server's; what is here is kept first
magpie s3 undo put back what the last restore replaced
magpie s3 upload put this computer's setup over a bucket file that isn't a backup
(empty, cut short, another app's); the bucket's is kept first
magpie s3 dismiss clear what the last sync said it replaced
magpie s3 off turn it off; the file in the bucket stays
@@ -168,6 +172,15 @@ func syncCmd(k syncKind, args []string) error {
fmt.Printf("put back: %s\n", strings.Join(parts, ", "))
}
return nil
case "upload":
if _, ok := davsync.Load(); !ok {
return fmt.Errorf("%s sync is off: %s", k.name, k.turnOn())
}
if err := davsync.Upload(context.Background()); err != nil {
return err
}
fmt.Println("this computer's setup is on the server now; what was there is kept in the sync folder")
return nil
case "dismiss":
if err := davsync.Dismiss(); err != nil {
return err