yetone
652e5e1ec5
The Race step took 690s on ubuntu and 908s on macOS, almost all of it internal/gateway: its 1046 tests can't call t.Parallel (t.Setenv rules it out), so the package ran on one core under the race detector. build/race-shards.sh compiles nothing itself: it takes the package's race test binary (go test -race -c), splits its tests round-robin by name over N processes run at once, prints each process's log in a folded group and fails when any does. The gateway's TestMain already isolates every process in its own temporary home, keychain stand-ins and PATH, so the processes share nothing. The step runs it with one process per core (getconf _NPROCESSORS_ONLN: 4 on ubuntu, 3 on macOS), the same -race binary and the same 20-minute timeout; redact keeps its own go test -race. Every test still runs under the race detector, once. Locally (Apple silicon, 4 processes) the gateway's race suite passed in 4:39 against 567s of test time run one after another; one test, TestLargeRequestEndpoints (seven 64 MB bodies, 198s), now bounds it.
32 lines
1.3 KiB
Bash
Executable File
32 lines
1.3 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# race-shards.sh DIR BIN N [FLAGS]: runs the tests of BIN, a package's test
|
|
# binary (go test -c), from the package's DIR in N processes at once, the
|
|
# tests split round-robin by name, and fails if any process does. FLAGS go
|
|
# to every process. A package's tests run one at a time unless they call
|
|
# t.Parallel, which t.Setenv rules out, so the gateway's suite under the
|
|
# race detector keeps one core busy; N processes keep N busy. Each process
|
|
# isolates its own home (the package's TestMain).
|
|
set -euo pipefail
|
|
pkg=$1 bin=$2 n=$3
|
|
shift 3
|
|
dir=$(mktemp -d)
|
|
trap 'rm -rf "$dir"' EXIT
|
|
(cd "$pkg" && "$bin" -test.list '.*') | grep -E '^(Test|Example|Fuzz)' | sort >"$dir/all"
|
|
echo "$(wc -l <"$dir/all" | tr -d ' ') tests in $n processes"
|
|
pids=()
|
|
for i in $(seq 0 $((n - 1))); do
|
|
awk -v n="$n" -v i="$i" 'NR % n == i' "$dir/all" >"$dir/s$i"
|
|
re="^($(paste -sd'|' "$dir/s$i"))\$"
|
|
(cd "$pkg" && "$bin" -test.run "$re" "$@" >"$dir/log$i" 2>&1) &
|
|
pids+=($!)
|
|
done
|
|
fail=0
|
|
for i in "${!pids[@]}"; do
|
|
if wait "${pids[$i]}"; then st=ok; else st=FAIL fail=1; fi
|
|
echo "::group::shard $i: $st ($(wc -l <"$dir/s$i" | tr -d ' ') tests)"
|
|
cat "$dir/log$i"
|
|
echo "::endgroup::"
|
|
if [ $st = FAIL ]; then grep -E '^(--- FAIL|FAIL|panic|WARNING: DATA RACE)' "$dir/log$i" || true; fi
|
|
done
|
|
exit $fail
|