Skip to content

Commit 94cf24e

Browse files
authored
Make the test fixture local and auditable (#241)
* Replace the network fixture clone with a local git bundle The test suite cloned https://github.com/gitonomy/foobar.git over the network on every run, and asserted on hardcoded commit SHAs from that external repository's history. This made the tests fail without network access, coupled them to a repository outside our control, and made it impossible for a contributor to add a new fixture scenario without push access to that separate repository. tests/fixtures/foobar.bundle is a git bundle of the same repository (all branches and tags, including the GPG-signed commit used by PushReferenceTest). Admin::cloneTo/cloneBranchTo/cloneRepository just shell out to `git clone <url> <path>`, so cloning from a local bundle file behaves identically to cloning the remote repository, but works offline and can't be affected by changes made upstream. * Guard the fixture bundle against unreviewable changes in CI Since tests/fixtures/foobar.bundle is a binary file, GitHub shows a PR touching it as an opaque "binary file not shown" diff: a contributor could regenerate it with extra refs or bloated content while keeping the commit SHAs the tests rely on intact, and a reviewer would have no way to see it from the diff alone. tests/fixtures/verify-bundle.sh runs `git bundle verify`, checks the ref list against an explicit allow-list of the 5 branches and 2 tags the fixture actually needs, and fails if the file grows past 200KB (current size: ~39KB). Any legitimate change to the fixture's shape or size requires editing this script in the same PR, making the change explicit and reviewable instead of silent. Wired into CI as a new verify-fixture job that runs on every push and pull request. * Document the test fixture bundle in CONTRIBUTING.md Explains what tests/fixtures/foobar.bundle is, how to regenerate it to add a new fixture scenario, and why tests/fixtures/verify-bundle.sh exists and needs updating alongside any intentional change to the bundle's refs or size. * Make the fixture regeneration recipe fully local The docs and comments still pointed at cloning gitonomy/foobar over the network to add a new fixture scenario, defeating the point of moving to a local bundle. The bundle already contains the full history (all branches and tags), so regenerating it only ever needs to clone tests/fixtures/foobar.bundle itself — verified this reproduces a byte-identical bundle with no network access.
1 parent b8ed9c0 commit 94cf24e

6 files changed

Lines changed: 98 additions & 1 deletion

File tree

.gitattributes

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1 +1,2 @@
11
* text=auto
2+
/tests/fixtures/*.bundle binary

.github/CONTRIBUTING.md

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,35 @@ $ vendor/bin/phpunit
3939

4040
* A script `test-git-versions.sh` is available in repository to test gitlib against many git versions.
4141
* The tests will be automatically run by [GitHub Actions](https://github.com/features/actions) against pull requests.
42+
* Tests run fully offline: no network access is required.
43+
44+
## Test fixtures
45+
46+
Most tests run against a fixture repository cloned from `tests/fixtures/foobar.bundle`,
47+
a local git bundle. Using a bundle instead of a network clone keeps the tests fast and
48+
fully offline.
49+
50+
If you need a new fixture scenario (a specific merge, encoding, or signed-commit shape,
51+
for example), regenerate the bundle locally, entirely from the one already in the repo:
52+
53+
```bash
54+
$ git clone tests/fixtures/foobar.bundle /tmp/foobar-fixture && cd /tmp/foobar-fixture
55+
$ for b in $(git branch -r | grep -v HEAD | sed 's#origin/##'); do
56+
$ git branch --track "$b" "origin/$b"
57+
$ done
58+
# ... add your commits, branches or tags ...
59+
$ git bundle create foobar.bundle \
60+
HEAD refs/heads/master refs/heads/new-feature refs/heads/diff-features \
61+
refs/heads/pagination refs/heads/path-resolving refs/tags/0.1 refs/tags/annotated
62+
$ cp foobar.bundle /path/to/gitlib/tests/fixtures/foobar.bundle
63+
```
64+
65+
Then update the commit SHA constants in `AbstractTestCase` to match, and run
66+
`tests/fixtures/verify-bundle.sh`. It checks the bundle's integrity, its ref list against
67+
an allow-list, and its size, since GitHub renders any change to this binary file as an
68+
opaque diff. If your change intentionally adds a ref or grows the file, update
69+
`ALLOWED_REFS` or `MAX_SIZE_KB` in that script as part of the same pull request, so the
70+
reason for the change is explicit and reviewable rather than a silent binary diff.
4271

4372
## Standard code
4473

.github/workflows/tests.yml

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,16 @@ on:
66
pull_request:
77

88
jobs:
9+
verify-fixture:
10+
name: Verify Test Fixture Bundle
11+
runs-on: ubuntu-24.04
12+
steps:
13+
- name: Checkout Code
14+
uses: actions/checkout@v7
15+
16+
- name: Verify fixture bundle integrity, refs and size
17+
run: tests/fixtures/verify-bundle.sh
18+
919
check-cs:
1020
name: Check Coding Standards
1121
runs-on: ubuntu-24.04

tests/Gitonomy/Git/Tests/AbstractTestCase.php

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,11 @@
1818

1919
abstract class AbstractTestCase extends TestCase
2020
{
21-
public const REPOSITORY_URL = 'https://github.com/gitonomy/foobar.git';
21+
/**
22+
* Local git bundle used as a fixture repository: cloning from it behaves like
23+
* cloning a remote repository, but requires no network access.
24+
*/
25+
public const REPOSITORY_URL = __DIR__.'/../../../fixtures/foobar.bundle';
2226

2327
public const NO_MESSAGE_COMMIT = '011cd0c1625190d2959ee9a8f9f822006d94b661';
2428
public const LONGFILE_COMMIT = '4f17752acc9b7c54ba679291bf24cb7d354f0f4f';

tests/fixtures/foobar.bundle

39.8 KB
Binary file not shown.

tests/fixtures/verify-bundle.sh

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
#!/bin/bash
2+
#
3+
# Guards tests/fixtures/foobar.bundle against unexpected changes: any pull
4+
# request touching it should be reviewed by hand (see CONTRIBUTING.md), but
5+
# this catches the obvious cases automatically:
6+
# - a corrupted or incomplete bundle
7+
# - refs that were not in the original fixture
8+
# - a file that has silently grown well past its expected size
9+
#
10+
# Run it locally with: tests/fixtures/verify-bundle.sh
11+
12+
set -euo pipefail
13+
14+
cd "$(dirname "${BASH_SOURCE[0]}")" > /dev/null
15+
16+
BUNDLE="foobar.bundle"
17+
MAX_SIZE_KB=200
18+
19+
ALLOWED_REFS="
20+
HEAD
21+
refs/heads/diff-features
22+
refs/heads/master
23+
refs/heads/new-feature
24+
refs/heads/pagination
25+
refs/heads/path-resolving
26+
refs/tags/0.1
27+
refs/tags/annotated
28+
"
29+
30+
echo "== Verifying $BUNDLE =="
31+
32+
VERIFY_OUTPUT="$(git bundle verify "$BUNDLE")"
33+
echo "$VERIFY_OUTPUT"
34+
35+
SIZE_KB=$(( $(stat -c%s "$BUNDLE") / 1024 ))
36+
echo "Size: ${SIZE_KB}KB (limit: ${MAX_SIZE_KB}KB)"
37+
if [ "$SIZE_KB" -gt "$MAX_SIZE_KB" ]; then
38+
echo "ERROR: $BUNDLE is ${SIZE_KB}KB, which exceeds the ${MAX_SIZE_KB}KB limit." >&2
39+
echo "If this growth is expected, bump MAX_SIZE_KB in $0 as part of the same PR." >&2
40+
exit 1
41+
fi
42+
43+
ACTUAL_REFS="$(grep -oE '(refs/[^ ]+|HEAD)$' <<< "$VERIFY_OUTPUT" | sort -u)"
44+
UNEXPECTED_REFS="$(comm -23 <(echo "$ACTUAL_REFS") <(sort -u <<< "$ALLOWED_REFS"))"
45+
46+
if [ -n "$UNEXPECTED_REFS" ]; then
47+
echo "ERROR: $BUNDLE contains refs that are not in the allow-list:" >&2
48+
echo "$UNEXPECTED_REFS" >&2
49+
echo "If this is expected, update ALLOWED_REFS in $0 as part of the same PR." >&2
50+
exit 1
51+
fi
52+
53+
echo "OK: bundle structure and size are within expected bounds."

0 commit comments

Comments
 (0)