fix/gateway: escape directory redirect url (#10649)

* fix/gateway: escape directory redirect URLs When a director gets redirected to a URL with a treailing slash, special chars in the directory name must be escaped in the redirect URL. This upgrades to a version of box that has that fix. Closes #10536 * Fix sharness test for new redirect URLs * Update to latest boxo * Use latest gateway-conformance
ipfs · Jan 4, 2025 · b7cc47a · b7cc47a
1 parent 016c2a7
commit b7cc47a
Show file tree

Hide file tree

Showing 9 changed files with 22 additions and 16 deletions.
diff --git a/.github/workflows/gateway-conformance.yml b/.github/workflows/gateway-conformance.yml
@@ -41,7 +41,7 @@ jobs:
     steps:
       # 1. Download the gateway-conformance fixtures
       - name: Download gateway-conformance fixtures
-        uses: ipfs/gateway-conformance/.github/actions/extract-fixtures@v0.6
+        uses: ipfs/gateway-conformance/.github/actions/extract-fixtures@v0.7
         with:
           output: fixtures
 
@@ -94,7 +94,7 @@ jobs:
 
       # 6. Run the gateway-conformance tests
       - name: Run gateway-conformance tests
-        uses: ipfs/gateway-conformance/.github/actions/test@v0.6
+        uses: ipfs/gateway-conformance/.github/actions/test@v0.7
         with:
           gateway-url: http://127.0.0.1:8080
           subdomain-url: http://localhost:8080
@@ -128,7 +128,7 @@ jobs:
     steps:
       # 1. Download the gateway-conformance fixtures
       - name: Download gateway-conformance fixtures
-        uses: ipfs/gateway-conformance/.github/actions/extract-fixtures@v0.6
+        uses: ipfs/gateway-conformance/.github/actions/extract-fixtures@v0.7
         with:
           output: fixtures
 
@@ -201,7 +201,7 @@ jobs:
 
       # 9. Run the gateway-conformance tests over libp2p
       - name: Run gateway-conformance tests over libp2p
-        uses: ipfs/gateway-conformance/.github/actions/test@v0.6
+        uses: ipfs/gateway-conformance/.github/actions/test@v0.7
         with:
           gateway-url: http://127.0.0.1:8092
           args: --specs "trustless-gateway,-trustless-ipns-gateway" -skip 'TestGatewayCar/GET_response_for_application/vnd.ipld.car/Header_Content-Length'

diff --git a/docs/changelogs/v0.33.md b/docs/changelogs/v0.33.md
@@ -15,6 +15,7 @@
   - [New options for faster writes: `WriteThrough`, `BlockKeyCacheSize`, `BatchMaxNodes`, `BatchMaxSize`](#new-options-for-faster-writes-writethrough-blockkeycachesize-batchmaxnodes-batchmaxsize)
   - [MFS stability with large number of writes](#mfs-stability-with-large-number-of-writes)
   - [📦️ Important dependency updates](#-important-dependency-updates)
+  - [Escape Redirect URL for Directory](#escape-redirect-url-for-directory)
 - [📝 Changelog](#-changelog)
 - [👨‍👩‍👧‍👦 Contributors](#-contributors)
 
@@ -89,6 +90,11 @@ We have fixed a number of issues that were triggered by writing or copying many
 - update `p2p-forge/client` to [v0.2.0](https://github.com/ipshipyard/p2p-forge/releases/tag/v0.2.0) (incl. [v0.1.0](https://github.com/ipshipyard/p2p-forge/releases/tag/v0.1.0))
 - update `ipfs-webui` to [v4.4.2](https://github.com/ipfs/ipfs-webui/releases/tag/v4.4.2) (incl. [v4.4.1](https://github.com/ipfs/ipfs-webui/releases/tag/v4.4.1))
 
+
+#### Escape Redirect URL for Directory
+
+When navigating to a subdirectory, served by the Kubo web server, a subdirectory without a trailing slash gets redirected to a URL with a trailing slash. If there are special characters such as "%" in the subdirectory name then these must be escaped in the redirect URL. Previously this was not being done and was preventing navigation to such subdirectories, requiring the user to manually add a trailing slash to the subdirectory URL. This is now fixed to handle the redirect to URLs with characters that must be escaped.
+
 ### 📝 Changelog
 
 ### 👨‍👩‍👧‍👦 Contributors
diff --git a/docs/examples/kubo-as-a-library/go.mod b/docs/examples/kubo-as-a-library/go.mod
@@ -7,7 +7,7 @@ go 1.23
 replace github.com/ipfs/kubo => ./../../..
 
 require (
-	github.com/ipfs/boxo v0.26.0
+	github.com/ipfs/boxo v0.26.1-0.20250103202834-5810fed5a3ac
 	github.com/ipfs/kubo v0.0.0-00010101000000-000000000000
 	github.com/libp2p/go-libp2p v0.38.1
 	github.com/multiformats/go-multiaddr v0.14.0

diff --git a/docs/examples/kubo-as-a-library/go.sum b/docs/examples/kubo-as-a-library/go.sum
@@ -304,8 +304,8 @@ github.com/ipfs-shipyard/nopfs/ipfs v0.25.0 h1:OqNqsGZPX8zh3eFMO8Lf8EHRRnSGBMqcd
 github.com/ipfs-shipyard/nopfs/ipfs v0.25.0/go.mod h1:BxhUdtBgOXg1B+gAPEplkg/GpyTZY+kCMSfsJvvydqU=
 github.com/ipfs/bbloom v0.0.4 h1:Gi+8EGJ2y5qiD5FbsbpX/TMNcJw8gSqr7eyjHa4Fhvs=
 github.com/ipfs/bbloom v0.0.4/go.mod h1:cS9YprKXpoZ9lT0n/Mw/a6/aFV6DTjTLYHeA+gyqMG0=
-github.com/ipfs/boxo v0.26.0 h1:RRxEon7rJMy8ScVaTLncSZ5/nA6majYhRSbzc80snO8=
-github.com/ipfs/boxo v0.26.0/go.mod h1:iHyc9cjoF7/zoiKVY65d2fBWRhoS2zx4cMk8hKgqrac=
+github.com/ipfs/boxo v0.26.1-0.20250103202834-5810fed5a3ac h1:RbM53YTWjs7nZAAiUV7GSIV7S8QOaq0YkGAuuNNGM7A=
+github.com/ipfs/boxo v0.26.1-0.20250103202834-5810fed5a3ac/go.mod h1:iHyc9cjoF7/zoiKVY65d2fBWRhoS2zx4cMk8hKgqrac=
 github.com/ipfs/go-bitfield v1.1.0 h1:fh7FIo8bSwaJEh6DdTWbCeZ1eqOaOkKFI74SCnsWbGA=
 github.com/ipfs/go-bitfield v1.1.0/go.mod h1:paqf1wjq/D2BBmzfTVFlJQ9IlFOZpg422HL0HqsGWHU=
 github.com/ipfs/go-bitswap v0.11.0 h1:j1WVvhDX1yhG32NTC9xfxnqycqYIlhzEzLXG/cU1HyQ=

diff --git a/go.mod b/go.mod
@@ -22,7 +22,7 @@ require (
 	github.com/hashicorp/go-version v1.7.0
 	github.com/ipfs-shipyard/nopfs v0.0.14
 	github.com/ipfs-shipyard/nopfs/ipfs v0.25.0
-	github.com/ipfs/boxo v0.26.0
+	github.com/ipfs/boxo v0.26.1-0.20250103202834-5810fed5a3ac
 	github.com/ipfs/go-block-format v0.2.0
 	github.com/ipfs/go-cid v0.4.1
 	github.com/ipfs/go-cidutil v0.1.0

diff --git a/go.sum b/go.sum
@@ -368,8 +368,8 @@ github.com/ipfs-shipyard/nopfs/ipfs v0.25.0 h1:OqNqsGZPX8zh3eFMO8Lf8EHRRnSGBMqcd
 github.com/ipfs-shipyard/nopfs/ipfs v0.25.0/go.mod h1:BxhUdtBgOXg1B+gAPEplkg/GpyTZY+kCMSfsJvvydqU=
 github.com/ipfs/bbloom v0.0.4 h1:Gi+8EGJ2y5qiD5FbsbpX/TMNcJw8gSqr7eyjHa4Fhvs=
 github.com/ipfs/bbloom v0.0.4/go.mod h1:cS9YprKXpoZ9lT0n/Mw/a6/aFV6DTjTLYHeA+gyqMG0=
-github.com/ipfs/boxo v0.26.0 h1:RRxEon7rJMy8ScVaTLncSZ5/nA6majYhRSbzc80snO8=
-github.com/ipfs/boxo v0.26.0/go.mod h1:iHyc9cjoF7/zoiKVY65d2fBWRhoS2zx4cMk8hKgqrac=
+github.com/ipfs/boxo v0.26.1-0.20250103202834-5810fed5a3ac h1:RbM53YTWjs7nZAAiUV7GSIV7S8QOaq0YkGAuuNNGM7A=
+github.com/ipfs/boxo v0.26.1-0.20250103202834-5810fed5a3ac/go.mod h1:iHyc9cjoF7/zoiKVY65d2fBWRhoS2zx4cMk8hKgqrac=
 github.com/ipfs/go-bitfield v1.1.0 h1:fh7FIo8bSwaJEh6DdTWbCeZ1eqOaOkKFI74SCnsWbGA=
 github.com/ipfs/go-bitfield v1.1.0/go.mod h1:paqf1wjq/D2BBmzfTVFlJQ9IlFOZpg422HL0HqsGWHU=
 github.com/ipfs/go-bitswap v0.11.0 h1:j1WVvhDX1yhG32NTC9xfxnqycqYIlhzEzLXG/cU1HyQ=

diff --git a/test/dependencies/go.mod b/test/dependencies/go.mod
@@ -120,7 +120,7 @@ require (
 	github.com/huin/goupnp v1.3.0 // indirect
 	github.com/inconshreveable/mousetrap v1.1.0 // indirect
 	github.com/ipfs/bbloom v0.0.4 // indirect
-	github.com/ipfs/boxo v0.26.0 // indirect
+	github.com/ipfs/boxo v0.26.1-0.20250103202834-5810fed5a3ac // indirect
 	github.com/ipfs/go-block-format v0.2.0 // indirect
 	github.com/ipfs/go-cid v0.4.1 // indirect
 	github.com/ipfs/go-datastore v0.6.0 // indirect

diff --git a/test/dependencies/go.sum b/test/dependencies/go.sum
@@ -324,8 +324,8 @@ github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2
 github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw=
 github.com/ipfs/bbloom v0.0.4 h1:Gi+8EGJ2y5qiD5FbsbpX/TMNcJw8gSqr7eyjHa4Fhvs=
 github.com/ipfs/bbloom v0.0.4/go.mod h1:cS9YprKXpoZ9lT0n/Mw/a6/aFV6DTjTLYHeA+gyqMG0=
-github.com/ipfs/boxo v0.26.0 h1:RRxEon7rJMy8ScVaTLncSZ5/nA6majYhRSbzc80snO8=
-github.com/ipfs/boxo v0.26.0/go.mod h1:iHyc9cjoF7/zoiKVY65d2fBWRhoS2zx4cMk8hKgqrac=
+github.com/ipfs/boxo v0.26.1-0.20250103202834-5810fed5a3ac h1:RbM53YTWjs7nZAAiUV7GSIV7S8QOaq0YkGAuuNNGM7A=
+github.com/ipfs/boxo v0.26.1-0.20250103202834-5810fed5a3ac/go.mod h1:iHyc9cjoF7/zoiKVY65d2fBWRhoS2zx4cMk8hKgqrac=
 github.com/ipfs/go-block-format v0.2.0 h1:ZqrkxBA2ICbDRbK8KJs/u0O3dlp6gmAuuXUJNiW1Ycs=
 github.com/ipfs/go-block-format v0.2.0/go.mod h1:+jpL11nFx5A/SPpsoBn6Bzkra/zaArfSmsknbPMYgzM=
 github.com/ipfs/go-cid v0.4.1 h1:A/T3qGvxi4kpKWWcPC/PgbvDA2bjVLO7n4UeVwnbs/s=

diff --git a/test/sharness/t0115-gateway-dir-listing.sh b/test/sharness/t0115-gateway-dir-listing.sh
@@ -40,7 +40,7 @@ test_expect_success "path gw: backlink on root CID should be hidden" '
 test_expect_success "path gw: redirect dir listing to URL with trailing slash" '
   curl -sD - http://127.0.0.1:$GWAY_PORT/ipfs/${DIR_CID}/ą/ę > list_response &&
   test_should_contain "HTTP/1.1 301 Moved Permanently" list_response &&
-  test_should_contain "Location: /ipfs/${DIR_CID}/%c4%85/%c4%99/" list_response
+  test_should_contain "Location: /ipfs/${DIR_CID}/%C4%85/%C4%99/" list_response
 '
 
 test_expect_success "path gw: Etag should be present" '
@@ -81,7 +81,7 @@ test_expect_success "subdomain gw: backlink on root CID should be hidden" '
 test_expect_success "subdomain gw: redirect dir listing to URL with trailing slash" '
   curl -sD - --resolve $DIR_HOSTNAME:$GWAY_PORT:127.0.0.1 http://$DIR_HOSTNAME:$GWAY_PORT/ą/ę > list_response &&
   test_should_contain "HTTP/1.1 301 Moved Permanently" list_response &&
-  test_should_contain "Location: /%c4%85/%c4%99/" list_response
+  test_should_contain "Location: /%C4%85/%C4%99/" list_response
 '
 
 test_expect_success "subdomain gw: Etag should be present" '
@@ -130,7 +130,7 @@ test_expect_success "dnslink gw: backlink on root CID should be hidden" '
 test_expect_success "dnslink gw: redirect dir listing to URL with trailing slash" '
   curl -sD - --resolve $DNSLINK_HOSTNAME:$GWAY_PORT:127.0.0.1 http://$DNSLINK_HOSTNAME:$GWAY_PORT/ą/ę > list_response &&
   test_should_contain "HTTP/1.1 301 Moved Permanently" list_response &&
-  test_should_contain "Location: /%c4%85/%c4%99/" list_response
+  test_should_contain "Location: /%C4%85/%C4%99/" list_response
 '
 
 test_expect_success "dnslink gw: Etag should be present" '