Skip to content

feat: implement gossipsub partial messages extension #577

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 58 commits into
base: sigp-gossipsub
Choose a base branch
from
Open

feat: implement gossipsub partial messages extension #577

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
58 commits
Select commit Hold shift + click to select a range
6313068
feat(gossipsub): implement gossipsub 1.3
jxs Jul 4, 2025
53cc7e8
implement test extesion
jxs Aug 7, 2025
4249ccb
Add PartialMessage trait implementation
jxs Aug 7, 2025
9aa7ead
implement sending and receiving partial messages at te protocol level
jxs Sep 10, 2025
bc45d3e
add partial_publish method
jxs Sep 12, 2025
8b1c086
update trait extend method to receive slice instead
jxs Sep 12, 2025
b1b8dbe
implement Partial message handling and publishing at the behaviour level
jxs Sep 15, 2025
e7250f5
Merge branch 'sigp-gossipsub' of github.com:sigp/rust-libp2p into gos…
jxs Sep 16, 2025
b433556
deps: bump Swatinem/rust-cache from 2.8.0 to 2.8.1
dependabot[bot] Sep 18, 2025
6370514
address Marco's review
jxs Sep 20, 2025
909a68f
feature gate partial messages
jxs Sep 20, 2025
046ad47
remove PartialIDONWANT from proto to follow spec
jxs Sep 24, 2025
69c2d95
update trait return types and remove non used impl
jxs Sep 25, 2025
45c2bc1
fixup! feature gate partial messages
jxs Sep 26, 2025
85b92f3
update to spec changes
jxs Sep 29, 2025
b6478d5
update extension message priority to highest
jxs Sep 29, 2025
5e98f1a
add missing proto field SubOpts partial
jxs Sep 29, 2025
f77490c
implement subscription options partial
jxs Sep 30, 2025
1bfbc55
publish only full messages to partial false peers
jxs Sep 30, 2025
33783bd
revert to only using TopicHash,
jxs Oct 1, 2025
438a297
add partial option to subscribe and unsubscribe methods
jxs Oct 1, 2025
e1929e0
chore(ci): remove usage of self-hosted runners (#6170)
galargh Oct 1, 2025
9d2eb00
chore: Address latest clippy lints
jxs Oct 2, 2025
e7fea6c
chore(ci): address cargo deny issues
jxs Oct 2, 2025
6cb5244
polish PR
jxs Oct 3, 2025
bc3d7ba
push publish method above for smaller diff
jxs Oct 3, 2025
79f097e
fix(gossipsub): remove duplicate config builder method
skaunov Oct 3, 2025
2fb2486
feat(gossipsub): upgrade internal message queue,
jxs Oct 7, 2025
4321429
address Daniel's review
jxs Oct 10, 2025
c3aa3e4
enum return value for `partial_message_bytes_from_metadata`
dknopik Oct 10, 2025
88806ad
update Partial implementation
jxs Oct 13, 2025
4ea2e45
address Daniel's review
jxs Oct 14, 2025
ea73a71
address missing review comment
jxs Oct 14, 2025
0e52451
fix(libp2p): delegate metrics feature to gossipsub when enabled
jxs Oct 14, 2025
c415f37
impl Daniel's suggestions for the public API
jxs Oct 15, 2025
3048649
borrow metadata mutably for `metadata.update`
dknopik Oct 15, 2025
1685aae
impl penalization for failed partials
jxs Oct 15, 2025
69cde69
address Marco review
jxs Oct 15, 2025
19e359d
cargo fmt
jxs Oct 15, 2025
22a7b74
Merge branch 'master' of github.com:libp2p/rust-libp2p into gossipsub…
jxs Oct 16, 2025
a753ab0
fix(gossipsub): Fix underflow when shuffling peers after prunning.
AgeManning Oct 17, 2025
0a81a74
chore(dpes): bump `prometheus-client`
hanabi1224 Oct 17, 2025
6b3c57a
chore(deps): bump quic-go/quic-go from 0.50.1 to 0.54.1 in /wasm-tests
dependabot[bot] Oct 20, 2025
dd3fb5d
chore(README.md): Add Links to mentioned tech
hoijui Oct 20, 2025
54228bf
Require Metadata to be Send + Sync
dknopik Oct 21, 2025
e0e6037
fix(gossisub): prevent mesh exceeding mesh_n_high
jxs Oct 24, 2025
61d436a
chore: prepare libp2p-upnp v0.6.0 (#6188)
jxs Oct 27, 2025
2220ae4
merge master@origin
jxs Oct 28, 2025
9545140
fix sending partial messages to peers not supporting them
jxs Oct 28, 2025
dd21893
update to latest spec changes
jxs Oct 29, 2025
d97ae1f
emmit gosipsup event whenever partial data was updated
jxs Oct 31, 2025
246fe26
update behaviour to cache partials
jxs Nov 9, 2025
a3ba227
experimental change
dknopik Nov 10, 2025
aa94317
add send and sync
dknopik Nov 10, 2025
4e59700
automatically reply to partial messages
jxs Nov 11, 2025
0a58619
Revert "add send and sync"
jxs Nov 12, 2025
1ea4ebd
Revert "experimental change"
jxs Nov 12, 2025
8c342f1
merge Daniel's experiments into main branch
jxs Nov 12, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/cache-factory.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ jobs:

- uses: dtolnay/rust-toolchain@stable

- uses: Swatinem/rust-cache@98c8021b550208e191a6a3145459bfc9fb29c4c0 # v2.8.0
- uses: Swatinem/rust-cache@f13886b937689c021905a6b90929199931d60db1 # v2.8.1
with:
shared-key: stable-cache

Expand Down
27 changes: 11 additions & 16 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,12 +18,7 @@ env:
jobs:
test:
name: Test ${{ matrix.crate }}
runs-on: ${{ fromJSON(
github.repository == 'libp2p/rust-libp2p' && (
(contains(fromJSON('["libp2p-webrtc", "libp2p"]'), matrix.crate) && '["self-hosted", "linux", "x64", "2xlarge"]') ||
(contains(fromJSON('["libp2p-quic", "libp2p-perf"]'), matrix.crate) && '["self-hosted", "linux", "x64", "xlarge"]') ||
'["self-hosted", "linux", "x64", "large"]'
) || '"ubuntu-latest"') }}
runs-on: ubuntu-latest
timeout-minutes: 10
needs: gather_published_crates
strategy:
Expand All @@ -41,7 +36,7 @@ jobs:

- uses: dtolnay/rust-toolchain@stable

- uses: Swatinem/rust-cache@98c8021b550208e191a6a3145459bfc9fb29c4c0 # v2.8.0
- uses: Swatinem/rust-cache@f13886b937689c021905a6b90929199931d60db1 # v2.8.1
with:
shared-key: stable-cache
save-if: false
Expand Down Expand Up @@ -150,7 +145,7 @@ jobs:

- uses: r7kamura/rust-problem-matchers@9fe7ca9f6550e5d6358e179d451cc25ea6b54f98 #v1.5.0

- uses: Swatinem/rust-cache@98c8021b550208e191a6a3145459bfc9fb29c4c0 # v2.8.0
- uses: Swatinem/rust-cache@f13886b937689c021905a6b90929199931d60db1 # v2.8.1
with:
key: ${{ matrix.target }}
save-if: ${{ github.ref == 'refs/heads/master' }}
Expand All @@ -175,7 +170,7 @@ jobs:

- uses: r7kamura/rust-problem-matchers@9fe7ca9f6550e5d6358e179d451cc25ea6b54f98 #v1.5.0

- uses: Swatinem/rust-cache@98c8021b550208e191a6a3145459bfc9fb29c4c0 # v2.8.0
- uses: Swatinem/rust-cache@f13886b937689c021905a6b90929199931d60db1 # v2.8.1
with:
save-if: ${{ github.ref == 'refs/heads/master' }}

Expand All @@ -195,7 +190,7 @@ jobs:

- uses: r7kamura/rust-problem-matchers@9fe7ca9f6550e5d6358e179d451cc25ea6b54f98 #v1.5.0

- uses: Swatinem/rust-cache@98c8021b550208e191a6a3145459bfc9fb29c4c0 # v2.8.0
- uses: Swatinem/rust-cache@f13886b937689c021905a6b90929199931d60db1 # v2.8.1
with:
key: ${{ matrix.features }}
save-if: ${{ github.ref == 'refs/heads/master' }}
Expand All @@ -212,7 +207,7 @@ jobs:

- uses: r7kamura/rust-problem-matchers@9fe7ca9f6550e5d6358e179d451cc25ea6b54f98 #v1.5.0

- uses: Swatinem/rust-cache@98c8021b550208e191a6a3145459bfc9fb29c4c0 # v2.8.0
- uses: Swatinem/rust-cache@f13886b937689c021905a6b90929199931d60db1 # v2.8.1
with:
save-if: ${{ github.ref == 'refs/heads/master' }}

Expand All @@ -238,7 +233,7 @@ jobs:

- uses: r7kamura/rust-problem-matchers@9fe7ca9f6550e5d6358e179d451cc25ea6b54f98 #v1.5.0

- uses: Swatinem/rust-cache@98c8021b550208e191a6a3145459bfc9fb29c4c0 # v2.8.0
- uses: Swatinem/rust-cache@f13886b937689c021905a6b90929199931d60db1 # v2.8.1
with:
save-if: ${{ github.ref == 'refs/heads/master' }}

Expand All @@ -254,7 +249,7 @@ jobs:

- uses: r7kamura/rust-problem-matchers@9fe7ca9f6550e5d6358e179d451cc25ea6b54f98 #v1.5.0

- uses: Swatinem/rust-cache@98c8021b550208e191a6a3145459bfc9fb29c4c0 # v2.8.0
- uses: Swatinem/rust-cache@f13886b937689c021905a6b90929199931d60db1 # v2.8.1
with:
save-if: ${{ github.ref == 'refs/heads/master' }}

Expand All @@ -273,7 +268,7 @@ jobs:

- uses: r7kamura/rust-problem-matchers@9fe7ca9f6550e5d6358e179d451cc25ea6b54f98 #v1.5.0

- uses: Swatinem/rust-cache@98c8021b550208e191a6a3145459bfc9fb29c4c0 # v2.8.0
- uses: Swatinem/rust-cache@f13886b937689c021905a6b90929199931d60db1 # v2.8.1
with:
shared-key: stable-cache
save-if: false
Expand Down Expand Up @@ -364,7 +359,7 @@ jobs:
steps:
- uses: actions/checkout@v5

- uses: Swatinem/rust-cache@98c8021b550208e191a6a3145459bfc9fb29c4c0 # v2.8.0
- uses: Swatinem/rust-cache@f13886b937689c021905a6b90929199931d60db1 # v2.8.1

- run: cargo install --version 0.10.0 pb-rs --locked

Expand All @@ -390,7 +385,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: Swatinem/rust-cache@98c8021b550208e191a6a3145459bfc9fb29c4c0 # v2.8.0
- uses: Swatinem/rust-cache@f13886b937689c021905a6b90929199931d60db1 # v2.8.1
- run: cargo metadata --locked --format-version=1 > /dev/null

cargo-deny:
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/interop-test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ jobs:
run-transport-interop:
name: Run transport interoperability tests
if: github.event_name == 'push' || github.event.pull_request.head.repo.full_name == github.repository
runs-on: ${{ fromJSON(github.repository == 'libp2p/rust-libp2p' && '["self-hosted", "linux", "x64", "4xlarge"]' || '"ubuntu-latest"') }}
runs-on: ubuntu-latest
strategy:
matrix:
flavour: [chromium, native]
Expand Down
Loading
Loading