Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: feed wrapping #4677

Merged
merged 44 commits into from
Sep 19, 2024
Merged
Changes from 1 commit
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
20c8878
chore: add todo for bzz
nugaon May 15, 2024
6332838
feat: getWrappedChunk with legacy payload handling
nugaon May 15, 2024
8c14c90
feat: add content length header for feeds path
nugaon May 15, 2024
8515af0
feat: remove payload structure for feeds
nugaon May 15, 2024
33f7b8b
chore: indicating bootstrap fixme
nugaon May 15, 2024
7edec9e
chore: update call arguments
nugaon May 15, 2024
d93cece
test: adjust feed testing
nugaon May 15, 2024
e0923f2
feat: return chunk payload on api
nugaon May 15, 2024
b6d382e
test: feed api
nugaon May 15, 2024
df01783
test: correct expected hashes
nugaon May 22, 2024
abe6896
feat: joiner and loadsaver with rootch init
nugaon May 22, 2024
ef05e2e
test: chunk wrapping and wrong legacy payload resolution
nugaon May 22, 2024
eccb5e4
test: changed feed response
nugaon May 22, 2024
a7a6e1e
chore: remove invalidFeedUpdate error
nugaon May 22, 2024
c768d88
test: make and copy wrappedRef
nugaon May 23, 2024
62acd85
fix: remove payload time related logic
nugaon May 23, 2024
c7ddac9
feat: give back whole data
nugaon May 24, 2024
dae4346
chore: resolve linter issues
nugaon May 25, 2024
27372fb
test: deactivate epoch based finder test
nugaon May 27, 2024
652a89e
feat: generate mock soc with span
nugaon May 27, 2024
7989e4d
test: resolve content that takes more than one chunk
nugaon May 27, 2024
d114125
chore: remove generated file
nugaon May 28, 2024
b8d6ee2
fix: epoch finder
nugaon May 28, 2024
1a11337
feat: only wrapped chunk
nugaon May 29, 2024
381f7bc
test: only wrapped chunk
nugaon May 29, 2024
c06cc24
refactor: only wrapped chunk to only root chunk
nugaon May 29, 2024
5215af9
docs: only root chunk
nugaon May 29, 2024
e8c2ff6
docs: add remaining redundancy related header options
nugaon May 29, 2024
0126076
fix: epoch ts def
nugaon May 29, 2024
4e1aea7
test: parallel
nugaon May 29, 2024
061bbca
fix: typeo on additional header set
nugaon Jun 6, 2024
ea0dfb3
refactor: remove joiner wrapper
nugaon Jun 6, 2024
9d99af1
feat: soc get api
nugaon Jun 6, 2024
df16c0c
fix: return on error and chunk get
nugaon Jun 7, 2024
7b15219
feat: swarm soc signature header in feed endpoint
nugaon Jun 27, 2024
f2cc8d8
refactor: swarm signature header
nugaon Jun 27, 2024
f4c117f
docs: update openapi
nugaon Jul 1, 2024
3d7de7c
fix: whitespace check
nugaon Jul 1, 2024
a6611df
docs: fix method namings
nugaon Sep 5, 2024
85a924f
Merge branch 'master' into feat/feed-wrapping
nugaon Sep 16, 2024
caed975
fix: params after merge
nugaon Sep 16, 2024
c347904
docs: fix duplicate keys
nugaon Sep 16, 2024
994bc2c
feat: add new headers to allowedHeaders
nugaon Sep 19, 2024
ba593c0
docs: typeo
nugaon Sep 19, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
feat: only wrapped chunk
nugaon committed May 29, 2024
commit 1a113373aa1b9762c4d00f991068ffd2f9013675
1 change: 1 addition & 0 deletions pkg/api/api.go
Original file line number Diff line number Diff line change
@@ -77,6 +77,7 @@ const (
SwarmErrorDocumentHeader = "Swarm-Error-Document"
SwarmFeedIndexHeader = "Swarm-Feed-Index"
SwarmFeedIndexNextHeader = "Swarm-Feed-Index-Next"
SwarmOnlyWrappedChunk = "Swarm-Only-Wrapped-Chunk"
SwarmCollectionHeader = "Swarm-Collection"
SwarmPostageBatchIdHeader = "Swarm-Postage-Batch-Id"
SwarmDeferredUploadHeader = "Swarm-Deferred-Upload"
22 changes: 22 additions & 0 deletions pkg/api/feed.go
Original file line number Diff line number Diff line change
@@ -5,9 +5,13 @@
package api

import (
"bytes"
"encoding/hex"
"errors"
"io"
"net/http"
"strconv"
"strings"
"time"

"github.com/ethereum/go-ethereum/common"
@@ -58,6 +62,14 @@ func (s *Service) feedGetHandler(w http.ResponseWriter, r *http.Request) {
queries.At = time.Now().Unix()
}

headers := struct {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we keep the queries above with At and After ? Also bellow there is a lookup using this queries.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it may be needed for epoch based feeds.
I created an issue about that #4830

OnlyWrappedChunk bool `map:"Swarm-Only-Wrapped-Chunk"`
}{}
if response := s.mapStructure(r.Header, &headers); response != nil {
response("invalid header params", logger, w)
return
}

f := feeds.New(paths.Topic, paths.Owner)
lookup, err := s.feedFactory.NewLookup(feeds.Sequence, f)
if err != nil {
@@ -118,6 +130,16 @@ func (s *Service) feedGetHandler(w http.ResponseWriter, r *http.Request) {
"Access-Control-Expose-Headers": {SwarmFeedIndexHeader, SwarmFeedIndexNextHeader},
}

if headers.OnlyWrappedChunk {
w.Header().Set(ContentLengthHeader, strconv.Itoa(len(wc.Data())))
// include additional headers
for name, values := range additionalHeaders {
w.Header().Set(name, strings.Join(values, "; "))
}
_, _ = io.Copy(w, bytes.NewReader(wc.Data()))
return
}

s.downloadHandler(logger, w, r, wc.Address(), additionalHeaders, true, false, wc)
}