A workaround for handling big response body?

[osxtest]

Hello,

While reviewing PR #1414, I found that large response bodies can be streamed and partially read (based on how you use bodyStream) by setting both StreamResponseBody and MaxResponseBodySize:

fasthttp/http_test.go

Lines 3017 to 3037 in d2dc36f

	t.Run("limit response body size", func(t *testing.T) {
	t.Parallel()
	client := Client{StreamResponseBody: true, MaxResponseBodySize: 20}
	resp := AcquireResponse()
	request := AcquireRequest()
	request.SetRequestURI(server.URL)
	if err := client.Do(request, resp); err != nil {
	t.Fatal(err)
	}
	stream := resp.BodyStream()
	defer func() {
	if err := resp.CloseBodyStream(); err != nil {
	t.Fatalf("close stream err: %v", err)
	}
	}()
	content, _ := io.ReadAll(stream)
	if string(content) != "hello world" {
	t.Fatalf("unexpected body content, got: %#v, want: %#v", string(content), "hello world")
	}
	})

When these two fields are set, the response body exceeding maxBodySize is returned as a stream:

fasthttp/http.go

Lines 1477 to 1481 in d2dc36f

	bodyBuf.B, err = readBody(r, contentLength, maxBodySize, bodyBuf.B)
	if err == ErrBodyTooLarge && resp.StreamBody {
	resp.bodyStream = acquireRequestStream(bodyBuf, r, &resp.Header)
	err = nil
	}

If this is a feasible workaround, should we update the doc or add examples for this specific use case?

Related doc and issues:

• fasthttp/README.md

  Lines 591 to 592 in d2dc36f

  	* net/http can stream both request and response bodies
  	* net/http can handle bigger bodies as it doesn't read the whole body into memory

• Partial read #1100
• A panic may occur when a large file download #1419
• Question: Add client.MaxResponseBody*Read*Size? #1724

[erikdubbelboer]

Pull requests with extra documentation and examples is always welcome!

[slicingmelon]

Hi @osxtest,

Please check this issue that I've opened: #1994