Exception not raised for chunked responses from Docker socket

[aidansteele]

This code will only raise an exception (via _result and _raise_for_status) for a non-chunked response:

docker-py/docker/api/client.py

Lines 339 to 358 in aaf68b7

	def _stream_helper(self, response, decode=False):
	"""Generator for data coming from a chunked-encoded HTTP response."""
	if response.raw._fp.chunked:
	if decode:
	yield from json_stream(self._stream_helper(response, False))
	else:
	reader = response.raw
	while not reader.closed:
	# this read call will block until we get a chunk
	data = reader.read(1)
	if not data:
	break
	if reader._fp.chunk_left:
	data += reader.read(reader._fp.chunk_left)
	yield data
	else:
	# Response isn't chunked, meaning we probably
	# encountered an error immediately
	yield self._result(response, json=decode)

This is problematic, because Docker for Mac appears to return errors with a chunked encoding:

POST /v1.35/build?t=repro%3Alatest&q=False&nocache=False&rm=True&forcerm=False&pull=False&dockerfile=Dockerfile&platform=linux%2Famd64 HTTP/1.1
Host: localhost
User-Agent: docker-sdk-python/4.2.2
Accept-Encoding: gzip, deflate
Accept: */*
Connection: keep-alive
Content-Type: application/tar
X-Registry-Config: eyIwN<trimmed>
Content-Length: 10240

<trimmed by me>

HTTP/1.1 500 Internal Server Error
Api-Version: 1.41
Content-Type: application/json
Date: Mon, 17 Apr 2023 04:46:09 GMT
Docker-Experimental: false
Ostype: linux
Server: Docker/20.10.23 (linux)
Transfer-Encoding: chunked


3d
{"message":"Cannot locate specified Dockerfile: Dockerfile"}

0

This means that exceptions are correctly raised on Linux (which doesn't return a chunked response), but not on Mac. Maybe self._raise_for_status(response) can be called unconditionally in _stream_helper, somewhere around line 341?

Docker version info:

Client:
 Cloud integration: v1.0.31
 Version:           20.10.23
 API version:       1.41
 Go version:        go1.18.10
 Git commit:        7155243
 Built:             Thu Jan 19 17:35:19 2023
 OS/Arch:           darwin/arm64
 Context:           default
 Experimental:      true

Server: Docker Desktop 4.17.0 (99724)
 Engine:
  Version:          20.10.23
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.18.10
  Git commit:       6051f14
  Built:            Thu Jan 19 17:31:28 2023
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.6.18
  GitCommit:        2456e983eb9e37e47538f59ea18f2043c9a73640
 runc:
  Version:          1.1.4
  GitCommit:        v1.1.4-0-g5fd4c4d
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

[sidhujus]

Hi, are there any plans to work on a fix for this? We changed to using the higher level images.build call because of this, but because that call is blocking we are unable to stream the build logs

[fade2black]

This is such a pain in the ass. When are you planning to fix it? It makes working with sam build nightmare. When an error occurs nothing is seen what and why!!!!

[tamnguyenvt]

it's a true nightmare