Skip to content

Commit 298d6fb

Browse files
lovelydinosaurlovelydinosaur
authored
Small, sharp, auditable HTTP parser. (#122)
* Add HTTPParser * Docs * Integrating HTTPParser into client side * Tidy up * Update docs * Update docs
1 parent f8a1497 commit 298d6fb

24 files changed

+1741
-169
lines changed

‎docs/connections.md‎

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -94,8 +94,6 @@ Individual HTTP connections can be managed directly with the `Connection` class.
9494
>>> r = await conn.request("GET", "/")
9595
```
9696

97-
Protocol handling is dealt with using [the `h11` package](https://h11.readthedocs.io/en/latest/), a rigorously designed HTTP/1.1 implementation which follows [the Sans-IO design pattern](https://sans-io.readthedocs.io/).
98-
9997
The `NetworkBackend` is responsible for managing the TCP stream, providing a raw byte-wise interface onto the underlying socket.
10098

10199
---
@@ -238,10 +236,10 @@ async with await ahttpx.open_connection("http://127.0.0.1:8080") as conn:
238236

239237
---
240238

241-
*Describe `.send()` rationale, and the `Transport` interface.*
239+
*Describe the `Transport` interface.*
242240

243241
---
244242

245243
<span class="link-prev">← [Streams](streams.md)</span>
246-
<span class="link-next">[Low Level Networking](networking.md) →</span>
244+
<span class="link-next">[Parsers](parsers.md) →</span>
247245
<span>&nbsp;</span>

‎docs/index.md‎

Lines changed: 6 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -61,26 +61,22 @@ $ pip install --pre ahttpx
6161
```{ .python .httpx }
6262
>>> import httpx
6363
64-
>>> def hello_world(request):
64+
>>> def app(request):
6565
... content = httpx.HTML('<html><body>hello, world.</body></html>')
6666
... return httpx.Response(200, content=content)
6767
68-
>>> with httpx.serve_http(hello_world) as server:
69-
... print(f"Serving on {server.url} (Press CTRL+C to quit)")
70-
... server.wait()
68+
>>> httpx.run(app)
7169
Serving on http://127.0.0.1:8080/ (Press CTRL+C to quit)
7270
```
7371

7472
```{ .python .ahttpx .hidden }
7573
>>> import ahttpx
7674
77-
>>> async def hello_world(request):
75+
>>> async def app(request):
7876
... content = httpx.HTML('<html><body>hello, world.</body></html>')
7977
... return httpx.Response(200, content=content)
8078
81-
>>> async with httpx.serve_http(hello_world) as server:
82-
... print(f"Serving on {server.url} (Press CTRL+C to quit)")
83-
... await server.wait()
79+
>>> await httpx.run(app)
8480
Serving on http://127.0.0.1:8080/ (Press CTRL+C to quit)
8581
```
8682

@@ -96,7 +92,9 @@ Serving on http://127.0.0.1:8080/ (Press CTRL+C to quit)
9692
* [URLs](urls.md)
9793
* [Headers](headers.md)
9894
* [Content Types](content-types.md)
95+
* [Streams](streams.md)
9996
* [Connections](connections.md)
97+
* [Parsers](parsers.md)
10098
* [Network Backends](networking.md)
10199

102100
---

‎docs/networking.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -377,5 +377,5 @@ Custom network backends can also be used to provide functionality such as handli
377377

378378
---
379379

380-
<span class="link-prev">← [Connections](connections.md)</span>
380+
<span class="link-prev">← [Parsers](parsers.md)</span>
381381
<span>&nbsp;</span>

‎docs/parsers.md‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
# Parsers
2+
3+
```python
4+
writer = io.BytesIO()
5+
reader = io.BytesIO(
6+
b"HTTP/1.1 200 OK\r\n"
7+
b"Content-Length: 23\r\n"
8+
b"Content-Type: application/json\r\n"
9+
b"\r\n"
10+
b'{"msg": "hello, world"}'
11+
)
12+
p = httpx.HTTPParser(writer, reader)
13+
14+
# Send the request...
15+
p.send_method_line(b"GET", b"/", b"HTTP/1.1")
16+
p.send_headers([(b"Host", b"example.com")])
17+
p.send_body(b'')
18+
19+
# Receive the response...
20+
protocol, code, reason_phase = p.recv_status_line()
21+
headers = p.recv_headers()
22+
body = b''
23+
while buffer := p.recv_body():
24+
body += buffer
25+
```
26+
27+
---
28+
29+
<span class="link-prev">← [Connections](connections.md)</span>
30+
<span class="link-next">[Low Level Networking](networking.md) →</span>

‎docs/streams.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ Additionally, the following properties are also defined...
1616
* `.size` - *(int or None)* Return an integer indicating the size of the stream, or `None` if the size is unknown. When working with HTTP this is used to either set a `Content-Length: <size>` header, or a `Content-Encoding: chunked` header.
1717
* `.content_type` - *(str or None)* Return a string indicating the content type of the data, or `None` if the content type is unknown. When working with HTTP this is used to optionally set a `Content-Type` header.
1818

19-
The `Stream` interface and `ContentType` interface are closely related, with streams being used as the abstraction for the bytewise representation, and content types being used to encapsulate the parsed data structure.
19+
The `Stream` interface and `ContentType` interface are related, with streams being used as the abstraction for the bytewise representation, and content types being used to encapsulate the parsed data structure.
2020

2121
For example, encoding some `JSON` data...
2222

‎scripts/docs‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,9 @@ pages = {
2222
'/urls': 'docs/urls.md',
2323
'/headers': 'docs/headers.md',
2424
'/content-types': 'docs/content-types.md',
25+
'/streams': 'docs/streams.md',
2526
'/connections': 'docs/connections.md',
27+
'/parsers': 'docs/parsers.md',
2628
'/networking': 'docs/networking.md',
2729
'/about': 'docs/about.md',
2830
}

‎scripts/unasync‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@ unasync.unasync_files(
88
"src/ahttpx/_client.py",
99
"src/ahttpx/_content.py",
1010
"src/ahttpx/_headers.py",
11+
"src/ahttpx/_parsers.py",
1112
"src/ahttpx/_pool.py",
1213
"src/ahttpx/_quickstart.py",
1314
"src/ahttpx/_response.py",

‎src/ahttpx/__init__.py‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@
33
from ._content import * # Content, File, Files, Form, HTML, JSON, MultiPart, Text
44
from ._headers import * # Headers
55
from ._network import * # NetworkBackend, NetworkStream, timeout
6+
from ._parsers import * # HTTPParser
67
from ._pool import * # Connection, ConnectionPool, Transport
78
from ._quickstart import * # get, post, put, patch, delete
89
from ._response import * # Response
@@ -29,6 +30,7 @@
2930
"get",
3031
"Headers",
3132
"HTML",
33+
"HTTPParser",
3234
"HTTPStream",
3335
"JSON",
3436
"MultiPart",

‎src/ahttpx/_network.py‎

Lines changed: 8 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -5,10 +5,13 @@
55

66
import certifi
77

8+
from ._streams import Stream
9+
10+
811
__all__ = ["NetworkBackend", "NetworkStream", "timeout"]
912

1013

11-
class NetworkStream:
14+
class NetworkStream(Stream):
1215
def __init__(
1316
self, reader: asyncio.StreamReader, writer: asyncio.StreamWriter, address: str = ''
1417
) -> None:
@@ -18,8 +21,10 @@ def __init__(
1821
self._tls = False
1922
self._closed = False
2023

21-
async def read(self, max_bytes: int = 64 * 1024) -> bytes:
22-
return await self._reader.read(max_bytes)
24+
async def read(self, size: int = -1) -> bytes:
25+
if size < 0:
26+
size = 64 * 1024
27+
return await self._reader.read(size)
2328

2429
async def write(self, buffer: bytes) -> None:
2530
self._writer.write(buffer)

0 commit comments

Comments
 (0)