Skip to content

HTTP compression support #30

@lovelydinosaur

Description

@lovelydinosaur

HTTP/1.1 compression support should be added at the Stream layer.

  • Include an appropriate Accept-Encoding header using the client defaults.
  • Inspect and automatically apply the Content-Encoding header on responses.
  • Ensure maximum decompression size is not open ended.

I'd suggest starting with just gzip, tho deflate, brotli, and zlib are also options here.

Here's how the .stream accessor might look...

>>> r = cli.get("https://www.example.com")
>>> len(r.body)
128823
>>> r.stream
<GZipStream [128KB unzipped from 100% of 64KB]>
>>> r.stream.raw
<HTTPByteStream [100% of 64KB]>

Related... currently the Connection class returns an AsyncIterByteStream here, tho a custom HTTPByteStream would be neater, even if that's just for the class name / making the providence clear.

We probably initially want just a GZipDecoder implementation, tho we might also choose to implement a GZipEncoder stream implementation for returning gzip responses from the server/application layer...

def compress_response(response):
    if response.headers.get('Content-Encoding') is None:
        status_code = response.status_code
        headers = response.headers.copy_set('Content-Encoding', 'gzip')
        content = GZipEncoder(response.stream)
        return Response(status_code, headers, content=content)
    return response

(In practice it's perhaps often smarter for that case to be handled by service gateway proxies, tho useful to have)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions