httpie -d does not work with gzip compressed content

Bắt đầu từ luồng tải xuống của CLI được thực thi bởi http -d -o và kiểm tra cách Content-Length của phản hồi được so sánh với số byte đã tải xuống khi Content-Encoding là gzip. Tái hiện ví dụ Azure Blob hoặc một phản hồi được nén tương đương, sau đó bổ sung coverage cho việc kiểm tra kích thước. Hoàn tất khi các lượt tải xuống được nén không còn báo tải xuống chưa hoàn tất khi đã nhận đủ payload được nén.

Checklist

• I've searched for similar issues.
• I'm using the latest version of HTTPie.

Minimal reproduction code and steps

Expose an API endpoint that respond with gzip compressed json such as an Azure storage account.

1. http get -d -o foo.json https://.../path/to/endpoint

Current result

+ http -d -I --check-status GET https://XXXX.blob.core.windows.net/invoices/p32132/1201394/input.json?sv=2021-12-02&se=2024-02-25T19%3A08%3A46Z&sr=c&sp=r&sig=XXXXX -o p32132/1201394/input.json
HTTP/1.1 200 OK
Accept-Ranges: bytes
Access-Control-Allow-Origin: *
Access-Control-Expose-Headers: x-ms-request-id,Server,x-ms-version,x-ms-resource-type,Content-Type,Content-Encoding,Last-Modified,ETag,x-ms-creation-time,Content-MD5,x-ms-lease-status,x-ms-lease-state,x-ms-blob-type,x-ms-server-encrypted,Accept-Ranges,x-ms-owner,x-ms-group,x-ms-permissions,Content-Length,Date,Transfer-Encoding
Content-Encoding: gzip
Content-Length: 55119
Content-MD5: IJ8A578M7DLY5/jW+7umKw==
Content-Type: application/json
Date: Fri, 26 Jan 2024 19:08:46 GMT
ETag: "0x8DBFA81678EA69D"
Last-Modified: Mon, 11 Dec 2023 19:43:09 GMT
Server: Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0
x-ms-blob-type: BlockBlob
x-ms-creation-time: Mon, 11 Dec 2023 19:36:34 GMT
x-ms-group: $superuser
x-ms-lease-state: available
x-ms-lease-status: unlocked
x-ms-owner: $superuser
x-ms-permissions: rw-r-----
x-ms-request-id: c3159c91-001e-0064-578b-5091a4000000
x-ms-resource-type: file
x-ms-server-encrypted: true
x-ms-version: 2021-12-02

Downloading to p32132/1201394/input.json
Done. 118.9 kB in 00:0.08901 (1.3 MB/s)

http: LogLevel.ERROR: Incomplete download: size=55119; downloaded=118911

…

Expected result

Httpie should consider the compressed size to validate that the entire content was downloaded, not consider the uncompressed size with the compressed gzip payload that was sent.

…

Debug output

Please re-run the command with --debug, then copy the entire command & output and paste both below:

+ http -d --debug -I --check-status GET https://XXXX.blob.core.windows.net/invoices/p32132/1201394/input.json?sv=2021-12-02&se=2024-02-25T19%3A10%3A41Z&sr=c&sp=r&sig=XXXX -o p32132/1201394/input.json
HTTPie 3.2.1
Requests 2.31.0
Pygments 2.17.2
Python 3.11.6 (main, Nov 14 2023, 18:04:26) [GCC 13.2.1 20230801]
/usr/bin/python3
Linux 6.7.0-zen3-1-zen

<Environment {'apply_warnings_filter': <function Environment.apply_warnings_filter at 0x78780a9fd120>,
 'args': Namespace(),
 'as_silent': <function Environment.as_silent at 0x78780a9fcfe0>,
 'colors': 256,
 'config': {'default_options': []},
 'config_dir': PosixPath('/home/root/.config/httpie'),
 'devnull': <property object at 0x78780a9ed4e0>,
 'is_windows': False,
 'log_error': <function Environment.log_error at 0x78780a9fd080>,
 'program_name': 'http',
 'quiet': 0,
 'rich_console': <functools.cached_property object at 0x78780a9f3050>,
 'rich_error_console': <functools.cached_property object at 0x78780a9f3150>,
 'show_displays': True,
 'stderr': <_io.TextIOWrapper name='<stderr>' mode='w' encoding='utf-8'>,
 'stderr_isatty': True,
 'stdin': <_io.TextIOWrapper name='<stdin>' mode='r' encoding='utf-8'>,
 'stdin_encoding': 'utf-8',
 'stdin_isatty': False,
 'stdout': <_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>,
 'stdout_encoding': 'utf-8',
 'stdout_isatty': True}>

<PluginManager {'adapters': [],
 'auth': [<class 'httpie.plugins.builtin.BasicAuthPlugin'>,
          <class 'httpie.plugins.builtin.DigestAuthPlugin'>,
          <class 'httpie.plugins.builtin.BearerAuthPlugin'>],
 'converters': [],
 'formatters': [<class 'httpie.output.formatters.headers.HeadersFormatter'>,
                <class 'httpie.output.formatters.json.JSONFormatter'>,
                <class 'httpie.output.formatters.xml.XMLFormatter'>,
                <class 'httpie.output.formatters.colors.ColorFormatter'>]}>

>>> requests.request(**{'auth': None,
 'data': RequestJSONDataDict(),
 'headers': <HTTPHeadersDict('User-Agent': b'HTTPie/3.2.1', 'Accept-Encoding': b'identity')>,
 'method': 'get',
 'params': <generator object MultiValueOrderedDict.items at 0x78780bca6a40>,
 'url': 'https://XXXX.blob.core.windows.net/invoices/p32132/1201394/input.json?sv=2021-12-02&se=2024-02-25T19%3A10%3A41Z&sr=c&sp=r&sig=XXXX'})

HTTP/1.1 200 OK
Accept-Ranges: bytes
Access-Control-Allow-Origin: *
Access-Control-Expose-Headers: x-ms-request-id,Server,x-ms-version,x-ms-resource-type,Content-Type,Content-Encoding,Last-Modified,ETag,x-ms-creation-time,Content-MD5,x-ms-lease-status,x-ms-lease-state,x-ms-blob-type,x-ms-server-encrypted,Accept-Ranges,x-ms-owner,x-ms-group,x-ms-permissions,Content-Length,Date,Transfer-Encoding
Content-Encoding: gzip
Content-Length: 55119
Content-MD5: IJ8A578M7DLY5/jW+7umKw==
Content-Type: application/json
Date: Fri, 26 Jan 2024 19:10:41 GMT
ETag: "0x8DBFA81678EA69D"
Last-Modified: Mon, 11 Dec 2023 19:43:09 GMT
Server: Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0
x-ms-blob-type: BlockBlob
x-ms-creation-time: Mon, 11 Dec 2023 19:36:34 GMT
x-ms-group: $superuser
x-ms-lease-state: available
x-ms-lease-status: unlocked
x-ms-owner: $superuser
x-ms-permissions: rw-r-----
x-ms-request-id: c8568e21-501e-0079-548b-509c18000000
x-ms-resource-type: file
x-ms-server-encrypted: true
x-ms-version: 2021-12-02

Downloading to p32132/1201394/input.json
Done. 118.9 kB in 00:0.08444 (1.4 MB/s)

http: LogLevel.ERROR: Incomplete download: size=55119; downloaded=118911

Additional information, screenshots, or code examples

…