Add support for setting resp.stream to a requests.Response or urllib3.HTTPResponse object
#1271 opened on May 12, 2018
Description
We already have support for streaming via a file-like project, but it is also common to fetch upstream resources via the requests library, and so it would be really helpful if Falcon supported this natively. Internally, the object can be tested to see if it is an instance of requests.Response or urllib3.HTTPResponse. These types can be optionally imported (and if the import fails, we know we never need to do this check).
Note that this requires passing stream=True when making the request (or preload_content=False when using urllib3 directly). This should be noted in the docstring for Response.stream.
One way of handling this would be to wrap the object in a closeable stream iterator and then return that to the WSGI server. Here's some proof of concept code (only tested on Python 3):
class CloseableStreamIterator(collections.Iterator):
"""Iterator that wraps a urllib3 response with support for release_conn().
Args:
resp (object): urllib3.HTTPResponse instance.
block_size (int): Number of bytes to read per iteration (default 64K).
decode_content (bool): If True, will attempt to decode the body
according to the Content-Encoding header (default False).
"""
__slots__ = [
'_resp',
'_stream',
]
def __init__(self, resp, block_size=2**16, decode_content=False):
self._resp = resp
self._stream = resp.stream(amt=block_size, decode_content=decode_content)
def __iter__(self):
return self
def __next__(self):
return next(self._stream)
def close(self):
self._resp.release_conn()