def test_stream_helper_decoding(self):
status_code, content = fake_api.fake_responses[url_prefix + 'events']()
content_str = json.dumps(content)
content_str = content_str.encode('utf-8')
body = io.BytesIO(content_str)
# mock a stream interface
raw_resp = urllib3.HTTPResponse(body=body)
setattr(raw_resp._fp, 'chunked', True)
setattr(raw_resp._fp, 'chunk_left', len(body.getvalue()) - 1)
# pass `decode=False` to the helper
raw_resp._fp.seek(0)
resp = response(status_code=status_code, content=content, raw=raw_resp)
result = next(self.client._stream_helper(resp))
assert result == content_str
# pass `decode=True` to the helper
raw_resp._fp.seek(0)
resp = response(status_code=status_code, content=content, raw=raw_resp)
result = next(self.client._stream_helper(resp, decode=True))
assert result == content
# non-chunked response, pass `decode=False` to the helper
setattr(raw_resp._fp, 'chunked', False)
raw_resp._fp.seek(0)
resp = response(status_code=status_code, content=content, raw=raw_resp)
result = next(self.client._stream_helper(resp))
assert result == content_str.decode('utf-8')
# non-chunked response, pass `decode=True` to the helper
raw_resp._fp.seek(0)
resp = response(status_code=status_code, content=content, raw=raw_resp)
result = next(self.client._stream_helper(resp, decode=True))
assert result == content
def load(self, key, entry):
file_path = self._file_path(key)
response = entry.response
response.raw = urllib3.HTTPResponse(
# The data that we write to disk is pre-decoding, which is good because it means in most cases we can have a gzipped
# cache without expanding CPU cycles for it. However it means that in order to provide the user with decoded data, we
# need to recreate an HTTPResponse object, since that's the object doing the decoding. Trying to pickle that got messy,
# so we reconstruct it like this, which isn't pretty, but works.
headers=urllib3._collections.HTTPHeaderDict(entry.raw_headers if entry.raw_headers is not None else response.headers),
status=response.status_code,
reason=response.reason,
request_method=response.request.method,
preload_content=False,
decode_content=False,
body=AutoClosingFile(self._open_local_file(response, file_path, 'r')),
)
response.raw._original_response = MockedHttplibResponse(response.raw)
response._content_consumed = False
response._content = False