Cant decode HTTP Response to JSON (Python3)

I am running a series of API tests and this one is giving me an error. As far as I can tell response.read() converts the HTTP response into binary and then decode() converts that binary into a string but it’s an empty string. When I copy paste the binary string into Python3 (b'{"error": {"code": "INVALID_TOKEN", "description": ""}}') and issue a decode() command it works fine. I can even import json and do a loads() call on that string.

Can anyone suggest why this is happening?

import http.client
import json

# Fake info
api = http.client.HTTPConnection('192.168.0.1', 12345)

def logout_user (p_token):
    headers = {"Content-type": "application/json",
               "Accept": "text/plain",
               'Authorization': 'Bearer {}'.format(p_token)}
    try:
        api.request(method='PATCH',
                    url='/logout_user',
                    headers=headers)

        response = api.getresponse()
        status = response.status
        reason = response.reason
        if status == 200:
            data = None
        else:
            logging.debug (response)
            logging.debug (response.read ())
            logging.debug (response.read ().decode('ascii'))
            logging.debug (type(response.read ()))
            logging.debug (type(response.read ().decode()))
            data = json.loads(response.read ().decode('ascii'))  # The error is caused at this line and I think it's because I am passing an empty string to loads

    # If there is some API error, log it, and re-raise a general Exception
    except Exception as e:
        logging.error (e)

        raise exceptions.api.UnknownApiError ()

    return (status, reason, data, )

Error:

[2021-02-20 17:43:02,398] <http.client.HTTPResponse object at 0x7f6891b29430>
[2021-02-20 17:43:02,399] b'{"error": {"code": "INVALID_TOKEN", "description": ""}}'
[2021-02-20 17:43:02,399]
[2021-02-20 17:43:02,399] <class 'bytes'>
[2021-02-20 17:43:02,399] <class 'str'>
[2021-02-20 17:43:02,399] Expecting value: line 1 column 1 (char 0)
Traceback (most recent call last):
  File "./test_api.py", line 168, in logout_user
    data = json.loads(response.read ().decode('ascii'))
  File "/usr/lib/python3.8/json/__init__.py", line 357, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python3.8/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python3.8/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "./test_api.py", line 4447, in <module>
    logout_user__fake_token ()
  File "./test_api.py", line 1093, in logout_user__fake_token
    status, reason, data = logout_user (token)
  File "./test_api.py", line 174, in logout_user
    raise exceptions.api.UnknownApiError ()
exceptions.api.UnknownApiError: Unable to make API call

Answer

The response object is a “file-like” object. You are trying to read the response twice, but at the second call you are getting an empty string.

You may want to change your code to something along the lines of:

content = response.read()
logging.debug (response)
logging.debug (content)
logging.debug (content.decode('ascii'))
data = json.loads(content.decode('ascii'))