Python client socket can’t receive data from a successful response

Socket can’t receive any data from the server, when there is a successful response, but with bad requests it can. Also server responds, just the socket can’t receive data (checked in WireShark)

import socket
import ssl

HOST, PORT = 'example.com', 443

sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

ssock = ssl.wrap_socket(sock)
ssock.connect((HOST, PORT))

raw_req = [f'GET / HTTP/1.1', 'Host: {HOST}', 'Connection: keep-alive']
req = 'n'.join(raw_req)

ssock.send(req.encode())

msg = ssock.recv(4096).decode()
print(msg)

ssock.close()

Answer

First, the HTTP GET expects a sequence of CR LF characters after each header line not just a single ‘n’ character and an extra CR LF after the last header line. Also, the join() adds the separator between each pair but not at the end so must append data with CR LF + CR LF to be a valid HTTP request.

Second, the 'Host: {HOST}' must be a f-string otherwise the “{HOST}” is not replaced.

import socket
import ssl

HOST, PORT = 'stackoverflow.com', 443

with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
    ssock = ssl.wrap_socket(sock)
    ssock.connect((HOST, PORT))

    raw_req = [f'GET / HTTP/1.1', f'Host: {HOST}', 'Connection: keep-alive']
    req = ('rn'.join(raw_req) + "rnrn").encode()
    print("Request:", req)

    ssock.send(req)

    msg = ssock.recv(4096).decode()
    print("nResponse:")
    print(msg)

Output:

Request: b'GET / HTTP/1.1rnHost: stackoverflow.comrnConnection: keep-alivernrn'

Response:
HTTP/1.1 200 OK
Connection: keep-alive
content-type: text/html; charset=utf-8
...

If the HTTP response is larger than 4096 bytes then you would need to call ssock.recv() in a loop until it returns a byte array of length 0.