Assuming that I can verify that a bunch of POST requests are in fact logically independent, how can I set up HTTP pipelining using python-requests and force it to allow POST requests in the pipeline?
Does someone have a working example?
P.S. for extra points, how to handle errors for outstanding requests if pipeline suddenly breaks?
P.P.S. grequests is not an option in this case.
Advertisement
Answer
The requests library does not support HTTP pipelining.
You can approximate pipelining by using grequests which makes it easier to run many requests in parallel, but each parallel request would still use a new TCP connection.
(requests does pool connections, keeping the TCP connection open if the remote server permits this, but that only helps for sequential connections, and request and response still have to alternate).