Skip to content
Advertisement

python-requests: is it possible to bypass HTTPS for speed

Is it possible to bypass the HTTPS on python3+requests to gain speed?

Profiling says SSL handling is the slowest part of my script:

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
      133    0.377    0.003    0.377    0.003 {method 'read' of '_ssl._SSLSocket' objects} <--- slowest part 380ms/web access
      232    0.131    0.001    0.131    0.001 {built-in method __new__ of type object at 0x000000007419C430}
      153    0.087    0.001    0.090    0.001 <frozen importlib._bootstrap_external>:830(get_data)
        1    0.023    0.023    0.023    0.023 {method 'do_handshake' of '_ssl._SSLSocket' objects}


import requests    
resp = requests.get("https://uk.finance.yahoo.com", verify=False)

verify=False just disables the certificate checking, but SSL/TLS still happens in the background.

I didn’t find any option to use the dumbest cipher (eg. 0bit) type to gain speed.

Security is not my goal in this script. I already upgraded my packages with pip. Environment Win10 x64. As I tested most http:// addresses only allow/redirect to https://

Advertisement

Answer

You cannot bypass the use of HTTPS if the URL you use says https://. The very meaning of https:// is that HTTPS gets used. You might try to use http:// instead of https://. But this will only work if the server actually exposes the resource in question with a simple http://. Typically today an access to http://... will just redirect to https://... and you end up at the same URL as you had before, only even slower due to the overhead of the additional HTTP request.

Advertisement