I was trying to send http/https requests via proxy (socks5), but I can’t understand if the problem is in my code or in the proxy.
I tried using this code and it gives me an error:
requests.exceptions.ConnectionError: SOCKSHTTPSConnectionPool(host='www.google.com', port=443): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.contrib.socks.SOCKSHTTPSConnection object at 0x000001B656AC9608>: Failed to establish a new connection: Connection closed unexpectedly'))
This is my code:
import requests url = "https://www.google.com" proxies = { "http":"socks5://fsagsa:sacesf241_country-darwedafs_session-421dsafsa@x.xxx.xxx.xx:31112", "https":"socks5://fsagsa:sacesf241_country-darwedafs_session-421dsafsa@x.xxx.xxx.xx:31112", } headers = { "Upgrade-Insecure-Requests": "1", "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.63 Safari/537.36", "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9", "Sec-Gpc": "1", "Sec-Fetch-Site": "same-origin", "Sec-Fetch-User": "?1", "Accept-Encoding": "gzip, deflate, br", "Sec-Fetch-Mode": "navigate", "Sec-Fetch-Dest": "document", "Accept-Language": "en-GB,en;q=0.9" } r = requests.get(url, headers = headers, proxies = proxies) print(r)
Then, I checked the proxy with an online tool The tool manages to send requests through the proxy. .
So the problem is in this code? I can’t figure out what’s wrong.
Edit (15/09/2021)
I added headers but the problem is still there.
Advertisement
Answer
Create a local server/mock to handle the request using pytest or some other testing framework with responses
library to eliminate variables external to your application/script. I’m quite sure Google will reject requests with empty headers. Also, ensure you installed the correct dependencies to enable SOCKS proxy support in requests (python -m pip install requests[socks]
). Furthermore, if you are making a remote request to connect to your proxy you must change socks5
to socks5h
in your proxies
dictionary.
References
pytest: https://docs.pytest.org/en/6.2.x/
responses: https://github.com/getsentry/responses
requests[socks]: https://docs.python-requests.org/en/master/user/advanced/#socks