Python urllib3.exceptions.NewConnectionError connecting to self-built API

I’ve built an API (with flask-restful) that stores data in its cache and exposes it to other applications. When I try to send a get request to this API from another app (also flask) it returns the following error

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/urllib3/connection.py", line 170, in _new_conn
    (self._dns_host, self.port), self.timeout, **extra_kw
  File "/usr/local/lib/python3.6/dist-packages/urllib3/util/connection.py", line 73, in create_connection
    for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
  File "/usr/lib/python3.6/socket.py", line 745, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 706, in urlopen
    chunked=chunked,
  File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 382, in _make_request
    self._validate_conn(conn)
  File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 1010, in _validate_conn
    conn.connect()
  File "/usr/local/lib/python3.6/dist-packages/urllib3/connection.py", line 353, in connect
    conn = self._new_conn()
  File "/usr/local/lib/python3.6/dist-packages/urllib3/connection.py", line 182, in _new_conn
    self, "Failed to establish a new connection: %s" % e
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7f6f965d9358>: Failed to establish a new connection: [Errno -2] Name or service not known

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/urllib3/util/retry.py", line 574, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='data-collector.cloud', port=443): Max retries exceeded with url: /sample_url (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f6f965d9358>: Failed to establish a new connection: [Errno -2] Name or service not known',))

I thought that this error occurred because I was sending too many requests with the same url to the API. I had then limited the number of API calls by adding decorators = [index.limiter.limit("60/minute")] to the API. The error still persisted however. Then I thought the error might be caused by the amount of calls accepted by the server. I thought I was not closing the connection properly after making an API call. So I added

from requests.packages.urllib3 import Retry, PoolManager

retries = Retry(connect=5, read=2, redirect=5)
with PoolManager(retries=retries) as http:
    response = http.request('GET', url)

But this also did not solve my issue. What am I missing here? I am using python3.8

EDIT: I found out that it’s not per se the query that is causing it, because if I try other queries, then the same message pops up. I’m still lost at how to debug this :/

Answer

The error says: “Name or service not known”. In other words, it’s a name resolution issue (DNS).

Cross check the target host (“data-collector.cloud”, which isn’t in the public DNS records.)

Advertisement

Answer