Python3: install github-based module in setup.py?

Installing with pip, I can write the following requirements.txt file:

git+https://repo@branch#egg=foo&subdirectory=this/bar/foo
numpy

And successfully install the requirements file:

python3 -m pip install -r requirements.tx

However, I have co-located in the directory a setup.py script that lists:

setuptools.setup(
  ...
  install_requires = get_lines('requirements.txt'),
  ...
)

And installing this submodule using pip involves pip running setup.py…which fails to handle the module link:

git+https://github.com/repo@branch#egg=foo&subdirectory=this/bar/foo

---

I can see a lot of ways around this, but it seems like there should be one non-ambiguous way to do this which changes as little as possible in the setup.py script.

Is there such a way?

Answer

You probably need to change the line in requirements.txt to something like:

foo @ git+https://repo@branch#egg=foo&subdirectory=this/bar/foo

References:

• https://pip.pypa.io/en/stable/reference/pip_install/#requirement-specifiers
• https://www.python.org/dev/peps/pep-0440/#direct-references

Although I am not entirely sure it will work. There might be subtle differences between the notations accepted in requirements.txt files, pip directly and setuptools. In particular I do not know how well things like egg and subdirectory are supported.

Advices:

• Avoid calling python setup.py install or python setup.py develop from now on, and make sure to call python -m pip install . or python -m pip install --editable . instead.
• I do consider reading requirements.txt from within setup.py as a red flag (or at least yellow). The contents of install_requires of setuptools and requirements.txt usually serve different purposes.