Struggling with python’s import mechanism

I am an experienced java enterprise developer but very new to python enterprise development shop. I am currently, struggling to understand why some imports work while others don’t.

Some background: Our dev team recently upgraded python from 3.6 to 3.10.5 and following is our package structure

src/
bunch of files (dockerfile, Pipfile, requrirements.txt, shell scripts, etc)
  package/
      __init__.py
      moduleA.py
      subpackage1/
          __init__.py
          moduleX.py
          moduleY.py
      subpackage2/
          __init__.py
          moduleZ.py
      tests/
          __init__.py
          test1.py
    

Now, inside the moduleA.py, I am trying to import subpackage2/moduleZ.py like so

from .subpackage2 import moduleZ

But, I get the error saying

ImportError: attempted relative import with no known parent package

The funny thing is that if I move moduleA.py out of package/ and into src/ then it is able to find everything. I am not sure why is this the case.

I run the moduleA.py by executiong python package/moduleA.py.

Now, I read that maybe there is a problem becasue you have you give a -m parameter if running a module as a script (or something on those lines). But, if I do that, I get the following error:

ModuleNotFoundError: No module names 'package/moduleA.py'

I even try running package1/moduleA and remove the .py, but that does not work either. I can understand why as I technically never installed it ?

All of this happened because apparently, the tests broke and to make it work they added relative imports. They changed the import from “from subpackage2 import moduleZ” to “from .subpackage2 import moduleZ” and the tests started working, but the app started failing.

Any understanding I can get would be much appreciated.

Answer

The -m parameter is used with the import name, not the path. So you’d use python3 -m package.moduleA (with . instead of /, and no .py), not python3 -m package/moduleA.py.

That said, it only works if package.moduleA is locatable from one of the roots in sys.path. Shy of installing the package, the simplest way to make it work is to ensure your working directory is src (so package exists in the working directory):

$ cd path/to/src
$ python3 -m package.moduleA

and, with your existing setup, if moduleA.py includes a from .subpackage2 import moduleZ, the import should work; Python knows package.moduleA is a module within package, so it can use a relative import to look for a sibling package to moduleA named subpackage2, and then inside it it can find moduleZ.

Obviously, this is brittle (it only works if you cd to the src root directory before running Python, or hack the path to src in PYTHONPATH, which is terrible hack if the code ever has to be run by anyone else); ideally you make this an installable package, install it (in global site-packages, user site-packages, or within a virtual environment created with the built-in venv module or the third-party virtualenv module), and then your working directory no longer matters (since the site-packages will be part of your sys.path automatically). For simple testing, as long as the working directory is correct (not sure what it was for you), and you use -m correctly (you were using it incorrectly), relative imports will work, but it’s not the long term solution.