How to use a module as part of a package and as a directly executable script?

Lets say i have a package folder called my_package. This directory contains a file called __init__.py and a module file called my_module.py among further module files. The module my_module.py contains the function do_something_with_file(file_path).

The function is used in other parts of the package but it would be practical if it could be called as a command line script that takes a file path as first positional argument and executes the function. My first naive approach was to implement that via:

if __name__ == "__main__":
    import sys
    do_something_with_file(sys.argv[1])

in the my_module.py file.

But that would require that the my_module.py is executed as main. But as part of a package it is using relative imports, which work only if the module is executed as part of a package but fail if the script is executed directly which makes this approach unattractive. It seems that a main block in this file is not the proper way to get the desired functionality.

What is good practice to expose the functionality as simple script?

I have no experience in packaging with python, I typically use scripts directly by executing python my_script.py but I’d like to get a better understanding of packages and exposing functionality with/via packages.

Answer

Exposing script entrypoints works best if you make your package installable. I’ll try to offer a more or less minimal way to do that using the basic tooling that comes with python itself, and list some third party packages in the end which make the whole process a lot more enjoyable if you have to maintain it for a while or end up writing a few more scripts.

---

To both turn your code into an installable package using setuptools as well as make specific functions within your code externally executable scripts, add a setup.py file to the top-level of your directory, like this:

my_project
├───setup.py  # right here, next to your source-code folder
└───my_package
    ├───__init__.py 
    └───my_module.py

With the setup.pys content being at least this (check here for some more possible keywords that may or may not make sense for your project):

import setuptools

setuptools.setup(
  name="my_package",        # your choice, but usually same as the package 
  version="0.1.0",          # obligatory
  packages=["my_package"],  # name of the folder relative to this file where the source code lives
  install_requires=[],      # your dependencies, if you have any
  entry_points = {          # this here is the magic that binds your function into a callable script
      'console_scripts': ['my_script=my_package.my_module:do_something_with_file'],
  }
)

The format of entry_points in conjunction with a 'console_scripts' key is:

['my_script=my_package.my_module:do_something_with_file']
| |        | └─ this is essentially just a slightly reformatted import of
| |        |    'from my_package.my_module import do_something_with_file'
| |        └─ this sign starts the parsing of the lookup string for the bound function 
| └─ the name you want to call your script by, can be chosen freely
└─ a list, since you can install multiple scripts with a single package

Once your setup is in place, you run pip install -e . next to it. Now your code is installed as a package, and your function is bound as a script. Run my_script location/to/some/file.txt from your console to run do_something_with_file with 'location/to/some/file.txt' in sys.argv[1].

---

Popular third party libraries to make script writing easier are click (quite old, but therefore super solid, and what I’d usually recommend) and typer (a lot more modern, makes use of recent builtin python features like type annotation).