Imported package searches for modules in my code

Can someone explain me what is going on here and how to prevent this?

I have a main.py with the following code:

import utils
import torch


if __name__ == "__main__":
    # Foo
    print("Foo")

    # Bar
    utils.bar()

    model = torch.hub.load("ultralytics/yolov5", "yolov5s")

I outsourced some functions into a module named utils.py:

def bar():
    print("Bar")

When I run this I get the following output:

(venv) jan@xxxxx test % python main.py
Foo
Bar
Using cache found in /Users/jan/.cache/torch/hub/ultralytics_yolov5_master
Traceback (most recent call last):
File "/Users/jan/PycharmProjects/test/main.py", line 12, in <module>
    model = torch.hub.load("ultralytics/yolov5", "yolov5s")
File "/Users/jan/PycharmProjects/test/venv/lib/python3.10/site-packages/torch/hub.py", line 540, in load
    model = _load_local(repo_or_dir, model, *args, **kwargs)
File "/Users/jan/PycharmProjects/test/venv/lib/python3.10/site-packages/torch/hub.py", line 569, in _load_local
    model = entry(*args, **kwargs)
File "/Users/jan/.cache/torch/hub/ultralytics_yolov5_master/hubconf.py", line 81, in yolov5s
    return _create('yolov5s', pretrained, channels, classes, autoshape, _verbose, device)
File "/Users/jan/.cache/torch/hub/ultralytics_yolov5_master/hubconf.py", line 31, in _create
    from models.common import AutoShape, DetectMultiBackend
File "/Users/jan/.cache/torch/hub/ultralytics_yolov5_master/models/common.py", line 24, in <module>
    from utils.dataloaders import exif_transpose, letterbox
ModuleNotFoundError: No module named 'utils.dataloaders'; 'utils' is not a package

So it seems like the torch package I imported has also a utils resource (package) and searches for a module named “utils.dataloaders”. Okay. But why is it searching in my utils module? And why isn’t it continuing searching in its own package if it doesn’t find a matching resource in my code? And last but not least: How can I prevent this situation?

I changed import utils to import utils as ut and call my function with ut.bar() but it doesn’t make any difference.

The only thing that worked is to rename my utils.py to something else but this cannot be the solution…

Thanks for your help. Cheers,

Jan

Answer

The other utils package does not belong to torch, it belongs to the yolov5 repository: /Users/jan/.cache/torch/hub/ultralytics_yolov5_master/utils.

Now, to explain the error: It seems that python would search for sys.modules first when you import utils. If you import your own utils first, it is registered in sys.modules, and python would search in your own utils for things needed for the yolov5 model. Same thing would happen if you create the model first and import your own utils later (at this time, utils of the yolov5 repo is registered in sys.modules, so you won’t be able to import your own utils).

The easiest solution would be to rename your utils.py, or put it under a folder, e.g.,

your_project/
  main.py
  some_name_other_than_utils/
    utils.py

and import it as

import some_name_other_than_utils.utils as utils

Then, some_name_other_than_utils.utils is registered in sys.modules so it won’t affect importing in yolov5.

Another solution is to copy the yolov5 folder as a subfolder of your project. So everything in the yolov5 folder is under another namespace, without conflicting with your files. But you may need to change some import statements, and replace torch.hub.load with your own model loading function.

---

In most cases, having utils in your own project shouldn’t conflict with third-party software that also has utils. If you try to import torch and then check sys.modules, you can see torch.utlis, torch.nn.utils, etc., rather than just utils. The following is an example of how this can be done with relative imports:

your_project/
  main.py
  utils.py
  some_3rdparty_lib/
    __init__.py
    utils.py

In some_3rdparty_lib/__init__.py:

# some_3rdparty_lib/__init__.py
from . import utils

Then in main.py:

# main.py
import utils   # your own utils
import some_3rdparty_lib.utils   # the one under some_3rdparty_lib

Note that some_3rdparty_lib does not have to be under your project directory. You can put it anywhere as long as the path is in sys.path. The imported utils would still belong to the namespace some_3rdparty_lib.