The initial situation is that I have two fast-api servers that access the same database. One is the real service for my application and the other is a service for loading data from different sources. Both services have their own Github repository, but use the same orm data model.
My question is: What are best practices to manage this orm data model without always editing it in both code bases?
The goal would be to have no duplicated code and to adapt changes to the database only in one place.
Current structure of the projects:
Service 1 (actual backend): . ├── app │ ├── __init__.py │ ├── main.py │ └── database │ ├── __init__.py │ ├── ModelItem.py # e.g. model item │ ├── ... # other models │ ├── SchemaItem.py # e.g. schema item │ └── ... # other schemas │ ├── routers │ └── ...
Service 2 (data loader): . ├── app │ ├── __init__.py │ ├── main.py │ └── database │ ├── __init__.py │ ├── ModelItem.py # e.g. model item │ ├── ... # other models │ ├── SchemaItem.py # e.g. schema item │ └── ... # other schemas │ ├── routers │ └── ...
sqlalchemy(.orm) library is used for the orm models and schemas.
I had thought about making another repository out of it or a library of my own. But then I would not know how best to import them and whether I won’t just produce more work.
Based on the answer, I created a private repository that I install via pip in the two servers.
- Structure of the package Python packaging projects
- Install the package from a private Github repository Installing Private Python Packages
Advertisement
Answer
Modularisation (avoiding code repetition) is a best practice, and so the ideal thing to do would indeed be to extract the model definition to a single file and import it where needed.
The problem is, when you deploy your two services both of them need to be able to ‘see’ the model file … otherwise they can’t import it. Because they are deployed separately, this isn’t going to be as easy as before (they presumably aren’t sharing a filesystem; if they were, it would be just as simple as before).
The usual solution to this is to make your model file available as a Python module and provide it via some server both services can reach over the network (for example, the public python package index, PyPI). You then install the module and import it in both deployed services.
This is inevitably going to be (a lot!) more work than just maintaining two copies. However, if you go to more than two copies that calculus could quickly change.