I’m using my complex jobs app in production and it works fine, with sqlite database. I’m trying to create new database from scratch and I cannot do that nor using migrations nor trying to make them once again:
When I’m trying to reuse my migrations:
rm db.sqlite3
python manage.py migrate
Traceback (most recent call last):
File "/backend/server/manage.py", line 22, in <module>
main()
File "/backend/server/manage.py", line 18, in main
execute_from_command_line(sys.argv)
File "/backend/env/lib/python3.10/site-packages/django/core/management/__init__.py", line 401, in execute_from_command_line
utility.execute()
File "/backend/env/lib/python3.10/site-packages/django/core/management/__init__.py", line 377, in execute
django.setup()
File "/backend/env/lib/python3.10/site-packages/django/__init__.py", line 24, in setup
apps.populate(settings.INSTALLED_APPS)
File "/backend/env/lib/python3.10/site-packages/django/apps/registry.py", line 122, in populate
app_config.ready()
File "/backend/env/lib/python3.10/site-packages/django/contrib/admin/apps.py", line 24, in ready
self.module.autodiscover()
File "/backend/env/lib/python3.10/site-packages/django/contrib/admin/__init__.py", line 24, in autodiscover
autodiscover_modules('admin', register_to=site)
File "/backend/env/lib/python3.10/site-packages/django/utils/module_loading.py", line 47, in autodiscover_modules
import_module('%s.%s' % (app_config.name, module_to_search))
File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 883, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "/backend/server/jobs/admin.py", line 57, in <module>
class TagCategoryDefaultImportanceAdmin(admin.TabularInline):
File "/backend/server/jobs/admin.py", line 59, in TagCategoryDefaultImportanceAdmin
extra = len(Role.objects.all())
File "/backend/env/lib/python3.10/site-packages/django/db/models/query.py", line 269, in __len__
self._fetch_all()
File "/backend/env/lib/python3.10/site-packages/django/db/models/query.py", line 1303, in _fetch_all
self._result_cache = list(self._iterable_class(self))
File "/backend/env/lib/python3.10/site-packages/django/db/models/query.py", line 53, in __iter__
results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)
File "/backend/env/lib/python3.10/site-packages/django/db/models/sql/compiler.py", line 1156, in execute_sql
cursor.execute(sql, params)
File "/backend/env/lib/python3.10/site-packages/django/db/backends/utils.py", line 98, in execute
return super().execute(sql, params)
File "/backend/env/lib/python3.10/site-packages/django/db/backends/utils.py", line 66, in execute
return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
File "/backend/env/lib/python3.10/site-packages/django/db/backends/utils.py", line 75, in _execute_with_wrappers
return executor(sql, params, many, context)
File "/backend/env/lib/python3.10/site-packages/django/db/backends/utils.py", line 79, in _execute
with self.db.wrap_database_errors:
File "/backend/env/lib/python3.10/site-packages/django/db/utils.py", line 90, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/backend/env/lib/python3.10/site-packages/django/db/backends/utils.py", line 84, in _execute
return self.cursor.execute(sql, params)
File "/backend/env/lib/python3.10/site-packages/django/db/backends/sqlite3/base.py", line 413, in execute
return Database.Cursor.execute(self, query, params)
django.db.utils.OperationalError: no such table: jobs_role
When I’ve tried to remove all migration files:
mv jobs/migrations jobs/old_migrations
rm -Rf db.sqlite3
python manage.py makemigrations
Traceback (most recent call last):
File "/backend/server/manage.py", line 22, in <module>
main()
File "/backend/server/manage.py", line 18, in main
execute_from_command_line(sys.argv)
File "/backend/env/lib/python3.10/site-packages/django/core/management/__init__.py", line 401, in execute_from_command_line
utility.execute()
File "/backend/env/lib/python3.10/site-packages/django/core/management/__init__.py", line 377, in execute
django.setup()
File "/backend/env/lib/python3.10/site-packages/django/__init__.py", line 24, in setup
apps.populate(settings.INSTALLED_APPS)
File "/backend/env/lib/python3.10/site-packages/django/apps/registry.py", line 122, in populate
app_config.ready()
File "/backend/env/lib/python3.10/site-packages/django/contrib/admin/apps.py", line 24, in ready
self.module.autodiscover()
File "/backend/env/lib/python3.10/site-packages/django/contrib/admin/__init__.py", line 24, in autodiscover
autodiscover_modules('admin', register_to=site)
File "/backend/env/lib/python3.10/site-packages/django/utils/module_loading.py", line 47, in autodiscover_modules
import_module('%s.%s' % (app_config.name, module_to_search))
File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 883, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "/backend/server/jobs/admin.py", line 57, in <module>
class TagCategoryDefaultImportanceAdmin(admin.TabularInline):
File "/backend/server/jobs/admin.py", line 59, in TagCategoryDefaultImportanceAdmin
extra = len(Role.objects.all())
File "/backend/env/lib/python3.10/site-packages/django/db/models/query.py", line 269, in __len__
self._fetch_all()
File "/backend/env/lib/python3.10/site-packages/django/db/models/query.py", line 1303, in _fetch_all
self._result_cache = list(self._iterable_class(self))
File "/backend/env/lib/python3.10/site-packages/django/db/models/query.py", line 53, in __iter__
results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)
File "/backend/env/lib/python3.10/site-packages/django/db/models/sql/compiler.py", line 1156, in execute_sql
cursor.execute(sql, params)
File "/backend/env/lib/python3.10/site-packages/django/db/backends/utils.py", line 98, in execute
return super().execute(sql, params)
File "/backend/env/lib/python3.10/site-packages/django/db/backends/utils.py", line 66, in execute
return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
File "/backend/env/lib/python3.10/site-packages/django/db/backends/utils.py", line 75, in _execute_with_wrappers
return executor(sql, params, many, context)
File "/backend/env/lib/python3.10/site-packages/django/db/backends/utils.py", line 79, in _execute
with self.db.wrap_database_errors:
File "/backend/env/lib/python3.10/site-packages/django/db/utils.py", line 90, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/backend/env/lib/python3.10/site-packages/django/db/backends/utils.py", line 84, in _execute
return self.cursor.execute(sql, params)
File "/backend/env/lib/python3.10/site-packages/django/db/backends/sqlite3/base.py", line 413, in execute
return Database.Cursor.execute(self, query, params)
django.db.utils.OperationalError: no such table: jobs_role
How can I investigate what is going on? Where is the problem with jobs_role table?
It figures out, that problem is related to my admin.py file
class TagCategoryDefaultImportanceAdmin(admin.TabularInline):
model = TagCategoryDefaultImportance
extra = len(Role.objects.all())
max_num = len(Role.objects.all())
formset = TagCategoryDefaultImportanceAdminFormse
The extra and max_num fields cause the issue. How can I handle that in a correct manner?
Advertisement
Answer
As the error indicates:
File "/backend/server/jobs/admin.py", line 59, in TagCategoryDefaultImportanceAdmin
extra = len(Role.objects.all())
This meansthat when interpreting the file, it will already run this query. This is thus also the case if the database is not (yet) fully migrated, hence the error.
As a (strong) rule of thumb, you should try to avoid running queries when interpreting the file (reading the file into memory), since these will thus run before the server starts running, and before the database might be properly set up. Even if that would work, it would mean that extra takes as value the number if items when you start the server: if you would later add/remove a role, then extra would not change.
Usually you should try to wrap the logic into a function such that it only will evaluate Role.objects.all() when it runs the function, and such admin functions are then typically only triggered when the user visits the admin pages.
As for this specific case, you can use the .get_extra(…) [Django-doc] and .get_max_num(…) methods [Django-doc]:
class TagCategoryDefaultImportanceAdmin(admin.TabularInline):
model = TagCategoryDefaultImportance
formset = TagCategoryDefaultImportanceAdminFormset
def get_max_num(request, obj=None, **kwargs):
return Role.objects.count()
def get_extra(self, request, obj=None, **kwargs):
return Role.objects.count()