How to fix a Mac base conda environment when sqlite3 is broken

I recently updated the Python version of my base conda environment from 3.8 to 3.9, using mamba update python=3.9, but I can no longer run IPython, because the sqlite3 package appears to be broken.

python
Python 3.9.15 | packaged by conda-forge | (main, Nov 22 2022, 08:55:37) 
[Clang 14.0.6 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sqlite3
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/rosborn/opt/miniconda3/lib/python3.9/sqlite3/__init__.py", line 57, in <module>
    from sqlite3.dbapi2 import *
  File "/Users/rosborn/opt/miniconda3/lib/python3.9/sqlite3/dbapi2.py", line 27, in <module>
    from _sqlite3 import *
ImportError: dlopen(/Users/rosborn/opt/miniconda3/lib/python3.9/lib-dynload/_sqlite3.cpython-39-darwin.so, 0x0002): Symbol not found: (_sqlite3_enable_load_extension)
  Referenced from: '/Users/rosborn/opt/miniconda3/lib/python3.9/lib-dynload/_sqlite3.cpython-39-darwin.so'
  Expected in: '/usr/lib/libsqlite3.dylib'

Since I had another Python 3.9 environment that is still functional, I tried copying over the envs/py39/lib/sqlite3.36.0 and envs/py39/lib/python3.9/sqlite3 directories, as well as envs/py39/lib/python3.9/lib-dynload/_sqlite3.cpython-39-darwin.so because I assumed the sqlite3 libraries had been incorrectly compiled, but that doesn’t fix the problem.

On the Homebrew Github, there was a related issue, where someone suggested checking whether the missing symbol was there. It seems to be all present and correct.

$ nm -gj /Users/rosborn/opt/miniconda3/lib/python3.9/lib-dynload/_sqlite3.cpython-39-darwin.so | grep enable_load_extension 
_sqlite3_enable_load_extension

I don’t know how Homebrew installs sqlite3, but the remaining fixes seemed to require checking the system libsqlite, which I don’t have administrative access to. In case it’s relevant, I am on an Intel Mac, so it’s not related to the M1 chip, as some related issues appear to be.

Does the conda distribution attempt to link to the system libsqlite? If so, why does this problem not affect the py39 environment?

Any tips will be welcome. If it were not the base environment, I would just delete the one with the problem and start again. I attempted a forced reinstall of sqlite3, but it appeared not to be installable as a separate package.

Answer

Following the suggestions by @merv, the solution to this problem was to force a reinstall of the libsqlite package.

$ mamba install libsqlite --force-reinstall

After updating Python, it seems that sqlite3 was linked to the Mac system library, /usr/lib/libsqlite3.dylib, rather than one installed by conda-forge. According to discussions elsewhere, it is likely that Apple disables the missing _sqlite3_enable_load_extension extension for security reasons, leading to the observed error message. I don’t know why the link error occurred in the first place, but fortunately, conda distributes libsqlite as a separate package, so the fix was simple to implement.