i installed airflow using bitnami repo: To install extra python packages i mounted an extra volume
I prepared my requirements.txt file and then i created a ConfigMap using kubectl create -n airflow configmap requirements –from-file=requirements.txt after this i upgraded airflow using helm upgrade….
But in my dags file, i’m still getting the error “ModuleNotFoundError: No module named ‘yfinance'”
Advertisement
Answer
Posting this as Community wiki for better visibility. Feel free to expand it.
As @Elpis wrote in the comments section, he followed this guide to install Apache Airflow on Kubernetes.
And he solved the problem by adding extraVolumeMounts
and extraVolumes
to the worker pod, and also to the web pod and to the scheduler pod.
extraVolumeMounts: - name: requirements mountPath: /bitnami/python/ ## Add extra volumes extraVolumes: - name: requirements configMap: # Provide the name of the ConfigMap containing the files you want # to add to the container name: requirements
After that, all extra Python packages were installed.