I have a google cloud composer environment. In my DAG I want to create a pod in GKE. When I come to deploy a simple app based on a docker container that doesn’t need any volume configuration or secrets, everything works fine, for example: But when I have an application that need to access to my GKE cluster volumes, I
Tag: google-cloud-composer
How to control the parallelism or concurrency of an Airflow installation?
In some of my Apache Airflow installations, DAGs or tasks that are scheduled to run do not run even when the scheduler doesn’t appear to be fully loaded. How can I increase the number of DAGs or tasks that can run concurrently? Similarly, if my installation is under high load and I want to limit how quickly my Airflow workers