![]() ![]() The core part of building a docker image is doing a pip install. It ensures maximum utilization of resources, unlike celery, which at any point must have a minimum number of workers running. Using KEDA with Airflow KEDA enables elastic from and to zero scaling of Airflow CeleryWorkers on Kubernetes Code samples and configuration can be found here. On completion of the task, the pod gets killed. The KubernetesExecutor natively runs any task in your DAG as a pod on Kubernetes. On scheduling a task with airflow Kubernetes executor, the scheduler spins up a pod and runs the tasks.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |