Skip to content

Commit 68c17db

Browse files
remove references to workers.celery.instances
Signed-off-by: Mathew Wicks <[email protected]>
1 parent a10ee66 commit 68c17db

File tree

2 files changed

+4
-9
lines changed

2 files changed

+4
-9
lines changed

charts/airflow/README.md

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -624,6 +624,10 @@ For a worker pod you can calculate it: `WORKER_CONCURRENCY * 200Mi`, so for `10
624624
In the following config if a worker consumes `80%` of `2Gi` (which will happen if it runs 9-10 tasks at the same time), an autoscaling event will be triggered, and a new worker will be added.
625625
If you have many tasks in a queue, Kubernetes will keep adding workers until maxReplicas reached, in this case `16`.
626626
```yaml
627+
airflow:
628+
config:
629+
AIRFLOW__CELERY__WORKER_CONCURRENCY: 10
630+
627631
workers:
628632
# the initial/minimum number of workers
629633
replicas: 2
@@ -649,8 +653,6 @@ workers:
649653
averageUtilization: 80
650654
651655
celery:
652-
instances: 10
653-
654656
## wait at most 9min for running tasks to complete before SIGTERM
655657
## WARNING:
656658
## - some cloud cluster-autoscaler configs will not respect graceful termination

charts/airflow/examples/google-gke/custom-values.yaml

Lines changed: 0 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -228,13 +228,6 @@ workers:
228228
## configs for the celery worker Pods
229229
##
230230
celery:
231-
## the number of tasks each celery worker can run at a time
232-
##
233-
## NOTE:
234-
## - sets AIRFLOW__CELERY__WORKER_CONCURRENCY
235-
##
236-
instances: 10
237-
238231
## if we should wait for tasks to finish before SIGTERM of the celery worker
239232
##
240233
gracefullTermination: true

0 commit comments

Comments
 (0)