Allow separate object and pod labels for chart components

[vyncint]

Add podLabels so workload object labels and pod labels can be configured separately, for apiServer, scheduler, workers.celery, triggerer, dagProcessor, flower, statsd, pgbouncer, redis and otelCollector.

When podLabels is unset, existing behavior remains unchanged: <component>.labels stays on the pods, while the Deployment or StatefulSet receives only global labels. Setting podLabels (including {}) opts into separation: <component>.labels applies to the workload object and <component>.podLabels applies to pods. Component values retain precedence over global labels.

otelCollector is the one exception: its Deployment already merged component labels onto the object, so only its pod side gains the new knob and its object labels are unchanged.

closes: #53190

Behaviour matrix, verified by rendering every affected workload:

podLabels	workload object labels	pod labels
unset (default ~)	global only — unchanged	component + global
set	component + global	podLabels + global
{}	component + global	global only

Tests:

• pytest helm_tests/airflow_aux/test_pod_labels.py helm_tests/dagprocessor — 38 passed. The new module parametrizes all three states over every component; dagProcessor keeps its existing dedicated module.
• pytest helm_tests/{dagprocessor,statsd,otel_collector,redis} helm_tests/airflow_core/test_{api_server,scheduler,triggerer,worker}.py helm_tests/other/test_{flower,pgbouncer}.py — 779 passed, 1 failed.
• pytest helm_tests/airflow_aux/test_{chart_quality,basic_helm_chart,annotations}.py — 114 passed, 9 failed.
• Backward compatibility: helm template with every component's labels set and podLabels unset is byte-identical to main across 3638 lines (random secrets pinned so the render is deterministic).

The 10 local failures are pre-existing and unrelated: they assert on Helm's JSON-schema error wording (value must be one of 'X') while local Helm 3.14.4 emits must be one of the following: "X", and they cover dags.persistence.accessMode, *.pullPolicy and apiServer.securityContext. Re-running the same modules on a clean checkout reproduces exactly the same counts, so this change adds no new failures.

---

Was generative AI tooling used to co-author this PR?

• Yes — Codex (GPT-5) for the initial dagProcessor commit, Claude Opus 5 for this revision

Generated-by: Codex (GPT-5), Claude Opus 5 following the guidelines

[boring-cyborg]

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide
Here are some useful points:

• Pay attention to the quality of your code (ruff, mypy and type annotations). Our prek-hooks will help you with that.
• In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example Dag that shows how users should use it.
• Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
• Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
• Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
• Be sure to read the Airflow Coding style.
• Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
  Apache Airflow is a community-driven project and together we are making it better 🚀.
  In case of doubts contact the developers at:
  Mailing List: dev@airflow.apache.org
  Slack: https://s.apache.org/airflow-slack

[Copilot]

Pull request overview

This PR aligns the Helm chart behavior with the documented contract that dagProcessor.labels applies to Dag processor Kubernetes objects (not just the Pods), by also merging those component-specific labels into the Dag Processor Deployment metadata labels while keeping component-over-global precedence.

Changes:

• Merge .Values.dagProcessor.labels with .Values.labels for Deployment.metadata.labels in the dag-processor deployment template.
• Add a regression test covering global-only, component-only, and colliding label keys for Deployment.metadata.labels.
• Add a chart newsfragment documenting the user-visible Helm chart enhancement.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File	Description
chart/templates/dag-processor/dag-processor-deployment.yaml	Applies the same global+component label merge logic to Deployment metadata labels as already used for Pod template labels.
chart/tests/helm_tests/dagprocessor/test_labels_deployment.py	Adds coverage to ensure merged labels appear on the Deployment metadata and component labels override global keys.
chart/newsfragments/53190.feature.rst	Records the chart feature change in release notes.

[Miretpl]

Could you please check discussions from previous proposed PRs for the relevant change #59290?

[Miretpl]

Requesting changes to block accidential merge

[vyncint]

Thanks @Miretpl — I read through #59290 and understand the hold now. That PR made the same change for the webserver and stalled on a design question that was never settled, so rather than re-land it for dag-processor I'd like a direction first.

Today per-component labels (e.g. dagProcessor.labels) only land on the pod template (spec.template.metadata.labels); the Deployment's own metadata.labels gets global labels only. Two ways to close that gap:

A) Merge component labels onto the Deployment metadata directly (what this PR does now). Simple, but it changes where existing users' labels land, so it's a backward-compatibility concern.

B) Add a per-component podLabels section (like the existing annotations / podAnnotations split): labels go to the Deployment, podLabels to the pod, and when podLabels is unset, labels fall back to the pod level to preserve today's behavior. This is the approach Miretpl proposed in #59290.

On scope, #59290 also concluded this should be applied across all components rather than one at a time, and target apiServer rather than webserver (webserver is going away with Airflow 2.x). Happy to broaden this PR accordingly.

Could a committer confirm (1) approach A or B, and (2) whether you want the full multi-component change in one PR? I'll rework and rebase onto main once there's a direction. cc @jscheffl @potiuk @jedcunningham @romsharon98

[Miretpl]

My idea #59290 (comment) after becoming a Committer (I wasn't back then) is still the same. Separation, like in annotations, gives more flexibility, and I see cases when it would be nice to separate labels between the pod and the deployment itself. And if there will be a user who will want to have the same labels, there will be a possibility to set the same value on both levels.

[vyncint]

Thanks @Miretpl — agreed, the separation is the cleaner model. I'll go with podLabels:

• labels -> Deployment metadata.labels
• podLabels -> pod spec.template.metadata.labels
• backward compatible: when podLabels is unset, labels stay on the pod level (today's behavior) and the Deployment keeps only global labels.

I'll implement it on dag-processor here first as the reference pattern (values.yaml + template + tests), and once you're happy with it extend the same pattern to the other components, using apiServer rather than webserver. Prefer everything in one PR, or dag-processor first then follow-ups?

[Miretpl]

I'll implement it on dag-processor here first as the reference pattern (values.yaml + template + tests), and once you're happy with it extend the same pattern to the other components, using apiServer rather than webserver. Prefer everything in one PR, or dag-processor first then follow-ups?

In general small PRs are better, but whole thing will rather not be big, so I will adjust. Do as it will be more comfortable for you.

using apiServer rather than webserver

The Webserver will also be needed, but for the 1.2x line (on main we have 2.0) only.

[vyncint]

Implemented the podLabels split requested in the review and earlier discussion:

• Leaving dagProcessor.podLabels unset preserves the existing behavior: dagProcessor.labels applies to pods, while the Deployment receives global labels only.
• Setting dagProcessor.podLabels (including {}) opts into separation: dagProcessor.labels applies to the Deployment and dagProcessor.podLabels applies to pods.
• Component labels keep precedence over global labels, and deepCopy prevents mustMerge from mutating the source values.

I also added values/schema documentation, a news fragment, and regression coverage for the fallback, separated-label, collision, and explicit-empty-map cases. The branch is rebased on current main.

Validation completed:

• breeze testing helm-tests --use-xdist — 2095 passed
• prek run --from-ref upstream/main --stage pre-commit
• prek run --from-ref upstream/main --stage manual
• breeze ci selective-check --commit-ref HEAD

---

Drafted-by: Codex (GPT-5) (no human review before posting)

[vyncint]

Verified on the rebased branch:

• podLabels unset — unchanged behavior (Deployment: global only, pods: component + global)
• podLabels set — Deployment: component + global, pods: podLabels + global
• podLabels: {} — pods get global only
• collisions — component wins on the Deployment, podLabels wins on pods

Backward compatibility: rendering the whole chart with the same values against current main and against this branch gives byte-identical output.

Tests: 7 passed in test_labels_deployment.py, 125 passed across the dagprocessor, redis, statsd and otel_collector helm suites.

Ready for review @Miretpl.

[vyncint]

@Miretpl extended the pattern to the remaining components in 177dc55: apiServer, scheduler, workers.celery, triggerer, flower, statsd, pgbouncer, redis and otelCollector. Title and description updated to match the wider scope.

otelCollector turned out to be the one exception. Its Deployment already merged component labels onto the object, so there is no backward-compatibility branch to preserve there — only its pod side gains the knob, and its object labels are unchanged. Every other component follows the dagProcessor shape exactly.

Verification:

• helm template with every component's labels set and podLabels unset is byte-identical to main across 3638 lines, with the random secrets pinned so the render is deterministic. That is the backward-compatibility proof.
• New parametrized module helm_tests/airflow_aux/test_pod_labels.py covers all three states (unset, set, {}) for every component: 38 passed together with the existing dagProcessor module. I put the new cases in one parametrized module rather than nine near-identical files, following test_annotations.py; happy to split them per component if you prefer that layout.
• Component suites for statsd, otel_collector, redis, api_server, scheduler, triggerer, worker, flower and pgbouncer: 779 passed, 1 failed.
• Chart quality, basic chart and annotations modules: 114 passed, 9 failed.

The 10 failures are pre-existing and unrelated. They assert on Helm's JSON-schema error wording, and local Helm 3.14.4 phrases it differently from what the tests expect; they cover dags.persistence.accessMode, pullPolicy and apiServer.securityContext. Running the same modules on a clean checkout of the base gives exactly the same counts, so nothing here is new. CI runs a pinned Helm, so I expect them green there.

[vyncint]

Hi @Miretpl,

Following up on this. The podLabels split you asked for is in, and I extended it to the remaining components in 177dc55: apiServer, scheduler, workers.celery, triggerer, flower, statsd, pgbouncer, redis and otelCollector.

Could you take another look when you have a moment? The changes-requested review is still blocking merge.

[Miretpl]

@vyncint Most of the people here are doing work voluntarily, so getting feedback may take some time, especially in the AI era where it is easy to generate a PR, but still people have to take a look at it if it is aligned with the project. Sooner or later, I or some other maintainer will take another look at the PR and give another round of feedback. Please do not mention people when the PR is stale for a day or two. If it is stale for around 2 weeks, I think it is a good time period to try to ping any maintainer for the review.

[github-actions]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.