cloudflare-worker: take back the labels that stopped being true

The bot applied "add package", "drop package" and release/<version>
whenever the condition held, and never took any of them off again. A
branch that dropped its new package during review, or was retargeted
from one release to another, kept a label saying otherwise - and the
nightly scan and everyone reading the list of pull requests believed
it. Only "stale" and the guidelines label were ever withdrawn.

All three are read straight off the pull request's own content, so the
bot now withdraws them on the same terms it applies them. Labels
matched from labeler.yml are deliberately left alone, the way GitHub's
own labeler action leaves them by default: a path label is as often put
on by hand as derived, and taking those back would fight whoever set
them.

A removal names the label the way the repository spells it rather than
the way the constant is written - openwrt/packages calls it "Add
package" - which the comparison, being case-insensitive, hid until now.

Deploying is also held back until the tests pass. deploy.yml and
tests.yml both triggered on a push to main, so they started in the same
second and raced: the Worker went live whether or not the suite ever
went green.

Signed-off-by: Josef Schlehofer <pepe.schlehofer@gmail.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Author: BKPepe

.github/workflows/deploy.yml

@@ -9,8 +9,31 @@ permissions:
   contents: read
 
 jobs:
+  # The tests also run from tests.yml on the same push, but a separate
+  # workflow cannot hold this one back: both start in the same second and the
+  # Worker would go live whether or not the suite ever went green. Running
+  # them here as well is what makes the deployment wait for an answer.
+  test:
+    name: Run Tests
+    runs-on: ubuntu-slim
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v7
+
+      - name: Setup Node.js
+        uses: actions/setup-node@v6
+        with:
+          node-version: 24
+
+      - name: Run Unit Tests
+        run: npm test
+        working-directory: cloudflare-worker
+        env:
+          FORCE_COLOR: 1
+
   deploy:
     name: Deploy Worker
+    needs: test
     runs-on: ubuntu-slim
     environment:
       name: production

README.md

@@ -69,8 +69,8 @@ Scans the contribution tree for nested downstream patch targets:
 ### Automated Triage & Stale PR Management
 
 *   **`not following guidelines`**: A high-visibility tag automatically attached to the PR if any critical validation check drops a failure blueprint. Clears itself upon a successful push.
-*   **`add package` / `drop package`**: Dynamically analyzes unified diff targets to label tracking trees introducing or purging software packages.
-*   **Stable Branch Tracking**: Auto-generates matching grey release tags (e.g., `release/24.10`, `release/25.12`) whenever a PR targets an active release backport branch.
+*   **`add package` / `drop package`**: Dynamically analyzes unified diff targets to label tracking trees introducing or purging software packages. Both are withdrawn again once the branch stops adding or dropping a package, so a label never outlives what it describes.
+*   **Stable Branch Tracking**: Auto-generates matching grey release tags (e.g., `release/24.10`, `release/25.12`) whenever a PR targets an active release backport branch, and removes a release tag that no longer matches after the pull request is retargeted. Labels matched from `labeler.yml` are only ever added, the way GitHub's own labeler action leaves them by default — a path label is as often set by hand as derived.
 *   **Issue Labeller**: Replaces the GitHub Actions `issue-labeller.yml` workflow. When a bug-report issue is opened with the trigger label, the bot validates form fields and applies labels based on a declarative `.github/issue-labeller.yml` configuration file (same spirit as `labeler.yml` for PRs — label name → list of conditions). Supports template variables (`{major}`, `{segment0}`, etc.), format validation (regex), existence checks (tag/path via GraphQL), substring matching, and presence checks. Falls back to sensible defaults if no config file exists. Disabled by default — enable per-repository with `"enable_issue_labeller": true`.
 *   **Stale PR Cleanup**: A daily scheduled cron task (05:30 UTC) scans all repositories where the App is installed. If explicitly enabled in a repository's configuration (\`"enable_stale_bot": true\`), it marks PRs containing the \`not following guidelines\` label as \`stale\` (with a warning comment) after 14 days of inactivity, and closes them after another 14 days of silence. Only contributor activity resets the countdown: pushed commits, force-pushes, reopens, and comments or reviews from people. Comments from GitHub Apps, `*[bot]` accounts and the machine accounts listed in `stale_ignored_users` are ignored, so an automated review can never keep a dead PR alive forever. Pushing new commits also removes the `stale` label immediately via the webhook, without waiting for the nightly scan. The scan asks GitHub for each repository's configuration and its labelled pull requests in two GraphQL queries, timelines included, so its request count no longer grows with the number of stale pull requests. A pull request whose timeline is longer than the fetched window (the most recent 100 entries) is left untouched rather than judged on incomplete history.
 

cloudflare-worker/src/index.js

@@ -1472,11 +1472,16 @@ async function handleWebhook(request, env) {
   }
 
   const currentPrLabels = new Set((data.pull_request?.labels || []).map(l => l.name.toLowerCase()));
+  // Comparisons are case-insensitive, but a removal has to name the label the
+  // way the repository spells it: openwrt/packages calls it "Add package",
+  // and the constant here is lower case.
+  const currentPrLabelNames = new Map((data.pull_request?.labels || []).map(l => [l.name.toLowerCase(), l.name]));
+  const spelledAsOnPr = (name) => currentPrLabelNames.get(name.toLowerCase()) || name;
 
   // New commits or a reopen are contributor activity: drop the stale marker
   // right away instead of waiting for the nightly scan to notice it.
   if ((data.action === 'synchronize' || data.action === 'reopened') && currentPrLabels.has('stale')) {
-    labelOperations.push(() => removeLabel('stale'));
+    labelOperations.push(() => removeLabel(spelledAsOnPr('stale')));
   }
 
   if (!allPassed) {
@@ -1487,27 +1492,49 @@ async function handleWebhook(request, env) {
   } else {
     // Delete validation failure label if present
     if (currentPrLabels.has(LABEL_GUIDELINES.toLowerCase())) {
-      labelOperations.push(() => removeLabel(LABEL_GUIDELINES));
+      labelOperations.push(() => removeLabel(spelledAsOnPr(LABEL_GUIDELINES)));
     }
   }
 
+  // The three labels below are read straight off the pull request's own
+  // content, so the bot both applies and withdraws them: a branch that
+  // dropped its new package, or was retargeted to another release, would
+  // otherwise keep a label that stopped being true. Labels matched from
+  // labeler.yml are left alone, the way GitHub's own labeler action leaves
+  // them by default - a path label is as often put on by hand as derived,
+  // and taking those back would fight whoever set them.
   if (CONFIG.add_package_label && state.isNewPackage && !currentPrLabels.has(LABEL_ADD_PACKAGE.toLowerCase())) {
     labelOperations.push(() => ensureLabel(LABEL_ADD_PACKAGE, '0e7490', 'Introduces a new package Makefile build script'));
     labelsToAdd.push(LABEL_ADD_PACKAGE);
   }
 
+  if (CONFIG.add_package_label && !state.isNewPackage && currentPrLabels.has(LABEL_ADD_PACKAGE.toLowerCase())) {
+    labelOperations.push(() => removeLabel(spelledAsOnPr(LABEL_ADD_PACKAGE)));
+  }
+
   if (CONFIG.drop_package_label && state.isDroppedPackage && !currentPrLabels.has(LABEL_DROP_PACKAGE.toLowerCase())) {
     labelOperations.push(() => ensureLabel(LABEL_DROP_PACKAGE, '3b82f6', 'Removes an existing package Makefile from the tracking tree'));
     labelsToAdd.push(LABEL_DROP_PACKAGE);
   }
 
-  if (CONFIG.branch_labeling && /^openwrt-\d{2}\.\d{2}$/.test(baseBranch)) {
-    const version = baseBranch.split('-')[1];
-    const labelName = `release/${version}`;
-    if (!currentPrLabels.has(labelName.toLowerCase())) {
+  if (CONFIG.drop_package_label && !state.isDroppedPackage && currentPrLabels.has(LABEL_DROP_PACKAGE.toLowerCase())) {
+    labelOperations.push(() => removeLabel(spelledAsOnPr(LABEL_DROP_PACKAGE)));
+  }
+
+  if (CONFIG.branch_labeling) {
+    const releaseMatch = baseBranch.match(/^openwrt-(\d{2}\.\d{2})$/);
+    const labelName = releaseMatch ? `release/${releaseMatch[1]}` : null;
+    if (labelName && !currentPrLabels.has(labelName.toLowerCase())) {
       labelOperations.push(() => ensureLabel(labelName, '6b7280', `Pull request targets the stable release branch ${labelName}`));
       labelsToAdd.push(labelName);
     }
+    // Retargeting a pull request to another branch leaves the old release
+    // label behind, saying it goes somewhere it no longer goes.
+    for (const existing of currentPrLabels) {
+      if (/^release\/\d{2}\.\d{2}$/.test(existing) && existing !== labelName?.toLowerCase()) {
+        labelOperations.push(() => removeLabel(spelledAsOnPr(existing)));
+      }
+    }
   }
 
   if (CONFIG.enable_labeler_yml && labelerText) {

cloudflare-worker/test/index.test.js

@@ -5896,6 +5896,99 @@ diff --git a/utils/mypkg/patches/00${i}-fix.patch b/utils/mypkg/patches/00${i}-f
     }
   });
 
+  // Runs one webhook on a pull request that already carries the given labels,
+  // and reports which labels the run took off.
+  async function labelsRemovedFor({ labels, baseRef = 'main', patch }) {
+    const payload = JSON.stringify({
+      action: 'synchronize',
+      pull_request: {
+        number: 123, title: 'mypkg: update to 1.2.3', body: 'Update',
+        labels: labels.map(name => ({ name })),
+        base: { ref: baseRef, sha: 'basesha' }, head: { ref: 'feature-branch', sha: 'headsha' },
+        user: { login: 'johndoe', type: 'User' },
+        commits_url: 'https://api.github.com/repos/test/repo/pulls/123/commits',
+        url: 'https://api.github.com/repos/test/repo/pulls/123'
+      },
+      installation: { id: 456 }, repository: { full_name: 'test/repo' }
+    });
+    const secret = 'mysecret';
+    const signature = await calculateHmac(secret, payload);
+    const removed = [];
+
+    fetchMock = async (url, options) => {
+      const method = options?.method || 'GET';
+      if (url.includes('/access_tokens')) return new Response(JSON.stringify({ token: 'mocktoken' }), { status: 200 });
+      if (url.includes('/formalities.json')) {
+        return new Response(JSON.stringify({ check_branch: false, enable_comments: false, require_linked_github_account: false, require_body: false, check_uci_config: false, check_pkg_release: false }), { status: 200 });
+      }
+      { const lr = graphqlLabelsHandler(url, options, labels); if (lr) return lr; }
+      if (url.includes('/pulls/123/commits')) {
+        return new Response(JSON.stringify([{
+          sha: 'sha123', html_url: 'https://github.com/test/repo/commit/sha123',
+          commit: { message: 'mypkg: update to 1.2.3\n\nA description of the change.\n\nSigned-off-by: John Doe <john@doe.com>', author: { name: 'John Doe', email: 'john@doe.com' }, committer: { name: 'John Doe', email: 'john@doe.com' } }
+        }]), { status: 200 });
+      }
+      if (url.match(/\/repos\/test\/repo\/commits\/sha123/)) return new Response(patch, { status: 200 });
+      if (url.includes('/issues/123/labels/') && method === 'DELETE') {
+        removed.push(decodeURIComponent(url.split('/labels/')[1]));
+        return new Response('[]', { status: 200 });
+      }
+      if (url.includes('/issues/123/comments')) return new Response(JSON.stringify([]), { status: 200 });
+      return new Response(JSON.stringify({}), { status: 201 });
+    };
+
+    const originalImportKey = crypto.subtle.importKey;
+    crypto.subtle.importKey = async (format, keyData, algorithm, extractable, keyUsages) =>
+      algorithm.name === 'RSASSA-PKCS1-v1_5' ? { type: 'private', extractable: false, algorithm, usages: keyUsages }
+        : originalImportKey.call(crypto.subtle, format, keyData, algorithm, extractable, keyUsages);
+    const originalSign = crypto.subtle.sign;
+    crypto.subtle.sign = async (algorithm, key, data) =>
+      algorithm === 'RSASSA-PKCS1-v1_5' ? new ArrayBuffer(256) : originalSign.call(crypto.subtle, algorithm, key, data);
+
+    try {
+      const response = await worker.fetch(new Request('http://localhost/webhook', {
+        method: 'POST', body: payload,
+        headers: { 'x-hub-signature-256': signature, 'x-github-event': 'pull_request' }
+      }), { WEBHOOK_SECRET: secret, APP_ID: '12345', PRIVATE_KEY: 'YW55Y29udGVudA==' }, {});
+      assert.strictEqual(response.status, 200, await response.text());
+      return removed;
+    } finally {
+      crypto.subtle.importKey = originalImportKey;
+      crypto.subtle.sign = originalSign;
+      fetchMock = null;
+    }
+  }
+
+  const plainPatch = 'diff --git a/README b/README\n--- a/README\n+++ b/README\n@@ -1 +1 @@\n-a\n+b\n';
+
+  test('takes back the "add package" label when the branch no longer adds one', async () => {
+    // Spelled the way openwrt/packages spells it, which is not how the
+    // constant in the bot is written.
+    const removed = await labelsRemovedFor({ labels: ['Add package'], patch: plainPatch });
+    assert.deepStrictEqual(removed, ['Add package']);
+  });
+
+  test('keeps the "add package" label while the branch still adds one', async () => {
+    const addsPackage = 'diff --git a/utils/mypkg/Makefile b/utils/mypkg/Makefile\nnew file mode 100644\n--- /dev/null\n+++ b/utils/mypkg/Makefile\n@@ -0,0 +1,2 @@\n+include $(TOPDIR)/rules.mk\n+PKG_NAME:=mypkg\n';
+    const removed = await labelsRemovedFor({ labels: ['Add package'], patch: addsPackage });
+    assert.deepStrictEqual(removed, []);
+  });
+
+  test('takes back a release label after the pull request is retargeted', async () => {
+    const removed = await labelsRemovedFor({ labels: ['release/24.10'], baseRef: 'openwrt-25.12', patch: plainPatch });
+    assert.deepStrictEqual(removed, ['release/24.10']);
+  });
+
+  test('keeps the release label that matches the base branch', async () => {
+    const removed = await labelsRemovedFor({ labels: ['release/25.12'], baseRef: 'openwrt-25.12', patch: plainPatch });
+    assert.deepStrictEqual(removed, []);
+  });
+
+  test('leaves a label matched from labeler.yml alone', async () => {
+    const removed = await labelsRemovedFor({ labels: ['target/ath79'], patch: plainPatch });
+    assert.deepStrictEqual(removed, [], 'a path label is as often set by hand as derived');
+  });
+
   test('labeler.yml integration: handles missing (404) .github/labeler.yml gracefully', async () => {
     const originalImportKey = crypto.subtle.importKey;
     crypto.subtle.importKey = async (format, keyData, algorithm, extractable, keyUsages) => {