Skip to content

osspckgs: pg_partman is registered but maintenance is never scheduled; repo_docker_pulls_daily runs out of partitions around 2026-10-01 #4565

Description

@AlexeyShalaev

What

The osspckgs migrations register three tables with pg_partman, but nothing ever runs pg_partman maintenance, so each table has a fixed runway of pre-created partitions that never advances. When a table reaches the end of its runway, rows land in its default partition — silently, no error — and from that point the partitions covering them can no longer be created.

repo_docker_pulls_daily reaches that point around 2026-10-01.

The scheduling step is missing

V1779710880__initial_schema.sql deferred the pg_partman setup and wrote down exactly what it needed, in a comment:

--   SELECT partman.create_parent(... p_premake => 3 ...);
--
--   -- pg_cron job to maintain partitions (also needs pg_cron enabled in OCI):
--   SELECT cron.schedule('partman-maintain', '0 1 * * *',
--       $$CALL partman.run_maintenance_proc()$$);
--
-- Without this setup, inserts into downloads_daily will fail with
-- "no partition found for row".

V1780231200__npm_worker.sql then implemented the first half — create_parent for downloads_daily and downloads_last_30d — and V1780928852__dockerhub_sync.sql did the same for repo_docker_pulls_daily. The cron.schedule half was never added.

Searching the repository for run_maintenance, pg_partman_bgw or pg_cron finds nothing outside those comments, and scripts/packages-db/Dockerfile installs postgresql-14-partman without configuring shared_preload_libraries = 'pg_partman_bgw'. So create_parent's initial premake window is all there will ever be.

The runway, per table

pg_partman's premake creates the current period plus N ahead (and N behind). Running your own create_parent call for repo_docker_pulls_daily verbatim today produces partitions from 2026-06-01 through 2026-12-31, plus a _default — current month ±3.

Applying that to the dates the migrations actually ran:

Table premake migration partitions run out
repo_docker_pulls_daily 3 2026-06-08 ~2026-10-01
downloads_daily 12 2026-05-31 ~2027-05
downloads_last_30d 3 2026-05-31 ~2026-09 (yearly interval — check yours)

What it actually looks like on your database:

SELECT c.relname AS parent,
       max((regexp_match(pg_get_expr(p.relpartbound, p.oid), 'TO \(''([^'']+)''\)'))[1]::date) AS covered_through
FROM pg_class c
JOIN pg_inherits i ON i.inhparent = c.oid
JOIN pg_class p ON p.oid = i.inhrelid
WHERE c.relkind = 'p'
  AND pg_get_expr(p.relpartbound, p.oid) <> 'DEFAULT'
GROUP BY 1;

Why it will not announce itself

The comment above predicts no partition found for row, which would at least be loud. That is not what happens, because pg_partman creates a default partition: an insert past the end of the runway simply succeeds into it.

INSERT INTO repo_docker_pulls_daily VALUES ('img', '2027-03-15', 1);
-- INSERT 0 1
-- rows in repo_docker_pulls_daily_default: 1

So the first symptom is not an error — it is one unpartitioned table quietly absorbing every subsequent row, with the query performance and retention behaviour that implies.

And it compounds, so the fix is cheaper now than later

Once rows for a window are sitting in the default partition, PostgreSQL refuses to attach a partition covering that window:

ERROR:  updated partition constraint for default partition "..._default"
        would be violated by some row

That means adding the scheduler after the runway has lapsed does not repair it — run_maintenance will fail for that parent until the default partition is drained by hand (detach it, create the covering partitions explicitly, move the rows back, reattach — all in one transaction). Adding the scheduler before 2026-10-01 avoids that entirely.

Fix

Any one of these; the first is the line already written in your own comment:

  • SELECT cron.schedule('partman-maintain', '0 1 * * *', $$CALL partman.run_maintenance_proc()$$); if pg_cron is available;
  • shared_preload_libraries = 'pg_partman_bgw' plus pg_partman_bgw.interval in scripts/packages-db/Dockerfile, which makes it work out of the box for anyone running that image;
  • a Temporal schedule calling run_maintenance_proc(), since you already run Temporal workers and that keeps it in code rather than in database configuration.

Whichever it is, worth also raising p_premake on the two tables sitting at 3: it is the margin for the scheduler being down, and 3 months of margin costs almost nothing in empty partitions.


Disclosure: I maintain a PostgreSQL partition-lifecycle library, which is why I was reading pg_partman setups across projects and found this. Nothing here involves it — the fix is a line of pg_cron you have already written.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions