48525 backend [ backend ] Upgrade Python version to 3.8.

[andrey-usov]

Problem

The backend runs on a Python release that stopped receiving security updates in June 2023, and on a Django release that predates 28 patch releases of security fixes. Any vulnerability found in the runtime or the framework stays unpatched, and the outdated runtime blocks upgrades of the libraries the product depends on, which slows down delivery of everything else. This is the first step of moving the platform onto a supported runtime.

Fix / Solution

Moved the backend runtime from Python 3.7.5 to 3.8.20, the final release of the 3.8 series, and Django from 2.2 to 2.2.28, the final 2.2 LTS patch, which is the first Django release with official Python 3.8 support.

• backend/Dockerfile base image python:3.7.11-bullseye → python:3.8.20-bullseye; .python-version and the pre-commit language version follow.
• django = "2.2" → django = "2.2.28", python = "^3.7.5" → python = "~3.8.20".
• Dropped 7 direct dependencies with zero usages in backend/src: recurly, ibm-cos-sdk, xmltodict, defusedxml, dj-database-url, python-slugify, iso8601. Also dropped the direct importlib-metadata pin, redundant because importlib.metadata is part of the standard library from 3.8 on.
• Relocked with poetry lock --no-update to keep the diff reviewable: 16 packages left poetry.lock and only Django changed version. Among the removed are the Python 2 backports futures, backports-functools-lru-cache and backports-ssl-match-hostname, which the removed SDKs were pulling in.
• backend/ruff.toml target-version py37 → py38. The only new finding it exposes is one UP035 in a test importing OrderedDict from typing_extensions; it now comes from typing. ruff check and pycodestyle are clean.

No application code was touched apart from that single import, and no cosmetic changes are included: the whole diff is 6 config files plus poetry.lock.

Verified on the built image: Python 3.8.20, Django 2.2.28, all C-extensions install without compiling from source on amd64 (psycopg2-binary, uvloop, httptools, websockets, twisted), importlib.metadata resolves from the standard library, and pytest collects 6161 tests with no collection errors.

Release notes

No user-facing changes. The service moves to a supported Python runtime and to the latest security patch of the current Django release, closing known vulnerabilities in the framework, including the admin privilege escalation CVE-2019-19118 and the SQL injections CVE-2021-35042, CVE-2022-28346 and CVE-2022-28347.

One API behaviour change comes with those patches. As part of the fix for CVE-2021-23336 (web cache poisoning), Django 2.2.18 stopped treating a semicolon as a query string parameter separator: QueryDict now splits on & only. A request such as GET /workflows?fields=api-name-1;template_id=5 used to be read as two filters and is now read as a single fields value of api-name-1;template_id=5. Standard &-separated requests are unaffected, and neither the web client nor any other project code builds semicolon-separated query strings.

Changes

API

No endpoints, serializers or business logic changed. Configuration only:

• backend/Dockerfile — base image
• backend/pyproject.toml, backend/poetry.lock — Python constraint, Django version, removed dependencies
• backend/ruff.toml — linter target version
• backend/README.md — local setup instructions
• .python-version, .pre-commit-config.yaml — local and hook runtime
• backend/src/processes/tests/test_webhooks/test_webhooks.py — OrderedDict imported from typing
• backend/src/processes/tests/test_views/test_workflow/test_list.py — 4 tests combined the fields and template_id filters with ;, which Django 2.2.28 no longer accepts as a separator; switched to &

Web-client

No changes.

Local verification

The stack was brought up from source on the new runtime (docker compose -f docker-compose.src.yml up -d --build backend celery celery-beat), which also exercises migrate, collectstatic, init_periodic_tasks and compilemessages on startup.

Check	Result
Versions inside the container	python 3.8.20, django 2.2.28
Containers	backend healthy, celery and celery-beat up
Unapplied migrations	0
gunicorn + uvicorn worker	starts, 2 workers, Application startup complete
/admin/login/	200
/api/schema (drf-spectacular)	200, 443 KB, openapi: 3.0.3
/workflows, /accounts/users, /templates, /v3/tasks with a token	200 with real data
WebSocket /ws/events	101 Switching Protocols
Celery worker	pong, 110 tasks processed, queue drained, no tracebacks
Celery beat	running, 11 periodic tasks, all enabled
Automated tests	6161 passed, 0 failed, run module by module across 29 modules (~1 h 49 min)
linux/arm64 image	builds and runs: Python 3.8.20, Django 2.2.28, all C-extensions import on aarch64

The full suite was run module by module rather than in a single invocation, so a failure points at a module immediately. The per-module totals add up to 6161, which matches pytest --collect-only, so the split covers every test. Longest modules: processes/test_services 1063 tests in 27 min, processes/test_views/test_tasks 330 in 24 min, processes/test_views/test_workflow 536 in 17 min.

The linux/arm64 image was built locally under qemu emulation to get a pre-merge signal, since CI only builds multi-arch on push to master. It succeeds without touching the Dockerfile: psycopg2-binary 2.8.5, uvloop 0.14.0, httptools 0.1.1, websockets 8.1 and twisted 20.3.0 have no aarch64 wheels for cp38 and are compiled from sdist, which the base image already supports — python:3.8.20-bullseye is the full variant and ships gcc 10.2.1 and libpq-fe.h. The same packages have no aarch64 wheels for cp37 either, so CI has been compiling them from source all along; this is not a change introduced by the upgrade.

Two things reviewers and QA should know about, neither introduced by this PR:

• New warning in the backend log on Python 3.8. Every worker boot now logs RuntimeWarning: line buffering (buffering=1) isn't supported in binary mode, the default buffer size will be used from os.py. gunicorn 19.9.0 opens a file descriptor in binary mode with line buffering, and Python 3.8 started warning about it; 3.7 did not. No functional impact, buffering just falls back to the default. It goes away only with gunicorn 20+, which is out of scope here. Please do not read it as a regression.
• Pre-existing migration drift. makemigrations --check --dry-run reports pending migrations in accounts (Meta on apikey, max_users on account), logs (event_type on accountevent), permissions (indexes and a constraint on userobjectpermission / groupobjectpermission) and processes (choices on workflowevent.type). The output is byte-for-byte identical on master, so the drift predates this branch. All of it is field metadata, no DDL. Left untouched here to keep the upgrade free of unrelated migrations.

Test cases

Django 2.2.28 changes admin inline permissions (CVE-2019-19118): an inline becomes read-only when the user only holds view on the parent model. That is the one behaviour change a user can notice, so admin scenarios come first.

API

Authorization	Test case	Expected result
Superuser	Open admin pages with inlines: users, accounts, groups, contacts, prices, menu items, AI prompts	Pages open, inline rows render, saving the parent object persists inline changes
Staff with view only on the parent model	Open the same admin pages	Inlines render read-only, no 500 error
Staff with change on the parent model	Edit and save an inline row	Change is saved
Account owner	Trigger an outgoing webhook: complete a task, complete a workflow	Webhook is delivered, JSON payload keeps the previous shape and date format
Customer.io	Send a webhook with a valid signature to the Customer.io endpoint	Signature passes, event is processed
Customer.io	Send a webhook with a broken signature	Rejected, no event created
Account owner	Create a template with kickoff and output fields of every type, then run it	Fields are parsed and validated, no errors on save or on run
Account owner	Complete a task with attachments, dropdown, date and user fields	Values are saved and returned in the task response unchanged
Anonymous	Sign up with reCAPTCHA enabled, then reset the password	Both flows pass, emails are queued
Anonymous	Follow each one-time link: password reset, email verification, unsubscribe, user transfer, payment confirmation	Each token is accepted, expired tokens are rejected with the previous error
Okta user	Sign in through Okta, then trigger back-channel logout from Okta	Sign-in succeeds, forced logout invalidates the session
Google, Microsoft, Auth0 user	Sign in through each provider	Sign-in succeeds
Account owner	Open the API documentation schema	Schema is generated without errors or warnings
Account owner	Run a workflow that triggers Celery work: notifications, digests, reminders, due-date recalculation	Workers pick the tasks up, queue does not grow, no task hangs or repeated retries
Account owner	Subscribe, change the plan and cancel through Stripe	Billing flows complete, webhooks from Stripe are processed
Account owner	Post a comment with a mention and a markdown table	Markdown renders as before, the mentioned user gets a notification
Account owner	Filter the workflow list by two query parameters at once: GET /workflows?fields=<api_name>&template_id=<id>	Both filters apply: only workflows of that template are returned, and each carries the requested field
—	Apply migrations on a copy of the production database	migrate applies cleanly, migrate --check reports no pending migrations

Web-client

Browser	Device	Authorization	Test scenario	Expected result
Chrome	Desktop	Account owner	Sign in, open the task list, open a task, complete it	Every step works, no errors in the console
Chrome	Desktop	Account owner	Keep two sessions open, complete a task in one	The second session receives the real-time notification over the websocket
Chrome	Desktop	Account owner	Run a workflow, upload a file to a task, download it back	Upload and download both work
Chrome	Desktop	Guest	Open a guest task link and complete the task	Guest access works, the task is completed
Chrome	Desktop	Account owner	Open the integrations page and the billing page	Pages load with the correct data
Firefox, Safari	Desktop	Account owner	Sign in and complete a task	Same behaviour as in Chrome
Chrome	Mobile	Account owner	Sign in, open the task list, complete a task	Same behaviour as on desktop

Detailed integration scenarios for QA, covering all 15 external integrations, are in the upgrade plan attached to the Pneumatic task, section 4.8.

Release note for the releaser: the backend, celery and celery-beat images have to be rebuilt, since the base image changed. Rollback is the previous image tag together with a revert of this branch.

Made with Cursor

Note

Upgrade backend Python version from 3.7 to 3.8

• Updates the Docker base image, pyenv config, pyproject.toml, ruff.toml, and pre-commit hooks to use Python 3.8.
• Bumps Django from 2.2 to 2.2.28 and removes several unused dependencies (recurly, dj-database-url, ibm-cos-sdk, xmltodict, python-slugify, defusedxml, iso8601, importlib-metadata).
• Fixes test query strings to use & instead of ; as parameter separator, and switches OrderedDict import from typing_extensions to typing, both required for Python 3.8 compatibility.
• Behavioral Change: Python 3.8 changed the default query string separator; tests now reflect the correct & separator behavior.

^{Macroscope summarized 2bfa14d.}

[cursor]

Bugbot is paused — on-demand spend limit reached

Bugbot uses usage-based billing for this team and has hit its on-demand spend limit.

A team admin can raise the spend limit in the Cursor dashboard, or wait for the next billing cycle to continue.