Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
81 changes: 81 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -1,13 +1,92 @@
name: CI

# A pull request runs the whole suite (build, postgres18 and browser, the checks
# master's rules require). A push to master, a merged pull request, runs the
# quick job alone: lint, type check and the tests the push changed. The release
# workflow calls this workflow (workflow_call) before each daily release, which
# runs the whole suite with coverage on master and uploads it to Codecov. A
# called workflow sees its caller's event (schedule or workflow_dispatch) in
# github.event_name, so the whole suite runs there.
on:
push:
branches: [ master ]
pull_request:
branches: [ master ]
workflow_call:
secrets:
CODECOV_TOKEN:
required: false

jobs:
quick:
if: github.event_name == 'push'
runs-on: ubuntu-latest
services:
postgres:
image: pgvector/pgvector:0.8.7-pg16
env:
POSTGRES_USER: adapter
POSTGRES_PASSWORD: adapter
POSTGRES_DB: adapter_test
ports:
- 5432:5432
options: >-
--health-cmd "pg_isready -U adapter -d adapter_test"
--health-interval 5s
--health-timeout 5s
--health-retries 10
env:
TZ: Europe/Berlin
POSTGRES_TEST_URL: postgresql://adapter:adapter@localhost:5432/adapter_test
SYNC_TEST_REQUIRE_POSTGRES: '1'
SYNC_TEST_POSTGRES_URL: postgresql://adapter:adapter@localhost:5432/adapter_test
steps:
- uses: actions/checkout@v5
with:
# The commit before the push, which the changed-tests run compares with.
fetch-depth: 0
- uses: actions/setup-node@v5
with:
node-version: '22'

# Seconds, and outside Vitest: the release workflow's suite runs only when
# the package changed, so a push that changes only the workflows is
# checked here.
- name: No workflow pushes to master
env:
DEFAULT_BRANCH: ${{ github.event.repository.default_branch }}
run: |
node --test .github/scripts/check-workflow-pushes.test.mjs
node .github/scripts/check-workflow-pushes.mjs

- name: Install dependencies
run: npm install

- name: Lint
run: npm run lint || true

- name: TypeCheck
run: npm run typecheck

# The specs whose files, or the modules they import, the push changed.
# Coverage is off here: vitest.config.ts turns it on with thresholds over
# all of src/, which a few specs alone need not meet. The release run's
# whole suite collects it.
- name: Run the tests the push changed
env:
BEFORE: ${{ github.event.before }}
run: |
since="$BEFORE"
if [ -z "$since" ] || [ "$since" = 0000000000000000000000000000000000000000 ] || ! git cat-file -e "${since}^{commit}" 2>/dev/null; then
echo "The commit before the push is not in the history; running every test"
npx vitest run --coverage.enabled=false
else
echo "Tests affected by the changes since ${since}"
npx vitest run --changed "$since" --passWithNoTests --coverage.enabled=false
fi

build:
if: github.event_name != 'push'
runs-on: ubuntu-latest
# The Postgres specs that need a server (createDatabase.postgres,
# postgres.integration, postgres.dialect.integration, postgres.segment,
Expand Down Expand Up @@ -252,6 +331,7 @@ jobs:


postgres18:
if: github.event_name != 'push'
runs-on: ubuntu-latest
services:
postgres:
Expand Down Expand Up @@ -284,6 +364,7 @@ jobs:
# key-value module, on the browsers' own IndexedDB, Web Locks and structured
# clone: Chromium, WebKit and Firefox, through Playwright.
browser:
if: github.event_name != 'push'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
Expand Down
133 changes: 117 additions & 16 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -1,10 +1,25 @@
name: Release sql-storage-adapter

# A release runs once a day, at 10:17 UTC, when master holds releasable
# changes, and whenever a maintainer starts this workflow by hand on master
# (Actions, Release sql-storage-adapter, Run workflow; or
# `gh workflow run release.yml --ref master`). A merge to master publishes
# nothing by itself: ci.yml's quick checks run on it, and the next daily run
# publishes what it changed. GitHub may start a scheduled run some minutes late
# when Actions is under load. RELEASING.md has the whole flow.
on:
push:
branches:
- master
schedule:
- cron: '17 10 * * *'
workflow_dispatch:
inputs:
force:
description: Skip the releasable-changes check and run the suite and the release job (the release job still decides the version, and publishes nothing when nothing changed)
type: boolean
default: false
dry_run:
description: Run everything up to the publish and stop there (no npm publish, tag or GitHub release)
type: boolean
default: false

# One release run at a time, in the order the runs began to wait: a run that
# starts while another runs waits for it, and up to 100 wait (queue: max), so a
Expand All @@ -16,11 +31,81 @@ concurrency:
cancel-in-progress: false
queue: max

permissions:
contents: read

jobs:
publish:
# Seconds, with git and Node alone: nothing is installed. It runs the release
# decision (scripts/release-version.mjs) and ends the run green when nothing
# is releasable.
gate:
name: Releasable changes
# A run started by hand on another branch publishes nothing.
if: github.ref == 'refs/heads/master'
runs-on: ubuntu-latest
permissions:
contents: read
outputs:
releasable: ${{ steps.decide.outputs.releasable }}
steps:
- name: Check out repository
uses: actions/checkout@v5
with:
# Every commit and tag: the decision finds the commit the newest
# version was published from and compares master with it.
fetch-depth: 0
persist-credentials: false

- name: Use Node.js
uses: actions/setup-node@v5
with:
node-version: 22

- name: Read the release decision
id: plan
run: node scripts/release-version.mjs

- name: Decide whether to run the suite and the release
id: decide
env:
PUBLISH: ${{ steps.plan.outputs.publish }}
VERSION: ${{ steps.plan.outputs.version }}
REPAIR: ${{ steps.plan.outputs.repair }}
FORCE: ${{ inputs.force == true }}
run: |
if [ "$FORCE" = true ]; then
releasable=true
reason="force is set, so the check is skipped; the release job decides the version again"
elif [ "$PUBLISH" = true ]; then
releasable=true
reason="master holds releasable changes: ${VERSION} would be published"
elif [ -n "$REPAIR" ]; then
releasable=true
reason="no tag names v${REPAIR}, the newest published version: the release job makes its GitHub release"
else
releasable=false
reason="nothing releasable on master since the newest published version; no install, build or test runs"
fi
echo "Releasable: ${releasable} (${reason})"
echo "releasable=${releasable}" >> "$GITHUB_OUTPUT"
echo "Releasable: ${releasable} (${reason})" >> "$GITHUB_STEP_SUMMARY"

# The full suite of ci.yml on this commit, with coverage uploaded to Codecov,
# which keeps master's coverage badge current.
suite:
needs: gate
if: needs.gate.outputs.releasable == 'true'
permissions:
contents: read
uses: ./.github/workflows/ci.yml
secrets:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}

publish:
needs: [gate, suite]
# Only after the suite passed on a releasable master.
if: needs.gate.outputs.releasable == 'true' && needs.suite.result == 'success'
runs-on: ubuntu-latest
permissions:
contents: write
packages: write
Expand All @@ -45,10 +130,11 @@ jobs:
node-version: 22
registry-url: https://registry.npmjs.org

# A push that changed a file the package ships publishes the newest
# published version with its patch raised by one, or package.json's
# version when that is higher; only the run of master's head publishes
# (RELEASING.md).
# The decision again, at publish time: a file the package ships changed
# since the newest published version publishes that version with its patch
# raised by one, or package.json's version when that is higher. Only a run
# of master's head publishes: when master moved while the suite ran, this
# run publishes nothing and the next one publishes (RELEASING.md).
- name: Decide the version to publish
id: plan
run: node scripts/release-version.mjs --notes "$RUNNER_TEMP"
Expand All @@ -57,10 +143,6 @@ jobs:
if: steps.plan.outputs.publish == 'true'
run: npm install

- name: Run tests
if: steps.plan.outputs.publish == 'true'
run: npx vitest run

- name: Build package
if: steps.plan.outputs.publish == 'true'
run: npm run build
Expand All @@ -72,19 +154,38 @@ jobs:
VERSION: ${{ steps.plan.outputs.version }}
run: npm version "$VERSION" --no-git-tag-version --allow-same-version

- name: Stop before the publish (dry run)
if: inputs.dry_run == true
env:
PUBLISH: ${{ steps.plan.outputs.publish }}
VERSION: ${{ steps.plan.outputs.version }}
REPAIR: ${{ steps.plan.outputs.repair }}
run: |
if [ "$PUBLISH" = true ]; then
npm pack --dry-run
line="Dry run: would publish ${VERSION}; nothing was published, tagged or released."
else
line="Dry run: nothing to publish."
fi
if [ -n "$REPAIR" ]; then
line="${line} Would make the missing GitHub release of v${REPAIR}."
fi
echo "$line"
echo "$line" >> "$GITHUB_STEP_SUMMARY"

# Trusted publishing needs npm 11.5.1 or later.
- name: npm 11.20.0 for trusted publishing
if: steps.plan.outputs.publish == 'true'
if: steps.plan.outputs.publish == 'true' && inputs.dry_run != true
run: npm install -g npm@11.20.0 && npm --version

- name: Publish to npm
if: steps.plan.outputs.publish == 'true'
if: steps.plan.outputs.publish == 'true' && inputs.dry_run != true
run: npm publish --access public --provenance
env:
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}

- name: Create GitHub release
if: steps.plan.outputs.release != ''
if: steps.plan.outputs.release != '' && inputs.dry_run != true
uses: ncipollo/release-action@v1
with:
token: ${{ secrets.GITHUB_TOKEN }}
Expand All @@ -98,7 +199,7 @@ jobs:
# The newest published version when no tag names it: a run that published
# it and failed before its release.
- name: Make the missing release of the newest published version
if: steps.plan.outputs.repair != ''
if: steps.plan.outputs.repair != '' && inputs.dry_run != true
uses: ncipollo/release-action@v1
with:
token: ${{ secrets.GITHUB_TOKEN }}
Expand Down
6 changes: 3 additions & 3 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -192,10 +192,10 @@ docs: update README with Capacitor examples

## Release Process

Every push to `master` that changes what the package ships publishes it. The [release workflow](https://github.com/framerslab/sql-storage-adapter/blob/master/.github/workflows/release.yml) decides the version before it installs anything: the newest published version with its patch raised by one, or `package.json`'s version when that is higher. It then installs, runs the tests, builds, publishes to npm and creates the GitHub release `v<version>`, which creates the tag. Nothing is committed to `master`. [RELEASING.md](./RELEASING.md) has the whole decision.
A release runs daily at 10:17 UTC when `master` holds changes to what the package ships since the newest published version; a merge to `master` runs quick checks and publishes nothing by itself. The [release workflow](https://github.com/framerslab/sql-storage-adapter/blob/master/.github/workflows/release.yml) decides the version before it installs anything: the newest published version with its patch raised by one, or `package.json`'s version when that is higher. It then runs the whole CI suite, builds, publishes to npm and creates the GitHub release `v<version>`, which creates the tag. Nothing is committed to `master`. To release now, start it by hand on `master` (`gh workflow run release.yml --ref master`; `-f dry_run=true` stops before the publish). [RELEASING.md](./RELEASING.md) has the whole decision.

- **A fix** needs no release step: its merge publishes the next patch. Add its notes to `CHANGELOG.md` under `## [Unreleased]`.
- **A new minor or major version**: raise `version` in `package.json` in the pull request, following [semantic versioning](https://semver.org/) (MAJOR for breaking changes, 1.0.0 to 2.0.0; MINOR for new features, backward compatible, 1.0.0 to 1.1.0), and add a section to `CHANGELOG.md` headed `## [x.y.z]` with what was added, changed, deprecated, removed and fixed. The merge publishes that version, with that section as its release notes.
- **A fix** needs no release step: the next daily run after its merge publishes the next patch. Add its notes to `CHANGELOG.md` under `## [Unreleased]`.
- **A new minor or major version**: raise `version` in `package.json` in the pull request, following [semantic versioning](https://semver.org/) (MAJOR for breaking changes, 1.0.0 to 2.0.0; MINOR for new features, backward compatible, 1.0.0 to 1.1.0), and add a section to `CHANGELOG.md` headed `## [x.y.z]` with what was added, changed, deprecated, removed and fixed. The next daily run after the merge publishes that version, with that section as its release notes.

Do not push a version tag by hand: the workflow creates it with the GitHub release.

Expand Down
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -1022,11 +1022,11 @@ TLS follows pg's, and so does the check of the server. pg's TLS is Node's, which
## CI, Releases, and Badges

- GitHub Actions workflows:
- `ci.yml` runs on pushes and pull requests to `master`: lint, type check, tests with coverage and the build, then it packs the package, installs the tarball into a new ES module project and type-checks a consumer with `--module nodenext --moduleResolution nodenext`. The job starts a PostgreSQL 16 service, and the specs that need a server read its URL from `POSTGRES_TEST_URL`.
- `release.yml` runs on every push to `master`: when a file the package ships changed since the newest published version, it publishes the next patch to npm (or `package.json`'s version when that is higher), tags the commit (`vX.Y.Z`) and creates the GitHub Release.
- `ci.yml` runs on pull requests to `master`: lint, type check, tests with coverage and the build, then it packs the package, installs the tarball into a new ES module project and type-checks a consumer with `--module nodenext --moduleResolution nodenext`. The job starts a PostgreSQL 16 service, and the specs that need a server read its URL from `POSTGRES_TEST_URL`. A push to `master` runs its quick job alone: lint, type check and the tests the push changed.
- `release.yml` runs daily at 10:17 UTC: when a file the package ships changed since the newest published version, it runs the whole of `ci.yml` (coverage uploaded to Codecov), then publishes the next patch to npm (or `package.json`'s version when that is higher), tags the commit (`vX.Y.Z`) and creates the GitHub Release. When nothing is releasable the run ends in seconds. Start it by hand to release now: `gh workflow run release.yml --repo framerslab/sql-storage-adapter --ref master` (inputs `force` and `dry_run`).
- Check badge health whenever builds fail:
- CI badge should be green for `master`.
- Codecov badge updates after coverage reports upload.
- Codecov badge updates when a release run uploads coverage for `master`.
- npm badge reflects the latest published version.
- Manual verification commands:
```bash
Expand Down
Loading
Loading