chore: unify dev environment with data-fair conventions

.github/workflows/reuse-build.yml

@@ -8,7 +8,7 @@ jobs:
     runs-on: ubuntu-latest
     steps:
     - name: Checkout git repository
-      uses: actions/checkout@v4
+      uses: actions/checkout@v5
 
     - name: Set up Docker Buildx
       uses: docker/setup-buildx-action@v3

.github/workflows/reuse-quality.yml

@@ -9,9 +9,9 @@ jobs:
     runs-on: ubuntu-latest
     steps:
     - name: Checkout git repository
-      uses: actions/checkout@v4
+      uses: actions/checkout@v5
 
-    - uses: actions/setup-node@v3
+    - uses: actions/setup-node@v5
       with:
         node-version: 24
         cache: 'npm'

.zellij.kdl

@@ -3,30 +3,30 @@ layout {
       split_direction "vertical"
       pane name="portals" borderless=true {
         command "bash"
-        args "-ic" "nvm use && bash"
+        args "-ic" "nvm use > /dev/null 2>&1 && bash"
       }
       pane name="deps" {
         command "bash"
         args "-ic" "npm run dev-deps"
       }
       pane name="mock ingress manager" {
         command "bash"
-        args "-ic" "nvm use && npm run dev-ingress-manager"
+        args "-ic" "nvm use > /dev/null 2>&1 && npm run dev-ingress-manager"
       }
     }
     pane {
       split_direction "vertical"
       pane name="manager ui" {
         command "bash"
-        args "-ic" "nvm use && npm -w ui run dev"
+        args "-ic" "nvm use > /dev/null 2>&1 && npm run dev-ui"
       }
       pane name="manager api" {
         command "bash"
-        args "-ic" "nvm use && npm -w api run dev"
+        args "-ic" "nvm use > /dev/null 2>&1 && npm run dev-api"
       }
       pane name="portal" {
         command "bash"
-        args "-ic" "nvm use && npm -w portal run dev"
+        args "-ic" "nvm use > /dev/null 2>&1 && npm run dev-portal"
       }
     }
     pane size=1 borderless=true {

AGENTS.md

@@ -11,14 +11,15 @@ The dev environment is managed by zellij (terminal multiplexer) and docker compo
 ### Checking status
 
 ```bash
-bash dev/status.sh
+bash dev/scripts/status.sh
 ```
 
 This shows the health of all services (nginx, API, UI, portal, docker services, databases) and lists log files with sizes and timestamps.
 
 ### Log files
 
 All dev processes write to `dev/logs/`:
+
 - `dev-api.log` — API server
 - `dev-ui.log` — UI dev server (Vite)
 - `dev-portal.log` — Portal (Nuxt)
@@ -27,21 +28,41 @@ All dev processes write to `dev/logs/`:
 
 ### Troubleshooting
 
-1. Run `bash dev/status.sh` to identify which services are down
+1. Run `bash dev/scripts/status.sh` to identify which services are down
 2. Read the relevant log file in `dev/logs/` for error details
 3. Report findings to the user — do not attempt to fix infrastructure issues yourself
 
 ### Port assignments
 
 Port numbers are defined in `.env`. Do not modify port assignments.
 
+### Git worktrees
+
+To work on a separate branch in an isolated environment (own `.env`, own
+docker compose project, randomized ports so it doesn't clash with the main
+checkout), **always use the project scripts** — never `git worktree add` by
+hand:
+
+```bash
+bash dev/worktree.sh <branch-name>     # create worktree + .env + npm ci + build-types + ui build
+bash dev/delete-worktree.sh <branch>   # stops/removes containers + volumes + images, then removes the worktree
+```
+
+`dev/worktree.sh` does the full setup the manual `git worktree add` skips
+(env file, dependencies, type and UI builds). `dev/delete-worktree.sh` runs
+`docker compose --profile dev --profile test down -v --remove-orphans
+--rmi local` so a deleted worktree leaves no leftover containers, named
+volumes, or locally-built images.
+
 ### Test and dev environment
 
 The environment allows for integration testing thanks to docker containers (see docker-compose.yml).
 Random ports are allocated and defined in `.env`.
 A nginx proxy is part of the containers and exposes all services, including the development API server (configured in dev/resources/nginx.conf.template).
 Test users are defined in dev/resources/users.json
 
+The test cleanup endpoint (`DELETE /api/test-env`) only removes data whose `owner.id` starts with `test_`. Use the dev-only superadmin `superadmin@dev.com` (id `dev_superadmin`, password `passwd`, member of org `dev_org`) to play around manually without seeing your data wiped by tests.
+
 ### Testing
 
 ```bash

CONTRIBUTING.md

@@ -4,22 +4,36 @@
 
 - A Javascript/Typescript IDE with [Vue.js](https://vuejs.org/) and [ESLint](https://marketplace.visualstudio.com/items?itemName=dbaeumer.vscode-eslint) support.
 - A recent [Docker](https://docs.docker.com/engine/install/) installation.
-- [Node.js v24+](https://nodejs.org/)
+- [Node.js v24+](https://nodejs.org/) — the project ships an `.nvmrc`, so if you use [nvm](https://github.com/nvm-sh/nvm) you can simply run `nvm use` from the project root to switch to the right version.
 
 ## Install dependencies
 
-1. Install npm dependencies for all workspaces :
+1. Generate the `.env` file (random ports, dev host) — only needed once on the main checkout, worktrees created via `./dev/worktree.sh` do this automatically :
+
+```sh
+./dev/init-env.sh
+```
+
+2. Install npm dependencies for all workspaces :
 
 ```sh
 npm i
 ```
 
-2. Build / Update the types based on schemas :
+3. Build / Update the types based on schemas :
 
 ```sh
 npm run build-types
 ```
 
+4. Build the UI once :
+
+```sh
+npm -w ui run build
+```
+
+> Known bug: the API currently fails to start without a built UI bundle, so this initial build is required even for pure API/dev work. It only needs to be re-run if you blow away `ui/dist`.
+
 ## Start the development environment
 
 ```sh
@@ -31,10 +45,11 @@ npm run dev-zellij
 <details>
 <summary>Services</summary>
 
-- **Dev dependencies** : `npm run dev-deps`
-- **Api** : `npm run dev-api`
+- **Dev dependencies** (docker compose stack) : `npm run dev-deps`
+- **API** : `npm run dev-api`
 - **UI** : `npm run dev-ui`
 - **Portal** : `npm run dev-portal`
+- **Mock ingress manager** : `npm run dev-ingress-manager`
 
 </details>
 
@@ -57,27 +72,28 @@ docker build --progress=plain --target=portal -t data-fair/portals/portal:dev .
 
 ## Running the tests
 
-First, you need to start the development dependencies
+First, you need the full dev environment up (the test suite hits the API/UI/portal as well as the docker compose services) :
 
 ```sh
-npm run dev-deps
+npm run dev-zellij
 ```
 
-Then, you can run the tests.
+Then, you can run the tests :
 
 ```sh
-npm run test
+npm run test           # full suite
+npm run test-unit      # unit tests only
+npm run test-api       # API tests only
+npm run test-e2e       # e2e tests only
 ```
 
-To run a specific test, you can mark it with `it.only` or `describe.only` in the test file, then run the tests with :
+To run a specific test file :
 
 ```sh
-npm run test-only test-it/file-name.ts
+npm run test -- path/to/file.spec.ts
 ```
 
-## Setup the development environment
-
-TODO
+You can also mark a test with `.only` (e.g. `it.only`, `test.only`, `describe.only`) and run the project it belongs to.
 
 ## Random information
 

api/config/development.js

@@ -1,5 +1,5 @@
 import dotenv from 'dotenv'
-dotenv.config({ path: import.meta.resolve('../../.env').replace('file://', '') })
+dotenv.config({ path: import.meta.resolve('../../.env').replace('file://', ''), quiet: true })
 
 if (!process.env.DEV_API_PORT) throw new Error('missing DEV_API_PORT env variable, use "source dev/init-env.sh" to init .env file')
 

api/config/test.js

@@ -1,5 +1,5 @@
 import dotenv from 'dotenv'
-dotenv.config({ path: import.meta.resolve('../../.env').replace('file://', '') })
+dotenv.config({ path: import.meta.resolve('../../.env').replace('file://', ''), quiet: true })
 
 if (!process.env.DEV_API_PORT) throw new Error('missing DEV_API_PORT env variable, use "source dev/init-env.sh" to init .env file')
 

api/package.json

@@ -2,7 +2,7 @@
   "name": "api",
   "type": "module",
   "scripts": {
-    "dev": "mkdir -p ../dev/logs && NODE_ENV=development DEBUG=upgrade* node --watch index.ts 2>&1 | tee ../dev/logs/dev-api.log"
+    "dev": "bash ../dev/scripts/log-tee.sh ../dev/logs/dev-api.log -- env NODE_ENV=development DEBUG=upgrade* node --watch index.ts"
   },
   "imports": {
     "#config": "./src/config.ts",

dev/delete-worktree.sh

@@ -1,10 +1,42 @@
 #!/bin/bash
+# Delete a worktree (and its docker containers/volumes/local images).
+#
+# Refuses to proceed if the worktree has uncommitted changes or commits not
+# pushed to any origin ref. Pass -f / --force to bypass these checks.
 
-BRANCH_NAME=$1
+set -e
+
+FORCE=false
+BRANCH_NAME=""
+
+while [ $# -gt 0 ]; do
+    case "$1" in
+        -f|--force) FORCE=true ;;
+        -h|--help)
+            echo "Usage: $0 <branch> [-f|--force]"
+            echo "  --force / -f  skip uncommitted/unpushed safety checks"
+            exit 0
+            ;;
+        -*)
+            echo "Error: unknown flag '$1'"
+            echo "Usage: $0 <branch> [-f|--force]"
+            exit 1
+            ;;
+        *)
+            if [ -z "$BRANCH_NAME" ]; then
+                BRANCH_NAME="$1"
+            else
+                echo "Error: too many positional arguments"
+                exit 1
+            fi
+            ;;
+    esac
+    shift
+done
 
 if [ -z "$BRANCH_NAME" ]; then
     echo "Error: Please provide a branch name."
-    echo "Usage: ./dev/delete-worktree.sh feat-xyz"
+    echo "Usage: $0 <branch> [-f|--force]"
     exit 1
 fi
 
@@ -16,12 +48,51 @@ if [ ! -d "$TARGET_DIR" ]; then
     exit 1
 fi
 
-echo "Stopping docker compose services in $TARGET_DIR"
+ORIGIN_DIR="$PWD"
+
+if [ "$FORCE" != true ]; then
+    echo "Checking $TARGET_DIR for uncommitted / unpushed work…"
+    cd "$TARGET_DIR"
+
+    # Refresh remote refs so the "unpushed" check is accurate
+    git fetch origin --quiet 2>/dev/null || true
+
+    DIRTY=$(git status --porcelain)
+    if [ -n "$DIRTY" ]; then
+        echo
+        echo "❌ Refusing to delete: $TARGET_DIR has uncommitted changes:"
+        echo "$DIRTY" | sed 's/^/    /'
+        echo
+        echo "Commit / push / stash them, or pass -f / --force to bypass."
+        exit 1
+    fi
+
+    # Commits reachable from HEAD but NOT reachable from any origin/* ref.
+    # Catches: never-pushed branches, branches ahead of upstream, detached HEAD with unique commits.
+    UNPUSHED_COUNT=$(git rev-list --count HEAD --not --remotes=origin 2>/dev/null || echo 0)
+    if [ "$UNPUSHED_COUNT" -gt 0 ]; then
+        echo
+        echo "❌ Refusing to delete: $TARGET_DIR has $UNPUSHED_COUNT commit(s) not present on any origin ref:"
+        git --no-pager log --oneline HEAD --not --remotes=origin | sed 's/^/    /'
+        echo
+        echo "Push them (e.g. 'git push -u origin $BRANCH_NAME'), or pass -f / --force to bypass."
+        exit 1
+    fi
+
+    cd "$ORIGIN_DIR"
+    echo "✓ working tree clean and all commits present on origin"
+fi
+
+echo "Stopping & removing docker compose services in $TARGET_DIR (with volumes and orphans)"
 cd "$TARGET_DIR"
-docker compose --profile dev down
+# --profile dev --profile test : catch every container the worktree may have started
+# -v : drop named volumes (mongo, elasticsearch, s3mock, clamav…) — data is worktree-scoped
+# --remove-orphans : sweep any container left over from a previous compose file
+# --rmi local : remove images built locally for this worktree (keeps remote-pulled images)
+docker compose --profile dev --profile test down -v --remove-orphans --rmi local
 
 echo "Removing git worktree at $TARGET_DIR"
-cd "$PWD"
+cd "$ORIGIN_DIR"
 git worktree remove "$TARGET_DIR"
 
 echo "-----------------------------------------------"

dev/resources/organizations.json

@@ -97,6 +97,10 @@
         "id": "albanm",
         "role": "admin"
       },
+      {
+        "id": "dev_superadmin",
+        "role": "admin"
+      },
       {
         "id": "dev_admin_dep",
         "role": "admin",

dev/resources/users.json

@@ -5,7 +5,7 @@
     "lastName": "Superadmin",
     "email": "test_superadmin@test.com",
     "password": {
-      "clear": "superpasswd"
+      "clear": "passwd"
     }
   },
   {
@@ -79,5 +79,14 @@
     "password": {
       "clear": "passwd"
     }
+  },
+  {
+    "id": "dev_superadmin",
+    "firstName": "Dev",
+    "lastName": "Superadmin",
+    "email": "superadmin@dev.com",
+    "password": {
+      "clear": "passwd"
+    }
   }
 ]

dev/scripts/dev-deps.sh

@@ -0,0 +1,25 @@
+#!/usr/bin/env bash
+# Bring up dev docker containers, stream their logs to a file in the background,
+# and watch a status table refreshed every 4 seconds.
+#
+# Containers are NOT stopped when this script exits (Ctrl+C in the pane just
+# stops the watch + log streamer). They are stopped via "npm run stop-dev-deps",
+# which is chained after zellij in the "dev-zellij" script so the cleanup only
+# happens when the whole zellij session is closed.
+
+set -e
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+PROJECT_DIR="$(dirname "$(dirname "$SCRIPT_DIR")")"
+cd "$PROJECT_DIR"
+
+mkdir -p dev/logs
+
+docker compose --profile dev up -d --wait
+
+docker compose --profile dev logs -f --no-color > dev/logs/docker-compose.log 2>&1 &
+LOG_PID=$!
+
+trap 'kill "$LOG_PID" 2>/dev/null || true' EXIT INT TERM
+
+watch -c -n 4 -t "docker compose ps --all --format 'table {{.Name}}\t{{.Status}}'"