Inject ReproStim-specific bids-inject fields into the BIDS scans.json data dictionary

.ai/bids/inject-spec.md

@@ -116,6 +116,29 @@ func/sub-qa_ses-20250814_task-rest_acq-p2_bold__dup-01.nii.gz        2025-08-14T
 > makes it possible to resume or validate a subsequent `bids-qr-sync` pass without
 > re-scanning the source video from scratch.
 
+**`scans.json` sidecar:** `src/reprostim/assets/bids/scans.json` provides a default BIDS
+data-dictionary sidecar (`LongName`/`Description`/`Units` per BIDS's tabular-file column
+metadata schema, the same shape used for `_events.json`) documenting `filename`, `acq_time`,
+and all four `reprostim_*` columns above.
+
+`_do_inject_scans_json(ctx)` (called by `_do_inject_all` as its first step, before any
+`_scans.tsv` is touched) keeps `<dataset_home>/scans.json` (`--dataset`/`-d`,
+`BiContext.dataset_home`, default `.`) in sync with this default sample:
+
+- **Missing** — `scans.json` doesn't exist under `dataset_home` yet: created verbatim from
+  the default sample (`_load_default_scans_json()`, read via `importlib.resources` from
+  `assets/bids/scans.json`).
+- **Present, complete** — every top-level field from the default sample is already present
+  (regardless of value): no-op, file is not rewritten.
+- **Present, incomplete** — one or more default fields are missing: only the missing fields
+  are appended (`existing.update(missing)`); fields already present — default or custom
+  (e.g. a hand-added `operator` entry) — are left untouched, never overwritten.
+- **Invalid JSON** — `scans.json` exists but fails to parse: reported as an error
+  (`ctx.summary.errors`/`n_errors`, `logger.error`, `out_func("ERROR: ...")`) and left
+  untouched — does not raise, does not block the rest of `_do_inject_all`.
+- **`--dry-run`** — logs/reports what would change (create vs. which fields would be
+  appended) but writes nothing, consistent with Dry-Run Mode below.
+
 ### C) QR codes file — BIDS _events-like .tsv
 
 If QR codes were parsed from the video (`--qr` mode is not `none`), the decoded QR records
@@ -162,6 +185,7 @@ reprostim bids-inject [OPTIONS] PATHS...
 | Option                                          | Type            | Default    | Description                                                                                                                                                                                                                                         |
 |-------------------------------------------------|-----------------|------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
 | `-f / --videos PATH`                            | Path            | required   | Path to `videos.tsv` produced by `video-audit`. Video file paths in the TSV are resolved relative to this file's location.                                                                                                                          |
+| `-d / --dataset PATH`                           | Path (dir)      | `.`        | Home directory of the BIDS dataset being injected into. Propagated into `BiContext.dataset_home`; `do_main` uses it to create/update `<dataset_home>/scans.json` as its first step (see `scans.json` sidecar note below). `do_main` re-validates it exists and is a directory even when called directly, bypassing the CLI's own `click.Path(exists=True)` check — reports the error via `out_func`/`logger.error` and returns `1`, same as any other `do_main` error (no exception raised). |
 | `-r / --recursive`                              | Flag            | False      | When a directory is given in PATHS, recurse into subdirectories to find all `*_scans.tsv` files.                                                                                                                                                    |
 | `-b / --buffer-before DURATION`                 | sec or ISO 8601 | `0`        | Extra video before scan onset.                                                                                                                                                                                                                      |
 | `-a / --buffer-after DURATION`                  | sec or ISO 8601 | `0`        | Extra video after scan end.                                                                                                                                                                                                                         |
@@ -172,7 +196,7 @@ reprostim bids-inject [OPTIONS] PATHS...
 | `-z / --reprostim-timezone TIMEZONE`            | String          | `local`    | Timezone of the ReproStim capture machine, applied to naive `videos.tsv` timestamps (see Timezone Handling below).                                                                                                                                  |
 | `-Z / --bids-timezone TIMEZONE`                 | String          | `local`    | Timezone assumed for naive BIDS `acq_time` values. When omitted, defaults to the value of `--reprostim-timezone` (see Timezone Handling below).                                                                                                    |
 | `-m / --match REGEX`                            | String          | `.*`       | Regular expression matched against the `filename` field of each scan record. Only records whose `filename` matches are processed; all others are skipped. Default `.*` matches every record. Example: `func/` to restrict to functional scans only. |
-| `-d / --dry-run`                                | Flag            | False      | Analyse BIDS data and resolve matches but do not call `split-video` or write any output files. Prints what would be done.                                                                                                                           |
+| `-n / --dry-run`                                | Flag            | False      | Analyse BIDS data and resolve matches but do not call `split-video` or write any output files. Prints what would be done.                                                                                                                           |
 | `-w / --overwrite [skip\|force\|always\|error]` | Choice          | `skip`     | Policy for handling existing output files (see Overwrite Mode below).                                                                                                                                                                               |
 | `-k / --lock [yes\|no]`                         | Choice          | `yes`      | Whether to acquire a file lock (`videos.tsv.lock`) before reading `videos.tsv`. Use `no` for dirty-read mode when the lock is held by another user (see Lock / Dirty-read Mode below).                                                              |
 | `-v / --verbose`                                | Flag            | False      | Increase verbosity.                                                                                                                                                                                                                                 |
@@ -278,6 +302,10 @@ reprostim bids-inject \
 
 ## Dry-Run Mode
 
+`--dry-run`'s short flag is `-n` (not `-d`), matching the `rsync`/`make` "no-op" convention —
+`-d` is used for the `--dataset` option instead (BIDS dataset root, default `.`, home of
+`scans.json`), which is used more frequently and deserves the more obvious mnemonic letter.
+
 When `--dry-run` is set, `bids-inject` performs all analysis steps — loading `videos.tsv`,
 discovering `*_scans.tsv` files, resolving scan durations, matching videos, determining output
 paths and media suffixes — but **skips the actual `split-video` call and writes no files**.

.ai/bids/inject-tasks.md

@@ -8,6 +8,11 @@ Tracks implementation progress against [inject-spec.md](inject-spec.md).
 
 - [x] `PATHS` argument — one or more `_scans.tsv` files or directories
 - [x] `-f / --videos` — path to `videos.tsv`
+- [x] `-d / --dataset` — BIDS dataset home directory, default `.`.
+- [x] `do_main` re-validates `dataset_home` exists and is a directory, independent of the
+      CLI's own `click.Path(exists=True)` check (covers direct callers that bypass the CLI) —
+      reports via `out_func`/`logger.error` and returns `1`, not an exception, so it surfaces
+      as a normal non-zero CLI exit code
 - [x] `-r / --recursive` — recurse into subdirectories
 - [x] `-b / --buffer-before` — extra video before scan onset
 - [x] `-a / --buffer-after` — extra video after scan end
@@ -18,7 +23,8 @@ Tracks implementation progress against [inject-spec.md](inject-spec.md).
 - [x] `-z / --reprostim-timezone` — timezone for `videos.tsv` timestamps
 - [x] `-Z / --bids-timezone` — timezone for BIDS `acq_time` values
 - [x] `-m / --match REGEX` — filter scan records by filename
-- [x] `-d / --dry-run`
+- [x] `-n / --dry-run` — short flag changed from `-d` to `-n` (rsync/make "no-op" convention);
+      `-d` now used for `--dataset` above
 - [x] `-w / --overwrite [skip|force|always|error]` — policy for existing output files
 - [x] `-k / --lock [yes|no]` — dirty-read mode for `videos.tsv`
 - [x] `-v / --verbose`
@@ -139,6 +145,22 @@ Tracks implementation progress against [inject-spec.md](inject-spec.md).
 - [x] Handle re-runs: update existing `reprostim_*` columns in-place (don't duplicate)
 - [x] Skip write-back in `--dry-run` mode
 - [x] `reprostim_path` stored relative to `videos.tsv` location (consistent with `videos.tsv` path convention)
+- [x] `src/reprostim/assets/bids/scans.json` — default BIDS data-dictionary sidecar documenting
+      `filename`, `acq_time`, and all four `reprostim_*` columns (`LongName`/`Description`/`Units`)
+
+### E) scans.json data-dictionary sync
+- [x] `_load_default_scans_json()` — reads the packaged default sample via `importlib.resources`
+- [x] `_do_inject_scans_json(ctx)` — called by `_do_inject_all` as its first step, before any
+      path in `paths` is processed
+- [x] Creates `<dataset_home>/scans.json` verbatim from the default sample when missing
+- [x] No-op (no rewrite) when the existing file already has every default field
+- [x] Appends only the missing default fields when the existing file has some but not all;
+      existing fields (default or custom, e.g. a hand-added `operator` entry) are never
+      overwritten
+- [x] Invalid/unparseable existing `scans.json` → reported via `ctx.summary.errors`/`n_errors`
+      and `out_func`/`logger.error`, file left untouched, does not raise and does not abort
+      the rest of `_do_inject_all`
+- [x] Honours `--dry-run` — logs/reports what would change, writes nothing
 
 ---
 
@@ -304,6 +326,20 @@ Test file location: `tests/bids/test_inject.py` (mirrors `tests/audio/test_audio
 - [x] Re-run (columns already present) → columns updated in-place, no duplication
 - [x] `--dry-run` → `_scans.tsv` not modified
 
+### scans.json data-dictionary sync tests (`_do_inject_scans_json`)
+- [x] Missing `scans.json` → created verbatim from `_load_default_scans_json()`
+- [x] Existing `scans.json` with every default field (plus a custom field) → byte-for-byte
+      untouched (no-op)
+- [x] Existing `scans.json` missing some default fields → only those appended; existing
+      default and custom fields preserved untouched
+- [x] `--dry-run`, missing file → nothing written, `[DRY-RUN] Would create ...` reported
+- [x] `--dry-run`, incomplete file → nothing written, `[DRY-RUN] Would add missing field ...`
+      reported
+- [x] Invalid JSON in existing `scans.json` → reported via `ctx.summary`/`out_func`, file left
+      untouched, no exception raised
+- [x] `do_main` end-to-end (`dry_run=False`) → `<dataset_home>/scans.json` created as the
+      pipeline's first step, before any `_scans.tsv` is processed
+
 ### Overwrite mode tests
 
 - [x] `skip` + existing output → 0 injected, files untouched, counted as skipped

src/reprostim/assets/bids/scans.json

@@ -0,0 +1,28 @@
+{
+  "filename": {
+    "Description": "Name of the nifti file"
+  },
+  "acq_time": {
+    "LongName": "Acquisition time",
+    "Description": "Acquisition time of the particular scan"
+  },
+  "reprostim_path": {
+    "LongName": "ReproStim source video path",
+    "Description": "Path to the source .mkv file the injected recording was sliced from, relative to the videos.tsv location"
+  },
+  "reprostim_offset": {
+    "LongName": "ReproStim buffer-segment offset",
+    "Description": "Offset of the buffer-segment start into the source video",
+    "Units": "s"
+  },
+  "reprostim_buffer_before": {
+    "LongName": "ReproStim buffer before scan onset",
+    "Description": "Actual buffer prepended before scan onset",
+    "Units": "s"
+  },
+  "reprostim_buffer_after": {
+    "LongName": "ReproStim buffer after scan end",
+    "Description": "Actual buffer appended after scan end",
+    "Units": "s"
+  }
+}

src/reprostim/bids/inject.py

@@ -20,6 +20,7 @@
 from datetime import datetime, time, timedelta, timezone, tzinfo
 from enum import Enum
 from functools import lru_cache
+from importlib.resources import files
 from typing import Callable, List, Optional, Tuple
 from zoneinfo import ZoneInfo
 
@@ -127,6 +128,11 @@ class BiSummary(BaseModel):
 class BiContext(BaseModel):
     """Context for bids-inject processing of scan records."""
 
+    dataset_home: str = Field(
+        default=".",
+        description="Home directory of the BIDS dataset being injected into "
+        "(e.g. contains scans.json). Defaults to the current directory.",
+    )
     dry_run: bool = Field(
         ..., description="Whether to skip actual file writes and print planned actions"
     )
@@ -1126,19 +1132,101 @@ def _do_inject_dir(ctx: BiContext, path: str):
             _do_inject_dir(ctx, entry.path)
 
 
+def _load_default_scans_json() -> dict:
+    """Load the packaged default BIDS ``scans.json`` data-dictionary sample.
+
+    :returns: Parsed contents of ``assets/bids/scans.json``.
+    :rtype: dict
+    """
+    text = (files("reprostim") / "assets" / "bids" / "scans.json").read_text()
+    return json.loads(text)
+
+
+def _do_inject_scans_json(ctx: BiContext) -> None:
+    """Ensure ``<dataset_home>/scans.json`` exists and has every default field.
+
+    Compares ``<dataset_home>/scans.json`` against the packaged default
+    sample (``assets/bids/scans.json``):
+
+    - If ``scans.json`` doesn't exist yet, it is created from the default
+      sample verbatim.
+    - If it exists and already has every top-level field from the default
+      sample, nothing is written.
+    - If it exists but is missing one or more default fields, those fields
+      (and only those — existing fields are left untouched) are appended
+      and the file is rewritten.
+
+    Honours ``ctx.dry_run``: when set, logs/reports what would change but
+    writes nothing. A ``scans.json`` that fails to parse as JSON is reported
+    as an error (via ``ctx.summary``) and left untouched rather than
+    overwritten or raised as an exception.
+
+    :param ctx: Processing context; uses ``ctx.dataset_home``, ``ctx.dry_run``,
+        ``ctx.out_func``, and ``ctx.summary``.
+    :type ctx: BiContext
+    """
+    scans_json_path = os.path.join(ctx.dataset_home, "scans.json")
+    default = _load_default_scans_json()
+
+    if not os.path.isfile(scans_json_path):
+        if ctx.dry_run:
+            msg = f"Would create {scans_json_path}"
+        else:
+            with open(scans_json_path, "w") as f:
+                json.dump(default, f, indent=2)
+                f.write("\n")
+            msg = f"Created {scans_json_path}"
+        logger.info(msg)
+        if ctx.out_func:
+            ctx.out_func(f"[DRY-RUN] {msg}" if ctx.dry_run else msg)
+        return
+
+    try:
+        with open(scans_json_path) as f:
+            existing = json.load(f)
+    except json.JSONDecodeError as e:
+        err_msg = f"Failed to parse {scans_json_path}: {e}"
+        logger.error(err_msg)
+        ctx.summary.errors.append(err_msg)
+        ctx.summary.n_errors += 1
+        if ctx.out_func:
+            ctx.out_func(f"ERROR: {err_msg}")
+        return
+
+    missing = {k: v for k, v in default.items() if k not in existing}
+    if not missing:
+        logger.debug(f"{scans_json_path} already has all default fields")
+        return
+
+    field_list = ", ".join(sorted(missing))
+    if ctx.dry_run:
+        msg = f"Would add missing field(s) to {scans_json_path}: {field_list}"
+    else:
+        existing.update(missing)
+        with open(scans_json_path, "w") as f:
+            json.dump(existing, f, indent=2)
+            f.write("\n")
+        msg = f"Added missing field(s) to {scans_json_path}: {field_list}"
+    logger.info(msg)
+    if ctx.out_func:
+        ctx.out_func(f"[DRY-RUN] {msg}" if ctx.dry_run else msg)
+
+
 def _do_inject_all(ctx: BiContext, paths: List[str]):
     """Dispatch injection across a mixed list of file and directory paths.
 
-    For each entry in *paths*: regular files are forwarded to
-    :func:`_do_inject_scans`; directories are forwarded to
-    :func:`_do_inject_dir` (which honours ``ctx.recursive``); anything else
-    is logged as a warning and skipped.
+    First ensures ``<dataset_home>/scans.json`` exists and has every default
+    field (see :func:`_do_inject_scans_json`). Then, for each entry in
+    *paths*: regular files are forwarded to :func:`_do_inject_scans`;
+    directories are forwarded to :func:`_do_inject_dir` (which honours
+    ``ctx.recursive``); anything else is logged as a warning and skipped.
 
     :param ctx: Processing context propagated to all subordinate calls.
     :type ctx: BiContext
     :param paths: Sequence of file or directory paths supplied by the caller.
     :type paths: List[str]
     """
+    _do_inject_scans_json(ctx)
 
     # iterate over paths and depending on whether it's a file or directory,
     # process accordingly
@@ -1371,6 +1459,7 @@ def dt_bids_to_reprostim(
 def do_main(
     paths: List[str],
     videos_tsv: str,
+    dataset_home: str,
     recursive: bool,
     match: str,
     buffer_before: str,
@@ -1397,6 +1486,9 @@ def do_main(
     :param videos_tsv: Path to ``videos.tsv`` produced by ``video-audit``.
         Video file paths inside the TSV are resolved relative to this file's location.
     :type videos_tsv: str
+    :param dataset_home: Home directory of the BIDS dataset being injected into
+        (e.g. contains ``scans.json``). Defaults to the current directory.
+    :type dataset_home: str
     :param recursive: When ``True``, recurse into subdirectories when searching
         for ``*_scans.tsv`` files.
     :type recursive: bool
@@ -1453,7 +1545,17 @@ def do_main(
     :returns: Exit code — ``0`` on success, non-zero on error.
     :rtype: int
     """
+    if not os.path.isdir(dataset_home):
+        err_msg = (
+            f"--dataset path does not exist or is not a directory: {dataset_home!r}"
+        )
+        logger.error(err_msg)
+        if out_func:
+            out_func(f"ERROR: {err_msg}")
+        return 1
+
     ctx: BiContext = BiContext(
+        dataset_home=dataset_home,
         dry_run=dry_run,
         recursive=recursive,
         match=match,