SQL support for LRR

[psilabs-dev]

Implementation of LANraragi API-compatible PostgreSQL data management layer for LRR with perl's DBI and DBD::pg.

• Integration testing branch: PostgreSQL integration tests aio-lanraragi#157 (dev-postgresql/main)

User Guide/Warning

A Postgres database is required; tested on docker "postgres:17.4". Redis 7 is also required, as it is used for configuration and caching. Docker is the only supported platform.

Example docker compose:

postgres:
    image: postgres:17.4
    environment:
        - POSTGRES_USER=koyomi
        - POSTGRES_PASSWORD=kamimamita
redis:
    image: redis:7

lanraragi:
    environment:
        - LRR_REDIS_ADDRESS=redis:6379
        - LRR_POSTGRES_DB=postgres
        - LRR_POSTGRES_HOST=postgres
        - LRR_POSTGRES_USER=koyomi
        - LRR_POSTGRES_PASSWORD=kamimamita
        - ...
    depends_on:
        - postgres
        - redis

Data Management

As this PR is in development and is downstream-only, schema changes may occur, and do not come with a migration script. Tables will not be amended with ALTER, and indexes may be added or deleted between commit hashes, resulting in breaking changes and database incompatibility.

Prepare to create/restore from backups, or prepare metadata via the API from a stable data source, and drop the postgres database entirely. This branch should not be the primary source of your archives' metadata management; assume that all data stored in this service is gone.

Terminology

A top-level entrypoint (TLE) is a subroutine/file which has no owning caller within the LRR application. Controllers, Shinobu, Minion, and LANraragi.pm are all top-level entrypoints.

An example of a file that is not a TLE is "lib/LANraragi/Model/Archive.pm", as get_title is called by "lib/LANraragi/Controller/Api/Archive.pm", which is within the project.

Minion queues are TLE's: although they accept jobs from API endpoints which live in the Controller layer, that layer is separated from the workers via the Minion library which is not within LRR, and the controller layer does not own the minion jobs.

A subroutine is:

• redis-specific if it (or any dependency) uses Redis as a database,
• SQL-specific if it (or any dependency) uses a SQL database,
• agnostic if it (and all dependencies) do not connect to a database.

A redis-specific function and SQL-specific function are equivalent, if their function signatures/contract logic are the same.

The sets of dependencies of all TLE's span the PR migration scope. By corollary, orphans and dead code do not fall under this spanning space, as they are not used by any TLE.

Agnostic subroutines do not require any migration.

Merge Conflict Mitigation

To minimize merge conflict risk, we will use model layer injection and namespace-based reservation to reserve specific downstream modules.

This PR shall not modify or remove perl modules under "lib/LANraragi/Utils" and "lib/LANraragi/Model". Instead, new files will be placed under the "PsilabsDev" downstream namespace under "Utils" or "Model" carrying new or equivalent subroutines/logic, and the top-level entrypoints will be re-wired to these downstream modules if required.

Put another way, we have the following mitigation properties:

• Apart from TLE's, no pre-existing upstream files require modification,
• All custom business or database logic does not conflict with upstream files,
• TLE conflicts are limited to re-wiring changes.

Equivalent subroutines must live in equivalent namespaced modules: for example, if a redis-specific subroutine lives in "lib/LANraragi/Utils/Abc.pm", then its postgres-equivalent subroutine must live in "lib/LANraragi/Utils/PsilabsDev/PgAbc.pm". An exception is made for novel logic not present in upstream.