WisePulse

WISE Loculus with V-Pipe – Infrastructure for start-to-end viral wastewater analysis

---

About

WisePulse is an end-to-end infrastructure for viral wastewater surveillance, combining Loculus for sequence data management with LAPIS-SILO, a high-performance genomic database – here deployed for the first time to Short-Read / amplicon sequences. It enables real-time querying and visualization of viral genomic data from wastewater samples.

Current Features

• Loculus instance for easy data sharing and collaboration
• srSILO API for querying amplicon sequence data

Data Processing Pipeline (external)

Upstream data processing is handled separately:

• NGS processing via V-pipe
• Database ingestion with sr2silo from V-pipe alignment outputs

Roadmap

• Raw data submission & V-pipe pipeline integration
• On-demand exploratory analysis tools

Quick Start

# One-time setup (user, directories, build tools)
ansible-playbook playbooks/srsilo/setup.yml -i inventory.ini --become  --ask-become-pass

# Run automated update pipeline (checks for new data, processes, updates API)
ansible-playbook playbooks/srsilo/update-pipeline.yml -i inventory.ini \
  --become --ask-become-pass \
  -e "@playbooks/srsilo/vars/test_vars.yml"

# Setup daily automated runs at 2 AM
ansible-playbook playbooks/srsilo/setup-timer.yml -i inventory.ini

# Check API status
curl http://localhost:8083/sample/info

API available at: http://localhost:8083/swagger-ui/index.html

Architecture

srSILO Pipeline

The srSILO pipeline is now fully managed by Ansible with:

• Low Downtime (API managed automatically)
• Self-healing (automatic rollback on failures)
• Smart execution (exits early if no new data)
• Retention policy (automatic cleanup of old indexes)

Full documentation: See docs/srsilo/ARCHITECTURE.md

Playbooks:

• playbooks/srsilo/setup.yml - One-time server setup
• playbooks/srsilo/update-pipeline.yml - Full automated update (7 phases)
• Future: playbooks/srsilo/rollback.yml - Manual rollback to previous index

Example Usage:

# Full automated update (recommended)
ansible-playbook playbooks/srsilo/update-pipeline.yml -i inventory.ini

# With custom configuration
ansible-playbook playbooks/srsilo/update-pipeline.yml \
  -i inventory.ini \
  -e "srsilo_retention_days=7" \
  -e "srsilo_fetch_days=30"

# Test specific phase (e.g., check for new data only)
ansible-playbook playbooks/srsilo/update-pipeline.yml \
  -i inventory.ini \
  --tags phase2

Loculus Deployment

Deploy W-ASAP Loculus to Kubernetes:

ansible-playbook playbooks/loculus/deploy-loculus.yml --ask-become-pass

Monitoring Stack

Deploy Prometheus + Grafana:

ansible-playbook playbooks/monitoring/full.yml

Nginx Reverse Proxy

Deploy Nginx as a reverse proxy with SSL termination for all services:

ansible-playbook playbooks/setup_nginx.yml -i inventory.ini --ask-become-pass

See docs/nginx/README.md for details.

Configuration

Key configuration in group_vars/srsilo/main.yml:

# Data retention
srsilo_retention_days: 7               # Delete indexes older than 7 days
srsilo_retention_min_keep: 2           # Always keep at least 2 indexes

# Fetch configuration  
srsilo_fetch_days: 90                  # Fetch last 90 days of data
srsilo_fetch_max_reads: 172500000      # 172.5M reads for production

# Processing (Production: 377GB RAM server)
srsilo_chunk_size: 1000000             # Large chunks for high-memory environment
srsilo_docker_memory_limit: 340g       # 90% of 377GB RAM

# For testing with constrained resources (8GB RAM):
# Use: -e "@playbooks/srsilo/vars/test_vars.yml"

See docs/srsilo/ARCHITECTURE.md for complete configuration reference.