SentryFlow AI

Real-time network intrusion detection system combining supervised and unsupervised machine learning with automated nftables-based threat mitigation.

Architecture

sentryflow/
├── src/
│   ├── main.py                 # Daemon orchestrator
│   ├── ingestion/              # Packet capture & dataset loading
│   │   ├── sniffer.py          # Scapy AsyncSniffer wrapper
│   │   └── dataset_manager.py  # PCAP/CSV chunked loader
│   ├── features/               # Feature engineering pipeline
│   │   ├── pipeline.py         # sklearn ColumnTransformer pipeline
│   │   └── aggregator.py       # Flow-level aggregation (idle/hard timeout)
│   ├── models/                 # Dual-engine detection
│   │   ├── supervised_engine.py   # LightGBM classifier
│   │   ├── unsupervised_engine.py # Isolation Forest
│   │   └── evaluator.py        # Metrics & visualization
│   ├── prevention/             # Automated response
│   │   └── firewall.py         # nftables rule management
│   ├── dashboard/              # Streamlit real-time UI
│   │   └── app.py              # Live telemetry & XAI dashboard
│   └── utils/
│       ├── config_loader.py    # YAML config (type-safe dataclasses)
│       ├── logger.py           # Rotating file + console logger
│       └── telemetry.py        # JSON-persisted singleton store
├── config/settings.yaml        # Application configuration
├── scripts/
│   ├── capture.py              # Live packet capture to PCAP
│   └── train_pipeline.py       # Model training pipeline
└── tests/

Quick Start

pip install -r requirements.txt

# Train models from PCAP data
python scripts/train_pipeline.py

# Run detection daemon
python -m src.main

# Launch explainability dashboard (separate terminal)
streamlit run src/dashboard/app.py

How It Works

1. Capture — Live packets are ingested via Scapy sniffer or loaded from PCAP/CSV sources
2. Aggregate — Raw packets are grouped into flows with idle/hard timeout boundaries
3. Engineer — 20+ network features extracted (payload entropy, flow rates, window size, TTL, etc.)
4. Detect — LightGBM (supervised) and Isolation Forest (unsupervised) run in parallel; an alert fires if either model flags a flow
5. Block — Offending source IPs are automatically added to nftables drop rules
6. Monitor — Streamlit dashboard displays live telemetry, detection events, and feature importance

Configuration

Edit config/settings.yaml to control capture interface, ML hyperparameters, simulation mode, flow timeouts, and feature selection.

Dependencies

Python 3.12+, scikit-learn, LightGBM, Scapy, pandas, Streamlit, nftables (Linux).