fluss-lab

Local lab for Apache Fluss with Flink, Spark, Lance, and an IoT streaming demo.

What Is Included

• infra/fluss: local Fluss cluster with ZooKeeper and MinIO/Lance storage.
• infra/flink: Flink 1.20 SQL environment connected to Fluss.
• infra/spark: Spark SQL environment connected to Fluss.
• example/iot-fluss-demo: Python IoT demo using generator, producer, Flink SQL jobs, and dashboard.

Quick Start

Add the shared local hostname once:

sudo sh -c 'echo "127.0.0.1 fluss-local" >> /etc/hosts'

Start Fluss:

cd infra/fluss
docker compose up -d

Start Flink:

cd infra/flink
docker compose up -d

Run the IoT demo:

cd example/iot-fluss-demo
uv sync
uv run python main.py state-job

In another terminal:

cd example/iot-fluss-demo
uv run python main.py generator --interval-ms 50 | uv run python main.py producer

Watch live data in Flink SQL:

cd infra/flink
docker exec -it flink-jobmanager bash
bin/sql-client.sh

CREATE CATALOG odc_fluss_catalog WITH (
  'type' = 'fluss',
  'bootstrap.servers' = 'fluss-local:9123'
);

USE CATALOG odc_fluss_catalog;
USE iot_demo;

SET 'execution.runtime-mode' = 'streaming';

SELECT device_id, sequence_no, temperature_c, status, alert_level, generated_at_ms, producer_sent_at_ms
FROM iot_telemetry_stream_v1 /*+ OPTIONS('scan.startup.mode' = 'latest') */;

More Docs

• Fluss infra
• Flink infra
• Spark infra
• IoT demo
• Dashboard