Longbridge Open Data Pipeline

A small, reliable downloader for Longbridge OpenAPI historical market data.

This repository is intentionally limited to the data acquisition layer. It does not include downstream analytics, private automation, private universes, or downloaded market data.

What It Does

• Uses the official longbridge Python SDK.
• Downloads historical candlesticks with checkpointed resume.
• Supports daily and intraday periods exposed by Longbridge.
• Handles SDK retries with exponential backoff and configurable pacing.
• Writes idempotent Parquet or CSV datasets with atomic replacement.
• Tracks runs and per-symbol progress in SQLite.
• Validates schema, duplicate timestamps, OHLC consistency, and nonnegative volume/turnover.
• Quarantines vendor rows that violate row-level invariants instead of dropping an entire symbol.
• Can download Longbridge market temperature and security static info.

What Is Not Included

• Downstream analytics or decision systems.
• Production schedules or private cloud automation.
• Private symbol lists, downloaded data, logs, local state, or credentials.

Setup

python3.12 -m venv .venv
.venv/bin/python -m pip install -e ".[dev]"
cp .env.example .env

Fill .env locally with your Longbridge credentials. .env is ignored by git.

Required API key variables:

LONGBRIDGE_APP_KEY=
LONGBRIDGE_APP_SECRET=
LONGBRIDGE_ACCESS_TOKEN=

Download Candlesticks

.venv/bin/lbdp download \
  --symbols AAPL.US,MSFT.US \
  --start 2024-01-01 \
  --end 2024-12-31 \
  --period day \
  --adjust no_adjust \
  --trade-sessions intraday \
  --out data

Using a symbols file:

.venv/bin/lbdp download \
  --symbols-file configs/symbols.example.txt \
  --start 2024-01-01 \
  --end 2024-12-31

Outputs are partitioned under:

data/candlesticks/period=day/adjust=no_adjust/trade_sessions=intraday/symbol=AAPL.US/data.parquet

Rows rejected by row-level quality gates are written under:

data/quarantine/candlesticks/run_id=<run_id>/symbol=<symbol>/invalid_rows.csv

State is stored in SQLite, by default:

state/pipeline.sqlite3

Auxiliary Data

Market temperature:

.venv/bin/lbdp temperature --market HK --start 2024-01-01 --end 2024-12-31

Security static info:

.venv/bin/lbdp static-info --symbols 700.HK,9988.HK

Validate Existing Output

.venv/bin/lbdp validate --out data

Test

.venv/bin/python -m pytest -q

References

• Longbridge docs: https://open.longbridge.com/docs
• Getting started: https://open.longbridge.com/docs/getting-started
• Historical candlesticks: https://open.longbridge.com/docs/quote/pull/history-candlestick
• Python SDK reference: https://longbridge.github.io/openapi/python/reference_all/

Notes

This project is a data acquisition utility, not a trading system. It does not submit orders and does not provide investment advice. Keep credentials in local secret storage and never commit real API keys.