Skip to content

docs: time series demo notebook #494

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
seayang-nv wants to merge 3 commits into
base: main
Choose a base branch
from
+360 −0
Open

docs: time series demo notebook #494

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
e58cb60
fix: upgrade xgrammar and use new structured gen method (#541)
mckornfield Jun 3, 2026
dcff1bf
added time series notebook
seayang-nv May 14, 2026
88d04d0
Merge branch 'main' into seayang/time-series-101-notebook
seayang-nv Jun 4, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/tutorials/index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ Interactive Jupyter notebook tutorials for NeMo Safe Synthesizer.

- [Safe Synthesizer 101](safe-synthesizer-101.ipynb) -- learn the fundamentals
- [Differential Privacy](differential-privacy.ipynb) -- enable differential privacy guarantees
- [Time Series](time-series.ipynb) -- learn to generate synthetic time series data with NSS

## Adding a Tutorial

Expand Down
358 changes: 358 additions & 0 deletions docs/tutorials/time-series.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,358 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"# 📈 NeMo Safe Synthesizer Tutorial: Time Series\n",
"\n",
"#### What you'll learn\n",
"\n",
"In this notebook, we'll explore how to use NeMo Safe Synthesizer for **time-series data**: loading a time-series classification dataset, configuring the synthesizer for temporal data, generating synthetic time series, and visualizing the synthetic version of the time series data.\n",
"\n",
"A full run takes about 20 minutes on an A100. If you have not yet completed the [Safe Synthesizer 101](safe-synthesizer-101.ipynb) tutorial, consider starting there first.\n",
"\n",
"### 🖥️ Prerequisites\n",
"\n",
"This notebook requires a Linux machine with an NVIDIA GPU (H100 recommended, A100 minimum) and CUDA 12.9+. It will not run on macOS, Windows, or Apple Silicon."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### ⚡ Install Safe Synthesizer\n",
"\n",
"Run the cell below to install NeMo Safe Synthesizer (engine and CUDA 12.9) and the `aeon` toolkit for loading the dataset and running TSTR evaluation."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"%%bash\n",
"# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.\n",
"# SPDX-License-Identifier: Apache-2.0\n",
"\n",
"if command -v uv > /dev/null 2>&1; then\n",
" uv pip install \"nemo-safe-synthesizer[engine,cu129]\" --index https://flashinfer.ai/whl/cu129 --index https://download.pytorch.org/whl/cu129 --index https://wheels.vllm.ai/88d34c6409e9fb3c7b8ca0c04756f061d2099eb1/cu129 --index-strategy unsafe-best-match\n",
" uv pip install aeon matplotlib\n",
"else\n",
" pip install \"nemo-safe-synthesizer[engine,cu129]\" --extra-index-url https://flashinfer.ai/whl/cu129 --extra-index-url https://download.pytorch.org/whl/cu129 --extra-index-url https://wheels.vllm.ai/88d34c6409e9fb3c7b8ca0c04756f061d2099eb1/cu129\n",
" pip install aeon matplotlib\n",
"fi\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 📥 Load and preview the ECG200 dataset\n",
"\n",
"We use the [ECG200](https://www.timeseriesclassification.com/description.php?Dataset=ECG200) (R. Olszewski and the UCR/TSML Archive) dataset from the UCR time-series archive. Each sample is a single-lead ECG recording (96 timesteps) classified as normal or abnormal heartbeat.\n",
"\n",
"| Detail | Value |\n",
"|--------|-------|\n",
"| Features/channels | 1 ECG amplitude feature (univariate) |\n",
"| Sequence length | 96 timesteps |\n",
"| Number of sequences (train / test) | 100 / 100 |\n",
"| Classes | 2 (-1: normal, 1: abnormal) |\n",
"\n",
"We load the dataset with [aeon](https://www.aeon-toolkit.org/) and convert it to the long-format DataFrame that Safe Synthesizer expects. Each time series becomes a group of rows identified by `group_id`, with a `timestep` column for the time index.\n",
"\n",
"> Note: This demo uses a univariate time series. Safe Synthesizer also supports multivariate time series with numeric, categorical, and text features.\n",
"\n",
"> Note: Data disclaimer: Each user is responsible for checking dataset content and applicable licenses, then determining whether the dataset is suitable for the intended use."
Comment thread
seayang-nv marked this conversation as resolved.
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import warnings\n",
"\n",
"import numpy as np\n",
"import pandas as pd\n",
"from aeon.datasets import load_classification\n",
"\n",
"\n",
"def aeon_to_long_df(X, y):\n",
" \"\"\"Convert aeon arrays (n_samples, n_channels, length) to NSS long-format DataFrame.\"\"\"\n",
" n_samples, _, length = X.shape\n",
" rows = []\n",
" for i in range(n_samples):\n",
" for timestep in range(length):\n",
" rows.append({\"timestep\": timestep, \"ecg\": X[i, 0, timestep], \"label\": int(y[i]), \"group_id\": i})\n",
" return pd.DataFrame(rows)\n",
"\n",
"\n",
"load_kwargs = {\"load_equal_length\": True, \"load_no_missing\": True}\n",
"with warnings.catch_warnings():\n",
" warnings.filterwarnings(\n",
" \"ignore\",\n",
" message=\"Call to deprecated function .*load_classification.*\",\n",
" category=FutureWarning,\n",
" )\n",
" X_train_raw, y_train = load_classification(\"ECG200\", split=\"train\", **load_kwargs)\n",
" X_test_raw, y_test = load_classification(\"ECG200\", split=\"test\", **load_kwargs)\n",
"\n",
"train_df = aeon_to_long_df(X_train_raw, y_train)\n",
"test_df = aeon_to_long_df(X_test_raw, y_test)\n",
"\n",
"print(f\"Train: {X_train_raw.shape[0]} series × {X_train_raw.shape[2]} timesteps = {len(train_df)} rows\")\n",
"print(f\"Test: {X_test_raw.shape[0]} series × {X_test_raw.shape[2]} timesteps = {len(test_df)} rows\")\n",
"train_df.head(10)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 🔍 Visualize real ECG traces\n",
"\n",
"Before generating synthetic data, let’s look at a few real ECG recordings from each class."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import matplotlib.pyplot as plt\n",
"\n",
"fig, axes = plt.subplots(1, 2, figsize=(12, 3.5), sharey=True)\n",
"class_labels = {-1: \"Normal (-1)\", 1: \"Abnormal (1)\"}\n",
"\n",
"for ax, cls in zip(axes, [-1, 1]):\n",
" group_ids = train_df[train_df[\"label\"] == cls][\"group_id\"].unique()[:5]\n",
" for gid in group_ids:\n",
" series = train_df[train_df[\"group_id\"] == gid]\n",
" ax.plot(series[\"timestep\"].values, series[\"ecg\"].values, alpha=0.6)\n",
" ax.set_title(f\"Class: {class_labels[cls]}\")\n",
" ax.set_xlabel(\"Timestep\")\n",
"\n",
"axes[0].set_ylabel(\"ECG amplitude\")\n",
"fig.suptitle(\"Real ECG200 traces (5 per class)\", fontsize=13)\n",
"plt.tight_layout()\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### ⚙️ Configure and run Safe Synthesizer\n",
"\n",
"Create the Safe Synthesizer builder and configure it for time-series synthesis:\n",
"\n",
"- `with_time_series` enables time-series mode and specifies the timestamp column.\n",
"- `with_data` tells the synthesizer that rows sharing the same `group_id` belong to a single time series. We set `holdout=0` so the tutorial trains on all 100 training sequences; TSTR evaluation uses the original ECG200 test split instead.\n",
"- `with_replace_pii(enable=False)` disables PII replacement because this numeric ECG dataset does not contain PII columns.\n",
"- `with_train` sets demo-specific training hyperparameters.\n",
"- `with_generate` enables timestamp fidelity enforcement. `num_records` is not applicable for time series mode as generation will generate all groups.\n",
"\n",
"We skip the built-in evaluation step because time-series-specific quality metrics (TSTR, similarity scores, etc.) are under active development and will be available in a future release. Instead, we demonstrate TSTR evaluation manually in a later section.\n",
"\n",
"Refer to the [configuration docs](../user-guide/configuration.md) for the full list of options."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from nemo_safe_synthesizer.sdk.library_builder import SafeSynthesizer\n",
"\n",
"builder = (\n",
" SafeSynthesizer()\n",
" .with_data_source(train_df)\n",
" .with_time_series(\n",
" is_timeseries=True,\n",
" timestamp_column=\"timestep\",\n",
" )\n",
" .with_data(\n",
" group_training_examples_by=\"group_id\",\n",
" max_sequences_per_example=None,\n",
" holdout=0,\n",
" )\n",
" .with_replace_pii(enable=False)\n",
" .with_train(\n",
" # These settings were selected for this small ECG200 demo.\n",
" num_input_records_to_sample=144000,\n",
" learning_rate=5e-4,\n",
" lora_r=32,\n",
" )\n",
" .with_generate(\n",
" enforce_timeseries_fidelity=True,\n",
" )\n",
" .with_evaluate(enabled=False)\n",
")\n",
"builder.run()\n",
"results = builder.results"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 📤 Retrieve synthetic data\n",
"\n",
"Inspect the generated synthetic data including row count and preview of the first rows."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"synth = results.synthetic_data\n",
Comment thread
seayang-nv marked this conversation as resolved.
"n_synth_groups = synth[\"group_id\"].nunique()\n",
"print(f\"Generated {n_synth_groups} synthetic time series ({len(synth)} rows)\")\n",
"synth.head(10)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 🔬 Visual comparison: Real vs. Synthetic\n",
"\n",
"Let’s compare real and synthetic ECG traces side by side for each class."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"fig, axes = plt.subplots(2, 2, figsize=(12, 7), sharey=True)\n",
"\n",
"for col, (source_name, source_df) in enumerate([(\"Real\", train_df), (\"Synthetic\", synth)]):\n",
" for row, cls in enumerate([-1, 1]):\n",
" ax = axes[row, col]\n",
" group_ids = source_df[source_df[\"label\"] == cls][\"group_id\"].unique()[:8]\n",
" for gid in group_ids:\n",
" series = source_df[source_df[\"group_id\"] == gid]\n",
" ax.plot(series[\"timestep\"].values, series[\"ecg\"].values, alpha=0.6)\n",
" ax.set_title(f\"{source_name} — {class_labels[cls]}\")\n",
" ax.set_xlabel(\"Timestep\")\n",
" axes[0, col].set_ylabel(\"ECG amplitude\")\n",
" axes[1, col].set_ylabel(\"ECG amplitude\")\n",
"\n",
"fig.suptitle(\"Real vs. Synthetic ECG200 traces (8 per class)\", fontsize=13)\n",
"plt.tight_layout()\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 🎯 Evaluate with TSTR (Train on Synthetic, Test on Real)\n",
"\n",
"[TSTR](https://arxiv.org/abs/1706.02633) is the standard way to measure whether synthetic data preserves the discriminative patterns of the original. The idea is simple:\n",
"\n",
"1. Baseline: Train a classifier on *real* data, evaluate on the held-out *real* test set.\n",
"2. TSTR: Train the same classifier on *synthetic* data, evaluate on the *real* test set.\n",
"\n",
"If the synthetic data captures the class-relevant temporal structure, the TSTR accuracy should be close to the baseline. We use [MiniRocket](https://arxiv.org/abs/2012.08791), a fast and accurate time-series classifier, for this comparison.\n",
"\n",
"The first three timesteps are used as prefill/context records during generation, so they are not part of the generated signal we want to score. We drop those prefill rows before TSTR training and apply the same trim to the real test split so train and test sequences have matching lengths.\n",
"\n",
"> Note: Built-in time-series evaluation metrics—including automated TSTR—are under active development and will be integrated into the Safe Synthesizer pipeline in a future release."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from aeon.transformations.collection.convolution_based import MiniRocket\n",
"from sklearn.linear_model import RidgeClassifierCV\n",
"from sklearn.pipeline import make_pipeline\n",
"\n",
"\n",
"def long_df_to_aeon(df, value_col=\"ecg\"):\n",
" \"\"\"Convert NSS long-format DataFrame back to aeon (n_samples, n_channels, length) arrays.\"\"\"\n",
" groups = df.groupby(\"group_id\")\n",
" X_list, y_list = [], []\n",
" for _, grp in groups:\n",
" grp = grp.sort_values(\"timestep\")\n",
" X_list.append(grp[value_col].values)\n",
" y_list.append(grp[\"label\"].iloc[0])\n",
" X = np.array(X_list)[:, np.newaxis, :] # (n_samples, 1, length)\n",
" y = np.array(y_list, dtype=str)\n",
" return X, y\n",
"\n",
"\n",
"prefill_timesteps = 3\n",
"train_eval_df = train_df[train_df[\"timestep\"] >= prefill_timesteps]\n",
"test_eval_df = test_df[test_df[\"timestep\"] >= prefill_timesteps]\n",
"synth_eval_df = synth[synth[\"timestep\"] >= prefill_timesteps]\n",
"\n",
"X_train_real, y_train_real = long_df_to_aeon(train_eval_df)\n",
"X_test_eval, y_test_eval = long_df_to_aeon(test_eval_df)\n",
"X_train_synth, y_train_synth = long_df_to_aeon(synth_eval_df)\n",
"\n",
"\n",
"# Baseline: train on REAL, test on real\n",
"pipe_real = make_pipeline(MiniRocket(), RidgeClassifierCV(alphas=np.logspace(-3, 3, 10)))\n",
"pipe_real.fit(X_train_real, y_train_real)\n",
"real_acc = pipe_real.score(X_test_eval, y_test_eval)\n",
"\n",
"# TSTR: train on SYNTHETIC, test on real\n",
"pipe_synth = make_pipeline(MiniRocket(), RidgeClassifierCV(alphas=np.logspace(-3, 3, 10)))\n",
"pipe_synth.fit(X_train_synth, y_train_synth)\n",
"synth_acc = pipe_synth.score(X_test_eval, y_test_eval)\n",
"\n",
"print(f\"Real-trained accuracy: {real_acc:.3f}\")\n",
"print(f\"Synthetic-trained accuracy: {synth_acc:.3f}\")\n",
"print(f\"TSTR gap: {real_acc - synth_acc:+.3f}\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 🚀 Interpreting results and next steps\n",
"\n",
"The synthetic-trained classifier will usually trail the real-trained baseline, especially in a small demo dataset like ECG200. A smaller TSTR gap suggests that the synthetic data preserved more class-relevant temporal structure; a larger gap is a signal to inspect the synthetic traces, class balance, and generated sequence lengths.\n",
"\n",
"To reduce the gap on your own dataset, try more training examples and tune generation quality with settings such as `learning_rate`, `lora_r`, and `num_input_records_to_sample`. See the [configuration docs](../user-guide/configuration.md) for the full parameter reference.\n",
"\n",
"Built-in time-series evaluation metrics are still under development. Until those are available, TSTR and visual inspection are useful manual checks for this workflow."
]
}
],
"metadata": {
"kernelspec": {
"display_name": ".venv",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
Comment thread
seayang-nv marked this conversation as resolved.
"version": "3.11.14"
}
},
"nbformat": 4,
"nbformat_minor": 4
}
1 change: 1 addition & 0 deletions mkdocs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -176,6 +176,7 @@ nav:
- tutorials/index.md
- Safe Synthesizer 101: tutorials/safe-synthesizer-101.ipynb
- Differential Privacy: tutorials/differential-privacy.ipynb
- Time Series: tutorials/time-series.ipynb
- User Guide:
- Getting Started: user-guide/getting-started.md
- Running Safe Synthesizer: user-guide/running.md
Expand Down
Loading