SEMPO is a novel time series foundation model with significantly reduced model size and pre-training scale, yet demonstrating superior generalization ability on diverse downstream forecasting tasks.
-
Oct 2025: Release of SEMPO library, along with SEMPO preprint now on arXiv.
-
Sep 2025: The SEMPO Paper has been accepted to NeurIPS 2025 as a Poster presentation!
This work proposes SEMPO, a novel lightweight foundation model that requires pretraining on relatively small-scale data, yet exhibits strong general time series forecasting. SEMPO comprises two key modules: 1) energy-aware SpEctral decomposition module, that substantially improves the utilization of pre-training data by modeling not only the high-energy frequency signals but also the low-energy yet informative frequency signals that are ignored in current methods; and 2) Mixture-of-PrOmpts enabled Transformer, that learns heterogeneous temporal patterns through small dataset-specific prompts and adaptively route time series tokens to these prompt-based experts for parameter-efficient model adaptation across different datasets and domains. Equipped with these modules, SEMPO significantly reduces both pre-training data scale and model size, while achieving strong generalization.
All pretraining datasets, except PEMS04 and PEMS07, use the numpy format UTSD. You can access the datasets from Google Drive, then place the downloaded contents under ./dataset.
Install Pytorch 2.1.2+cu118 with Python 3.10, and then install the dependencies:
pip install -r requirements.txt
To pretrain the model(s) in the paper, run the follow command, which supports both single-GPU and multi-GPU execution on a single node. For convenience, we provide a single-GPU pretrained model in the folder ./checkpoints/.
bash ./scripts/time_series_forecasting/pretrain/sempo_utsd.sh
To fine-tune the model(s) in the paper, use the few-shot examples in the folder ./scripts/time_series_forecasting/few_shot. Run with --is_pretraining 0, --is_training 1, and --is_zeroshot 0, using two configurations of 5% and 10%. For example:
bash ./scripts/time_series_forecasting/few_shot/sempo_ETTh1.sh
To evaluate the model(s) in the paper, use the zero-shot examples in the folder ./scripts/time_series_forecasting/zero_shot. Run with --is_pretraining 0, --is_training 0, and --is_zeroshot 1. For example:
bash ./scripts/time_series_forecasting/zero_shot/sempo_weather.sh
To evaluate other advanced foundation models such as Chronos-Bolt, first download their pretrained weights from HuggingFace and place the downloaded weights under ./models. Then use the zero-shot examples above and run with:
--task_name long_term_forecast_chronos --model Chronos
If you're using this repo in your research or applications, please cite our paper:
@misc{he2025sempolightweightfoundationmodels,
title={SEMPO: Lightweight Foundation Models for Time Series Forecasting},
author={Hui He and Kun Yi and Yuanchi Ma and Qi Zhang and Zhendong Niu and Guansong Pang},
year={2025},
eprint={2510.19710},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2510.19710},
}
We appreciate the following GitHub repos for providing valuable code bases and efforts.
- Time-MoE [repo]
- TTM [repo]
- uni2ts [repo]
- chronos-forecasting [repo]
- Large-Time-Series-Model [repo]
- gift-eval [repo]
- FilterNet [repo]
- TSLANet [repo]
This project is licensed under the Apache-2.0 License.