- Introduction
- TSG Methods
- Datasets
- Evaluation Measures
- Benchmarking Results
- Getting Started with TSGBench
TSGBench is the inaugural comprehensive benchmark designed for the Time Series Generation (TSG) task.
TSGBench surveys a diverse range of Time Series Generation (TSG) methods by different backbone models and their specialties. The table below provides an overview of these methods along with their references.
TSGBench selects ten real-world datasets from various domains, ensuring a wide coverage of scenarios for TSG evaluation.
TSGBench considers the following evaluation measures, ranking analysis, and a novel generalization test by Domain Adaptation (DA).
-
Model-based measures
- Discriminitive Score (DS)
- Predictive Score (PS)
- Contextual-FID (C-FID)
-
Feature-based measures
- Marginal Distribution Difference (MDD)
- AutoCorrelation Difference (ACD)
- Skewness Difference (SD)
- Kurtosis Difference (KD)
-
Distance-based measures
- Euclidean Distance (ED)
- Dynamic Time Warping (DTW)
-
Visualization
- t-SNE
- Distribution Plot
-
Training Efficiency
Folder ./data
contains the input data, which has the shape of
Folder ./data/method_name
contains the generated data, which has the shape of
We recommend using conda
to create a virtual environment.
conda create -n tsgbench python=3.7
conda activate tsgbench
conda install --file requirements.txt
First, users can first preprocess the data by
python preprocessing/data_preprocess.py
Users can also preprocess the data for the generalization test by
python preprocessing/data_preprocess_generalization.py
Then, users can then generate time series and store the generated time series in ./data/method_name/
.
Finally, users can run evaluations by
cd eval
# DS & PS
python ds_ps.py --method_name METHOD_NAME --dataset_name DATASET_NAME --dataset_state DATASET_STATE --gpu_id 0 --gpu_fraction 0.99
# C-FID
python c_fid/c_fid.py --method_name METHOD_NAME --dataset_name DATASET_NAME --dataset_state DATASET_STATE --gpu_id 0
# MDD, ACD, SD, KD, ED, DTW
python feature_distance_eval.py --method_name METHOD_NAME --dataset_name DATASET_NAME --dataset_state DATASET_STATE --gpu_id 0
# t-SNE, Distribution Plot
python visualization.py --method_name METHOD_NAME --dataset_name DATASET_NAME --dataset_state DATASET_STATE
Please consider citing our work if you use TSGBench in your research:
@article{ang2023tsgbench,
title = {TSGBench: Time Series Generation Benchmark},
author = {Ang, Yihao and Huang, Qiang and Bao, Yifan and Tung, Anthony KH and Huang, Zhiyong},
journal = {Proc. {VLDB} Endow.},
volume = {17},
number = {3},
pages = {305--318},
year = {2023}
}