TSGBench is the inaugural TSG benchmark designed for the Time Series Generation (TSG) task. We are excited to share that TSGBench has received the Best Research Paper Award Nomination at VLDB 2024 🏆

TSGAssist is an interactive assistant that integrates the strengths of TSGBench and utilizes Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG) for TSG recommendations and benchmarking 🤖📊

We are actively exploring industrial collaborations in time series analytics. Please feel free to reach out (yihao_ang AT comp.nus.edu.sg) if interested 🤝✨

• Overview of TSGBench
  • Time Series Generation (TSG)
  • TSG Methods
  • TSG Datasets
  • TSG Evaluation Measures
  • Benchmarking Results
    • Main Results
    • Visualization
    • Generalization Test
• Overview of TSGAssist
• Getting Started with TSGBench
  • Configuration
  • Running TSGBench

Time Series Generation (TSG) is crucial in a range of applications, including data augmentation, anomaly detection, and privacy preservation. Given an input time series, TSG aims to produce time series akin to the original, preserving temporal dependencies and dimensional correlations while ensuring the generated time series remains useful for various downstream tasks.

TSGBench surveys a diverse range of Time Series Generation (TSG) methods by different backbone models and their specialties. The table below provides an overview of these methods along with their references.

TSGBench selects ten real-world datasets from various domains, ensuring a wide coverage of scenarios for TSG evaluation. Here, $R$ is the number of sub-matrics after preprocessing, $l$ is the series length, and $N$ is the number of series in the sub-matrics.

TSGBench considers the following evaluation measures, ranking analysis, and a novel generalization test by Domain Adaptation (DA).

1. Model-based Measures
   • Discriminitive Score (DS)
   • Predictive Score (PS)
   • Contextual-FID (C-FID)
2. Feature-based Measures
   • Marginal Distribution Difference (MDD)
   • AutoCorrelation Difference (ACD)
   • Skewness Difference (SD)
   • Kurtosis Difference (KD)
3. Distance-based Measures
   • Euclidean Distance (ED)
   • Dynamic Time Warping (DTW)
4. Visualization
   • t-SNE
   • Distribution Plot
5. Training Efficiency
   • Training Time

TSGAssist is an interactive assistant harnessing LLMs and RAG for time series generation recommendations and benchmarking.

• It offers multi-round personalized recommendations through a conversational interface that bridges the cognitive gap,
• It enables the direct application and instant evaluation of users' data, providing practical insights into the effectiveness of various methods.

We recommend using conda to create a virtual environment for TSGBench.

conda create -n tsgbench python=3.7
conda activate tsgbench
conda install --file requirements.txt

The configuration file ./config/config.yaml contains various settings to run TSGBench. It is structured into the following sections:

• Preprocessing: Configures data preprocessing. Specify the input data path using the preprocessing.original_data_path and the output path for processed data using preprocessing.output_ori_path.
• Generation: Contains the settings related to data generation.
• Evaluation: Includes the parameters required for evaluating the model's performance.

1. Set Input Data: Update the preprocessing.original_data_path in config.yaml to specify the location of your input data.

2. Run TSGBench: Execute the main script by running python ./main.py. By default, this will run the preprocessing, generation, and evaluation stages in sequence. You can skip or adjust these steps by modifying the relevant sections in the configuration file. In particular,

   (1) Preprocessing: During preprocessing, data is processed and saved to the path specified by preprocessing.output_ori_path in the configuration file.

   (2) Generation: Place your designated model structure under the ./model directory. In ./src/generation, point the model entry to your model. If necessary, provide pretrained parameters by specifying them under generation.pretrain_path. Generated data will be saved at generation.output_gen_path.

   (3) Evaluation: Select specific evaluation measures by updating the evaluation.method_list in the configuration file. The evaluation results will be saved to the path specified in evaluation.result_path.

Please consider citing our work if you use TSGBench (and/or TSGAssist) in your research:

# TSGBench
@article{ang2023tsgbench,
  title        = {TSGBench: Time Series Generation Benchmark},
  author       = {Ang, Yihao and Huang, Qiang and Bao, Yifan and Tung, Anthony KH and Huang, Zhiyong},
  journal      = {Proc. {VLDB} Endow.},
  volume       = {17},
  number       = {3},
  pages        = {305--318},
  year         = {2023}
}

# TSGAssist
@article{ang2024tsgassist,
  title        = {TSGAssist: An Interactive Assistant Harnessing LLMs and RAG for Time Series Generation Recommendations and Benchmarking},
  author       = {Ang, Yihao and Bao, Yifan and Huang, Qiang and Tung, Anthony KH and Huang, Zhiyong},
  journal      = {Proc. {VLDB} Endow.},
  volume       = {17},
  number       = {12},
  pages        = {4309--4312},
  year         = {2024}
}