Time series imputation single sample

[quasirng]

To use this tool for imputing values from long multivariate time series data (single observable path for each feature/variable), I presume we need to utilize some sort of sliding window approach to create samples? If there is another approach please describe? The worry is when we have large gaps to fill ... there could be multiple samples with all missing samples. Would this pose a problem?

[WenjieDu]

Hi @quasirng 👋,

Thank you so much for your attention to PyPOTS! You can follow me on GitHub to receive the latest news of PyPOTS. If you find PyPOTS helpful to your work, please star⭐️ this repository. Your star is your recognition, which can help more people notice PyPOTS and grow PyPOTS community. It matters and is definitely a kind of contribution to the community.

I have received your message and will respond ASAP. Thank you for your patience! 😃

Best,
Wenjie

[WenjieDu]

Yes, you need the sliding window, which is also provided in pypots, try pypots.data.utils.sliding_window.

You should ensure that the samples at least have some observed data point, though sparse. Otherwise, all-missing samples will lead to the imputation results being not credible. You can try to increase the window length of the sliding_window to ensure that generated samples have data at the head or tail.

[quasirng]

Thanks for clarifications and suggestions! To be clear, in some boundary cases we will have some features which have full histories and some features to have very large gaps. But I suspect your point may still hold if samples are completely empty in particular feature it may not credible! But these are indeed just some tough boundary sub-cases where things are definitely missing not-at-random.. otherwise super helpful tools here. Thanks!

[WenjieDu]

According to your clarification, imputation could still work for such boundary cases, because deep learning methods like SAITS can infer the missing features on the basis of observed features. BTW, our time series imputation benchmark https://arxiv.org/abs/2406.12747 may be helpful to you, which is the first comprehensive benchmark in the literature. You can also follow me on GitHub to receive future updates about PyPOTS ;-)

[quasirng]

Interesting, thanks!