Roy, S. B., & Yuan, M. (2023, October 16). Multi-attention based Feature Embedding for Irregular Asynchronous Time Series Modelling. IECON 2023- 49th Annual Conference of the IEEE Industrial Electronics Society. https://doi.org/10.1109/iecon51785.2023.10312633
Abstract:
Forecasting time series values based on historic
covariates has been an active area of research in statistics and
machine learning. With the availability of computation resources
and big data infrastructure supporting massive volume, velocity
and variety, the algorithms have evolved from classic statistical
learning to neural-network driven loss minimisation techniques.
While state of the art attention and self-attention-transformers
have shown promise of improved performance with sufficient
training data, most of them fail to generalise to different problems
of time-series modelling (such as classification and extremum
forecasting) with asynchronously sampled covariates. This paper
introduces the concept of a generalised time series embedding
and transfer learning for time series (analogous to token-tovector
or image-to-vector embeddings in language and vision
models respectively) that allow joint training with a unified
interface. The major benefit of this work is a unified embedding
model employing multi-attention for feature representation which
enables benchmark performance against state of the art models
from recent literature.
License type:
Publisher Copyright
Funding Info:
This research / project is supported by the A*STAR - Supply Chain (SC) 4.0 – Digital Supply Chain Development via Platform Technologies Programme
Grant Reference no. : M21J6a0080