Financial Time Series Forecasting with PatchTST

This repository contains the implementation and experimental analysis of financial time series forecasting using a PatchTST Transformer backbone, developed as part of the Master of Science in Data Science program at the University of Luxembourg.

The project investigates whether additional input features and custom time-series–aware cross-validation strategies improve forecasting performance compared to standard training and validation setups.

Project Overview

The main research questions addressed in this project are:

• Can additional features beyond historical prices improve Transformer-based forecasting?
• Does a custom cross-validation strategy better capture the temporal structure of non-stationary financial time series?
• How do univariate and multivariate forecasting tasks compare under different data configurations?

To answer these questions, a PatchTST-based Transformer model is evaluated across multiple experimental setups.

Data

The study is based on historical stock market data obtained via the Alpha Vantage API.

Assets

• Oracle (ORCL)
• Cisco Systems (CSCO)
• Qualcomm (QCOM)

Input Features

• Daily closing prices
• Trading volume
• Technical indicators:
  • MACDEXT
  • RSI
  • Bollinger Bands
• Quarterly fundamental data:
  • Cash flow
  • Earnings
  • Income statements

Processed datasets are stored under data/raw/ and data/preprocessed/.

Model

The core model used in this project is PatchTST (Patch-based Time Series Transformer).

Architecture

• Channel-independent Transformer encoder
• Patch-based tokenization of time series
• Transformer backbone
• Custom prediction head for:
  • Regression (price forecasting)
  • Classification (price movement labels)

Labeling Strategy

In addition to regression, a three-class classification task is defined based on relative price changes:

• Class 0: price decrease below −5%
• Class 1: price change between −5% and +5%
• Class 2: price increase above +5%

This allows joint evaluation of regression accuracy and directional prediction performance.

Training and Cross-Validation

Two validation strategies are compared:

Default Split

• Single train/validation split
• Validation set size: 20%

Custom Time-Series Cross-Validation

• Multiple temporally ordered train/validation windows
• Preserves chronological structure
• Validation ratio approximately equal to default setup

Training notebooks for different tasks and configurations are stored under: training/.

Each notebook corresponds to a specific experiment:

• Univariate vs multivariate forecasting
• Price-only vs enriched feature sets
• Default vs custom validation splits
• TSai-based training variants

Results Summary

Key findings from the experiments:

• Models trained with default train/validation splits consistently outperform those trained with custom cross-validation
• Using only historical prices yields better results than adding technical and fundamental features
• Multivariate forecasting does not outperform univariate setups
• Additional features introduce noise rather than improving predictive performance

Overall, none of the proposed enhancements lead to statistically meaningful improvements over the baseline configuration.

Detailed quantitative results are available in the accompanying paper.

Inference

Inference notebooks and saved prediction outputs are located under: inference/.

These include:

• Model predictions
• Stored inference results in JSON format
• Jupyter notebooks for result inspection

Academic Context

• Course: Deep Learning
• Program: Master of Science in Data Science
• Institution: University of Luxembourg

This project was conducted as part of graduate coursework and is shared for educational and research purposes.

DISCLAIMER

This repository is intended for educational and reference use only.

It should not be used for financial decision-making or trading purposes.

AUTHOR

Anton Zaitsev