This repository contains a quantitative finance project aimed at developing a machine learning-based trading strategy to consistently outperform the S&P 500 index. Built as a rigorous data science pipeline, the system extracts signals from technical indicators, validates models using time-series specific cross-validation, and backtests financial performance.
The architecture follows a strict pipeline to prevent data leakage and ensure statistical robustness:
- Feature Engineering: Calculation of technical indicators (Bollinger Bands, RSI, MACD) on S&P 500 constituent OHLCV data without look-ahead bias. The target is defined as
sign(return(D+1, D+2)). - Cross-Validation: Implementation of Time Series Split / Blocking Time Series Split to handle the non-stationary nature of financial data and identify distinct market regimes.
- Machine Learning Pipeline: A complete model training phase (Imputation, Scaling, Dimensionality Reduction, and ML Model) utilizing Out-of-Fold (OOF) predictions to generate untainted trading signals.
- Strategy Backtesting: Conversion of ML probability signals into actionable financial positions (e.g., Long/Short, Stock Picking), evaluated via strict PnL and Maximum Drawdown metrics against the S&P 500 benchmark.
To isolate the predictive power of the machine learning models and simplify the backtesting mechanics, this project assumes the following constraints:
- Strict 1-Day Holding Period: The strategy executes exactly one action per day per asset. If a buy signal is generated based on Day D's data, the asset is purchased on Day D+1 and strictly sold on Day D+2 to measure the exact 24-hour predictive validity.
- Fixed Position Sizing (Equal Weighting): Portfolio allocation does not scale with model confidence. The strategy assumes a fixed investment of exactly $1 per day into each stock that triggers an active signal.
- Static Index Composition: The dataset assumes the S&P 500 constituents remain constant over the 5-year period. In reality, index composition frequently changes; ignoring delisted companies introduces Survivorship Bias into the backtest results.
The model relies on a mix of momentum and mean-reversion indicators to capture different market regimes:
- Relative Strength Index (RSI): Measures the magnitude of recent price changes to evaluate overbought or oversold conditions (Mean Reversion).
- MACD (Moving Average Convergence Divergence): Captures accelerating trend momentum (Momentum).
- Bollinger Bands: Provides dynamic support and resistance levels based on standard deviations of moving averages (Volatility & Mean Reversion).
- Look-Ahead Bias Prevention: Targets are strictly shifted
(D+1 to D+2)so the model only learns from data strictly available at 11:59 PM on Day D. - Expanding Window Time-Series Split: Prevents data leakage by training strictly on historical data and predicting forward. Splitting is done by Calendar Date to prevent same-day macroeconomic leakage across the 500 stocks (Panel Data).
- Out-of-Fold (OOF) Signal Generation: The final trading signal is generated by stitching together validation predictions from 10 distinct folds. This guarantees every probability was generated by a model completely blind to that specific day.
- Risk Management (Stop-Loss): To prevent catastrophic tail-risk (e.g., theoretically infinite losses from short selling), the backtester implements a strict
-5%daily stop-loss on individual trades. - Compounding Equity: Portfolio performance is measured using cumulative product (
cumprod) assuming a $1.00 base investment, preventing the mathematical illusions associated with additive cumulative sums. - Maximum Drawdown: Evaluated as the ultimate measure of risk (peak-to-trough decline) alongside Final PnL.
project/
├── data/
│ ├── HistoricalData.csv # SP500 index OHLC data
│ └── all_stocks_5yr.csv # SP500 constituents OHLCV data
├── requirements.txt # Pip fallback dependencies
├── pyproject.toml # Poetry dependency management
├── README.md # Project documentation
├── CONTRIBUTING.md # Code quality and PR guidelines
├── results/
│ ├── cross-validation/ # CV metrics, plots, and feature importance
│ ├── selected-model/ # Pickled model, hyperparameters, and ML signal
│ └── backtest/ # Markdown report, PnL plots, and backtest results
└── scripts/
├── features_engineering.py # Data processing and indicator generation
├── indicators.py # Decoupled technical indicator engine
├── model_selection.py # Time-series cross-validation logic
├── gridsearch.py # Hyperparameter tuning and model training
├── create_signal.py # OOF prediction generation
└── backtest.py # Financial strategy conversion and backtesting
This project enforces strict environment constraints to ensure reproducibility.
- Python: Standard Python 3.12+ is required. (Note for Windows users: Ensure standard Python is prioritized in your PATH and Windows App Execution Aliases for
pythonare disabled to avoid conflicts with MSYS2 environments). - Dependency Management: We use Poetry for resolving dependencies and maintaining a locked environment.
- Git Configuration: Ensure your local Git identity is configured properly before committing (check
git config --global user.nameanduser.email).
- Clone the repository:
git clone [https://github.com/mahdikheirkhah/sp500-strategies.git](https://github.com/mahdikheirkhah/sp500-strategies.git)
cd sp500-strategies
- Initialize the environment: Using Poetry (Recommended):
poetry install
Alternatively, using pip:
pip install -r requirements.txt
- Data Placement:
Ensure
HistoricalData.csvandall_stocks_5yr.csvare placed inside thedata/directory.
To execute the full quantitative pipeline, run the scripts in the following sequence from the root directory:
1. Data Processing & Feature Engineering Processes the raw OHLCV data, shifts targets to prevent leakage, and calculates indicators.
poetry run python -m scripts.features_engineering
2. Model Selection & Cross-Validation Generates the cross-validation folds and visualizations.
poetry run python -m scripts.model_selection
3. Pipeline Training & Grid Search
Trains the ML models, logs metrics, and saves the best pipeline to results/selected-model/.
poetry run python -m scripts.gridsearch
4. Signal Generation Produces the out-of-fold machine learning signals and saves them as a double-indexed CSV.
poetry run python -m scripts.create_signal
5. Strategy Backtesting Converts the generated ML signal into a simulated, risk-managed financial portfolio, calculating compounding PnL, Drawdown, and generating plots.
poetry run python -m scripts.backtest
This project adheres to clean code standards and OOP principles. Standard print() statements are prohibited; all application flow, warnings, and errors are handled via Loguru. Before submitting any Pull Requests, ensure code is formatted with Black (poetry run black .).
Mohammad Mahdi Kheirkhah Student at grit:lab, Åland
Email: mahdikheirkhah060@gmail.com