🤖 Binary Options ML Pipeline

A production-ready machine learning pipeline for predicting 15-minute binary option direction on M5 forex candles. Trained and validated on 10+ years of EURUSD, GBPUSD, and USDJPY data.

---

📋 Table of Contents

• Overview
• Results at a Glance
• How It Works
• Pipeline Architecture
• Feature Engineering
• Edge Discovery
• Performance Deep Dive
• Quick Start
• Project Structure
• Bring Your Own Data
• Risk Disclaimer

---

🎯 Overview

This project builds a complete end-to-end ML pipeline that finds a statistically validated edge in binary options trading on forex M5 charts.

The core strategy: At every M5 candle close, compute a LightGBM probability score. When the model is confident enough (|prob − 0.5| ≥ 0.06), place a 15-minute binary option in the predicted direction. Only ~9.5% of candles qualify — but those that do win at 58.47%, beating the 55.56% breakeven.

Key Innovations

What	Why it matters
15-min expiry target (close[T+3] > close[T])	Longer horizon has more directional signal than single 5-min candles
Confidence filtering (|prob − 0.5| ≥ 0.06)	Eliminates low-signal noise trades, lifts accuracy from 52% → 58.47%
Walk-forward validation	10/12 windows profitable before committing to final model
Multi-pair training	EURUSD + GBPUSD + USDJPY trained jointly → more data, better generalization
Mean-reversion features	M5 forex is slightly mean-reverting (autocorr = −0.02); model exploits this

---

📊 Results at a Glance

Test Period: Dec 2024 → May 2026  |  Truly Out-of-Sample

Metric	Our Model	Benchmark	Meaning
🎯 Win Rate	58.47%	55.56%	Minimum to be profitable at 80% payout
💰 Net P&L	+668.6 units	0 units	Break-even
📅 Profitable Months	15 / 18	9 / 18	Coin-flip baseline (50%)
📉 Max Drawdown	80.4 units	—	Lower is better
⚖️ Profit Factor	1.126	1.000	Below 1.0 = losing money
🔢 Total Trades	12,763	134,479 (unfiltered)	Confidence filter removes 90.5% of noise
📆 Avg Trades/Month	~709 (~23/day)	~7,471 (unfiltered)	Selective = higher quality signals
🗂️ Coverage	9.5% of candles	100%	Only trade top 9.5% confident signals

💵 Real-Money Translation

Stake per Trade	Monthly Avg	18-Month Total
$1	~$37	~$668
$10	~$371	~$6,686
$50	~$1,857	~$33,430
$100	~$3,714	~$66,860

P&L formula: Win = +80% of stake, Loss = −100% of stake

---

⚙️ How It Works

flowchart LR
    A[🕯️ M5 Candle\nCloses] --> B[Compute\n260 Features]
    B --> C{Model\nProbability}
    C -->|prob > 0.56| D[📈 CALL\n15-min expiry]
    C -->|prob < 0.44| E[📉 PUT\n15-min expiry]
    C -->|0.44 ≤ prob ≤ 0.56| F[⏭️ Skip\nLow confidence]
    D --> G[Wait 3 M5\nCandles = 15 min]
    E --> G
    G --> H{close T+3\nvs close T}
    H -->|+0.80 units| I[✅ Win]
    H -->|-1.00 units| J[❌ Loss]

Loading

The Confidence Gate

All candles: 134,479   →   52.2% accuracy   →   −8,038 units  ❌
Filtered 9.5%: 12,763  →   58.47% accuracy  →   +668.6 units  ✅

The model assigns a probability to every candle. Only signals where the model is most "sure" (distance from 50% ≥ 6%) are acted on — the rest are ignored.

Threshold  │  Coverage  │  Accuracy  │  Net P&L
───────────┼────────────┼────────────┼──────────
  0.00     │   100.0%   │   51.88%   │  −13,063
  0.04     │    21.0%   │   56.72%   │   +869
  0.06 ✅  │     9.3%   │   60.55%   │  +1,647   ← Optimal on Val
  0.08     │     4.9%   │   63.82%   │  +1,432
  0.10     │     2.5%   │   67.67%   │  +1,060

---

🏗️ Pipeline Architecture

flowchart TD
    subgraph DATA["📁 Data Layer"]
        R1[Raw EURUSD\nM5/M1/H1/H4/D]
        R2[Raw GBPUSD\nM5/M1/H1/H4/D]
        R3[Raw USDJPY\nM5/M1/H1/H4/D]
    end

    subgraph FEAT["🔧 Feature Layer"]
        F1[260 Features\nper pair]
    end

    subgraph SPLIT["✂️ Split Layer"]
        TR[Train\n≤ 2022-12-31\n845k rows]
        VA[Val\n2023–2024\n201k rows]
        TE[Test\n2025+\n136k rows]
    end

    subgraph MODEL["🧠 Model Layer"]
        LG[LightGBM\nlgbm_v2\n276 trees]
    end

    subgraph INFER["🎯 Inference"]
        CF[Confidence\nFilter ≥ 0.06]
        SIG[Trade Signal\nCALL / PUT]
    end

    DATA --> FEAT --> SPLIT
    TR --> MODEL
    VA -->|"Threshold\nTuning"| MODEL
    TE -->|"Final\nEvaluation"| MODEL
    MODEL --> INFER

Loading

The 6-Step Pipeline

Step	Script	Description	Output
1️⃣	01_prepare_data.py	Load, validate & clean EURUSD raw data across all timeframes	eurusd_*_clean.parquet
2️⃣	02_build_features.py	Compute all 248 base features for EURUSD (M5, M1, MTF, temporal, volatility)	eurusd_features.parquet
3️⃣	03_prepare_multi_pair.py	Same as Step 1, for GBPUSD + USDJPY	gbpusd/usdjpy_*_clean.parquet
4️⃣	04_build_features_multi_pair.py	Same as Step 2, for GBPUSD + USDJPY	gbpusd/usdjpy_features.parquet
5️⃣	05_combine_and_split.py	Merge all 3 pairs, add target_1c, time-based train/val/test split	train/val/test.parquet
6️⃣	06_train_final_model.py	Add target_3c + exhaustion features, train lgbm_v2, sweep threshold, evaluate	lgbm_v2.joblib

---

🔬 Feature Engineering

260 features across 7 groups, all strictly backward-looking (zero leakage):

Group	Count	Examples
📊 M5 Price/Technical	~80	RSI-14, MACD, EMA-8/21/50, Bollinger, candle body ratio, streak counts
⏱️ M1 Micro-structure	~25	M1 momentum within current M5, micro volume, tick direction
🕐 Multi-Timeframe (HTF)	~60	H1/H4/Daily closed candle direction, alignment signals
🔄 Partial HTF	~30	In-progress H1/H4 cumulative stats (only from closed M5s)
🕒 Temporal	~20	Hour, session (Asian/London/NY), day-of-week, is-session-open
💥 Volatility	~25	ATR-14, Parkinson vol, regime flags, vol-of-vol
🔁 Mean-Reversion (new)	~17	RSI exhaustion, cumulative return vs ATR, wick asymmetry, swing distance

Mean-Reversion Features (the edge)

# After N consecutive same-direction candles, mean reversion probability rises
cum_ret_3   = (close - close.shift(3)) / close.shift(3)     # 3-candle cumulative return
move_vs_atr = abs(close - close.shift(5)) / ATR_14          # Move size vs recent volatility
rsi_dist_from_70 = 70 - RSI_14                              # Distance from overbought
pct_from_top_20  = (swing_high_20 - close) / range_20       # Position within 20-bar range
wick_asymmetry   = (upper_wick - lower_wick) / total_wick   # Rejection signal

---

🧪 Edge Discovery

The honest finding: Predicting every M5 candle direction gives ~52% accuracy — unprofitable. The real edge only appears when combining a longer expiry target with high-confidence filtering.

xychart-beta
    title "Walk-Forward Accuracy by Approach (22 windows)"
    x-axis ["1-candle (5min)", "3-candle (15min)", "conf≥0.08 + 3c", "conf≥0.10 + 3c"]
    y-axis "Accuracy %" 48 --> 60
    bar [51.53, 51.77, 55.34, 56.82]
    line [55.56, 55.56, 55.56, 55.56]

Loading

Walk-Forward Results Summary

Approach	Windows Profitable	Mean Accuracy	Mean P&L	Verdict
1-candle target (5 min)	0 / 12	51.53%	−3,451	❌ No Edge
3-candle target (15 min)	0 / 12	51.77%	−3,243	❌ No Edge
Session-open only	0 / 12	50.64%	−405	❌ No Edge
conf ≥ 0.08 + 3c	5 / 12	55.34%	−42	⚠️ Marginal
conf ≥ 0.10 + 3c	10 / 12	56.82%	+78	✅ Edge Found

Walk-forward setup: Train on 18 months → test on next 6 months → slide by 6 months. 12 total out-of-sample windows from 2019–2026.

---

📈 Performance Deep Dive

Monthly P&L Breakdown (Test Set)

Month	Trades	Win Rate	P&L
2025-01	753	54.85%	−9.6	🔴
2025-02	651	57.91%	+27.6	🟢
2025-03	771	55.90%	+4.8	🟢
2025-04	802	57.23%	+24.2	🟢
2025-05	781	61.33%	+81.2	🟢
2025-06	882	53.74%	−28.8	🔴
2025-07	908	58.48%	+47.8	🟢
2025-08	757	59.97%	+60.2	🟢
2025-09	760	65.26%	+132.8	🟢
2025-10	810	59.38%	+55.8	🟢
2025-11	701	57.92%	+29.8	🟢
2025-12	763	62.52%	+95.6	🟢
2026-01	796	55.28%	−4.0	🔴
2026-02	776	56.31%	+10.6	🟢
2026-03	846	57.33%	+27.0	🟢
2026-04	771	61.61%	+84.0	🟢
2026-05	222	62.16%	+26.4	🟢

Per-Pair Breakdown

Pair	Trades	Accuracy	Net P&L	Profit Factor
🇪🇺 EURUSD	4,491	57.63%	+167.4	1.088	✅
🇬🇧 GBPUSD	4,355	60.39%	+379.0	1.220	✅
🇯🇵 USDJPY	3,917	57.29%	+122.2	1.073	✅
Combined	12,763	58.47%	+668.6	1.126	✅

---

🚀 Quick Start

Prerequisites

Python 3.10+

Installation

git clone https://github.com/yourusername/binary-options-ml.git
cd binary-options-ml
pip install -r requirements.txt

Data Setup

Place your raw parquet files in data/raw/ following this naming convention:

data/raw/
├── EURUSD_M5.parquet
├── EURUSD_M1.parquet
├── EURUSD_H1.parquet
├── EURUSD_H4.parquet
├── EURUSD_D.parquet
├── GBPUSD_M5.parquet   # same structure
└── USDJPY_M5.parquet   # same structure

Run the Full Pipeline

# Step 1-2: Process EURUSD (takes ~10-15 min for M1 features)
python scripts/01_prepare_data.py
python scripts/02_build_features.py

# Step 3-4: Process GBPUSD + USDJPY
python scripts/03_prepare_multi_pair.py
python scripts/04_build_features_multi_pair.py

# Step 5: Combine & split
python scripts/05_combine_and_split.py

# Step 6: Train model + full evaluation (takes ~5 min)
python scripts/06_train_final_model.py

Use the Trained Model

import joblib
import pandas as pd
import numpy as np

# Load model
model = joblib.load("models/lgbm_v2.joblib")
with open("models/lgbm_v2_features.txt") as f:
    feat_cols = [l.strip() for l in f if l.strip()]

CONF_THRESHOLD = 0.06   # from lgbm_v2_config.txt

# At each M5 candle close, after computing features:
X = df[feat_cols].values[-1:]          # current candle features
prob = model.predict_proba(X)[0, 1]    # probability of UP
conf = abs(prob - 0.5)

if conf >= CONF_THRESHOLD:
    direction = "CALL" if prob > 0.5 else "PUT"
    print(f"→ Place {direction}  |  prob={prob:.3f}  |  conf={conf:.3f}")
    print(f"  Expiry: 15 minutes from now")
else:
    print("→ Skip (low confidence)")

---

📁 Project Structure

binary-options-ml/
│
├── 📜 config.py                        # All settings (paths, payout, dates, params)
│
├── 📂 scripts/                         # Pipeline — run in order 01 → 06
│   ├── 01_prepare_data.py              # Clean & validate EURUSD raw data
│   ├── 02_build_features.py            # Build 248 features for EURUSD
│   ├── 03_prepare_multi_pair.py        # Clean GBPUSD + USDJPY
│   ├── 04_build_features_multi_pair.py # Features for GBPUSD + USDJPY
│   ├── 05_combine_and_split.py         # Merge pairs → train/val/test split
│   └── 06_train_final_model.py         # Train lgbm_v2, evaluate, save
│
├── 📂 src/                             # Reusable modules
│   ├── data_loader.py                  # Raw data loading & validation
│   └── features/
│       ├── m5_features.py              # Price, technical, momentum features
│       ├── m1_features.py              # M1 micro-structure features
│       ├── mtf_features.py             # Multi-timeframe (H1/H4/D) features
│       ├── partial_htf_features.py     # In-progress HTF candle stats
│       ├── temporal.py                 # Time-of-day, session features
│       └── volatility.py              # ATR, Parkinson, regime features
│
├── 📂 models/
│   ├── lgbm_v2.joblib                  # Trained LightGBM model
│   ├── lgbm_v2_features.txt            # 260 feature names (ordered)
│   └── lgbm_v2_config.txt             # Strategy config (threshold, expiry)
│
├── 📂 reports/
│   └── lgbm_v2_backtest.png            # Static backtest chart
│
└── 📂 data/
    ├── raw/                            # Raw parquet files (not committed)
    └── processed/                      # Processed features & splits

---

📦 Bring Your Own Data

Where to Get Data

Source	Free	Format	Notes
Dukascopy	✅	CSV/JForex	Best free tick & OHLCV source
MetaTrader 4/5	✅	CSV export	Export from History Center
TrueFX	✅	CSV	Tick data only — needs resampling
HistData.com	✅	CSV	M1 OHLCV, needs resampling to M5
Polygon.io	💰	JSON/CSV	Paid, clean, good API

Required File Format

Each raw file must be saved as Parquet in data/raw/ with this exact naming:

data/raw/
├── EURUSD_M5.parquet
├── EURUSD_M1.parquet
├── EURUSD_H1.parquet
├── EURUSD_H4.parquet
├── EURUSD_D.parquet
├── GBPUSD_M5.parquet   ← same 5 files per pair
├── GBPUSD_M1.parquet
... etc

Required Columns

Column	Type	Example	Notes
time	datetime (UTC)	2023-01-02 08:00:00+00:00	Must be UTC timezone-aware
open	float64	1.07012
high	float64	1.07045
low	float64	1.07001
close	float64	1.07038
volume	float64	1842.0	Tick volume is fine

Minimum data required: 2013 onwards gives ~10 years of training data. Less than 5 years will degrade model quality.

Convert CSV → Parquet

If your data is in CSV format, convert it with this snippet:

import pandas as pd

# Load your CSV (adjust sep and column names to match your source)
df = pd.read_csv("EURUSD_M5.csv", parse_dates=["time"])

# Rename columns to match expected format (if needed)
df = df.rename(columns={
    "Date": "time",   "Open": "open",
    "High": "high",   "Low":  "low",
    "Close": "close", "Volume": "volume"
})

# Ensure UTC timezone
df["time"] = pd.to_datetime(df["time"]).dt.tz_localize("UTC")  # if naive
# df["time"] = pd.to_datetime(df["time"]).dt.tz_convert("UTC") # if already tz-aware

# Keep only required columns and sort
df = df[["time", "open", "high", "low", "close", "volume"]].sort_values("time")

# Save as parquet
df.to_parquet("data/raw/EURUSD_M5.parquet", index=False)
print(df.head())
print(f"Shape: {df.shape}  |  Range: {df['time'].min()} → {df['time'].max()}")

Resample M1 → M5 (if needed)

If your source only provides M1 data:

import pandas as pd

m1 = pd.read_parquet("data/raw/EURUSD_M1.parquet").set_index("time")

m5 = m1.resample("5min").agg({
    "open":   "first",
    "high":   "max",
    "low":    "min",
    "close":  "last",
    "volume": "sum"
}).dropna().reset_index()

m5.to_parquet("data/raw/EURUSD_M5.parquet", index=False)

---

🔑 Key Configuration

# config.py — the only file you need to change
PAYOUT      = 0.80          # Your broker's payout rate
TRAIN_END   = "2022-12-31"  # Train/Val/Test split boundary
VAL_END     = "2024-12-31"
PAIR        = "EURUSD"      # Primary pair (for steps 01-02)
RANDOM_SEED = 42

---

🧠 Model Details

Parameter	Value
Algorithm	LightGBM GBDT
Trees	276 (early stopping on val)
Max Depth	6
Num Leaves	50
Min Child Samples	300
Learning Rate	0.01
Features Used	260
Training Samples	816,592
Target	close[T+3] > close[T] (15-min direction)
Inference Threshold	|prob − 0.5| ≥ 0.06

---

⚠️ Risk Disclaimer

This project is for research and educational purposes only.

• Past performance on backtested data does not guarantee future results
• Binary options are high-risk financial instruments; many retail traders lose money
• The +668.6 unit backtest result covers only ~18 months of test data — real-world performance may differ due to:
  • Broker spread & slippage not modeled
  • Internet latency and execution delays
  • Regime changes in forex markets
• Never trade with money you cannot afford to lose
• Always start with a demo account before using real funds
• Maximum drawdown of 80.4 units means sizing matters — risk at most 0.5–1% of account per trade

---

📄 License

This project is licensed under the MIT License — see LICENSE for details.

---

Built with ❤️ using Python, LightGBM, and rigorous walk-forward validation

If this helped you, give it a ⭐

💬 Questions or collaboration? Reach out on Telegram: @usmanch069