Skip to content

Commit d8a5b55

Browse files
author
Michael Heller
committed
feat(schemas): add stochastic-process discrimination + process->regime->option->archetype crosswalk
Builds on the merged MemoryRegimeCharacterization (#50); reuses its Hurst / Lyapunov / kurtosis / receipt estimators (memory_regime_estimators unchanged). - process_family_estimators.py: Ornstein-Uhlenbeck generator + estimator (theta, long-run mu, half-life = ln2/theta; OU autocorr decays exponentially => canonical short_decaying / mean-reverting, anti-persistent H<0.5), fractional-OU (mean-reversion + long memory), a variance-time diffusion Hurst, seeded GBM / regime-switching / Hawkes / jump-Levy generators, and a six-family discriminator (brownian_gbm, markov_regime_switching, ornstein_uhlenbeck, fractional_brownian_motion, hawkes, jump_levy). - ProcessFamilyCharacterization schema + ProcessRegimeCrosswalk schema (the total process<->regime<->option-family<->archetype mapping), fixtures, and one validator with teeth both ways: VERIFIES seeded OU -> theta>0/finite half-life/anti-persistent, fBm -> stays long_memory, GBM/Hawkes/jump-Levy -> their families; REJECTS OU-as-long_memory, theta<=0 called mean-reverting, fBm-as-memoryless, OU missing ou_params, crosswalk OU<->long_memory / fBm<->memoryless, non-total crosswalk; option-family anchors (OU->Vasicek/Heston, fBm->rough-vol) asserted. - rough-vol tie makes the Hurst axis govern volatility itself, unifying the long-memory work with the vol surface (economic-prophet #44) by reference. - Makefile target + aggregate validate, CI workflow, README + arch doc. - proof-artifact-spine SHA-256 receipts (FIPS-180-4 algorithm; not FIPS-140).
1 parent a3b3285 commit d8a5b55

21 files changed

Lines changed: 1721 additions & 2 deletions
Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
name: process-family-crosswalk
2+
3+
on:
4+
pull_request:
5+
paths:
6+
- "schemas/process-family-characterization.schema.json"
7+
- "schemas/process-regime-crosswalk.schema.json"
8+
- "examples/process-family/**"
9+
- "examples/process-regime-crosswalk/**"
10+
- "scripts/validate_process_family_crosswalk.py"
11+
- "scripts/process_family_estimators.py"
12+
- "scripts/memory_regime_estimators.py"
13+
- "docs/architecture/process-regime-crosswalk.md"
14+
- ".github/workflows/process-family-crosswalk.yml"
15+
- "Makefile"
16+
push:
17+
branches:
18+
- main
19+
paths:
20+
- "schemas/process-family-characterization.schema.json"
21+
- "schemas/process-regime-crosswalk.schema.json"
22+
- "examples/process-family/**"
23+
- "examples/process-regime-crosswalk/**"
24+
- "scripts/validate_process_family_crosswalk.py"
25+
- "scripts/process_family_estimators.py"
26+
- "scripts/memory_regime_estimators.py"
27+
- "docs/architecture/process-regime-crosswalk.md"
28+
- ".github/workflows/process-family-crosswalk.yml"
29+
- "Makefile"
30+
workflow_dispatch:
31+
32+
jobs:
33+
validate-process-family-crosswalk:
34+
runs-on: ubuntu-latest
35+
permissions:
36+
contents: read
37+
steps:
38+
- uses: actions/checkout@v4
39+
- uses: actions/setup-python@v5
40+
with:
41+
python-version: "3.11"
42+
- name: Install validation dependency
43+
run: python -m pip install jsonschema
44+
- name: Validate process-family + crosswalk contracts (VERIFIES + REJECTS teeth)
45+
run: make validate-process-family-crosswalk

Makefile

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
PYTHON ?= python
22

3-
.PHONY: validate-upstreams validate-python validate-deploy-assets validate-agent-learning-proposal validate-scenario-learning-binding validate-governed-learning-lifecycle validate-workspace-recall-promotion validate-channel-provenance-write-gate validate-wallguard-memory-compartment-gate validate-memory-distribution-grant validate-gateway-call-audit validate-fsms-memory-schemas validate-memory-regime-characterization validate-prophet-mesh-scope-mirror validate local-preflight local-up local-smoke local-debug local-down
3+
.PHONY: validate-upstreams validate-python validate-deploy-assets validate-agent-learning-proposal validate-scenario-learning-binding validate-governed-learning-lifecycle validate-workspace-recall-promotion validate-channel-provenance-write-gate validate-wallguard-memory-compartment-gate validate-memory-distribution-grant validate-gateway-call-audit validate-fsms-memory-schemas validate-memory-regime-characterization validate-process-family-crosswalk validate-prophet-mesh-scope-mirror validate local-preflight local-up local-smoke local-debug local-down
44

55
validate-upstreams:
66
$(PYTHON) scripts/validate_upstreams.py third_party/upstreams.lock.yaml
@@ -46,7 +46,10 @@ validate-fsms-memory-schemas:
4646
validate-memory-regime-characterization:
4747
$(PYTHON) scripts/validate_memory_regime_characterization.py
4848

49-
validate: validate-upstreams validate-python validate-deploy-assets validate-wallguard-memory-compartment-gate validate-memory-distribution-grant validate-gateway-call-audit validate-fsms-memory-schemas validate-memory-regime-characterization validate-prophet-mesh-scope-mirror
49+
validate-process-family-crosswalk:
50+
$(PYTHON) scripts/validate_process_family_crosswalk.py
51+
52+
validate: validate-upstreams validate-python validate-deploy-assets validate-wallguard-memory-compartment-gate validate-memory-distribution-grant validate-gateway-call-audit validate-fsms-memory-schemas validate-memory-regime-characterization validate-process-family-crosswalk validate-prophet-mesh-scope-mirror
5053

5154
local-preflight:
5255
bash deploy/local/scripts/preflight-podman-m2.sh

README.md

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -203,6 +203,28 @@ make validate-memory-regime-characterization
203203

204204
The teeth fire in both directions: **VERIFIES** seeded generators classify correctly (fGn H=0.7 → `long_memory`/`power_law` with H>0.5; i.i.d. Gaussian → `memoryless`/`gaussian` with H≈0.5; AR(1) → `short_decaying`/`exponential`; logistic map → `chaotic` with λ>0); **REJECTS** refuses every incoherent record (regime/estimate contradiction, chaotic with non-positive λ, multifractal without a spectrum, `D``2−H`, sub-min-n not `provisional`, incoherent memory-type binding, tampered receipt). Each record carries a proof-artifact-spine SHA-256 hash-chain receipt (SHA-256 is the FIPS-180-4 algorithm; not a FIPS-140 module). The `risk_distribution_F` block exposes a fat-tailed / long-memory F for the RAROC `risk(F, reference, kernel, horizon)` interface (PR in flight on `economic-prophet`).
205205

206+
## Stochastic-process discrimination and crosswalk
207+
208+
Building on the regime characterizer (reusing its estimators), Memory Mesh classifies a process **path/observable** among six stochastic-process families and carries a total **process → regime → option-model family → behavioral archetype** crosswalk. It adds an Ornstein-Uhlenbeck estimator (θ, long-run μ, **half-life = ln2/θ**; OU autocorrelation decays exponentially so OU is the canonical `short_decaying` / mean-reverting, anti-persistent regime) and a variance-time diffusion Hurst, distinguishing continuous mean-reverting OU from discrete-state Markov regime-switching (Hamilton). Families: `brownian_gbm`, `markov_regime_switching`, `ornstein_uhlenbeck`, `fractional_brownian_motion`, `hawkes`, `jump_levy`. The `fractional_brownian_motion`**rough volatility** tie makes the Hurst axis govern volatility itself, unifying the long-memory work with the vol surface (`economic-prophet` #44) by reference.
209+
210+
The estimators, schemas, examples, validator, docs, and workflow live at:
211+
212+
- `scripts/process_family_estimators.py` (reuses `memory_regime_estimators.py`)
213+
- `schemas/process-family-characterization.schema.json`, `schemas/process-regime-crosswalk.schema.json`
214+
- `examples/process-family/`, `examples/process-regime-crosswalk/`
215+
- `scripts/validate_process_family_crosswalk.py`
216+
- `docs/architecture/process-regime-crosswalk.md`
217+
- `.github/workflows/process-family-crosswalk.yml`
218+
219+
Validate locally:
220+
221+
```bash
222+
python -m pip install jsonschema
223+
make validate-process-family-crosswalk
224+
```
225+
226+
Teeth: **VERIFIES** a seeded OU estimates θ>0, finite half-life, and classifies `ornstein_uhlenbeck`/`short_decaying`/anti-persistent (variance-time H<0.5); a seeded fBm(H=0.7) stays `fractional_brownian_motion`/`long_memory`; GBM/Hawkes/jump-Lévy paths classify to their families. **REJECTS** an OU labeled `long_memory`, a θ≤0 process called mean-reverting, an fBm labeled `memoryless`, a crosswalk mapping OU↔`long_memory` or fBm↔`memoryless`, and a non-total crosswalk; the anchor option families (OU→Vasicek/Heston, fBm→rough-vol) are asserted.
227+
206228
## Repository semantics
207229

208230
This is the canonical public repository for the runtime and deployment work. It is not a disposable starter artifact.
Lines changed: 100 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,100 @@
1+
# Process-family ↔ memory-regime ↔ option-family ↔ archetype crosswalk
2+
3+
## Status
4+
5+
Contract v0.1. Fixture-backed and CI-validated with teeth in both directions.
6+
Builds on the merged MemoryRegimeCharacterization contract (#50) and **reuses**
7+
its Hurst / Lyapunov / kurtosis estimators — it does not duplicate them. Two
8+
records: `ProcessFamilyCharacterization` (classify a process path/observable)
9+
and `ProcessRegimeCrosswalk` (the total typed mapping). Deterministic,
10+
stdlib-only.
11+
12+
## Why this exists
13+
14+
#50 reads the *memory regime* of a series. This layer names the **stochastic
15+
process** that produced it and ties it to how the process is priced (option-model
16+
family) and how it shows up as trader behaviour (archetype). The Hurst axis is
17+
the spine: it runs from mean-reverting (H<0.5) through efficient (H=0.5) to
18+
persistent long-memory (H>0.5).
19+
20+
## Ornstein-Uhlenbeck vs discrete-state Markov
21+
22+
Both are "Markov", but they are different regimes:
23+
24+
- **Ornstein-Uhlenbeck**`dX = θ(μ − X)dt + σ dW`. A *continuous* mean-reverting
25+
diffusion. Its autocorrelation decays **exponentially** (`e^{−θτ}`) with a
26+
finite timescale, so OU is the canonical `short_decaying` / mean-reverting
27+
regime, and it is **anti-persistent**: the variance of a k-step move
28+
*saturates* instead of growing like a random walk, so its variance-time Hurst
29+
is `< 0.5`. We estimate θ (mean-reversion speed), the long-run mean μ, and the
30+
**half-life = ln2/θ**. Seeded check: OU(θ=0.7) recovers θ̂≈0.74,
31+
half-life≈0.93, variance-time H≈0.05. Rate models (Vasicek, Hull-White, CIR),
32+
stochastic-vol models (Heston, Stein-Stein), and stat-arb spreads are OU.
33+
- **Markov regime-switching (Hamilton)** — memoryless jumps between *discrete*
34+
states (e.g. bull/bear, greed/fear). There is no single reversion timescale;
35+
the "memory" is the current state, so its regime is `memoryless` (state-only).
36+
- **Fractional-OU** — OU driven by fractional Brownian motion: the hybrid =
37+
mean-reversion **plus** long memory. Provided as a generator
38+
(`process_family_estimators.fractional_ou`).
39+
40+
## How the discriminator works
41+
42+
Feed the process **path / observable**. It reuses the #50 estimators and adds an
43+
OU fit and a variance-time (aggregated-variance) Hurst (slope of
44+
`log Var[X_{t+k}−X_t]` vs `log k` = 2H). Priority order:
45+
46+
1. **hawkes** — persistent volatility clustering (mean ACF(1..5) of squared
47+
increments ≥ 0.15) with variance-time H≈0.5. Checked *before* the jump branch
48+
because clustering itself fattens the tails.
49+
2. **jump_levy** — fat-tailed increments (excess kurtosis ≥ 5) with rare isolated
50+
jumps and no clustering.
51+
3. **ornstein_uhlenbeck** — θ>0 (AR(1) coefficient in (0,1)) and anti-persistent
52+
(variance-time H < 0.45).
53+
4. **fractional_brownian_motion** — persistent (variance-time H ≥ 0.58).
54+
5. **markov_regime_switching** — discrete-state separation with no mean reversion
55+
(heuristic; represented in the crosswalk, not asserted by a seeded verify).
56+
6. **brownian_gbm** — efficient random walk (H≈0.5, thin tails, no reversion).
57+
58+
## The crosswalk
59+
60+
`examples/process-regime-crosswalk/crosswalk.canonical.valid.json` is the
61+
machine-checked form of this table.
62+
63+
| process family | memory regime | kernel | option-model family (anchor) | behavioral archetype |
64+
|---|---|---|---|---|
65+
| `brownian_gbm` | `memoryless` | delta | Black-Scholes, Black-76 | efficient-market / no-edge |
66+
| `markov_regime_switching` | `memoryless` (state) | delta | regime-switching option models, Hamilton-filter pricing | greed/fear & bull/bear shift |
67+
| `ornstein_uhlenbeck` | `short_decaying` | exponential | **Vasicek**, Hull-White, CIR, **Heston**, Stein-Stein | contrarian / value / market-maker (fade extremes) |
68+
| `fractional_brownian_motion` | `long_memory` | power_law | **rough volatility** (Gatheral-Jaisson-Rosenbaum), rBergomi, MMAR | trend-follower / momentum / Elliott-impulse |
69+
| `hawkes` | `short_decaying` (self-exciting) | exponential | Hawkes self-exciting jump models, jump-clustering / contagion | herd / FOMO / reflexive |
70+
| `jump_levy` | `memoryless` (fat-tail jumps) | delta | Merton-jump, variance-gamma, CGMY, smile / tail models | shock / crash / tail-hunter |
71+
72+
### Rough volatility unifies the long-memory work with the vol surface
73+
74+
The `fractional_brownian_motion`**rough volatility** tie is load-bearing: in
75+
the Gatheral-Jaisson-Rosenbaum result, *volatility itself* is a fractional
76+
process with H≈0.1, so the Hurst axis governs the volatility surface, not just
77+
the price path. This unifies the long-memory / fractional work here with the
78+
volatility-surface work in `economic-prophet` #44 **by reference** — the
79+
`ProcessFamilyCharacterization.risk_distribution_F` for an fBm carries the Hurst
80+
H that a rough-vol calibration consumes, the same F the RAROC
81+
`risk(F, reference, kernel, horizon)` interface (economic-prophet, from #50) takes.
82+
83+
## Teeth
84+
85+
`scripts/validate_process_family_crosswalk.py`, wired as
86+
`make validate-process-family-crosswalk` and into the aggregate `make validate`;
87+
CI in `.github/workflows/process-family-crosswalk.yml`.
88+
89+
**VERIFIES** (seeded, deterministic): OU(θ∈{0.7,0.3,1.0}) → `ornstein_uhlenbeck`
90+
/ `short_decaying` with θ̂>0, finite half-life, anti-persistent variance-time
91+
H<0.5; fBm(H=0.7) → `fractional_brownian_motion` / `long_memory`; plus
92+
`brownian_gbm`, `hawkes`, `jump_levy` seeded paths classify to their families.
93+
94+
**REJECTS** (one fixture per tooth): an OU labeled `long_memory`; a process with
95+
θ≤0 called `mean_reverting`; an fBm labeled `memoryless`; an OU record missing
96+
`ou_params`; a crosswalk mapping OU↔`long_memory` or fBm↔`memoryless`; a
97+
non-total crosswalk (a process family missing). The crosswalk is asserted total
98+
and coherent, and the anchor option-model families (OU→Vasicek/Heston,
99+
fBm→rough-vol) are checked. Every record carries a proof-artifact-spine SHA-256
100+
hash-chain receipt (FIPS-180-4 algorithm; not a FIPS-140 module).
Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
{
2+
"schemaVersion": "memory-mesh.process-family-characterization.v0.1",
3+
"recordType": "ProcessFamilyCharacterization",
4+
"characterizationId": "pfc-brownian_gbm",
5+
"inputDescriptor": {
6+
"source": "seeded_generator",
7+
"n": 1024,
8+
"generator": {
9+
"family": "brownian_gbm",
10+
"seed": 12345
11+
}
12+
},
13+
"process_family": "brownian_gbm",
14+
"regime": "memoryless",
15+
"autocorr_kernel": "delta",
16+
"estimates": {
17+
"hurst_variance_time": 0.4962,
18+
"anti_persistent": true,
19+
"lyapunov_lambda": 0.1808,
20+
"increment_excess_kurtosis": 0.0159,
21+
"jump_fraction": 0.0,
22+
"vol_clustering": -0.0017
23+
},
24+
"crosswalk_binding": {
25+
"memory_regime": "memoryless",
26+
"option_model_family": [
27+
"Black-Scholes",
28+
"Black-76"
29+
],
30+
"behavioral_archetype": "efficient-market / no-edge"
31+
},
32+
"consumer_binding": {
33+
"memory_class": "working",
34+
"rationale": "brownian_gbm => memoryless regime => working tier."
35+
},
36+
"receipt": {
37+
"algo": "sha256",
38+
"hashAlgorithmStandard": "FIPS-180-4",
39+
"fipsModuleValidated": false,
40+
"prevReceiptHash": "GENESIS",
41+
"contentHash": "da227b8359999cbed3109fa125b5adac4637320db633f41586dc99c3f3b38996",
42+
"issuedAt": "2026-08-03T00:00:00Z"
43+
}
44+
}
Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
{
2+
"schemaVersion": "memory-mesh.process-family-characterization.v0.1",
3+
"recordType": "ProcessFamilyCharacterization",
4+
"characterizationId": "pfc-r3-fbm-as-memoryless",
5+
"inputDescriptor": {
6+
"source": "seeded_generator",
7+
"n": 1024,
8+
"generator": {
9+
"family": "fractional_brownian_motion",
10+
"seed": 111,
11+
"params": {
12+
"H": 0.7
13+
}
14+
}
15+
},
16+
"process_family": "fractional_brownian_motion",
17+
"regime": "memoryless",
18+
"autocorr_kernel": "delta",
19+
"estimates": {
20+
"hurst_variance_time": 0.6472,
21+
"anti_persistent": false,
22+
"lyapunov_lambda": 0.2156,
23+
"increment_excess_kurtosis": 0.3227,
24+
"jump_fraction": 0.0,
25+
"vol_clustering": 0.0487
26+
},
27+
"crosswalk_binding": {
28+
"memory_regime": "memoryless",
29+
"option_model_family": [
30+
"rough volatility (Gatheral-Jaisson-Rosenbaum)",
31+
"rBergomi",
32+
"MMAR"
33+
],
34+
"behavioral_archetype": "trend-follower / momentum / Elliott-impulse"
35+
},
36+
"consumer_binding": {
37+
"memory_class": "working",
38+
"rationale": "x"
39+
},
40+
"risk_distribution_F": {
41+
"family": "fractional_brownian_motion",
42+
"params": {
43+
"hurst_H": 0.6472
44+
},
45+
"tail_class": "fat_tailed",
46+
"kernel": "power_law"
47+
},
48+
"receipt": {
49+
"algo": "sha256",
50+
"hashAlgorithmStandard": "FIPS-180-4",
51+
"fipsModuleValidated": false,
52+
"prevReceiptHash": "GENESIS",
53+
"contentHash": "13cfe45554b51a79a9ce4de5c55359c53942e1332b4552dffe3cda157ac2875f",
54+
"issuedAt": "2026-08-03T00:00:00Z"
55+
}
56+
}
Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
{
2+
"schemaVersion": "memory-mesh.process-family-characterization.v0.1",
3+
"recordType": "ProcessFamilyCharacterization",
4+
"characterizationId": "pfc-fractional_brownian_motion",
5+
"inputDescriptor": {
6+
"source": "seeded_generator",
7+
"n": 1024,
8+
"generator": {
9+
"family": "fractional_brownian_motion",
10+
"seed": 111,
11+
"params": {
12+
"H": 0.7
13+
}
14+
}
15+
},
16+
"process_family": "fractional_brownian_motion",
17+
"regime": "long_memory",
18+
"autocorr_kernel": "power_law",
19+
"estimates": {
20+
"hurst_variance_time": 0.6472,
21+
"anti_persistent": false,
22+
"lyapunov_lambda": 0.2156,
23+
"increment_excess_kurtosis": 0.3227,
24+
"jump_fraction": 0.0,
25+
"vol_clustering": 0.0487
26+
},
27+
"crosswalk_binding": {
28+
"memory_regime": "long_memory",
29+
"option_model_family": [
30+
"rough volatility (Gatheral-Jaisson-Rosenbaum)",
31+
"rBergomi",
32+
"MMAR"
33+
],
34+
"behavioral_archetype": "trend-follower / momentum / Elliott-impulse"
35+
},
36+
"consumer_binding": {
37+
"memory_class": "long_term",
38+
"rationale": "fractional_brownian_motion => long_memory regime => long_term tier."
39+
},
40+
"risk_distribution_F": {
41+
"family": "fractional_brownian_motion",
42+
"params": {
43+
"hurst_H": 0.6472
44+
},
45+
"tail_class": "fat_tailed",
46+
"kernel": "power_law"
47+
},
48+
"receipt": {
49+
"algo": "sha256",
50+
"hashAlgorithmStandard": "FIPS-180-4",
51+
"fipsModuleValidated": false,
52+
"prevReceiptHash": "GENESIS",
53+
"contentHash": "9b284a916aec5f373963c1490b1cd7b0ad27c5fab732a4143a2f828ba2290652",
54+
"issuedAt": "2026-08-03T00:00:00Z"
55+
}
56+
}
Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
{
2+
"schemaVersion": "memory-mesh.process-family-characterization.v0.1",
3+
"recordType": "ProcessFamilyCharacterization",
4+
"characterizationId": "pfc-hawkes",
5+
"inputDescriptor": {
6+
"source": "seeded_generator",
7+
"n": 1024,
8+
"generator": {
9+
"family": "hawkes",
10+
"seed": 7
11+
}
12+
},
13+
"process_family": "hawkes",
14+
"regime": "short_decaying",
15+
"autocorr_kernel": "exponential",
16+
"estimates": {
17+
"hurst_variance_time": 0.4655,
18+
"anti_persistent": true,
19+
"lyapunov_lambda": 0.1867,
20+
"increment_excess_kurtosis": 9.9757,
21+
"jump_fraction": 0.00391,
22+
"vol_clustering": 0.2049
23+
},
24+
"crosswalk_binding": {
25+
"memory_regime": "short_decaying",
26+
"option_model_family": [
27+
"Hawkes self-exciting jump models",
28+
"jump-clustering / contagion"
29+
],
30+
"behavioral_archetype": "herd / FOMO / reflexive"
31+
},
32+
"consumer_binding": {
33+
"memory_class": "short_term",
34+
"rationale": "hawkes => short_decaying regime => short_term tier."
35+
},
36+
"receipt": {
37+
"algo": "sha256",
38+
"hashAlgorithmStandard": "FIPS-180-4",
39+
"fipsModuleValidated": false,
40+
"prevReceiptHash": "GENESIS",
41+
"contentHash": "e73dab29101482f117acc0f575a41c09567e13ff9f4e9cca5c527327d2c2b4ce",
42+
"issuedAt": "2026-08-03T00:00:00Z"
43+
}
44+
}

0 commit comments

Comments
 (0)