Loci

A research workbench for Alzheimer's disease mechanism discovery.

Loci means "places" in Latin: the places where memory lives, and the places where disease leaves its mark.

This repository is evolving from an undergraduate Alzheimer MRI classifier into a research-oriented system for reasoning over Alzheimer's disease data. The original project asked whether an uploaded scan looked consistent with a disease stage. Loci asks a harder set of questions: which mechanisms are implicated, what evidence supports them, what data is missing, and which research hypotheses should be investigated next.

Loci is not a medical device and is not a diagnostic product. The current product target is Loci Workbench: an auditable research workspace for clinical researchers, neurologists, computational biologists, and medical practitioners who need evidence review rather than opaque prediction output. Loci Engine is the later target, after real ADNI-backed modeling, retrieval, and mechanism graph work are implemented.

Why This Exists

Most AI work in Alzheimer's disease stops at classification. A model predicts cognitively normal, MCI, or AD; sometimes it adds a probability score or a heatmap. That can be useful, but it does not explain disease mechanics, expose uncertainty clearly, or help a researcher decide what to test next.

Alzheimer's disease is not a single-process condition. Amyloid pathology, tau pathology, neurodegeneration, vascular dysfunction, synaptic loss, inflammatory signaling, functional decline, and genetic risk interact over years. A useful research system needs to preserve that complexity without pretending to know more than the available evidence supports.

Loci is built around three research questions:

1. Why? Which disease pathways are most consistent with the available clinical and biomarker evidence?
2. What next? Which missing measurements or hypotheses should be prioritized?
3. How certain? Where is the system confident, incomplete, or limited by missing data?

Evolution Of The Project

The repository began as a CNN-based image classification project with a Flask and Streamlit demo around a saved .h5 model. That work is preserved in legacy_classification/ as Phase 0 because it remains useful as historical baseline work and as a reminder of the project origin.

The current direction is different. Loci treats the classifier as one possible input to a larger research engine, not as the final product. The first serious implementation layer is deliberately lightweight: structured subject and cohort records, missingness-aware pathway scoring, deterministic reports, and citation-aware hypothesis templates. This gives the project a stable research core before adding ADNI ingestion, trained models, imaging encoders, retrieval, or LLM-assisted hypothesis generation.

The working proposal is in docs/loci_proposal_v2.md. The normalized data contract is in docs/data_contract.md. The ADNI ingestion guide is in docs/adni_ingestion_guide.md, with the ingestion bundle format in docs/ingestion_bundle.md, ADNI table adapter plan in docs/adni_table_adapter_plan.md, quality gate policy in docs/quality_gate.md, ADNI missingness strategy in docs/adni_missingness_strategy.md, and real ADNI checklist in docs/adni_real_run_checklist.md. The evaluation policy is in docs/evaluation_plan.md, the Phase 1 closure definition is in docs/phase1_completion.md, and the archived classifier model card is in docs/model_card_legacy_classifier.md.

Current Capabilities

The current Loci core can:

• Load a structured patient profile from JSON.
• Load a cohort from CSV.
• Normalize an ADNI-like tabular export or a private multi-table ADNI freeze into the Loci cohort contract.
• Audit label balance, deterministic splits, feature-set readiness, and missingness before modeling.
• Track missing clinical and biomarker fields explicitly.
• Score curated Alzheimer's pathway signals from available evidence.
• Generate deterministic pathway reports with caveats.
• Recommend next measurements likely to reduce report uncertainty.
• Produce a templated, citation-aware hypothesis queue.
• Summarize pathway signals and missingness across a cohort.
• Render reproducible Loci Workbench artifacts for subject and cohort review.

This is intentionally not yet a trained ADNI benchmark. ADNI is the intended primary research dataset for the next modeling phase, but the project first defines the schema, reporting behavior, and audit trail that ADNI exports should map into.

Architecture

flowchart TD
    A[Legacy MRI Classifier] --> B[Phase 0 Archive]
    C[Patient JSON] --> D[Loci Data Contract]
    E[Cohort CSV] --> D
    F[ADNI-like Export Adapter] --> D
    P[Private ADNI Freeze Normalizer] --> D
    D --> G[Structured Patient Records]
    G --> H[Missingness Analysis]
    G --> I[Pathway Scoring]
    G --> S[Dataset Audit and Splits]
    H --> J[Measurement Recommendations]
    I --> K[Pathway Report]
    I --> L[Hypothesis Queue]
    J --> K
    K --> M[Researcher Workbench]
    L --> M
    S --> N[Tabular Baseline Harness]
    N --> Q[Phase 2 Quality Gate]
    O[Future Literature Retrieval] --> L

Loading

The present implementation covers the center of the diagram: data contract, adapters, private ADNI freeze normalization, structured records, missingness analysis, dataset audit, pathway scoring, reports, recommendations, hypothesis templates, a tabular baseline harness, and a Phase 2 quality gate. The future phases attach stronger feature-specific models, imaging encoders, calibration, and literature retrieval.

Research Workflow

A researcher or clinician can start with a synthetic or structured subject profile:

python -m loci.cli examples/demo_patient.json

The output is a pathway report: strongest mechanism signals, supporting evidence, missing inputs, recommended next measurements, caveats, and hypothesis cards.

Render the Phase 1 Workbench view:

python -m loci.cli --workbench examples/demo_patient.json
python -m loci.cli --cohort --workbench examples/demo_cohort.csv

Canonical rendered examples are checked in under examples/outputs/.

For cohort-level review:

python -m loci.cli --cohort examples/demo_cohort.csv

Audit a cohort before modeling:

python -m loci.cli --cohort --audit examples/demo_cohort.csv

Run the dependency-free tabular baseline harness:

python -m loci.cli --cohort --baseline examples/demo_cohort.csv

Write reproducible baseline run artifacts:

python -m loci.cli --cohort --baseline examples/demo_cohort.csv --run-dir runs

Normalize an ADNI-like tabular export into the Loci data contract:

python -m loci.cli examples/adni_like_export.csv --adapter adni_tabular --mapping-report
python -m loci.cli examples/adni_like_export.csv --adapter adni_tabular --output examples/adni_like_normalized.csv

Normalize a private ADNI freeze directory after downloaded tables are placed under data/raw/adni/<freeze>/exports/:

python -m loci.cli data/raw/adni/adni_2026_07_30 --adni-freeze --output data/interim/adni_2026_07_30_normalized.csv
python -m loci.cli --cohort --audit data/interim/adni_2026_07_30_normalized.csv
python -m loci.cli --cohort --feature-sets data/interim/adni_2026_07_30_normalized.csv
python -m loci.cli --cohort --baseline --feature-set biomarker_available data/interim/adni_2026_07_30_normalized.csv
python -m loci.cli --cohort --quality-gate --feature-set biomarker_available data/interim/adni_2026_07_30_normalized.csv
python -m loci.cli --cohort --workbench data/interim/adni_2026_07_30_normalized.csv

Create a full Phase 2 ingestion bundle:

python -m loci.cli examples/adni_like_export.csv --adapter adni_tabular --preflight --source-manifest examples/adni_source_manifest.example.json
python -m loci.cli examples/adni_like_export.csv --adapter adni_tabular --ingestion-bundle --run-dir runs --run-name adni_like_ingestion
python -m loci.cli examples/adni_like_export.csv --adapter adni_tabular --ingestion-bundle --source-manifest examples/adni_source_manifest.example.json --run-dir runs --run-name adni_like_ingestion

Project Structure

Alzheimer-Detection-Research/
|-- docs/
|   |-- data_contract.md         # Normalized structured input contract
|   |-- adni_source_manifest.md
|   |-- adni_ingestion_guide.md  # ADNI export and normalization workflow
|   |-- adni_missingness_strategy.md
|   |-- evaluation_plan.md       # Evaluation and run artifact policy
|   |-- ingestion_bundle.md
|   |-- mechanism_resource_review.md
|   |-- phase1_completion.md
|   |-- quality_gate.md
|   |-- model_card_legacy_classifier.md
|   `-- loci_proposal_v2.md      # Grounded implementation proposal
|-- examples/
|   |-- adni_like_export.csv     # Synthetic ADNI-like export
|   |-- demo_patient.json        # Synthetic patient profile
|   |-- demo_cohort.csv          # Synthetic cohort profile
|   `-- outputs/                 # Canonical rendered Workbench and mapping artifacts
|-- legacy_classification/
|   |-- Alzheimer.ipynb          # Original CNN training notebook
|   |-- alzheimers_cnn_model.h5  # Original saved image classifier
|   |-- app.py                   # Legacy Flask image upload demo
|   |-- streamlit_app.py         # Legacy Streamlit image upload demo
|   |-- requirements.txt         # Legacy TensorFlow/Streamlit dependencies
|   |-- static/                  # Legacy Flask assets
|   `-- templates/               # Legacy Flask templates
|-- loci/
|   |-- schema.py                # Core dataclasses and report schemas
|   |-- adapters.py              # External flat export normalization adapters
|   |-- adni_multitable.py        # Private ADNI freeze normalizer
|   |-- io.py                    # Patient JSON loading
|   |-- cohort.py                # Cohort CSV loading and cohort reports
|   |-- pathways.py              # Transparent pathway scoring rules
|   |-- recommendations.py       # Missing-measurement prioritization
|   |-- evaluation.py            # Dataset audit and deterministic splits
|   |-- feature_sets.py          # Candidate modeling task readiness reports
|   |-- quality.py               # Phase 2 model-claim readiness checks
|   |-- adni_tables.py           # ADNI source table role planning
|   |-- baseline.py              # Dependency-free tabular baseline harness
|   |-- artifacts.py             # Reproducible run artifact writing
|   |-- ingestion.py             # Phase 2 ingestion bundle generation
|   |-- model_card.py            # Phase 2 tabular baseline model card renderer
|   |-- reports.py               # Patient report generation
|   |-- hypotheses.py            # Citation-aware hypothesis templates
|   |-- workbench.py             # Phase 1 Markdown workbench renderer
|   |-- cli.py                   # Command-line entrypoint
|   |-- demo.py                  # Hardcoded deterministic demo
|   `-- resources/               # Versioned pathway rules and templates
`-- tests/
    `-- test_loci_core.py        # Unit coverage for the Loci core

Implementation Roadmap

Phase 0: Baseline Preservation

• Preserve the legacy MRI classifier and demos in legacy_classification/.
• Document provenance and metric limitations.
• Remove unsupported claims until reproducible evaluation exists.
• Add a model card and baseline evaluation checklist.

Phase 1: Structured Loci Core

• Define patient and cohort schemas.
• Support clinical and biomarker fields first.
• Track missingness and recommend next measurements.
• Generate deterministic pathway reports.
• Keep the core independent of TensorFlow and other heavy ML dependencies.

Phase 2: ADNI-Backed Baseline

• Build explicit adapters from ADNI exports into the normalized data contract.
• Maintain the dependency-free tabular baseline as a harness before adding heavier ML tooling.
• Separate clinical sanity-check baselines from biomarker and imaging feature sets.
• Train tabular clinical and biomarker baselines before image fusion.
• Report held-out metrics, calibration, split definitions, and missingness analysis.
• Treat diagnosis labels as research labels, not clinical outputs.

Phase 3: Multimodal Modeling

• Add MRI/PET-derived features and imaging encoders.
• Quantify whether each modality improves prediction and calibration.
• Add uncertainty estimation and model cards for every trained artifact.

Phase 4: Research Reasoning Workbench

• Curate a citation-backed Alzheimer's mechanism graph.
• Add retrieval over PubMed or a local literature corpus.
• Rank hypotheses by mechanistic coherence, evidence gap, and tractability.
• Keep generated prose citation-constrained and auditable.

Legacy Classification Demo

The original image demos now live in legacy_classification/. They require TensorFlow and the saved .h5 model:

streamlit run legacy_classification/streamlit_app.py

or:

python legacy_classification/app.py

These demos are educational research prototypes only. They should not be used for diagnosis, screening, triage, or clinical decision-making.

Tests

python -m unittest discover -s tests

License

MIT.