Automated X-Ray Analysis · Fracture Detection · Wound Classification · Clinical Report Generation
⚠️ Educational Platform Only — This is an AI-assisted learning tool. Not a certified medical device or substitute for professional radiological evaluation.
XRayVision AI is a full-stack web application that allows medical students, radiologists, and healthcare professionals to upload medical images and receive an AI-powered diagnostic report in seconds.
The system uses a Hybrid Multi-Model Ensemble:
- 🫁 DenseNet121 — detects 18 chest pathologies (trained on 700K+ clinical X-rays)
- 🦴 YOLOv8 — localises fractures with bounding boxes drawn on the image
- 🩹 ViT (Vision Transformer) — classifies external wounds into 6 categories
- 🤖 GLM 4.5 Air (LLM) — synthesizes all model findings into a clinical paragraph
Built as a Final Year Project (FYP) at Minhaj University Lahore — BSSE 8th Semester, May 2026.
|
|
┌──────────────────────────────────────────────────────────────┐
│ Frontend (React 19) │
│ TanStack Start + Router + Query · TailwindCSS · Shadcn │
│ Deployed → Cloudflare Workers │
└───────────────────────┬──────────────────────────────────────┘
│ HTTPS (JWT Auth)
┌───────────────────────▼──────────────────────────────────────┐
│ Backend (FastAPI) │
│ POST /analyze · POST /chat · POST /diet · GET /stats │
│ Deployed → Hugging Face Spaces (Docker) │
└──────┬────────────┬─────────────┬───────────────┬────────────┘
│ │ │ │
┌──────▼──┐ ┌──────▼──┐ ┌──────▼──┐ ┌────────▼──────┐
│DenseNet │ │ YOLOv8 │ │ ViT │ │ OpenRouter │
│ (18 │ │(Fracture│ │ (Wound │ │ GLM 4.5 Air │
│ labels) │ │ bbox) │ │ 6cls) │ │ (Synthesis) │
└─────────┘ └─────────┘ └─────────┘ └───────────────┘
│ │
┌──────▼────────────────────────────────────────▼────────────┐
│ Supabase │
│ PostgreSQL (scans, profiles, chat) · Storage (images) │
└─────────────────────────────────────────────────────────────┘
| Requirement | Version | Notes |
|---|---|---|
| Node.js | ≥ 18.x | Frontend runtime |
| Python | ≥ 3.11 | Backend runtime |
| Supabase account | — | Free tier works |
| OpenRouter API key | — | Free tier (GLM 4.5 Air is free) |
| Hugging Face token | — | For ViT model download |
| YOLO weights | — | fracture_yolov8.pt in backend/models/ |
git clone https://github.com/ZohaibCodez/x-ray-vision-board.git
cd x-ray-vision-board# Install dependencies
npm install
# Create environment file
echo "VITE_API_URL=http://localhost:8000" > .env.local
# Start development server
npm run dev
# → http://localhost:5173cd backend
# Create virtual environment
python -m venv venv
venv\Scripts\activate # Windows
source venv/bin/activate # macOS / Linux
# Install dependencies
pip install -r requirements.txt
# Configure environment
cp .env.example .env
# Fill in your keys (see Environment Variables section below)
# Start the server
uvicorn app.main:app --reload --port 8000
# → API docs at http://localhost:8000/docs-- In your Supabase project → SQL Editor
-- Run: backend/supabase_schema.sql
-- Then create storage buckets:
-- 1. xray-images (public)
-- 2. avatars (public)# Supabase
SUPABASE_URL=https://your-project.supabase.co
SUPABASE_KEY=your-service-role-key
SUPABASE_ANON_KEY=your-anon-key
# OpenRouter (LLM synthesis)
OPENROUTER_API_KEY=sk-or-...
OPENROUTER_MODEL=z-ai/glm-4.5-air:free
# Authentication
JWT_SECRET=your-minimum-32-char-random-secret
# Hugging Face
HF_TOKEN=hf_...
# AI Models
YOLO_WEIGHTS_PATH=models/fracture_yolov8.pt
CONFIDENCE_THRESHOLD=0.40
# CORS
FRONTEND_URL=http://localhost:5173
# Performance (set true on free-tier <512MB RAM hosts)
DISABLE_PRELOAD=falseVITE_API_URL=http://localhost:8000Live API documentation: http://localhost:8000/docs (Swagger UI)
| Method | Endpoint | Description | Rate Limit |
|---|---|---|---|
POST |
/auth/register |
Create new account | — |
POST |
/auth/login |
Sign in → returns JWT | — |
POST |
/auth/forgot-password |
Send password reset email | — |
GET |
/auth/me |
Get current user profile | — |
POST |
/analyze |
Upload & analyse image (core feature) | 10/min |
GET |
/scans |
List scan history | — |
GET |
/scans/{id} |
Get specific scan result | — |
GET |
/scans/{id}/report.pdf |
Download PDF report | — |
GET |
/scans/{id}/export.json |
Export raw findings JSON | — |
DELETE |
/scans/{id} |
Delete scan | — |
POST |
/chat |
Health chatbot message | 20/min |
GET |
/chat/sessions |
List chat sessions | — |
POST |
/diet |
Generate diet plan | 10/min |
GET |
/clinics |
Find nearby clinics (GPS) | — |
GET |
/stats |
Dashboard statistics | — |
GET |
/health |
Health check | — |
curl -X POST "http://localhost:8000/analyze" \
-H "Authorization: Bearer YOUR_JWT_TOKEN" \
-F "file=@chest_xray.jpg" \
-F "scan_type=chest" \
-F "notes=Patient reports chest pain for 3 days"{
"id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
"scan_type": "chest",
"urgency": "high",
"findings": [
{
"name": "Lung Opacity",
"confidence": 74.2,
"severity": "high",
"model": "DenseNet121",
"region": "Lung parenchyma",
"icd_code": "R91.8",
"color": "destructive"
}
],
"agent_synthesis": {
"urgency": "high",
"synthesis_text": "The imaging reveals bilateral lung opacity consistent with pneumonia. Urgent pulmonologist review is recommended.",
"recommended_actions": [
"Urgent Pulmonologist consultation",
"Order CBC and CRP blood tests",
"Consider CT chest for further characterization"
],
"specialist": "Pulmonologist"
},
"image_url": "https://...supabase.co/storage/v1/object/public/xray-images/...",
"created_at": "2026-06-04T10:30:00Z"
}| Model | Source | Task | Input | Threshold |
|---|---|---|---|---|
| DenseNet121 | TorchXRayVision | 18 chest pathologies | 224×224 grayscale | 60% |
| YOLOv8 | Ultralytics (custom trained) | Fracture bounding boxes | Original size BGR | 15% |
| ViT | PayamFard123/dermaintel-wound-classifier |
6 wound types | RGB PIL Image | Top-1 |
| GLM 4.5 Air | OpenRouter (z-ai/glm-4.5-air:free) | Clinical report synthesis | Structured prompt | — |
Atelectasis · Cardiomegaly · Consolidation · Edema · Effusion · Emphysema · Fibrosis · Hernia · Infiltration · Mass · Nodule · Pleural Thickening · Pneumonia · Pneumothorax · Enlarged Cardiomediastinum · Lung Opacity · Lung Lesion · Fracture
# Build production bundle
npm run build
# Deploy via Wrangler CLI
npx wrangler deploy
# Or connect GitHub repo to Cloudflare Pages for auto-deploycd backend
# Build Docker image
docker build -t xrayvision-backend .
# Test locally
docker run -p 7860:7860 --env-file .env xrayvision-backend
# Deploy: push to HuggingFace Spaces repo
# All env vars are set as HF Spaces secrets| Component | Platform | URL |
|---|---|---|
| Frontend | Cloudflare Workers | x-ray-vision-board.vercel.app |
| Backend API | Hugging Face Spaces | *.hf.space |
| Database | Supabase PostgreSQL | Managed cloud |
| Image Storage | Supabase Storage | Managed cloud |
| AI Models | HF Hub (downloaded at start) | huggingface.co |
x-ray-vision-board/
├── src/ # Frontend (React 19 + TanStack)
│ ├── routes/ # File-based routing (one .tsx = one page)
│ │ ├── index.tsx # Landing page
│ │ ├── dashboard.tsx # Analytics dashboard
│ │ ├── analyze.tsx # Image upload & analysis
│ │ ├── results.$scanId.tsx # Results with bbox overlays
│ │ ├── history.tsx # Scan history
│ │ ├── chat.tsx # Health chatbot
│ │ ├── diet.tsx # Diet plan generator
│ │ └── clinics.tsx # GPS clinic locator
│ ├── components/
│ │ ├── app/AppShell.tsx # Sidebar + header layout
│ │ ├── landing/ # Landing page components
│ │ └── ui/ # 50+ Shadcn/Radix components
│ ├── hooks/
│ │ ├── use-analyze.ts # POST /analyze mutation
│ │ ├── use-scans.ts # Scan history & stats queries
│ │ └── use-chat.ts # Chatbot session queries
│ └── lib/
│ ├── api.ts # Central HTTP client (ALL API calls)
│ ├── auth-context.tsx # Global auth state (useAuth hook)
│ └── types.ts # TypeScript types
│
├── backend/ # Python FastAPI backend
│ └── app/
│ ├── main.py # App bootstrap, CORS, rate limiting
│ ├── config.py # Pydantic Settings (env vars)
│ ├── models/schemas.py # All Pydantic request/response models
│ ├── routers/
│ │ ├── analyze.py # Core AI pipeline endpoint
│ │ ├── auth.py # Auth endpoints
│ │ ├── scans.py # Scan CRUD + PDF/JSON export
│ │ ├── chat.py # Chatbot endpoints
│ │ ├── diet.py # Diet plan generator
│ │ ├── stats.py # Dashboard statistics
│ │ └── clinics.py # GPS clinic search
│ ├── services/
│ │ ├── image_preprocess.py # CLAHE, resize, DICOM support
│ │ ├── chest_model.py # DenseNet121 inference
│ │ ├── fracture_model.py # YOLOv8 inference
│ │ ├── wound_model.py # ViT wound classifier
│ │ ├── openrouter_agent.py # LLM synthesis engine
│ │ ├── chatbot_service.py # Chatbot (EN/UR system prompts)
│ │ └── diet_service.py # Diet plan generation
│ └── utils/supabase_client.py # All Supabase DB operations
│
├── LICENSE # MIT License
├── CONTRIBUTING.md # Contribution guidelines
├── CODE_OF_CONDUCT.md # Community standards
├── SECURITY.md # Security policy & reporting
└── README.md # This file
- JWT Authentication — HS256 tokens, 24-hour expiry, server refuses default secret
- Row Level Security — Supabase RLS on all tables (users see only their own data)
- Rate Limiting — slowapi: 200/min global, 10/min for AI endpoints
- CORS — Production locked to configured frontend domain only
- Input Validation — Pydantic validates all request bodies before processing
See SECURITY.md for vulnerability reporting instructions.
We welcome contributions! Please read our Contributing Guidelines first.
- Fork the repository
- Create a feature branch (
git checkout -b feature/amazing-feature) - Commit your changes (
git commit -m 'feat: add amazing feature') - Push to the branch (
git push origin feature/amazing-feature) - Open a Pull Request
See CONTRIBUTING.md for detailed setup, coding standards, and submission process.
This project is licensed under the MIT License — see LICENSE for details.
Medical Disclaimer: This software is for educational purposes only. It is NOT a certified medical device and must NOT be used for clinical diagnosis or treatment decisions. Always consult a qualified medical professional.
| Name | Role | ID |
|---|---|---|
| Muhammad Ali Raza | Developer | 2022F-MUL-BSSWE-017 |
| Hamza Afzal | Developer | 2022F-MUL-BSSWE-027 |
Supervisor: Maam Misbah — Lecturer, School of Software Engineering
Institution: Minhaj University Lahore
Programme: BSSE 8th Semester — Final Year Project (May 2026)
- TorchXRayVision — DenseNet121 pretrained on clinical X-rays
- Ultralytics YOLOv8 — Object detection framework
- PayamFard123/dermaintel-wound-classifier — ViT wound classifier
- OpenRouter — Free LLM API access (GLM 4.5 Air)
- Supabase — Database, Auth, and Storage
- TanStack — React Start, Router, and Query
- Shadcn/ui — Beautiful UI components
Made with ❤️ at Minhaj University Lahore
