📚 Classroom Pulse — AI-Powered Smart Classroom Management System

Classroom Pulse is a comprehensive, AI-driven classroom management platform built for Smart India Hackathon 2025 (SIH '25). It integrates real-time mood & engagement analysis, face-recognition-based automated attendance, geofenced QR attendance, a faculty LMS dashboard, and video conferencing into a single unified system — enabling data-driven teaching and enhanced student engagement.

🏆 Built for SIH 2025 — Solving real-time classroom monitoring and intelligent attendance management at scale.

🏗 Architecture Overview

┌─────────────────────────────────────────────────────────────────────────────┐
│                    Faculty LMS Dashboard (React + Vite)                      │
│         Timetable │ Attendance │ Marks │ Mood Analytics │ Reports            │
└──────────┬──────────────────────────────┬───────────────────────────────────┘
           │ REST / SSE                   │
┌──────────▼──────────┐     ┌─────────────▼──────────────────────────────────┐
│  QR Attendance      │     │  Classroom Pulse Backend                       │
│  (Flask + ngrok)    │     │  (FastAPI / Flask)                             │
│                     │     │                                                │
│  ┌───────────────┐  │     │  ┌──────────────┐   ┌────────────────────────┐ │
│  │ QR Generation │  │     │  │ YOLOv8-Face  │   │  FER / DeepFace        │ │
│  │ + Geofencing  │  │     │  │ Detection    │──▶│  Emotion Recognition   │ │
│  │ (Haversine)   │  │     │  └──────────────┘   └────────────────────────┘ │
│  └───────────────┘  │     │  ┌──────────────┐   ┌────────────────────────┐ │
│                     │     │  │ MediaPipe    │   │  Engagement Scoring    │ │
│                     │     │  │ FaceMesh     │──▶│  (Valence / Arousal)   │ │
│                     │     │  └──────────────┘   └────────────────────────┘ │
│                     │     │  ┌──────────────┐   ┌────────────────────────┐ │
│                     │     │  │ Head Pose    │   │  Yawn / Blink / Gaze   │ │
│                     │     │  │ (PnP Solve)  │──▶│  Detection (MAR/EAR)   │ │
│                     │     │  └──────────────┘   └────────────────────────┘ │
└─────────────────────┘     └────────────────────────────────────────────────┘

┌─────────────────────┐     ┌────────────────────────────────────────────────┐
│  Video Conferencing │     │  Smart Kiosk (Face Recognition Attendance)      │
│  (Express + Jitsi)  │     │  DeepFace SFace + YOLO + Live Analytics UI     │
└─────────────────────┘     └────────────────────────────────────────────────┘

✨ Key Features

Feature	Description
Real-Time Mood Detection	7-emotion classification (happy, sad, angry, surprise, fear, disgust, neutral) using FER / DeepFace on live webcam feeds
Engagement Index	Weighted Valence-Arousal model producing a 0–100 engagement score with EMA smoothing
Head Pose Estimation	PnP-based 3D head pose (pitch, yaw, roll) to detect "Listening" vs. "Looking Down" states
Yawn Detection	Mouth Aspect Ratio (MAR) based real-time yawning detection via MediaPipe FaceMesh
Blink Detection	Eye Aspect Ratio (EAR) analysis for drowsiness / attention monitoring
YOLOv8 Face Detection	Fast, accurate multi-face detection optimized for classroom-scale (50+ faces)
Face Recognition Attendance	DeepFace (SFace model) for automatic check-in/check-out via smart kiosk
QR + Geofence Attendance	QR code scan with GPS verification (Haversine formula, configurable radius)
Faculty LMS Dashboard	React + Vite frontend with timetable, attendance records, marks, and mood analytics views
Video Conferencing	Jitsi Meet integration with role-based settings (host/student) and bandwidth optimization
Live Video Stream	MJPEG streaming with annotated face boxes, emotion labels, and attention overlays
SSE / WebSocket Analytics	Real-time analytics push to frontend dashboards via Server-Sent Events and WebSocket
Per-Emotion Headcount Charts	Live Matplotlib bar charts and engagement time-series (standalone mode)

🧩 Modules

1. Classroom Pulse — Real-Time Mood & Engagement Analytics

The core analytics engine that processes live webcam feeds to generate classroom-level mood and engagement metrics.

Two operational modes:

Mode	File	Face Detection	Server
Standalone (OpenCV Window)	`classroom_yolo.py` / `realtime_pulse_preview.py`	YOLOv8-Face / MediaPipe	None (runs locally with Matplotlib charts)
Server Mode (API + Dashboard)	`app/main.py` + `app/pulse.py`	FER built-in detector	FastAPI with WebSocket + MJPEG stream

Key capabilities:

Multi-face emotion detection (FER library on per-face crops)
MediaPipe FaceMesh for 468-landmark tracking
PnP-based head pose estimation with EMA smoothing
Yawning detection via Mouth Aspect Ratio (MAR > 0.60)
Listening / Looking Down classification
Engagement scoring: E = 100 × (0.5 × (happy + surprise) + 0.3 × (1 − neutral) + 0.2 × arousal)
Real-time Chart.js dashboard via WebSocket push

2. Smart Kiosk — Face Recognition Attendance

A classroom-entrance kiosk system that uses DeepFace for automated face-recognition-based attendance.

Pipeline:

Camera Frame
    │
    ├──► YOLOv8-Face → face bounding boxes
    │
    ├──► DeepFace.find() → match against student dataset (SFace model)
    │       └──► Auto check-in / check-out with cooldown timer
    │
    ├──► DeepFace.analyze() → per-face emotion classification
    │
    └──► MediaPipe FaceMesh → MAR (yawning) + EAR (blink) detection

Features:

Automatic student identification from pre-registered face database
Check-in / check-out with configurable cooldown (default: 5 min)
Live UI overlay with sidebar showing attendance log and analytics
Backend sync via REST API for persistent attendance records

3. QR Code Attendance (Geofenced)

A lightweight Flask app for QR-based attendance with GPS geofencing to prevent proxy attendance.

How it works:

Faculty generates a QR code linked to a public ngrok URL
Students scan the QR → opens a verification page on their phone
Browser captures GPS coordinates
Server validates location using Haversine distance (configurable radius, default: 20m)
Attendance logged as PRESENT or ABSENT to CSV

4. Faculty LMS Frontend

A React 18 + Vite + TypeScript single-page application for faculty to manage their classes.

Pages:

Page	Description
`LoginPage`	Faculty authentication
`DashboardPage`	Timetable grid + upcoming classes overview
`AttendancePage`	Student attendance records and management
`MarksPage`	Marks entry and management
`AnalyticsPage`	Class performance analytics
`MoodAnalyticsPage`	Live mood & engagement visualizations from Classroom Pulse

Tech: React Router, Tailwind CSS, Recharts, jsPDF + html2canvas for report export.

5. Video Conferencing

An Express.js server that wraps Jitsi Meet to provide seamless video conferencing for online/hybrid classes.

Features:

Room creation with unique IDs (class-<random>)
Role-based join URLs (host vs. student)
Auto-configured Jitsi settings: prejoin disabled, students muted by default, 360p bandwidth cap, P2P disabled for stability

🛠 Tech Stack

AI / Computer Vision

Technology	Purpose
YOLOv8n-Face	Fast multi-face detection (Ultralytics)
MediaPipe FaceMesh	468-landmark facial tracking, head pose, MAR/EAR
FER	Lightweight 7-emotion classifier
DeepFace	Face recognition (SFace) + emotion analysis
TensorFlow / Keras	Custom CNN for drowsiness detection (eye/yawn classification)
OpenCV	Video capture, frame processing, PnP pose estimation
NumPy / SciPy	Numerical computation, Haversine distance

Backend

Technology	Purpose
FastAPI	Classroom Pulse API server (WebSocket + REST)
Flask	QR attendance server + Kiosk analytics backend
Express.js	Video conferencing room management
ngrok	Public URL tunneling for QR attendance

Frontend

Technology	Purpose
React 18	UI framework
Vite	Build tool and dev server
TypeScript	Type safety
Tailwind CSS	Styling
Recharts	Data visualization charts
Chart.js	Real-time analytics dashboard (Pulse MVP)
jsPDF + html2canvas	PDF report generation
Axios	HTTP client

Infrastructure

Technology	Purpose
Jitsi Meet	Open-source video conferencing
Matplotlib	Standalone analytics charts (OpenCV mode)
CSV	Lightweight attendance logging

📁 Project Structure

classroom-pulse/
│
├── classroom_pulse_mvp/           # 🧠 Core real-time analytics module
│   ├── app/
│   │   ├── main.py                # FastAPI server (WebSocket, MJPEG, REST)
│   │   ├── pulse.py               # PulseEngine — threaded emotion + engagement pipeline
│   │   ├── config.py              # Tunable parameters (intervals, thresholds, labels)
│   │   └── static/
│   │       └── index.html         # Chart.js real-time dashboard
│   ├── classroom_yolo.py          # Standalone mode — YOLO + FER + head pose + charts
│   ├── realtime_pulse_preview.py  # Standalone mode — MediaPipe + FER + charts
│   ├── requirements.txt           # Python dependencies
│   └── yolov8n-face.pt            # YOLOv8 face detection weights
│
├── github-new/attendance-marker/
│   └── SIH/
│       ├── backend/
│       │   ├── server.py          # Flask server — KioskRuntime + MJPEG + SSE analytics
│       │   ├── app.py             # Flask app routes
│       │   ├── models.py          # Data models
│       │   └── routes.py          # REST API routes
│       ├── kiosk/
│       │   ├── kiosk_app.py       # Smart kiosk — DeepFace recognition + live UI
│       │   ├── register_students.py # Student face enrollment
│       │   ├── hardware_test.py   # Camera/hardware diagnostics
│       │   └── students.csv       # Registered student records
│       └── requirements.txt       # Python dependencies
│
├── qr-code/                       # 📱 QR-based geofenced attendance
│   ├── app.py                     # Flask server — QR generation + location verification
│   ├── qr_utils.py                # QR code image generator
│   ├── location_utils.py          # Haversine distance calculator
│   ├── location_logs.csv          # Attendance log
│   ├── templates/                 # HTML templates (index, verify)
│   └── static/                    # QR images
│
├── faculty-lms-frontend/          # 🖥 Faculty dashboard (React + Vite)
│   ├── src/
│   │   ├── App.tsx                # Router and layout
│   │   ├── pages/                 # Login, Dashboard, Attendance, Marks, Analytics, Mood
│   │   ├── components/            # Navbar, TimetableGrid, UpcomingClasses, charts
│   │   ├── context/               # Auth context
│   │   ├── services/              # API clients
│   │   ├── router/                # Route configuration
│   │   └── types/                 # TypeScript type definitions
│   ├── package.json
│   └── vite.config.ts
│
├── video-conferencing/            # 🎥 Jitsi-based video conferencing
│   ├── server.cjs                 # Express.js — room creation + join URL generation
│   ├── public/                    # Static assets
│   └── package.json
│
├── dataset_new/                   # Training data (eye/yawn classification)
│   ├── train/
│   └── test/
│
├── mood_recog.py                  # 🧪 CNN training script for drowsiness detection
├── yolomodel.py                   # YOLO face weights downloader utility
├── yolov8n-face.pt                # YOLOv8 face detection model weights
└── kaggle.json                    # Kaggle API credentials (for dataset downloads)

⚙️ Installation

Prerequisites

Python 3.10+ with pip
Node.js 18+ with npm
Webcam (required for real-time analysis)
GPU (optional, falls back to CPU — recommended for DeepFace)

1. Clone the Repository

git clone https://github.com/<your-username>/classroom-pulse.git
cd classroom-pulse

2. Set Up Classroom Pulse (Analytics Engine)

cd classroom_pulse_mvp

# Create and activate virtual environment
python -m venv .venv

# Windows
.\.venv\Scripts\activate

# Linux/macOS
source .venv/bin/activate

# Install dependencies
pip install -r requirements.txt
cd ..

3. Set Up Smart Kiosk (Face Recognition Attendance)

cd github-new/attendance-marker/SIH

# Create virtual environment and install
python -m venv .venv
.\.venv\Scripts\activate     # Windows
pip install -r requirements.txt
cd ../../..

4. Set Up QR Attendance

cd qr-code
pip install flask qrcode[pil]
cd ..

5. Set Up Faculty LMS Frontend

cd faculty-lms-frontend
npm install
cd ..

6. Set Up Video Conferencing

cd video-conferencing
npm install
cd ..

7. Download YOLO Face Weights

The YOLOv8n-Face weights are auto-downloaded on first run, or you can manually place yolov8n-face.pt in the relevant directories.

🚀 Usage

Option A: Standalone Real-Time Analytics (No server needed)

cd classroom_pulse_mvp
python classroom_yolo.py

This opens a fullscreen camera window with real-time face detection, emotion labels, attention tracking, and live Matplotlib charts showing per-emotion headcount and engagement over time.

Controls:

Press F — Toggle fullscreen
Press Q — Quit

Option B: Server Mode (Dashboard + API)

Terminal 1 — Analytics Backend:

cd classroom_pulse_mvp
uvicorn app.main:app --host 0.0.0.0 --port 8000 --reload

Terminal 2 — Faculty Frontend:

cd faculty-lms-frontend
npm run dev

Terminal 3 — Video Conferencing (optional):

cd video-conferencing
node server.cjs

Access Points

Service	URL
Pulse Dashboard	http://localhost:8000
Camera Preview	http://localhost:8000/preview
Live Video Stream	http://localhost:8000/video
Analytics Snapshot	http://localhost:8000/snapshot
Faculty LMS	http://localhost:5173
Video Conferencing	http://localhost:3000

Option C: QR Attendance

cd qr-code

# Start ngrok (in a separate terminal)
./ngrok-v3-stable-windows-amd64/ngrok http 5000

# Update the public_url in app.py with your ngrok URL, then:
python app.py

Option D: Smart Kiosk

cd github-new/attendance-marker/SIH

# Terminal 1 — Start the backend
cd backend
python server.py

# Terminal 2 — Start the kiosk
cd kiosk
python kiosk_app.py

🔬 Detection & Analysis Pipeline

Emotion & Engagement Pipeline

Webcam Frame (960×540)
    │
    ├──► YOLOv8-Face Detection
    │       └──► Bounding boxes (padded 10%) for each face
    │
    ├──► FER Emotion Classification (per-face crop)
    │       └──► Top emotion + confidence score
    │       └──► Thresholded (conf > 0.40, else → "neutral")
    │
    ├──► MediaPipe FaceMesh (468 landmarks)
    │       │
    │       ├──► PnP Head Pose (6-point solve)
    │       │       ├──► Pitch > 15° → "Looking Down"
    │       │       └──► |Pitch| < 10° & |Yaw| < 25° → "Listening"
    │       │
    │       └──► Mouth Aspect Ratio (MAR)
    │               └──► MAR > 0.60 → "Yawning"
    │
    └──► Aggregation (every 0.1–0.5s interval)
            ├──► Per-emotion headcount distribution
            ├──► Weighted Valence = Σ (weight_i × valence_i)
            ├──► Weighted Arousal = Σ (weight_i × arousal_i)
            └──► Engagement Index (0–100) with EMA smoothing

Face Recognition Pipeline (Kiosk)

Camera Frame
    │
    ├──► YOLO Detect → face boxes
    │
    ├──► DeepFace.find(SFace model) → match against enrolled dataset
    │       ├──► Match found → check-in / check-out (with cooldown)
    │       └──► POST /api/attendance/log
    │
    ├──► DeepFace.analyze() → dominant emotion per face
    │
    └──► FaceMesh → yawn count (MAR) + blink count (EAR)
            └──► Kiosk UI overlay with live analytics sidebar

Geofenced QR Attendance Pipeline

Faculty generates QR → Student scans
    │
    └──► Browser geolocation API → (lat, lon)
            │
            └──► Haversine distance to classroom center
                    ├──► distance ≤ 20m → PRESENT ✅
                    └──► distance > 20m → ABSENT ❌

📡 API Endpoints

Classroom Pulse (FastAPI — port 8000)

Method	Endpoint	Description
`GET`	`/`	Chart.js analytics dashboard
`GET`	`/preview`	Live camera preview (MJPEG)
`GET`	`/video`	Raw MJPEG video stream
`GET`	`/snapshot`	Latest metrics JSON
`GET`	`/healthz`	Health check
`WS`	`/ws`	WebSocket — real-time metrics push

Snapshot Response:

{
  "emotion_dist": { "happy": 0.45, "neutral": 0.30, "surprise": 0.15, "sad": 0.10 },
  "valence": 0.3412,
  "arousal": 0.6234,
  "engagement": 72.5,
  "n_faces": 12,
  "timestamp": 1747412345.678
}

Smart Kiosk Backend (Flask — port 5000)

Method	Endpoint	Description
`GET`	`/api/stream`	MJPEG annotated video stream
`GET`	`/api/kiosk/analytics/stream`	SSE — real-time analytics events
`GET`	`/api/kiosk/analytics/summary`	JSON snapshot of current analytics

QR Attendance (Flask — port 5000)

Method	Endpoint	Description
`GET`	`/`	QR display page
`GET`	`/verify`	Student verification page
`POST`	`/store_location`	Submit GPS for attendance verification

Video Conferencing (Express — port 3000)

Method	Endpoint	Description
`POST`	`/courses/:courseId/meetings`	Create a new Jitsi meeting room
`GET`	`/meetings/:id/join-url`	Get role-based join URL

📸 Screenshots

Screenshots can be added here after deployment.

🙏 Acknowledgements

Ultralytics YOLOv8 — Object detection framework
MediaPipe — Face mesh and landmark tracking
FER — Facial Expression Recognition
DeepFace — Face recognition and analysis
Jitsi Meet — Open-source video conferencing
FastAPI — Modern async Python web framework
Vite — Next-generation frontend build tool
Recharts — React charting library

📄 License

This project is developed for Smart India Hackathon 2025 and academic purposes.
See LICENSE for details.

_{Built with 🎓 by the Classroom Pulse Team — SIH 2025}

Name		Name	Last commit message	Last commit date
Latest commit History 1 Commit
classroom_pulse_mvp		classroom_pulse_mvp
faculty-lms-frontend		faculty-lms-frontend
github-new/attendance-marker		github-new/attendance-marker
qr-code		qr-code
video-conferencing		video-conferencing
.gitignore		.gitignore
README.md		README.md
mood_recog.py		mood_recog.py
yolomodel.py		yolomodel.py

Folders and files

Latest commit

History

Repository files navigation

📚 Classroom Pulse — AI-Powered Smart Classroom Management System

📋 Table of Contents

🏗 Architecture Overview

✨ Key Features

🧩 Modules

1. Classroom Pulse — Real-Time Mood & Engagement Analytics

2. Smart Kiosk — Face Recognition Attendance

3. QR Code Attendance (Geofenced)

4. Faculty LMS Frontend

5. Video Conferencing

🛠 Tech Stack

AI / Computer Vision

Backend

Frontend

Infrastructure

📁 Project Structure

⚙️ Installation

Prerequisites

1. Clone the Repository

2. Set Up Classroom Pulse (Analytics Engine)

3. Set Up Smart Kiosk (Face Recognition Attendance)

4. Set Up QR Attendance

5. Set Up Faculty LMS Frontend

6. Set Up Video Conferencing

7. Download YOLO Face Weights

🚀 Usage

Option A: Standalone Real-Time Analytics (No server needed)

Option B: Server Mode (Dashboard + API)

Access Points

Option C: QR Attendance

Option D: Smart Kiosk

🔬 Detection & Analysis Pipeline

Emotion & Engagement Pipeline

Face Recognition Pipeline (Kiosk)

Geofenced QR Attendance Pipeline

📡 API Endpoints

Classroom Pulse (FastAPI — port 8000)

Smart Kiosk Backend (Flask — port 5000)

QR Attendance (Flask — port 5000)

Video Conferencing (Express — port 3000)

📸 Screenshots

🙏 Acknowledgements

📄 License

About

Topics

Resources

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Contributors

Uh oh!

Languages

Packages