README_EN.md

Smart Crib Guard

中文 | English

An intelligent baby crib monitoring system based on multimodal large language models and edge-cloud integrated architecture, enabling real-time baby status recognition, risk early warning, and intelligent interaction.

✨ Key Features

• 🎯 Baby Behavior Multi-classification：Based on Qwen2.5-VL-7B LoRA fine-tuning, covering 9 typical behaviors

  • Rolling, climbing crib rails, crying, spitting up, sneezing, yawning, turning, sleeping, quiet state
  • Training Loss converged to 0.0175, excellent model accuracy

• 📊 Automated Data Processing Pipeline

  • Video frame extraction with automatic sampling
  • Multiple data augmentation techniques: brightness adjustment, angle transformation, random cropping, flipping
  • Standardized JSON annotation format supporting batch processing

• 🚀 Efficient Inference Deployment

  • Support for single image and batch video inference
  • Real-time classification results with visualization
  • Confusion matrix and classification report generation

• 🔧 Edge-Cloud Integrated Architecture

  • Local inference combined with cloud-based fine-tuning
  • Flexible configuration management system
  • Support for model hot-updates

🚀 Quick Start

Prerequisites

• Python 3.11+
• CUDA 12.0+ (GPU recommended)
• 8GB+ VRAM (16GB+ recommended)

Installation

# Clone the repository
git clone https://github.com/Zsyyxrs/smart-crib-guard.git
cd smart-crib-guard

# Create virtual environment
python -m venv venv
source venv/bin/activate  # Windows: venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

Quick Inference Examples

Single Image Inference

python scripts/inference/infer_image.py \
  --image_path "path/to/image.jpg" \
  --model_name_or_path "Qwen/Qwen2.5-VL-7B" \
  --adapter_name_or_path "path/to/lora/adapter"

Video Inference

python scripts/inference/infer_video.py \
  --video_path "path/to/video.mp4" \
  --model_name_or_path "Qwen/Qwen2.5-VL-7B" \
  --adapter_name_or_path "path/to/lora/adapter" \
  --output_dir "./output"

Build Dataset

python scripts/data/build_dataset.py \
  --input_dir "path/to/raw/data" \
  --output_json "dataset.json"

📖 Project Structure

smart-crib-guard/
├── README.md                    # Chinese documentation (main)
├── README_EN.md                 # English documentation
├── LICENSE                      # MIT License
├── requirements.txt             # Dependencies
├── pyproject.toml              # Project configuration
│
├── src/
│   └── crib_guard/             # Core package
│       ├── __init__.py
│       ├── data/               # Data processing module
│       │   ├── augmentation.py # Data augmentation
│       │   ├── dataset.py      # Dataset handling
│       │   └── processing.py   # Data preprocessing
│       ├── models/             # Model module
│       │   └── inference.py    # Inference utilities
│       └── utils/              # Utility functions
│           └── helpers.py      # Helper functions
│
├── scripts/                     # Scripts directory
│   ├── train/                  # Training scripts
│   │   ├── train_lora.py       # LoRA fine-tuning
│   │   └── config/             # Training configs
│   ├── data/                   # Data processing scripts
│   │   ├── build_dataset.py    # Build dataset
│   │   ├── augment.py          # Data augmentation
│   │   └── merge.py            # Merge datasets
│   └── inference/              # Inference scripts
│       ├── infer_image.py      # Image inference
│       └── infer_video.py      # Video inference
│
├── result/                      # Inference results directory
│   ├── .gitkeep
│   └── README.md
│
├── output/                      # Model output directory
│   ├── .gitkeep
│   └── README.md
│
└── data/                        # Data directory (local only)
    ├── .gitkeep
    └── README.md

📊 Core Achievements

Model Performance

Metric	Value
Base Model	Qwen2.5-VL-7B
Fine-tuning Method	LoRA (r=64, alpha=16)
Training Loss	0.0175 ↓
Classification Classes	9 types
Inference Latency	~1.2s/image (GPU)

Behavior Classes

ID	Behavior	Risk Level	Description
1	Rolling	Low	Baby self-initiated rolling
2	Climbing Rails	High	Baby attempting to climb crib rails
3	Crying	Medium	Baby crying state
4	Spitting Up	Medium	Baby spitting up
5	Sneezing	Low	Physiological reflex
6	Yawning	Low	Normal sleep indicator
7	Turning	Low	Body turning motion
8	Sleeping	Low	Normal sleep state
9	Quiet	Low	Awake and quiet

🏗 Technical Architecture

Data Processing Pipeline

Raw Video
    ↓
[Frame Extraction] → Frame Sequence
    ↓
[Data Augmentation] → Diverse Dataset
  • Brightness Adjustment
  • Angle Transformation
  • Random Cropping
  • Flipping
    ↓
[JSON Annotation] → Standardized Format
    ↓
[Dataset Building] → Training Ready

Model Fine-tuning Pipeline

Qwen2.5-VL-7B (Base Model)
    ↓
[LoRA Adapter]
    ↓
[Supervised Fine-tuning] (SFT)
  • Learning Rate: 1e-4
  • Batch Size: 32
  • Optimizer: AdamW
    ↓
[Convergence Verification] Loss: 0.0175
    ↓
[Model Merging] → Production Model

Inference Deployment

Input (Image/Video)
    ↓
[Preprocessing] → Qwen2.5-VL Format
    ↓
[Model Inference] → Classification Result
    ↓
[Post-processing] → Risk Warning
    ↓
Output (Classification + Visualization)

📝 Usage Examples

1. Build Dataset

from src.crib_guard.data import build_dataset

# Build dataset from raw videos
dataset = build_dataset(
    video_dir="./data/videos",
    annotation_file="./data/annotations.json",
    output_dir="./output/dataset"
)

2. Batch Inference

from src.crib_guard.models import InferenceEngine

engine = InferenceEngine(
    model_name="Qwen/Qwen2.5-VL-7B",
    adapter_path="./output/adapter",
    device="cuda:0"
)

# Inference
results = engine.infer_video("path/to/video.mp4")

# Visualization
engine.visualize_results(results, output_path="./output/results.mp4")

3. Performance Evaluation

python scripts/inference/infer_video.py \
  --video_path "test_video.mp4" \
  --generate_report \
  --save_confusion_matrix

🔧 Configuration Guide

Training Configuration (scripts/train/config)

# Qwen2.5-VL-7B LoRA Fine-tuning Configuration
model_name_or_path: Qwen/Qwen2.5-VL-7B
adapter_name_or_path: null
lora_target: q_proj,v_proj,k_proj,o_proj,gate_proj,up_proj,down_proj
lora_rank: 64
lora_alpha: 16
lora_dropout: 0.1
dataset: baby_behavior
template: qwen_vl
learning_rate: 1e-4
num_train_epochs: 3
batch_size: 32

Inference Configuration

# Inference parameters
config = {
    "model_name": "Qwen/Qwen2.5-VL-7B",
    "adapter_path": "./output/adapter",
    "temperature": 0.3,
    "max_tokens": 128,
    "device": "cuda:0"
}

🤝 Contributing

Contributions are welcome! Please feel free to submit issues and pull requests.

1. Fork the repository
2. Create your feature branch (git checkout -b feature/AmazingFeature)
3. Commit your changes (git commit -m 'Add some AmazingFeature')
4. Push to the branch (git push origin feature/AmazingFeature)
5. Open a Pull Request

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

👤 Author

Shangyi Zhu

• GitHub: @Zsyyxrs
• Email: y5bcgb98fr@privaterelay.appleid.com

📚 References

• Qwen2.5-VL Official Documentation
• PEFT LoRA Fine-tuning Guide
• LLaMA-Factory

🎯 Future Plans

• Support for more vision models (LLaVA, GPT-4V)
• Real-time inference web service
• Mobile deployment adaptation
• Large-scale dataset training
• Behavior prediction and anomaly detection

---

Last Updated: 2026-05-29

If you have any questions, please open an issue on GitHub Issues.