Skip to content
BranchesTags

Repository files navigation

Remote-Piloted Delivery Robot System

A sophisticated teleoperation platform for remotely piloted micro-delivery robots featuring real-time video streaming, predictive routing, hazard detection, and comprehensive pilot controls.

Operational State: Monitoring Tech Stack: Node.js + React Database: PostgreSQL

Features

🎮 Pilot Control Interface

  • Operational Modes: Monitoring, Assist, Manual Control, Emergency Halt
  • 360° Camera System: Multi-view switching (Front, Rear, Left, Right, Top, Panoramic)
  • Desktop Controls: WASD keyboard input + auxiliary controls
  • Mobile Controls: Virtual joystick + touch-optimized interface

📡 Real-Time Telemetry

  • Battery level monitoring
  • Network strength indicators
  • Motor temperature tracking
  • Video stream latency metrics
  • Live GPS positioning

⚠️ Hazard Detection & Alerts

  • Real-time hazard feed with severity classification
  • Object detection alerts (Pedestrians, Construction, Potholes, Traffic)
  • Visual hazard markers on map
  • Timestamp tracking and resolution status

🗺️ Navigation & Routing

  • Interactive mini-map with live robot positioning
  • Active route visualization
  • Alternative route suggestions
  • Distance and ETA calculations
  • Predictive guidance with confidence scores

💾 Session Management

  • Telemetry logging to PostgreSQL
  • Command history tracking
  • Session analytics (distance, speed, interventions)
  • Full audit trail

Tech Stack

Frontend

  • React - UI framework
  • TypeScript - Type safety
  • Tailwind CSS - Styling
  • Shadcn UI - Component library
  • TanStack Query - Data fetching & caching
  • Wouter - Lightweight routing
  • WebSocket - Real-time communication

Backend

  • Node.js + Express - API server
  • TypeScript - Type safety
  • PostgreSQL - Database (via Neon)
  • Drizzle ORM - Type-safe database queries
  • WebSocket (ws) - Real-time bidirectional communication
  • Zod - Runtime validation

Infrastructure (Current - Mock)

  • Simulated WebRTC - Video feed placeholder
  • In-memory telemetry - Mock sensor data generation
  • PostgreSQL - Persistent data storage

Infrastructure (Production - See Migration Guide)

  • LiveKit / Janus - WebRTC media server
  • Mosquitto MQTT - Robot telemetry transport
  • AWS Rekognition / TensorFlow.js - Computer vision
  • Mapbox - Routing and maps

Quick Start

Prerequisites

  • Node.js 20+
  • PostgreSQL database (automatically provisioned on Replit)

Installation

# Install dependencies
npm install

# Push database schema
npm run db:push

# Start development server
npm run dev

The application will:

  1. Start on port 5000
  2. Automatically initialize demo data (1 robot with hazards and routes)
  3. Begin simulating telemetry updates every 2 seconds

Access the pilot dashboard at: http://localhost:5000

Project Structure

├── client/                    # Frontend React application
│   ├── src/
│   │   ├── components/        # Reusable UI components
│   │   │   ├── hazard-feed.tsx
│   │   │   ├── mini-map.tsx
│   │   │   ├── operational-state-banner.tsx
│   │   │   ├── predictive-guidance.tsx
│   │   │   ├── robot-health-panel.tsx
│   │   │   ├── teleoperation-controls.tsx
│   │   │   ├── video-interface.tsx
│   │   │   └── ui/            # Shadcn UI primitives
│   │   ├── pages/
│   │   │   └── pilot-dashboard.tsx
│   │   ├── lib/               # Utilities
│   │   ├── App.tsx
│   │   └── index.css          # Global styles + design tokens
│   └── index.html
│
├── server/                    # Backend Node.js application
│   ├── index.ts               # Express server setup
│   ├── routes.ts              # API endpoints + WebSocket server
│   ├── storage.ts             # Database abstraction layer
│   ├── db.ts                  # Drizzle database connection
│   └── mock-data.ts           # Demo data generators
│
├── shared/                    # Shared types between frontend/backend
│   └── schema.ts              # Database schema + TypeScript types
│
├── docs/                      # Documentation
│   ├── ARCHITECTURE.md        # System design & data flow
│   └── PRODUCTION_MIGRATION_GUIDE.md  # Deploy to production
│
└── design_guidelines.md       # UI/UX specifications

API Documentation

REST Endpoints

Robots

  • GET /api/robots - List all robots
  • GET /api/robots/:id - Get robot details
  • POST /api/robots - Create new robot
  • PATCH /api/robots/:id - Update robot state

Hazards

  • GET /api/hazards/:robotId - Get hazards for robot
  • POST /api/hazards - Create hazard alert
  • PATCH /api/hazards/:id/resolve - Mark hazard resolved

Routes

  • GET /api/routes/:robotId - Get routes for robot
  • POST /api/routes - Create new route
  • POST /api/routes/:robotId/activate/:routeId - Set active route

Commands

  • GET /api/commands/:robotId - Get command history
  • POST /api/commands - Submit new command

Utilities

  • POST /api/init-demo-data - Initialize demo robot (auto-called on first run)

WebSocket API

Connection: ws://localhost:5000/ws

Subscribe to robot updates:

{
  "type": "subscribe",
  "robotId": "robot-id-here"
}

Send command:

{
  "type": "command",
  "robotId": "robot-id-here",
  "commandType": "move",
  "payload": { "direction": "forward", "speed": 2.0 }
}

Receive telemetry update:

{
  "type": "telemetry_update",
  "data": { /* full robot state */ }
}

Database Schema

Core Tables

  • robots - Robot fleet inventory & current state
  • telemetry_sessions - Pilot session tracking
  • telemetry_logs - Historical sensor data
  • hazards - Detected obstacles & warnings
  • routes - Navigation paths & waypoints
  • commands - Command history & audit log

See shared/schema.ts for full schema definitions.

Development Workflow

1. Make Schema Changes

Edit shared/schema.ts to add/modify tables or columns.

2. Push to Database

npm run db:push

3. Update Storage Layer

Add new methods to IStorage interface in server/storage.ts and implement in DatabaseStorage.

4. Create API Endpoints

Add routes in server/routes.ts with proper validation using Zod schemas.

5. Update Frontend

Create/modify React components in client/src/components/ and connect to API using TanStack Query.

Mock vs Production

Current Implementation (Mock)

✅ Fully functional pilot interface
✅ Real-time telemetry simulation
✅ PostgreSQL data persistence
✅ WebSocket communication
✅ Comprehensive UI with all features

⏳ Simulated Services:

  • Video streaming (CSS-based animation)
  • GPS coordinates (random within San Francisco)
  • Hazard detection (pre-generated mock data)
  • Routing (static mock routes)

Production Migration

See docs/PRODUCTION_MIGRATION_GUIDE.md for step-by-step instructions to replace mock services with:

  1. LiveKit / Janus - Real-time video streaming (<200ms latency)
  2. Mosquitto MQTT - Robot telemetry transport
  3. AWS Rekognition / TensorFlow.js - Computer vision & hazard detection
  4. Mapbox Directions API - Real GPS routing
  5. Production PostgreSQL - Scaled database with replicas

All mock components use adapter patterns for seamless swapping.

Key Components

🎥 Video Interface

  • Multi-camera view switching (360°, Front, Rear, Left, Right, Top)
  • Zoom controls (1x - 3x digital zoom)
  • Latency indicator with color-coded warnings
  • Fullscreen mode
  • Simulated WebRTC feed (production: LiveKit integration)

🏥 Robot Health Panel

  • Battery level with progress bar
  • Network strength indicator
  • Motor temperature monitoring
  • Video stream latency display
  • Color-coded status (green/yellow/red thresholds)

⚠️ Hazard Feed

  • Real-time scrollable alert list
  • Severity classification (Critical, Warning, Info)
  • Hazard types: Pedestrian, Construction, Pothole, Traffic, Low Light, Curb
  • Timestamp tracking
  • Resolution status

🗺️ Mini-Map

  • Live robot position with pulsing marker
  • Active route visualization
  • Alternative route suggestions
  • Distance and ETA display
  • Hazard markers on map

🧭 Predictive Guidance

  • Next 3 upcoming turns/instructions
  • Distance to next action (meters)
  • Direction icons (straight, left, right)
  • AI confidence scores with progress bars
  • Hazard warnings on route

🎮 Teleoperation Controls

  • Desktop: WASD keyboard controls, mode buttons, auxiliary controls
  • Mobile: Virtual joystick, touch-optimized buttons
  • Modes: Monitoring, Assist, Manual, Emergency Halt
  • Actions: Stop, Reverse, Resume Autonomy, Lights, Horn

Testing

Frontend E2E Tests

# Test pilot workflow
npm run test:e2e

Backend Integration Tests

# Test API endpoints
npm run test:api

Load Testing

# Simulate 100 concurrent connections
npm run test:load

Environment Variables

Development (Replit - Auto-configured)

DATABASE_URL=postgresql://...
PGHOST=...
PGPORT=...
PGUSER=...
PGPASSWORD=...
PGDATABASE=...
SESSION_SECRET=...
PORT=5000

Production (Required)

# Database
DATABASE_URL=postgresql://...

# WebRTC
LIVEKIT_URL=wss://...
LIVEKIT_API_KEY=...
LIVEKIT_API_SECRET=...

# MQTT
MQTT_BROKER_URL=mqtts://...
MQTT_USERNAME=...
MQTT_PASSWORD=...

# Computer Vision
AWS_REGION=us-east-1
AWS_ACCESS_KEY_ID=...
AWS_SECRET_ACCESS_KEY=...

# Routing
MAPBOX_ACCESS_TOKEN=...
VITE_MAPBOX_TOKEN=...  # Public token for frontend

Performance Targets

Metric Target Current (Mock)
Video Latency <200ms Simulated
Telemetry Rate 10-20 Hz 0.5 Hz (demo)
Command Response <100ms ~50ms
API Response <200ms ~20ms

Design Guidelines

This project follows mission-critical operator interface design principles:

  • Information Primacy - Video feed is hero element
  • Glanceable Telemetry - Critical metrics visible without focus interruption
  • State Clarity - Operational mode unmistakable at all times
  • Zero Ambiguity - Every control's function immediately obvious

See design_guidelines.md for complete specifications.

Browser Compatibility

  • ✅ Chrome 90+
  • ✅ Firefox 88+
  • ✅ Safari 14+
  • ✅ Edge 90+
  • ⚠️ Mobile Safari (virtual joystick optimized)

Contributing

  1. Fork the repository
  2. Create a feature branch (git checkout -b feature/amazing-feature)
  3. Commit your changes (git commit -m 'Add amazing feature')
  4. Push to the branch (git push origin feature/amazing-feature)
  5. Open a Pull Request

Architecture

See docs/ARCHITECTURE.md for:

  • System design diagrams
  • Data flow documentation
  • Service integration points
  • Database relationships
  • Scaling strategies

License

MIT License - See LICENSE file for details

Support

For technical questions or migration support:

Roadmap

Phase 1: ✅ MVP (Current)

  • Pilot dashboard with all core features
  • Mock data generators
  • PostgreSQL persistence
  • WebSocket real-time communication

Phase 2: 🚧 Production Integration

  • LiveKit video streaming
  • MQTT broker deployment
  • Computer vision hazard detection
  • Mapbox routing integration

Phase 3: 📋 Planned

  • Multi-robot fleet management
  • Pilot authentication & roles
  • Session recording & playback
  • Mobile native apps (iOS/Android)
  • Advanced analytics dashboard
  • Predictive maintenance
  • AR/VR pilot interface

Built for mission-critical robotics teleoperation with maximum reliability and minimal latency.

About

No description, website, or topics provided.

Resources

Readme
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

Packages

Contributors

Languages