Skip to content

Latest commit

 

History

History
106 lines (91 loc) · 3.76 KB

File metadata and controls

106 lines (91 loc) · 3.76 KB

Changelog

All notable changes to the Arya Pub-Sub System will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

[1.0.0] - 2024-12-19

Added

  • Initial release of Arya pub-sub system
  • WebSocket-based real-time pub-sub functionality
  • REST API for topic management and monitoring
  • Thread-safe in-memory storage with asyncio locks
  • Message history replay (configurable per subscription)
  • Backpressure handling with bounded message queues
  • Comprehensive error handling with proper error codes
  • Health monitoring and system statistics endpoints
  • Docker support for containerized deployment
  • Extensive test suite with 90%+ coverage
  • Detailed documentation and API examples
  • Demo test client with colored output
  • Production-ready Dockerfile with security best practices

Features

  • Core Functionality

    • Create/delete topics via REST API
    • Subscribe/unsubscribe via WebSocket
    • Publish messages via WebSocket
    • Real-time message broadcasting to subscribers
    • Message history with configurable retention (default: 100 messages)
  • WebSocket Protocol

    • JSON-based message format
    • Request/response correlation with request_id
    • Heartbeat/ping-pong for connection health
    • Graceful error handling with error codes
    • Connection lifecycle management
  • Performance & Reliability

    • Thread-safe operations with asyncio locks
    • Bounded message queues (100 messages per connection)
    • Slow consumer detection and disconnection
    • Concurrent connection handling
    • Memory-efficient message storage
  • Monitoring & Operations

    • Health check endpoint with system metrics
    • Detailed statistics per topic
    • Structured logging with different levels
    • Graceful shutdown handling
    • Docker containerization
  • Developer Experience

    • Comprehensive test suite (unit + integration)
    • Type hints throughout codebase
    • Detailed API documentation
    • Example clients in Python and JavaScript
    • Docker Compose for local development

Technical Details

  • Framework: FastAPI with WebSocket support
  • Language: Python 3.11+ with asyncio
  • Storage: In-memory with thread-safe data structures
  • Concurrency: AsyncIO-based with proper locking
  • Testing: pytest with asyncio support
  • Containerization: Docker with multi-stage builds
  • Documentation: Comprehensive README with examples

Architecture

  • PubSubManager: Core singleton managing topics and subscriptions
  • WebSocketManager: Per-connection handler with message routing
  • Topic: Message storage with configurable history limits
  • Models: Pydantic validation for all data structures
  • Error Handling: Structured error responses with proper codes

Performance Characteristics

  • Throughput: 1000+ messages/second (depends on hardware)
  • Connections: Limited by system resources
  • Memory: ~1KB per message in history
  • Latency: Sub-millisecond message routing
  • Scalability: Single-instance, in-memory design

Security Features

  • Input validation with Pydantic models
  • Sanitized error messages
  • Resource limits (bounded queues, message history)
  • Non-root Docker user
  • No sensitive data exposure in logs

Known Limitations

  • In-memory only (no persistence)
  • Single instance (no clustering)
  • No built-in authentication
  • No message size limits (beyond JSON parsing)
  • No cross-instance communication

Future Enhancements

  • Persistent storage backends
  • Horizontal scaling support
  • Authentication and authorization
  • Message size limits and validation
  • Metrics and monitoring integration
  • Rate limiting per client
  • Message TTL and expiration
  • Topic-based access control