A distributed, real-time video transcoding and HLS streaming service built with NestJS, MongoDB, and RabbitMQ. Accepts MP4 source URLs, transcodes them to HLS format using FFmpeg, and serves the resulting .m3u8 playlists directly from a scalable transcoder pool.
- Overview
- Architecture
- API Reference
- Prerequisites
- Setup
- Environment Variables
- Running Services Individually
- Testing
- Original Assignment
When a stream is started via the API, the MP4 source URL is validated with FFmpeg, persisted in MongoDB, and placed in a processing queue. A queue processor picks up pending jobs and dispatches them to a pool of transcoder workers. Each transcoder runs ffmpeg -re to generate HLS segments and serves them over HTTP. Streams can be stopped at any time — a fanout broadcast kills the FFmpeg process on the responsible transcoder and removes all generated media files.
┌─────────────────────────────────────────────────────┐
│ Client │
└──────────────────────┬──────────────────────────────┘
│ HTTP
┌──────────────────────▼──────────────────────────────┐
│ API Gateway (NestJS) │
│ │
│ POST /stream — start a stream │
│ PATCH /stream/:id/stop — stop a stream │
│ GET /stream — list streams │
│ GET /api — Swagger UI │
└───────┬─────────────────────────────────┬───────────┘
│ MongoDB (persist/read) │ RabbitMQ (publish stop)
│ │
┌───────▼───────────┐ ┌───────────────▼─────────────────┐
│ MongoDB │ │ Queue Processor │
│ streams │◄────│ Polls pending streams │
│ collection │ │ Dispatches to input queue │
└───────────────────┘ │ Processes transcoding results │
└───────────────┬─────────────────┘
│ RabbitMQ
┌───────────────▼─────────────────┐
│ Transcoder Pool │
│ ┌──────────┐ ┌──────────┐ │
│ │transcoder│ │transcoder│ ... │
│ │ :3001 │ │ :3002 │ │
│ │ ffmpeg │ │ ffmpeg │ │
│ │ /media/* │ │ /media/* │ │
│ └──────────┘ └──────────┘ │
└─────────────────────────────────┘
| Service | Entry Point | Description |
|---|---|---|
| API Gateway | src/main.ts |
Handles HTTP requests, validates inputs, persists streams, publishes stop commands |
| Queue Processor | src/queue-processor.ts |
Polls MongoDB for pending streams, enqueues them for transcoding, processes results |
| Transcoder | src/transcoder.ts |
Runs FFmpeg to generate HLS segments, serves media files via ServeStatic |
Two persistent RabbitMQ channels coordinate the services:
| Channel | Type | Direction | Purpose |
|---|---|---|---|
processing_input |
Durable queue (prefetch=1) | Queue Processor → Transcoder | Dispatches transcoding tasks one at a time |
processing_output |
Durable queue | Transcoder → Queue Processor | Reports transcoding results (success/failure, port) |
processing_control |
Fanout exchange | API Gateway → All Transcoders | Broadcasts stop and cleanup commands |
PENDING ──► QUEUED ──► STREAMING
└──► FAILED
STREAMING ──► STOPPED
Interactive Swagger documentation is available at http://localhost:<PORT>/api.
Validates the provided MP4 URL with FFmpeg, persists a stream record in MongoDB, and enqueues it for transcoding. Returns immediately with the stream ID; transcoding begins asynchronously.
Request body:
{
"url": "https://example.com/video.mp4"
}Response 201 Created:
{
"id": "64f1a2b3c4d5e6f7a8b9c0d1"
}Error responses:
| Status | Description |
|---|---|
400 Bad Request |
URL is missing or points to a non-video resource |
500 Internal Server Error |
Failed to persist the stream record |
Broadcasts a stop command to all transcoders via the fanout exchange. The transcoder responsible for this stream kills its FFmpeg process and removes all generated HLS files. The stream record is updated to STOPPED.
URL parameter: :id — the stream ID returned by POST /stream
Response 200 OK:
{
"success": true
}Error responses:
| Status | Description |
|---|---|
400 Bad Request |
ID parameter is missing |
404 Not Found |
No stream found with the given ID |
500 Internal Server Error |
Failed to publish the stop command or update the record |
Returns a paginated list of stream records with their current status and HLS playlist URL.
Query parameters:
| Parameter | Type | Description |
|---|---|---|
page |
number |
Page number, zero-indexed (default: 0) |
limit |
number |
Records per page (default: no limit) |
Response 200 OK:
[
{
"id": "64f1a2b3c4d5e6f7a8b9c0d1",
"url": "https://example.com/video.mp4",
"createdAt": "2024-01-15T12:00:00.000Z",
"updatedAt": "2024-01-15T12:00:05.000Z",
"playlistUrl": "http://localhost:3001/media/64f1a2b3c4d5e6f7a8b9c0d1/video.m3u8"
}
]The playlistUrl points directly to the transcoder instance that served the stream and can be opened in VLC, ffplay, or any HLS-capable media player.
- Docker and Docker Compose
- Node.js ≥ 16 (for local development and IDE support only)
- FFmpeg is bundled inside the Docker image — no local installation required
# 1. Copy environment configuration
cp .env.example .env
# 2. (Optional) Install dependencies for local IDE support
npm install
# 3. Build the image and start all services
docker-compose up --buildThe API will be available at http://localhost:${PORT} (default 3000).
Swagger UI is at http://localhost:${PORT}/api.
RabbitMQ management console is at http://localhost:15672.
Copy .env.example to .env and adjust as needed.
| Variable | Default | Description |
|---|---|---|
PORT |
3000 |
API Gateway HTTP port |
TRANSCODER1_PORT |
3001 |
Transcoder instance 1 HTTP port |
TRANSCODER2_PORT |
3002 |
Transcoder instance 2 HTTP port |
TRANSCODER3_PORT |
3003 |
Transcoder instance 3 HTTP port |
MONGO_URL |
mongodb://db_mongo:27017/hls |
MongoDB connection string |
AMQP_URL |
amqp://rabbitmq:5672 |
RabbitMQ connection string |
PROCESSING_INPUT_QUEUE |
processing_input |
Queue name for transcoding tasks |
PROCESSING_OUTPUT_QUEUE |
processing_output |
Queue name for transcoding results |
PROCESSING_CONTROL_EXCHANGE |
processing_control |
Fanout exchange name for stop commands |
MEDIA_PATH |
/data |
Filesystem path where HLS segments are stored and served |
To run the API on a custom port (e.g., 8080), set PORT=8080 in .env before starting.
Each service can be started independently for local development:
# API Gateway
npm run start:dev
# Queue Processor
npm run queue-processor:start:dev
# Transcoder (PORT env var determines the HTTP serve port)
PORT=3001 npm run transcoder:start:dev# Run all unit tests
npm test
# Run with coverage report
npm run test:cov
# Watch mode (re-runs on file changes)
npm run test:watchThe following is an English translation of the original assignment, which was written in Ukrainian.
Build a REST backend in TypeScript with Swagger (or an equivalent API documentation service) that exposes three endpoints: start, stop, and list.
Accepts: a url pointing to an MP4 file. Launches ffmpeg with the appropriate command to generate HLS output (the command is easy to find online — it must include the -re flag) and serves the generated files. There is no need to download the source file locally, and no need to use wrappers like fluent-ffmpeg — use only Node's child_process module.
Returns: id — an identifier tied to the request, and playlistUrl — a link to the .m3u8 playlist (hostname derived from the incoming request), which can be used to watch the video with VLC, ffplay, or any HLS-capable player.
Accepts: the id returned by the start endpoint. Stops the associated HLS generation process and cleans up its temporary files.
Returns: { success: true }.
Returns a list of active HLS generators. The response structure is left to the implementor's discretion.
- Swagger (or equivalent) must be driven entirely by TypeScript annotations in the source code — not a separate configuration file.
- Implement robust error handling; all error responses must also be described via Swagger annotations.
- The entire application must be Dockerized with
docker-compose, with support for configuring the backend port via an environment variable (e.g.,PORT=8080). SSL is not required in the backend itself — it is typically handled externally via an nginx reverse proxy.
CC-BY-4.0