Track 4 — Autopilot Agent | Qwen Cloud Global AI Hackathon
A self-healing control plane for autonomous Kubernetes infrastructure, powered by the Qwen model family. NeuroScale Autopilot continuously monitors your cluster, detects incidents, diagnoses root causes with Qwen-Max, plans remediations via RAG runbooks, executes fixes with circuit-breaker protection, and escalates to humans — autonomously closing the ops loop without manual intervention.
Click to watch the 2:44 demo — full pipeline walkthrough, Qwen models in action, MCP server, and 17/17 tests.
NeuroScale Autopilot runs a continuous self-healing loop on your Kubernetes cluster:
Metrics → Detect → Analyze (Qwen-Max) → Plan (Qwen-Embedding RAG) → Execute → Escalate (Qwen-Turbo)
↑ ↓
└──────────────── Self-healing feedback loop ─────────────────────────────┘
- Detector — Polls Prometheus/mock metrics; fires alerts on anomaly thresholds
- Analyzer — Sends alert context to Qwen-Max for root cause analysis + risk scoring
- Planner — Uses Qwen-Embedding to retrieve the most relevant runbook via semantic search; produces a structured remediation plan
- Executor — Runs kubectl commands with circuit-breaker protection; dry-run by default
- Escalation — Qwen-Turbo generates a concise Slack notification; human-in-the-loop approval for high-risk actions
- MCP Server — 8 Model Context Protocol tools expose the agent to external AI clients
- Alibaba Cloud ECS — Native ECS/STS client for cloud-layer remediation
Full pipeline: Kubernetes/Kyverno/OpenCost events → 5 autonomous agents → MCP Server → Alibaba Cloud ECS. Orchestrator handles alert deduplication and human-approval timeout (300s).
ASCII fallback
┌──────────────────────────────────────────────────────────────┐
│ NeuroScale Autopilot │
│ │
│ ┌─────────┐ ┌──────────────┐ ┌──────────────────────┐ │
│ │Detector │──▶│Analyzer │──▶│Planner │ │
│ │ │ │Qwen-Max LLM │ │Qwen-Embedding + RAG │ │
│ │Prometheus│ │RCA + Scoring │ │Runbook Retrieval │ │
│ └─────────┘ └──────────────┘ └──────────┬───────────┘ │
│ │ │
│ ┌─────────────────────────┐ ┌─────────────▼───────────┐ │
│ │Escalation Agent │◀──│Executor │ │
│ │Qwen-Turbo Summary │ │kubectl + Circuit Breaker│ │
│ │Slack + Approval Flow │ │Alibaba Cloud ECS │ │
│ └─────────────────────────┘ └─────────────────────────┘ │
│ │
│ ┌─────────────────────────────────────────────────────────┐ │
│ │MCP Server (8 tools) — FastAPI REST + SSE │ │
│ └─────────────────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────────┘
Real screenshots from a live running instance — captured from the actual server with simulated incidents.
Monitoring Overview — Stat Cards + Agent Pipeline
Active Incident Log — 2 Incident Types
Expanded Incident — Qwen Analysis + Remediation Plan + Human Approval
| Component | Model | Purpose |
|---|---|---|
| Analyzer | qwen-max |
Root cause analysis, risk scoring, confidence |
| Planner | text-embedding-v3 |
Runbook semantic search (RAG) |
| Escalation | qwen-turbo |
Human-readable Slack incident summaries |
All models served via Alibaba Cloud DashScope (dashscope.aliyuncs.com/compatible-mode/v1).
- Python 3.11+
- Docker & Docker Compose (optional)
- Qwen API key from DashScope Console
kubectlconfigured (or use mock mode)
git clone https://github.com/sodiq-code/neuroscale-autopilot.git
cd neuroscale-autopilot
cp .env.example .env
# Edit .env — set your QWEN_API_KEYpip install -r requirements.txt
python main.pyThe agent starts in dry-run mode by default — no real kubectl commands are executed.
docker-compose up --buildServices:
http://localhost:8000— MCP Server API + Healthhttp://localhost:3000— React Monitoring Dashboard
| Variable | Required | Default | Description |
|---|---|---|---|
QWEN_API_KEY |
✅ | — | DashScope API key |
QWEN_BASE_URL |
❌ | https://dashscope.aliyuncs.com/compatible-mode/v1 |
Qwen endpoint |
QWEN_MODEL_MAX |
❌ | qwen-max |
Analyzer model |
QWEN_MODEL_TURBO |
❌ | qwen-turbo |
Escalation model |
QWEN_MODEL_EMBEDDING |
❌ | text-embedding-v3 |
Embedding model |
SLACK_WEBHOOK_URL |
❌ | — | Slack webhook for notifications |
KUBECONFIG |
❌ | ~/.kube/config |
Kubeconfig path |
DRY_RUN |
❌ | true |
Disable real kubectl execution |
ALIBABA_ACCESS_KEY_ID |
❌ | — | ECS cloud remediation |
ALIBABA_ACCESS_KEY_SECRET |
❌ | — | ECS cloud remediation |
ALIBABA_REGION_ID |
❌ | cn-hangzhou |
ECS region |
POLL_INTERVAL_SECONDS |
❌ | 30 |
Metric polling frequency |
The MCP server exposes 8 tools for external AI clients:
| Tool | Description |
|---|---|
get_cluster_status |
Current health summary of the cluster |
list_active_alerts |
All active alerts with severity + age |
get_alert_detail |
Full detail for a specific alert |
trigger_remediation |
Manually trigger remediation for an alert |
get_remediation_status |
Status of a running remediation job |
approve_action |
Human approval for pending high-risk actions |
get_runbook |
Retrieve runbook content by name |
get_metrics_summary |
Raw metric summary for a namespace |
neuroscale-autopilot/
├── agents/
│ ├── detector/ # Prometheus poller + alert generation
│ ├── analyzer/ # Qwen-Max RCA engine
│ ├── planner/ # Qwen-Embedding RAG + remediation planner
│ ├── executor/ # kubectl runner + circuit breaker
│ └── escalation/ # Qwen-Turbo + Slack + approval flow
├── mcp_server/ # FastAPI MCP server (8 tools)
├── alibaba_cloud/ # ECS/STS client for cloud remediation
├── dashboard/ # React monitoring dashboard
├── runbooks/ # Markdown runbooks for RAG
├── k8s/ # Kubernetes manifests (deploy to ECS K8s)
├── tests/ # Pytest smoke + integration tests
├── .github/workflows/ # CI pipeline
├── main.py # Entry point
├── Dockerfile
└── docker-compose.yml
Deploy to Alibaba Cloud Container Service for Kubernetes (ACK):
# Apply namespace + RBAC
kubectl apply -f k8s/namespace.yaml
kubectl apply -f k8s/rbac.yaml
# Create secret with your API key
kubectl create secret generic neuroscale-secrets \
--from-literal=QWEN_API_KEY=<your-key> \
-n neuroscale-autopilot
# Deploy
kubectl apply -f k8s/deployment.yaml
kubectl apply -f k8s/service.yaml1. Detector polls Prometheus every 30s
2. Anomaly detected → Alert fired (severity: info/warning/critical)
3. Analyzer sends alert to Qwen-Max → returns RCA + risk score
4. Planner embeds RCA with text-embedding-v3 → finds closest runbook
5. Planner builds RemediationPlan (steps + requires_approval flag)
6. If requires_approval=True:
→ Qwen-Turbo generates summary → Slack notification sent
→ System waits up to 5 min for human approval
→ Auto-rejects on timeout (safety-first)
7. If approved (or auto-approved):
→ Executor runs kubectl steps with circuit breaker
→ On consecutive failures → breaker OPEN → no more attempts
8. Result logged → Detector re-polls → loop continues
MIT — see LICENSE
Sodiq Jimoh — Platform Engineer
LinkedIn
Built for the Qwen Cloud Global AI Hackathon — Track 4: Autopilot Agent