Skip to content
This repository was archived by the owner on Jun 8, 2026. It is now read-only.

Commit fa87ddf

Browse files
nyagamunenenyagamunene
authored
NOISSUE - Add magistrala agent docs (#105)
* add agent docs Signed-off-by: nyagamunene <stevenyaga2014@gmail.com> * fix meta.json Signed-off-by: nyagamunene <stevenyaga2014@gmail.com> --------- Signed-off-by: nyagamunene <stevenyaga2014@gmail.com>
1 parent f9ee086 commit fa87ddf

2 files changed

Lines changed: 370 additions & 0 deletions

File tree

‎content/docs/dev-guide/agent.mdx‎

Lines changed: 369 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,369 @@
1+
---
2+
title: Agent
3+
description: Magistrala IoT Agent runs on edge devices, connects them to Magistrala over MQTT, exposes a local HTTP API, manages Node-RED flows, executes commands, tracks local services, and serves as the bridge between local workloads and the cloud.
4+
keywords:
5+
- Agent
6+
- Magistrala
7+
- MQTT
8+
- Node-RED
9+
- Edge
10+
- Terminal
11+
- Configuration
12+
image: /img/mg-preview.png
13+
---
14+
15+
The **Magistrala IoT Agent** runs in the project Docker environment and acts as a bridge between local services and the Magistrala platform. Inside the container stack it loads runtime configuration, connects to MQTT, subscribes to remote commands,
16+
exposes a local HTTP API, manages Node-RED flows, executes shell commands, maintains terminal sessions, and tracks heartbeat messages from local services.
17+
18+
## Configuration
19+
20+
The agent can start from a local `config.toml` file or bootstrap itself from Magistrala using an external ID and key. Environment variables are merged with file-based configuration at startup.
21+
22+
| Variable | Description | Default |
23+
| --- | --- | --- |
24+
| `MG_AGENT_CONFIG_FILE` | Path to the runtime config file | `config.toml` |
25+
| `MG_AGENT_LOG_LEVEL` | Log level | `info` |
26+
| `MG_AGENT_HTTP_PORT` | HTTP API port | `9999` |
27+
| `MG_AGENT_PORT` | Alias for the HTTP API port | |
28+
| `MG_AGENT_BROKER_URL` | FluxMQ / AMQP broker URL used for heartbeats | `amqp://guest:guest@localhost:5682/` |
29+
| `MG_AGENT_MQTT_URL` | MQTT broker URL | `localhost:1883` |
30+
| `MG_AGENT_MQTT_SKIP_TLS` | Skip MQTT TLS verification | `true` |
31+
| `MG_AGENT_MQTT_MTLS` | Enable MQTT mutual TLS | `false` |
32+
| `MG_AGENT_MQTT_CA` | CA certificate path | `ca.crt` |
33+
| `MG_AGENT_MQTT_CLIENT_CERT` | Client certificate path | `client.cert` |
34+
| `MG_AGENT_MQTT_CLIENT_KEY` | Client key path | `client.key` |
35+
| `MG_AGENT_MQTT_QOS` | MQTT QoS | `0` |
36+
| `MG_AGENT_MQTT_RETAIN` | MQTT retain flag | `false` |
37+
| `MG_AGENT_NODERED_URL` | Node-RED API base URL | `http://localhost:1880/` |
38+
| `MG_AGENT_HEARTBEAT_INTERVAL` | Expected heartbeat interval for local services | `10s` |
39+
| `MG_AGENT_TERMINAL_SESSION_TIMEOUT` | Idle timeout for terminal sessions | `60s` |
40+
| `MG_AGENT_BOOTSTRAP_URL` | Bootstrap endpoint | |
41+
| `MG_AGENT_BOOTSTRAP_ID` | Bootstrap external ID | |
42+
| `MG_AGENT_BOOTSTRAP_KEY` | Bootstrap external key | |
43+
| `MG_AGENT_BOOTSTRAP_RETRIES` | Bootstrap retry count | `5` |
44+
| `MG_AGENT_BOOTSTRAP_RETRY_DELAY_SECONDS` | Delay between bootstrap retries | `10` |
45+
| `MG_AGENT_BOOTSTRAP_SKIP_TLS` | Skip TLS verification for bootstrap fetch | `false` |
46+
47+
## How Agent Works
48+
49+
### Provisioning comes first
50+
51+
Before the agent container is started, the project provisions the Magistrala resources the agent depends on. Running `make provision` or `make run_provision` executes `scripts/provision.sh`, which creates the required cloud-side resources and writes the resulting runtime values into `configs/config.toml`.
52+
53+
The provisioning flow creates or configures:
54+
55+
- a Magistrala client for the agent
56+
- a channel used for request and response messaging
57+
- a bootstrap configuration for the device
58+
- a Rule Engine rule used by the example setup
59+
60+
By default the provisioning script creates the agent client with the name `agent-mock-device`, so the MQTT examples in this project use that same provisioned client name.
61+
62+
The provisioning script supports both local and cloud targets:
63+
64+
- Local Docker target: if `MG_API` is not set, provisioning uses the local service defaults such as `http://localhost:9006`, `http://localhost:9005`, `http://localhost:9008`, `http://localhost:9013`, and the MQTT default `ssl://host.docker.internal:8883`
65+
- Cloud target: if `MG_API=https://cloud.magistrala.absmach.eu/api` is set, the provisioning requests are sent to Magistrala Cloud and the default MQTT target becomes `ssl://messaging.magistrala.absmach.eu:8883`
66+
67+
This is why provisioning is the first step in the Docker workflow: the agent container expects its channel, MQTT credentials, domain information, and bootstrap data to exist before it starts.
68+
69+
### Startup flow
70+
71+
At startup the agent:
72+
73+
1. Starts inside the Docker Compose stack.
74+
2. Loads environment-based defaults.
75+
3. Reads `config.toml` when available.
76+
4. Optionally fetches bootstrap configuration from Magistrala.
77+
5. Merges the provisioned values into its runtime config.
78+
6. Validates required runtime values such as `domain_id`, `channels.id`, `mqtt.url`, `mqtt.username`, `mqtt.password`, and `server.broker_url`.
79+
7. Connects to FluxMQ for heartbeat subscriptions.
80+
8. Connects to MQTT for remote request and response messaging.
81+
9. Starts the local HTTP server.
82+
83+
### Messaging flow
84+
85+
The agent subscribes to:
86+
87+
- `m/<domain-id>/c/<channel-id>/req` for incoming remote commands.
88+
- `heartbeat.<service-name>.<service-type>` through the local pub/sub broker for service heartbeat tracking.
89+
90+
The agent publishes responses to:
91+
92+
- `m/<domain-id>/c/<channel-id>/res` for command results and status messages.
93+
- terminal response topics while an interactive terminal session is active.
94+
95+
All remote MQTT commands use SenML JSON array payloads. The `n` field selects the subsystem:
96+
97+
| `n` | Purpose |
98+
| --- | --- |
99+
| `exec` | Execute a shell command on the device |
100+
| `config` | View or save runtime configuration |
101+
| `term` | Open, write to, and close terminal sessions |
102+
| `nodered` | Deploy, inspect, and manage Node-RED flows |
103+
| `control` | Internal response channel used by the agent |
104+
105+
### Core responsibilities
106+
107+
| Capability | How it works |
108+
| --- | --- |
109+
| Configuration management | Reads and writes TOML config and can update config remotely |
110+
| Remote command execution | Runs shell commands via `sh -c` in the current working directory |
111+
| Terminal sessions | Maintains interactive sessions with idle timeout control |
112+
| Node-RED integration | Talks to the local Node-RED HTTP API to deploy or inspect flows |
113+
| Service discovery | Tracks local services from heartbeat subjects and exposes them over HTTP |
114+
| HTTP API | Provides local endpoints for config, exec, services, and Node-RED management |
115+
| Web UI | Exposes a browser-based interface for local management of the running containerized agent |
116+
117+
### Node-RED behavior
118+
119+
The agent supports these Node-RED commands:
120+
121+
| Command | Behavior |
122+
| --- | --- |
123+
| `nodered-deploy,<base64-flow>` | Replaces all currently running flows with the provided flow JSON |
124+
| `nodered-add-flow,<base64-flow>` | Adds a new flow tab alongside the existing running flows |
125+
| `nodered-flows` | Fetches the current flows from Node-RED |
126+
| `nodered-state` | Returns the current Node-RED runtime state |
127+
| `nodered-ping` | Checks whether Node-RED is reachable |
128+
129+
When flows are sent through `nodered-deploy` or `nodered-add-flow`, the payload is base64-decoded, normalized, and then forwarded to Node-RED. For imported single-flow payloads, the agent rekeys Node-RED IDs to avoid duplicate ID conflicts. The agent also patches MQTT-related flow values so deployed flows use the runtime device/channel context.
130+
131+
## Deployment
132+
133+
The documented flow for this project is container-based.
134+
135+
### 1. Provision Magistrala resources
136+
137+
Set the required Magistrala values and run provisioning:
138+
139+
```bash
140+
export MG_PAT=<personal-access-token>
141+
export MG_DOMAIN_ID=<domain-id>
142+
make provision
143+
```
144+
145+
This step creates the required Magistrala client, channel, bootstrap config, and example rule, then updates `configs/config.toml` with the provisioned values used by the container setup.
146+
147+
For a cloud deployment target, set the cloud API base before provisioning:
148+
149+
```bash
150+
export MG_API=https://cloud.magistrala.absmach.eu/api
151+
export MG_PAT=<personal-access-token>
152+
export MG_DOMAIN_ID=<domain-id>
153+
make provision
154+
```
155+
156+
When `MG_API` points to `https://cloud.magistrala.absmach.eu/api`, the provisioning flow targets Magistrala Cloud. In that mode,
157+
`MG_AGENT_MQTT_URL=ssl://messaging.magistrala.absmach.eu:8883` also points the agent at the cloud MQTT broker instead of the local Docker-accessible one.
158+
159+
### 2. Build the Docker images
160+
161+
```bash
162+
git clone https://github.com/absmach/agent
163+
cd agent
164+
165+
make all && make dockers_dev
166+
```
167+
168+
### 3. Start the Docker stack
169+
170+
```bash
171+
make run
172+
```
173+
174+
This starts the agent together with the supporting containers such as Node-RED, the Agent UI, and the local messaging components used by the development setup.
175+
176+
### 4. Stop the stack
177+
178+
```bash
179+
make stop
180+
make clean_volumes
181+
```
182+
183+
## Agent UI
184+
185+
The Docker stack also starts a web-based management UI at `http://localhost:3002`.
186+
187+
The UI is used for local interaction with the running agent container and provides:
188+
189+
- **Configuration** to view and save agent settings such as `server`, `channels`, `mqtt`, `nodered`, and `log`
190+
- **Node-RED** tools to ping Node-RED, inspect state, fetch flows, deploy flows, and add a single flow tab from a local JSON file
191+
- **Services** to view heartbeat-discovered local services
192+
- **Execute Command** to run shell commands and inspect terminal-style output
193+
194+
The UI is built with Elm and served as part of the Docker environment, so it is part of the normal development workflow rather than a separate optional component.
195+
196+
## HTTP API
197+
198+
Base URL defaults to `http://localhost:9999`.
199+
200+
| Endpoint | Description |
201+
| --- | --- |
202+
| `POST /exec` | Execute a shell command and return output |
203+
| `GET /config` | Read the current runtime configuration |
204+
| `POST /config` | Save updated runtime configuration |
205+
| `GET /services` | List services discovered from heartbeats |
206+
| `POST /nodered` | Run a Node-RED operation |
207+
| `POST /pub` | Publish an arbitrary topic/payload pair |
208+
| `GET /health` | Health check endpoint |
209+
| `GET /metrics` | Prometheus metrics |
210+
211+
### Execute a shell command
212+
213+
```bash
214+
curl -X POST http://localhost:9999/exec \
215+
-H "Content-Type: application/json" \
216+
-d '{
217+
"bn": "req-1:",
218+
"n": "exec",
219+
"vs": "pwd"
220+
}'
221+
```
222+
223+
### View current config
224+
225+
```bash
226+
curl http://localhost:9999/config
227+
```
228+
229+
### Deploy Node-RED flows
230+
231+
```bash
232+
FLOWS=$(base64 -w 0 examples/nodered/speed-flow.json)
233+
234+
curl -X POST http://localhost:9999/nodered \
235+
-H "Content-Type: application/json" \
236+
-d "{\"command\":\"nodered-deploy\",\"flows\":\"$FLOWS\"}"
237+
```
238+
239+
### Add a flow tab without replacing existing flows
240+
241+
```bash
242+
FLOWS=$(base64 -w 0 examples/nodered/modbus-flow.json)
243+
244+
curl -X POST http://localhost:9999/nodered \
245+
-H "Content-Type: application/json" \
246+
-d "{\"command\":\"nodered-add-flow\",\"flows\":\"$FLOWS\"}"
247+
```
248+
249+
### Ping Node-RED
250+
251+
```bash
252+
curl -X POST http://localhost:9999/nodered \
253+
-H "Content-Type: application/json" \
254+
-d '{"command":"nodered-ping"}'
255+
```
256+
257+
### View Node-RED runtime state
258+
259+
```bash
260+
curl -X POST http://localhost:9999/nodered \
261+
-H "Content-Type: application/json" \
262+
-d '{"command":"nodered-state"}'
263+
```
264+
265+
### Fetch current flows
266+
267+
```bash
268+
curl -X POST http://localhost:9999/nodered \
269+
-H "Content-Type: application/json" \
270+
-d '{"command":"nodered-flows"}'
271+
```
272+
273+
### View registered services
274+
275+
```bash
276+
curl http://localhost:9999/services
277+
```
278+
279+
## MQTT Commands
280+
281+
Remote control happens over MQTT using SenML payloads published to `m/<domain-id>/c/<channel-id>/req`.
282+
283+
### Execute a command
284+
285+
```bash
286+
mosquitto_pub \
287+
-h <mqtt-host> -p 8883 --capath /etc/ssl/certs \
288+
-u <client-id> -P <client-secret> \
289+
-t "m/<domain-id>/c/<channel-id>/req" \
290+
-m '[{"bn":"req-1:","n":"exec","vs":"ls,-la"}]'
291+
```
292+
293+
### View config
294+
295+
```bash
296+
mosquitto_pub \
297+
-h <mqtt-host> -p 8883 --capath /etc/ssl/certs \
298+
-u <client-id> -P <client-secret> \
299+
-t "m/<domain-id>/c/<channel-id>/req" \
300+
-m '[{"bn":"req-1:","n":"config","vs":"view"}]'
301+
```
302+
303+
### Deploy Node-RED flows
304+
305+
```bash
306+
FLOWS=$(base64 -w 0 examples/nodered/speed-flow.json)
307+
308+
mosquitto_pub \
309+
-h <mqtt-host> -p 8883 --capath /etc/ssl/certs \
310+
-I "agent-mock-device" \
311+
-u <client-id> -P <client-secret> \
312+
-t "m/<domain-id>/c/<channel-id>/req" \
313+
-m "[{\"bn\":\"req-1:\",\"n\":\"nodered\",\"vs\":\"nodered-deploy,$FLOWS\"}]"
314+
```
315+
316+
### Add a flow tab without replacing existing flows
317+
318+
```bash
319+
FLOWS=$(base64 -w 0 examples/nodered/modbus-flow.json)
320+
321+
mosquitto_pub \
322+
-h <mqtt-host> -p 8883 --capath /etc/ssl/certs \
323+
-I "agent-mock-device" \
324+
-u <client-id> -P <client-secret> \
325+
-t "m/<domain-id>/c/<channel-id>/req" \
326+
-m "[{\"bn\":\"req-1:\",\"n\":\"nodered\",\"vs\":\"nodered-add-flow,$FLOWS\"}]"
327+
```
328+
329+
### Ping Node-RED
330+
331+
```bash
332+
mosquitto_pub \
333+
-h <mqtt-host> -p 8883 --capath /etc/ssl/certs \
334+
-I "agent-mock-device" \
335+
-u <client-id> -P <client-secret> \
336+
-t "m/<domain-id>/c/<channel-id>/req" \
337+
-m '[{"bn":"req-2:","n":"nodered","vs":"nodered-ping"}]'
338+
```
339+
340+
### View Node-RED runtime state
341+
342+
```bash
343+
mosquitto_pub \
344+
-h <mqtt-host> -p 8883 --capath /etc/ssl/certs \
345+
-I "agent-mock-device" \
346+
-u <client-id> -P <client-secret> \
347+
-t "m/<domain-id>/c/<channel-id>/req" \
348+
-m '[{"bn":"req-3:","n":"nodered","vs":"nodered-state"}]'
349+
```
350+
351+
### Fetch current flows
352+
353+
```bash
354+
mosquitto_pub \
355+
-h <mqtt-host> -p 8883 --capath /etc/ssl/certs \
356+
-I "agent-mock-device" \
357+
-u <client-id> -P <client-secret> \
358+
-t "m/<domain-id>/c/<channel-id>/req" \
359+
-m '[{"bn":"req-4:","n":"nodered","vs":"nodered-flows"}]'
360+
```
361+
362+
## Best Practices
363+
364+
- Run provisioning before starting the Docker stack so the agent container has valid client, channel, and bootstrap data.
365+
- Keep `channels.id`, `domain_id`, and MQTT credentials aligned, because the agent depends on them to subscribe and publish correctly.
366+
- Route all Node-RED flow deployment through the agent when flow payloads need runtime patching.
367+
- Use heartbeat publishing for local services that should appear in the agent UI and `/services` API.
368+
- Keep terminal session timeouts short on production devices.
369+
- Restrict shell command access to trusted operators only.

‎content/docs/dev-guide/meta.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@
22
"title": "Dev Guide",
33
"pages": [
44
"introduction",
5+
"agent",
56
"architecture",
67
"getting-started",
78
"entities",

0 commit comments

Comments
 (0)