Skip to content

[Enhancement] Initial public release of Ubiquiti UniFi integration #14566

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions packages/ubnt_unifi/_dev/build/build.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
dependencies:
ecs:
reference: "[email protected]"
112 changes: 112 additions & 0 deletions packages/ubnt_unifi/_dev/build/docs/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,112 @@
# Ubiquiti UniFi

This integration is for [Ubiquiti UniFi](https://ui.com) equipment event logs. The package processes events collected from Ubiquiti Unifi devices.

## Data Streams

The Ubiquiti UniFi integration collects the following event types:

- **logs**, Logs produced via UDP syslog from a Unifi controller, application or device.

This includes CEF logs, iptables firewall logs, and other Unix/Linux style syslog messages that may be produced.

You can use Elastic Agent to read files of logs if you already have a syslog aggregation system that is already collecting UniFi syslog output. Or alternatively you can configure your UniFi systems to log directly to a UDP listener on an Elastic Agent.

- **webhooks**, Events produced by Unifi Alarm Manager as webhooks, aka. HTTP POST's with a JSON body.

The Ubiquiti UniFi Alarm Manager and webhook based alarms are very new features and the content currently included in the body of a webhook is highly variable in terms of quality and field completeness.

## Related Integrations

**NOTE**: Ubiquiti UniFi now supports NetFlow based traffic logging. If network flow visibility is desired you can and should utilise the existing Elastic [Netflow](https://www.elastic.co/docs/reference/integrations/netflow) integration using NetFlow Version 9 to collect flow records from your Ubiquiti UniFi equipment. Refer to [https://community.ui.com/releases](https://community.ui.com/releases) for further documentation regarding NetFlow support and configuration instructions.

**NOTE**: Ubiquiti UniFi produces iptables "style" firewall logs with a slightly different format to the firewall logs previously produced by other Ubiquiti systems. You do not need to, and should not, install or utilise existing Ubiquiti support within the [iptables](https://www.elastic.co/docs/reference/integrations/iptables) integration as it will not work for firewall logs produced by UniFi systems. You should utilise this integration to collect Ubiquiti UniFi firewall logs independently of other non-UniFi Ubiquiti equipment.

**NOTE**: Ubiquiti UniFi components produce iptables style firewall logs, *some* CEF format logs for configuration activity and events on UniFi consoles and within applications, as well as some common *nix style logs. While at times these are sent with a syslog prefix at other times they are not sent with a syslog prefix. At present not all CEF logs produced by UniFi components are conformant to the Common Event Format (CEF) specification. You do not need to, and should not, attempt to utilise the existing Elastic [CEF](https://www.elastic.co/docs/reference/integrations/cef) integration to process Ubiquiti UniFi logs in any way. This Ubiquiti UniFi integration includes Elastic Agent beat level content fixes for the format problems that are often produced by Ubiquiti UniFi components at present.

## Requirements

For `logs` based event collection Elastic Agent *MUST* be utilised due to the pre-processing and filtering that occurs at the agent level. For example CEF parsing is completed by the Elastic Agent, as this is the only component that natively supports CEF parsing, when logs are first received from the network or read from file. A number of content fixes are applied.

If `logs` are received/aggregated or otherwise handled by something else and delivered to Elasticsearch for indexing, without passing thru an Elastic Agent, you should replicate the Elastic Agent behaviour, including content fixes, CEF parsing, as well as appropriate tagging.

`webhooks` events from the Ubiquiti UniFi Alarm Manager feature/s require no special Elastic Agent based pre-processing and can be delivered to Elasticsearch for indexing via any method that is suitable for your environment; provided you tag the events appropriately.

For more details and installation instructions, please refer to the [Elastic Agent Installation Guide](https://www.elastic.co/guide/en/fleet/current/elastic-agent-installation.html).

Your Ubiquiti UniFi infrastructure should consist of:
- Ubiquiti UniFi OS `4.0.0` or higher, if running a Ubquiti Unifi Cloud Gateway or similar appliance.
- Ubiquiti UniFi Applications, e.g. Network, `9.0.0` or higher, either on a Ubquiti Unifi Cloud Gateway or self hosted.

Refer to [https://community.ui.com/releases](https://community.ui.com/releases) for current release information, upgrade instructions and further documentation.

**NOTE**: This integration has been tested with Ubiquiti UniFi Cloud Gateways only, self-hosted versions of UniFi applications should work but have not been tested.

**NOTE**: This integration has only been tested with Ubiquiti UniFi Network and Protect applications at this time.

### Installing and managing an Elastic Agent:

There are several options for installing and managing Elastic Agent:

### Install a Fleet-managed Elastic Agent (recommended):

With this approach, you install Elastic Agent and use Fleet in Kibana to define, configure, and manage your agents in a central location. We recommend using Fleet management because it makes the management and upgrade of your agents considerably easier.

### Install Elastic Agent in standalone mode (advanced users):

With this approach, you install Elastic Agent and manually configure the agent locally on the system where it’s installed. You are responsible for managing and upgrading the agents. This approach is reserved for advanced users only.

### Install Elastic Agent in a containerized environment:

You can run Elastic Agent inside a container, either with Fleet Server or standalone. Docker images for all versions of Elastic Agent are available from the Elastic Docker registry, and we provide deployment manifests for running on Kubernetes.

Please note, there are minimum requirements for running Elastic Agent. For more information, refer to the [Elastic Agent Minimum Requirements](https://www.elastic.co/guide/en/fleet/current/elastic-agent-installation.html#elastic-agent-installation-minimum-requirements).


### Enabling the integration in Elastic:

1. In Kibana navigate to Management > Integrations.
2. In "Search for integrations" top bar, search for `Ubiquiti UniFi`.
3. Select the "Ubiquiti UniFi" integration from the search results.
4. Select "Add Ubiquiti UniFi" to add the integration.
5. Add all the required integration configuration parameters.
6. Select "Save and continue" to save the integration.

The default syslog based log collection configuration is likely suitable for most environments, e.g.

![Default Integration Configuration](../img/add-integration-defaults.png)

### Enabling SIEM integration in Ubiquiti UniFi:

Logging for UnifiOS and Unifi applications can be configured via,

1. Login to your Unifi system, navigate to Settings, typically found via the gear icon in the menu bar to the left
2. Click on "Control Plane" in the second level menu to the left of the screen
3. Click on "Integrations" in the third level menu near the top of the screen
4. Select "SIEM Server" next to "Activity Logging (Syslog)"
5. Select Activity Log Categories as appropriate, note that "UniFi OS" categories will be for admin activity and other system events, while "Network" categories can be used to enable traffic logging including logging of traffic that matches the default firewally policy.
6. Enter the IP address and port that your Elastic Agent Ubiquiti UniFi syslog integration listener has been configured to use
7. Optionally click "Send Test Event" and ensure ingest to Elastic is occurring
8. Click "Save" to save the configuration

Additional logging options may be available via other screens.

![Control Plane SIEM Integration Configuration](../img/configure-unifi-siem-integration.png)

## Logs

### Ubiquiti UniFi Logs

The `logs` dataset collects Ubiquiti Unifi logs sent via syslog.

{{event "logs"}}

{{fields "logs"}}

### Ubiquiti UniFi Webhooks

The `webhooks` dataset collects Ubiquiti Unifi events producted by Alarm Manager configurations which send alarms as HTTP POST requests with a JSON body.

{{event "webhooks"}}

{{fields "webhooks"}}
20 changes: 20 additions & 0 deletions packages/ubnt_unifi/_dev/deploy/docker/docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
services:
test-filestream:
image: alpine
volumes:
- ./sample_logs:/sample_logs:ro
- ${SERVICE_LOGS_DIR}:/var/log
command: /bin/sh -c "cp /sample_logs/* /var/log/"
test-udp-syslog:
image: docker.elastic.co/observability/stream:v0.18.0
volumes:
- ./sample_logs:/sample_logs:ro
command: log --start-signal=SIGHUP --delay=5s --addr elastic-agent:9514 -p=udp /sample_logs/logs-udp-syslog.log
test-http_endpoint:
image: docker.elastic.co/observability/stream:v0.18.0
volumes:
- ./sample_logs:/sample_logs:ro
environment:
- STREAM_PROTOCOL=webhook
- STREAM_ADDR=http://elastic-agent:10002/
command: log --start-signal=SIGHUP --delay=5s /sample_logs/logs-webhooks.ndjson
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
Jul 5 14:16:05 unifi CEF: 0|Ubiquiti|UniFi OS|4.3.5|admins|1|msg=Some User changed the SSH access setting from "undefined" to "disabled". Source IP: 192.168.0.167
Jul 5 04:17:21 unifi.fqdn 2025-07-05T04: 17:21.976Z unifi CEF:0|Ubiquiti|UniFi Network|9.3.33|544|Admin Accessed UniFi Network|1|UNIFIcategory=System UNIFIsubCategory=Admin UNIFIhost=unifi UNIFIaccessMethod=web UNIFIadmin=Some User src=192.168.0.167 msg=Some User accessed UniFi Network using the web. Source IP: 192.168.0.167
Jul 5 04:21:59 unifi.fqdn 2025-07-05T04: 21:59.127Z unifi CEF:0|Ubiquiti|UniFi Network|9.3.33|549|Admin Removed Config|3|UNIFIcategory=System UNIFIsubCategory=Admin UNIFIhost=unifi UNIFIsettingsChanges=logging: UNIFIaccessMethod=web UNIFIsettingsSection=FIREWALL_POLICY UNIFIsettingsEntry=[CUSTOM2_LAN]Block All Traffic UNIFIadmin=Some User src=192.168.0.167 msg=Some User removed [CUSTOM2_LAN]Block All Traffic Firewall Policy. Source IP: 192.168.0.167
Jul 5 04:29:04 unifi.fqdn 2025-07-05T04: 29:04.222Z unifi CEF:0|Ubiquiti|UniFi Network|9.3.33|202|Honeypot Triggered|6|UNIFIcategory=Security UNIFIsubCategory=Honeypot UNIFIhost=unifi UNIFIdeviceMac=01:23:45:67:89:0a UNIFIdeviceName=unifi UNIFIdeviceModel=UniFi Dream Machine PRO SE UNIFIdeviceIp=192.168.0.1 UNIFIdeviceVersion=4.3.5 UNIFIclientAlias=ClientDescription UNIFIclientHostname=client UNIFIclientMac=01:23:45:67:89:0a msg=Honeypot triggered by ClientDescription.
Jul 5 04:29:36 unifi.fqdn 2025-07-05T04: 29:36.878Z unifi CEF:0|Ubiquiti|UniFi Network|9.3.33|201|Threat Detected and Blocked|9|proto=TCP src=192.168.0.16 spt=60700 dst=192.168.0.2 dpt=8000 UNIFIcategory=Security UNIFIsubCategory=Intrusion Prevention UNIFIhost=unifi UNIFIdeviceMac=d0:21:f9:89:c2:43 UNIFIdeviceName=unifi UNIFIdeviceModel=UniFi Dream Machine PRO SE UNIFIdeviceIp=192.168.0.1 UNIFIdeviceVersion=4.3.5 UNIFIrisk=high UNIFIipsSessionId=255132502100797 UNIFIipsSignature=ET SCAN Possible Nmap User-Agent Observed UNIFIipsSignatureId=2024364 msg=A network intrusion attempt from 192.168.0.16 to 192.168.0.2 has been detected and blocked.
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Jul 5 13:58:28 unifi [VPN_LOCAL-A-2147483647] DESCR="[VPN_LOCAL]Allow All Traffic" IN=wgsrv1 OUT= MAC= SRC=192.168.0.167 DST=192.168.1.1 LEN=64 TOS=00 PREC=0x00 TTL=63 ID=0 DF PROTO=TCP SPT=64395 DPT=443 SEQ=4043694216 ACK=0 WINDOW=65535 SYN URGP=0 MARK=1a0000
Jul 5 14:17:39 unifi [VPN_LAN-A-10002] DESCR="Allow VPN to Internal - TCP Services" IN=wgsrv1 OUT=br2 MAC= SRC=192.168.0.167 DST=192.168.2.16 LEN=64 TOS=00 PREC=0x00 TTL=62 ID=0 DF PROTO=TCP SPT=57425 DPT=22122 SEQ=1288244791 ACK=0 WINDOW=65535 SYN URGP=0 MARK=1a0000
Jul 5 14:18:04 unifi [LAN_GUEST-D-10000] DESCR="Block Internal to Hotspot" IN=br2 OUT=br999 MAC=01:23:45:67:89:0a:01:23:45:67:89:0a:08:00 SRC=192.168.2.16 DST=192.168.4.4 LEN=84 TOS=00 PREC=0x00 TTL=63 ID=3868 DF PROTO=ICMP TYPE=8 CODE=0 ID=1 SEQ=1 MARK=1a0000
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
Jul 5 13:58:32 unifi unifi systemd[1]: Reloading.
Jul 5 13:58:33 unifi unifi systemd[1]: Starting UniFi Directory Service...
Jul 5 13:58:33 unifi unifi pre-start.sh[1464721]: cat: /data/unifi-directory/tmp/.restore_status: No such file or directory
Jul 5 13:58:33 unifi unifi sudo[1464738]: root : PWD=/ ; USER=postgres ; COMMAND=/usr/bin/psql -d unifi-directory -U postgres -c select 1
Jul 5 13:58:52 unifi unifi mcad[5277]: udapi_cache.udapi_cache_set_global_update_interval(): Bumping global update interval :: interval=1000msec->10000msec
Jul 5 13:58:54 unifi unifi earlyoom[1722]: mem avail: 685 of 3946 MiB (17.38%), swap free: 6045 of 7167 MiB (84.34%)
Jul 5 14:01:35 unifi unifi ubios-udapi-server[1475278]: ; <<>> DiG 9.16.50-Debian <<>> google.com -p 5053 +retry=3 +time=1 +noall
Jul 5 14:01:45 unifi unifi odhcp6c[5241]: Got a valid REPLY after 12ms
Jul 5 14:01:45 unifi unifi odhcp6c[5241]: IA_NA 0001 T1 300 T2 480
Jul 5 14:02:10 unifi unifi dpi-flow-stats[3687]: ubnt-dpi-util: mdns data: Error reading file
Jul 5 14:02:50 unifi unifi dnsmasq[1208700]: inotify: /run/dnsmasq.dns.conf.d/hosts.d//leases new or modified
Jul 5 14:02:54 unifi unifi earlyoom[1722]: mem avail: 620 of 3946 MiB (15.72%), swap free: 6043 of 7167 MiB (84.31%)
Jul 5 14:17:20 unifi unifi ubnt-idsips-daemon[9482]: 2025-07-05T14:17:20.410+1000#011Info: Subscription is active: true
Jul 5 14:29:11 unifi unifi ubios-udapi-server[3687]: [error] ubnt-dpi-util: mdns data: Error reading file
Jul 5 14:29:36 unifi unifi ubnt-idsips-daemon[9482]: 2025-07-05T14:29:36.390+1000#011Warn: error handling event: ipset[ips] add failed ip1:192.168.0.16, port1:45006, ip2:192.168.0.2, port2:80, proto:tcp, err1:, err2:ipset v7.10: Element cannot be added to the set: it's already added, ignore
Jul 5 14:29:36 unifi unifi ubnt-idsips-daemon[9482]: 2025-07-05T14:29:36.391+1000#011Warn: error handling event: add event version err: <nil>, add counterpart hostname err: no public ip found, add reference url err: <nil>, add out iface err: <nil>
Jul 5 14:03:48 ap2 01234567890b,U7-Pro-8.0.49+16814: syswrapper[7933]: Trigger rrm scan(1): sleep 3;iwpriv ath10 acsrrm 11; sleep 1;
Jul 5 14:01:17 ap3 01234567890a,U6-Lite-6.7.17+15512: libubnt[15024]: mcad[15024]: wireless_agg_stats.log_sta_anomalies(): bssid=01:23:45:67:89:0a radio=ra0 vap=ra2 sta=01:23:45:67:89:0a satisfaction_now=77 anomalies=tcp_latency
Jul 5 14:01:20 ap3 01234567890a,U6-Lite-6.7.17+15512: kernel: [1105260.277109] ApSiteSurveyNew_by_wdev : bandidx :0!!
Jul 5 14:01:21 ap3 01234567890a,U6-Lite-6.7.17+15512: syswrapper[9363]: Trigger rrm scan(3): sleep 2;iwpriv rai0 set ApScanChannel=active:36:120; sleep 1;
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
Jul 03 13:58:21 unifi.localdomain %{MESSAGE}%
Jul 03 13:58:21 2025-07-03T13:58:21.557Z unifi.localdomain %{MESSAGE}%
2025-07-03T13:58:21.557Z 2025-07-03T13:58:21.557Z unifi.localdomain %{MESSAGE}%
Jul 3 01:56:54 unifi.localdomain 2025-07-03T01:56:54.222Z unifi %{MESSAGE}%
Jul 3 01:56:54 unifi.localdomain 2025-07-03T01: 56:54.222Z unifi %{MESSAGE}%
hostname-switch 1234567890,MODEL-1.2.3.456: %{MESSAGE}%
<27>Jul 03 13:58:21 unifi.localdomain %{MESSAGE}%
<27>Jul 03 13:58:21 2025-07-03T13:58:21.557Z unifi.localdomain %{MESSAGE}%
<27>2025-07-03T13:58:21.557Z 2025-07-03T13:58:21.557Z unifi.localdomain %{MESSAGE}%
<27>Jul 3 01:56:54 unifi.localdomain 2025-07-03T01:56:54.222Z unifi %{MESSAGE}%
<27>Jul 3 01:56:54 unifi.localdomain 2025-07-03T01: 56:54.222Z unifi %{MESSAGE}%
<27>hostname-switch 1234567890,MODEL-1.2.3.456: %{MESSAGE}%
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
{"alarm":{"conditions":[{"condition":{"source":"device_issue","type":"is"}},{"condition":{"source":"device_adoption_state_changed","type":"is"}},{"condition":{"source":"device_discovery","type":"is"}},{"condition":{"source":"admin_access","type":"is"}},{"condition":{"source":"admin_recording_clips_manipulations","type":"is"}},{"condition":{"source":"admin_geolocation","type":"is"}},{"condition":{"source":"admin_settings_change","type":"is"}},{"condition":{"source":"device_update_status_change","type":"is"}},{"condition":{"source":"camera_utilization_limit","type":"is"}},{"condition":{"source":"application_issue","type":"is"}}],"name":"Elastic - System - All","sources":[],"triggers":[{"device":"nvr","eventId":"6865498302c5a803e4234efe","key":"admin_access","timestamp":1751468419711}]},"timestamp":1751468420734}
{"events":[{"alert_id":"68654963897bb377dc0f6479","alert_key":"ADMIN_ACCESS","id":"event.admin_accessed_unifi_network","scope":{"site_id":"1234567890abcdef12345678"}}]}
{"events":[{"alert_id":"68654a6c897bb377dc0f64d0","alert_key":"CLIENT_DISCONNECTED_WIRELESS_2","id":"event.client_disconnected","scope":{"client_device_id":"01:23:45:67:89:0a","site_id":"1234567890abcdef12345678"}}]}
{"events":[{"alert_id":"68654cc4897bb377dc0f65d1","alert_key":"CLIENT_DISCONNECTED_WIRED_2","id":"event.client_disconnected","scope":{"client_device_id":"01:23:45:67:89:0a","site_id":"1234567890abcdef12345678"}}]}
{"events":[{"alert_id":"68654e54897bb377dc0f6670","alert_key":"CLIENT_CONNECTED_WIRELESS_2","id":"event.client_connected","scope":{"client_device_id":"01:23:45:67:89:0a","site_id":"1234567890abcdef12345678"}}]}
{"events":[{"alert_id":"6865d1cc897bb377dc0f916d","alert_key":"HONEYPOT_HIT_DETECTED_KNOWN_CLIENT","id":"event.honeypot_triggered","scope":{"site_id":"1234567890abcdef12345678"}}]}
12 changes: 12 additions & 0 deletions packages/ubnt_unifi/changelog.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# newer versions go on top
# newer versions go on top
- version: "0.1.6"
changes:
- description: Update README to fix typo and missing field descriptions, ran elastic-package format
type: enhancement
link: https://github.com/elastic/integrations/pull/14566
- version: "0.1.5"
changes:
- description: Initial public release
type: enhancement
link: https://github.com/elastic/integrations/pull/14566
Loading