diff --git a/packages/ubnt_unifi/_dev/build/build.yml b/packages/ubnt_unifi/_dev/build/build.yml new file mode 100644 index 00000000000..d8553567e9c --- /dev/null +++ b/packages/ubnt_unifi/_dev/build/build.yml @@ -0,0 +1,3 @@ +dependencies: + ecs: + reference: "git@v8.17.0" diff --git a/packages/ubnt_unifi/_dev/build/docs/README.md b/packages/ubnt_unifi/_dev/build/docs/README.md new file mode 100644 index 00000000000..5a43b6f770c --- /dev/null +++ b/packages/ubnt_unifi/_dev/build/docs/README.md @@ -0,0 +1,112 @@ +# Ubiquiti UniFi + +This integration is for [Ubiquiti UniFi](https://ui.com) equipment event logs. The package processes events collected from Ubiquiti Unifi devices. + +## Data Streams + +The Ubiquiti UniFi integration collects the following event types: + +- **logs**, Logs produced via UDP syslog from a Unifi controller, application or device. + +This includes CEF logs, iptables firewall logs, and other Unix/Linux style syslog messages that may be produced. + +You can use Elastic Agent to read files of logs if you already have a syslog aggregation system that is already collecting UniFi syslog output. Or alternatively you can configure your UniFi systems to log directly to a UDP listener on an Elastic Agent. + +- **webhooks**, Events produced by Unifi Alarm Manager as webhooks, aka. HTTP POST's with a JSON body. + +The Ubiquiti UniFi Alarm Manager and webhook based alarms are very new features and the content currently included in the body of a webhook is highly variable in terms of quality and field completeness. + +## Related Integrations + +**NOTE**: Ubiquiti UniFi now supports NetFlow based traffic logging. If network flow visibility is desired you can and should utilise the existing Elastic [Netflow](https://www.elastic.co/docs/reference/integrations/netflow) integration using NetFlow Version 9 to collect flow records from your Ubiquiti UniFi equipment. Refer to [https://community.ui.com/releases](https://community.ui.com/releases) for further documentation regarding NetFlow support and configuration instructions. + +**NOTE**: Ubiquiti UniFi produces iptables "style" firewall logs with a slightly different format to the firewall logs previously produced by other Ubiquiti systems. You do not need to, and should not, install or utilise existing Ubiquiti support within the [iptables](https://www.elastic.co/docs/reference/integrations/iptables) integration as it will not work for firewall logs produced by UniFi systems. You should utilise this integration to collect Ubiquiti UniFi firewall logs independently of other non-UniFi Ubiquiti equipment. + +**NOTE**: Ubiquiti UniFi components produce iptables style firewall logs, *some* CEF format logs for configuration activity and events on UniFi consoles and within applications, as well as some common *nix style logs. While at times these are sent with a syslog prefix at other times they are not sent with a syslog prefix. At present not all CEF logs produced by UniFi components are conformant to the Common Event Format (CEF) specification. You do not need to, and should not, attempt to utilise the existing Elastic [CEF](https://www.elastic.co/docs/reference/integrations/cef) integration to process Ubiquiti UniFi logs in any way. This Ubiquiti UniFi integration includes Elastic Agent beat level content fixes for the format problems that are often produced by Ubiquiti UniFi components at present. + +## Requirements + +For `logs` based event collection Elastic Agent *MUST* be utilised due to the pre-processing and filtering that occurs at the agent level. For example CEF parsing is completed by the Elastic Agent, as this is the only component that natively supports CEF parsing, when logs are first received from the network or read from file. A number of content fixes are applied. + +If `logs` are received/aggregated or otherwise handled by something else and delivered to Elasticsearch for indexing, without passing thru an Elastic Agent, you should replicate the Elastic Agent behaviour, including content fixes, CEF parsing, as well as appropriate tagging. + +`webhooks` events from the Ubiquiti UniFi Alarm Manager feature/s require no special Elastic Agent based pre-processing and can be delivered to Elasticsearch for indexing via any method that is suitable for your environment; provided you tag the events appropriately. + +For more details and installation instructions, please refer to the [Elastic Agent Installation Guide](https://www.elastic.co/guide/en/fleet/current/elastic-agent-installation.html). + +Your Ubiquiti UniFi infrastructure should consist of: +- Ubiquiti UniFi OS `4.0.0` or higher, if running a Ubquiti Unifi Cloud Gateway or similar appliance. +- Ubiquiti UniFi Applications, e.g. Network, `9.0.0` or higher, either on a Ubquiti Unifi Cloud Gateway or self hosted. + +Refer to [https://community.ui.com/releases](https://community.ui.com/releases) for current release information, upgrade instructions and further documentation. + +**NOTE**: This integration has been tested with Ubiquiti UniFi Cloud Gateways only, self-hosted versions of UniFi applications should work but have not been tested. + +**NOTE**: This integration has only been tested with Ubiquiti UniFi Network and Protect applications at this time. + +### Installing and managing an Elastic Agent: + +There are several options for installing and managing Elastic Agent: + +### Install a Fleet-managed Elastic Agent (recommended): + +With this approach, you install Elastic Agent and use Fleet in Kibana to define, configure, and manage your agents in a central location. We recommend using Fleet management because it makes the management and upgrade of your agents considerably easier. + +### Install Elastic Agent in standalone mode (advanced users): + +With this approach, you install Elastic Agent and manually configure the agent locally on the system where it’s installed. You are responsible for managing and upgrading the agents. This approach is reserved for advanced users only. + +### Install Elastic Agent in a containerized environment: + +You can run Elastic Agent inside a container, either with Fleet Server or standalone. Docker images for all versions of Elastic Agent are available from the Elastic Docker registry, and we provide deployment manifests for running on Kubernetes. + +Please note, there are minimum requirements for running Elastic Agent. For more information, refer to the [Elastic Agent Minimum Requirements](https://www.elastic.co/guide/en/fleet/current/elastic-agent-installation.html#elastic-agent-installation-minimum-requirements). + + +### Enabling the integration in Elastic: + +1. In Kibana navigate to Management > Integrations. +2. In "Search for integrations" top bar, search for `Ubiquiti UniFi`. +3. Select the "Ubiquiti UniFi" integration from the search results. +4. Select "Add Ubiquiti UniFi" to add the integration. +5. Add all the required integration configuration parameters. +6. Select "Save and continue" to save the integration. + +The default syslog based log collection configuration is likely suitable for most environments, e.g. + +![Default Integration Configuration](../img/add-integration-defaults.png) + +### Enabling SIEM integration in Ubiquiti UniFi: + +Logging for UnifiOS and Unifi applications can be configured via, + +1. Login to your Unifi system, navigate to Settings, typically found via the gear icon in the menu bar to the left +2. Click on "Control Plane" in the second level menu to the left of the screen +3. Click on "Integrations" in the third level menu near the top of the screen +4. Select "SIEM Server" next to "Activity Logging (Syslog)" +5. Select Activity Log Categories as appropriate, note that "UniFi OS" categories will be for admin activity and other system events, while "Network" categories can be used to enable traffic logging including logging of traffic that matches the default firewally policy. +6. Enter the IP address and port that your Elastic Agent Ubiquiti UniFi syslog integration listener has been configured to use +7. Optionally click "Send Test Event" and ensure ingest to Elastic is occurring +8. Click "Save" to save the configuration + +Additional logging options may be available via other screens. + +![Control Plane SIEM Integration Configuration](../img/configure-unifi-siem-integration.png) + +## Logs + +### Ubiquiti UniFi Logs + +The `logs` dataset collects Ubiquiti Unifi logs sent via syslog. + +{{event "logs"}} + +{{fields "logs"}} + +### Ubiquiti UniFi Webhooks + +The `webhooks` dataset collects Ubiquiti Unifi events producted by Alarm Manager configurations which send alarms as HTTP POST requests with a JSON body. + +{{event "webhooks"}} + +{{fields "webhooks"}} \ No newline at end of file diff --git a/packages/ubnt_unifi/_dev/deploy/docker/docker-compose.yml b/packages/ubnt_unifi/_dev/deploy/docker/docker-compose.yml new file mode 100644 index 00000000000..92d25383b78 --- /dev/null +++ b/packages/ubnt_unifi/_dev/deploy/docker/docker-compose.yml @@ -0,0 +1,20 @@ +services: + test-filestream: + image: alpine + volumes: + - ./sample_logs:/sample_logs:ro + - ${SERVICE_LOGS_DIR}:/var/log + command: /bin/sh -c "cp /sample_logs/* /var/log/" + test-udp-syslog: + image: docker.elastic.co/observability/stream:v0.18.0 + volumes: + - ./sample_logs:/sample_logs:ro + command: log --start-signal=SIGHUP --delay=5s --addr elastic-agent:9514 -p=udp /sample_logs/logs-udp-syslog.log + test-http_endpoint: + image: docker.elastic.co/observability/stream:v0.18.0 + volumes: + - ./sample_logs:/sample_logs:ro + environment: + - STREAM_PROTOCOL=webhook + - STREAM_ADDR=http://elastic-agent:10002/ + command: log --start-signal=SIGHUP --delay=5s /sample_logs/logs-webhooks.ndjson diff --git a/packages/ubnt_unifi/_dev/deploy/docker/sample_logs/logs-filestream-cef.log b/packages/ubnt_unifi/_dev/deploy/docker/sample_logs/logs-filestream-cef.log new file mode 100644 index 00000000000..2651f6d95e0 --- /dev/null +++ b/packages/ubnt_unifi/_dev/deploy/docker/sample_logs/logs-filestream-cef.log @@ -0,0 +1,5 @@ +Jul 5 14:16:05 unifi CEF: 0|Ubiquiti|UniFi OS|4.3.5|admins|1|msg=Some User changed the SSH access setting from "undefined" to "disabled". Source IP: 192.168.0.167 +Jul 5 04:17:21 unifi.fqdn 2025-07-05T04: 17:21.976Z unifi CEF:0|Ubiquiti|UniFi Network|9.3.33|544|Admin Accessed UniFi Network|1|UNIFIcategory=System UNIFIsubCategory=Admin UNIFIhost=unifi UNIFIaccessMethod=web UNIFIadmin=Some User src=192.168.0.167 msg=Some User accessed UniFi Network using the web. Source IP: 192.168.0.167 +Jul 5 04:21:59 unifi.fqdn 2025-07-05T04: 21:59.127Z unifi CEF:0|Ubiquiti|UniFi Network|9.3.33|549|Admin Removed Config|3|UNIFIcategory=System UNIFIsubCategory=Admin UNIFIhost=unifi UNIFIsettingsChanges=logging: UNIFIaccessMethod=web UNIFIsettingsSection=FIREWALL_POLICY UNIFIsettingsEntry=[CUSTOM2_LAN]Block All Traffic UNIFIadmin=Some User src=192.168.0.167 msg=Some User removed [CUSTOM2_LAN]Block All Traffic Firewall Policy. Source IP: 192.168.0.167 +Jul 5 04:29:04 unifi.fqdn 2025-07-05T04: 29:04.222Z unifi CEF:0|Ubiquiti|UniFi Network|9.3.33|202|Honeypot Triggered|6|UNIFIcategory=Security UNIFIsubCategory=Honeypot UNIFIhost=unifi UNIFIdeviceMac=01:23:45:67:89:0a UNIFIdeviceName=unifi UNIFIdeviceModel=UniFi Dream Machine PRO SE UNIFIdeviceIp=192.168.0.1 UNIFIdeviceVersion=4.3.5 UNIFIclientAlias=ClientDescription UNIFIclientHostname=client UNIFIclientMac=01:23:45:67:89:0a msg=Honeypot triggered by ClientDescription. +Jul 5 04:29:36 unifi.fqdn 2025-07-05T04: 29:36.878Z unifi CEF:0|Ubiquiti|UniFi Network|9.3.33|201|Threat Detected and Blocked|9|proto=TCP src=192.168.0.16 spt=60700 dst=192.168.0.2 dpt=8000 UNIFIcategory=Security UNIFIsubCategory=Intrusion Prevention UNIFIhost=unifi UNIFIdeviceMac=d0:21:f9:89:c2:43 UNIFIdeviceName=unifi UNIFIdeviceModel=UniFi Dream Machine PRO SE UNIFIdeviceIp=192.168.0.1 UNIFIdeviceVersion=4.3.5 UNIFIrisk=high UNIFIipsSessionId=255132502100797 UNIFIipsSignature=ET SCAN Possible Nmap User-Agent Observed UNIFIipsSignatureId=2024364 msg=A network intrusion attempt from 192.168.0.16 to 192.168.0.2 has been detected and blocked. diff --git a/packages/ubnt_unifi/_dev/deploy/docker/sample_logs/logs-filestream-firewall.log b/packages/ubnt_unifi/_dev/deploy/docker/sample_logs/logs-filestream-firewall.log new file mode 100644 index 00000000000..85f03ea292f --- /dev/null +++ b/packages/ubnt_unifi/_dev/deploy/docker/sample_logs/logs-filestream-firewall.log @@ -0,0 +1,3 @@ +Jul 5 13:58:28 unifi [VPN_LOCAL-A-2147483647] DESCR="[VPN_LOCAL]Allow All Traffic" IN=wgsrv1 OUT= MAC= SRC=192.168.0.167 DST=192.168.1.1 LEN=64 TOS=00 PREC=0x00 TTL=63 ID=0 DF PROTO=TCP SPT=64395 DPT=443 SEQ=4043694216 ACK=0 WINDOW=65535 SYN URGP=0 MARK=1a0000 +Jul 5 14:17:39 unifi [VPN_LAN-A-10002] DESCR="Allow VPN to Internal - TCP Services" IN=wgsrv1 OUT=br2 MAC= SRC=192.168.0.167 DST=192.168.2.16 LEN=64 TOS=00 PREC=0x00 TTL=62 ID=0 DF PROTO=TCP SPT=57425 DPT=22122 SEQ=1288244791 ACK=0 WINDOW=65535 SYN URGP=0 MARK=1a0000 +Jul 5 14:18:04 unifi [LAN_GUEST-D-10000] DESCR="Block Internal to Hotspot" IN=br2 OUT=br999 MAC=01:23:45:67:89:0a:01:23:45:67:89:0a:08:00 SRC=192.168.2.16 DST=192.168.4.4 LEN=84 TOS=00 PREC=0x00 TTL=63 ID=3868 DF PROTO=ICMP TYPE=8 CODE=0 ID=1 SEQ=1 MARK=1a0000 diff --git a/packages/ubnt_unifi/_dev/deploy/docker/sample_logs/logs-filestream-other.log b/packages/ubnt_unifi/_dev/deploy/docker/sample_logs/logs-filestream-other.log new file mode 100644 index 00000000000..04bf357bdb7 --- /dev/null +++ b/packages/ubnt_unifi/_dev/deploy/docker/sample_logs/logs-filestream-other.log @@ -0,0 +1,20 @@ +Jul 5 13:58:32 unifi unifi systemd[1]: Reloading. +Jul 5 13:58:33 unifi unifi systemd[1]: Starting UniFi Directory Service... +Jul 5 13:58:33 unifi unifi pre-start.sh[1464721]: cat: /data/unifi-directory/tmp/.restore_status: No such file or directory +Jul 5 13:58:33 unifi unifi sudo[1464738]: root : PWD=/ ; USER=postgres ; COMMAND=/usr/bin/psql -d unifi-directory -U postgres -c select 1 +Jul 5 13:58:52 unifi unifi mcad[5277]: udapi_cache.udapi_cache_set_global_update_interval(): Bumping global update interval :: interval=1000msec->10000msec +Jul 5 13:58:54 unifi unifi earlyoom[1722]: mem avail: 685 of 3946 MiB (17.38%), swap free: 6045 of 7167 MiB (84.34%) +Jul 5 14:01:35 unifi unifi ubios-udapi-server[1475278]: ; <<>> DiG 9.16.50-Debian <<>> google.com -p 5053 +retry=3 +time=1 +noall +Jul 5 14:01:45 unifi unifi odhcp6c[5241]: Got a valid REPLY after 12ms +Jul 5 14:01:45 unifi unifi odhcp6c[5241]: IA_NA 0001 T1 300 T2 480 +Jul 5 14:02:10 unifi unifi dpi-flow-stats[3687]: ubnt-dpi-util: mdns data: Error reading file +Jul 5 14:02:50 unifi unifi dnsmasq[1208700]: inotify: /run/dnsmasq.dns.conf.d/hosts.d//leases new or modified +Jul 5 14:02:54 unifi unifi earlyoom[1722]: mem avail: 620 of 3946 MiB (15.72%), swap free: 6043 of 7167 MiB (84.31%) +Jul 5 14:17:20 unifi unifi ubnt-idsips-daemon[9482]: 2025-07-05T14:17:20.410+1000#011Info: Subscription is active: true +Jul 5 14:29:11 unifi unifi ubios-udapi-server[3687]: [error] ubnt-dpi-util: mdns data: Error reading file +Jul 5 14:29:36 unifi unifi ubnt-idsips-daemon[9482]: 2025-07-05T14:29:36.390+1000#011Warn: error handling event: ipset[ips] add failed ip1:192.168.0.16, port1:45006, ip2:192.168.0.2, port2:80, proto:tcp, err1:, err2:ipset v7.10: Element cannot be added to the set: it's already added, ignore +Jul 5 14:29:36 unifi unifi ubnt-idsips-daemon[9482]: 2025-07-05T14:29:36.391+1000#011Warn: error handling event: add event version err: , add counterpart hostname err: no public ip found, add reference url err: , add out iface err: +Jul 5 14:03:48 ap2 01234567890b,U7-Pro-8.0.49+16814: syswrapper[7933]: Trigger rrm scan(1): sleep 3;iwpriv ath10 acsrrm 11; sleep 1; +Jul 5 14:01:17 ap3 01234567890a,U6-Lite-6.7.17+15512: libubnt[15024]: mcad[15024]: wireless_agg_stats.log_sta_anomalies(): bssid=01:23:45:67:89:0a radio=ra0 vap=ra2 sta=01:23:45:67:89:0a satisfaction_now=77 anomalies=tcp_latency +Jul 5 14:01:20 ap3 01234567890a,U6-Lite-6.7.17+15512: kernel: [1105260.277109] ApSiteSurveyNew_by_wdev : bandidx :0!! +Jul 5 14:01:21 ap3 01234567890a,U6-Lite-6.7.17+15512: syswrapper[9363]: Trigger rrm scan(3): sleep 2;iwpriv rai0 set ApScanChannel=active:36:120; sleep 1; diff --git a/packages/ubnt_unifi/_dev/deploy/docker/sample_logs/logs-udp-syslog.log b/packages/ubnt_unifi/_dev/deploy/docker/sample_logs/logs-udp-syslog.log new file mode 100644 index 00000000000..55361a54323 --- /dev/null +++ b/packages/ubnt_unifi/_dev/deploy/docker/sample_logs/logs-udp-syslog.log @@ -0,0 +1,12 @@ +Jul 03 13:58:21 unifi.localdomain %{MESSAGE}% +Jul 03 13:58:21 2025-07-03T13:58:21.557Z unifi.localdomain %{MESSAGE}% +2025-07-03T13:58:21.557Z 2025-07-03T13:58:21.557Z unifi.localdomain %{MESSAGE}% +Jul 3 01:56:54 unifi.localdomain 2025-07-03T01:56:54.222Z unifi %{MESSAGE}% +Jul 3 01:56:54 unifi.localdomain 2025-07-03T01: 56:54.222Z unifi %{MESSAGE}% +hostname-switch 1234567890,MODEL-1.2.3.456: %{MESSAGE}% +<27>Jul 03 13:58:21 unifi.localdomain %{MESSAGE}% +<27>Jul 03 13:58:21 2025-07-03T13:58:21.557Z unifi.localdomain %{MESSAGE}% +<27>2025-07-03T13:58:21.557Z 2025-07-03T13:58:21.557Z unifi.localdomain %{MESSAGE}% +<27>Jul 3 01:56:54 unifi.localdomain 2025-07-03T01:56:54.222Z unifi %{MESSAGE}% +<27>Jul 3 01:56:54 unifi.localdomain 2025-07-03T01: 56:54.222Z unifi %{MESSAGE}% +<27>hostname-switch 1234567890,MODEL-1.2.3.456: %{MESSAGE}% diff --git a/packages/ubnt_unifi/_dev/deploy/docker/sample_logs/logs-webhooks.ndjson b/packages/ubnt_unifi/_dev/deploy/docker/sample_logs/logs-webhooks.ndjson new file mode 100644 index 00000000000..4c746f89b8e --- /dev/null +++ b/packages/ubnt_unifi/_dev/deploy/docker/sample_logs/logs-webhooks.ndjson @@ -0,0 +1,6 @@ +{"alarm":{"conditions":[{"condition":{"source":"device_issue","type":"is"}},{"condition":{"source":"device_adoption_state_changed","type":"is"}},{"condition":{"source":"device_discovery","type":"is"}},{"condition":{"source":"admin_access","type":"is"}},{"condition":{"source":"admin_recording_clips_manipulations","type":"is"}},{"condition":{"source":"admin_geolocation","type":"is"}},{"condition":{"source":"admin_settings_change","type":"is"}},{"condition":{"source":"device_update_status_change","type":"is"}},{"condition":{"source":"camera_utilization_limit","type":"is"}},{"condition":{"source":"application_issue","type":"is"}}],"name":"Elastic - System - All","sources":[],"triggers":[{"device":"nvr","eventId":"6865498302c5a803e4234efe","key":"admin_access","timestamp":1751468419711}]},"timestamp":1751468420734} +{"events":[{"alert_id":"68654963897bb377dc0f6479","alert_key":"ADMIN_ACCESS","id":"event.admin_accessed_unifi_network","scope":{"site_id":"1234567890abcdef12345678"}}]} +{"events":[{"alert_id":"68654a6c897bb377dc0f64d0","alert_key":"CLIENT_DISCONNECTED_WIRELESS_2","id":"event.client_disconnected","scope":{"client_device_id":"01:23:45:67:89:0a","site_id":"1234567890abcdef12345678"}}]} +{"events":[{"alert_id":"68654cc4897bb377dc0f65d1","alert_key":"CLIENT_DISCONNECTED_WIRED_2","id":"event.client_disconnected","scope":{"client_device_id":"01:23:45:67:89:0a","site_id":"1234567890abcdef12345678"}}]} +{"events":[{"alert_id":"68654e54897bb377dc0f6670","alert_key":"CLIENT_CONNECTED_WIRELESS_2","id":"event.client_connected","scope":{"client_device_id":"01:23:45:67:89:0a","site_id":"1234567890abcdef12345678"}}]} +{"events":[{"alert_id":"6865d1cc897bb377dc0f916d","alert_key":"HONEYPOT_HIT_DETECTED_KNOWN_CLIENT","id":"event.honeypot_triggered","scope":{"site_id":"1234567890abcdef12345678"}}]} \ No newline at end of file diff --git a/packages/ubnt_unifi/changelog.yml b/packages/ubnt_unifi/changelog.yml new file mode 100644 index 00000000000..367542acb49 --- /dev/null +++ b/packages/ubnt_unifi/changelog.yml @@ -0,0 +1,12 @@ +# newer versions go on top +# newer versions go on top +- version: "0.1.6" + changes: + - description: Update README to fix typo and missing field descriptions, ran elastic-package format + type: enhancement + link: https://github.com/elastic/integrations/pull/14566 +- version: "0.1.5" + changes: + - description: Initial public release + type: enhancement + link: https://github.com/elastic/integrations/pull/14566 diff --git a/packages/ubnt_unifi/data_stream/logs/_dev/test/pipeline/test-logs-filestream-cef.json b/packages/ubnt_unifi/data_stream/logs/_dev/test/pipeline/test-logs-filestream-cef.json new file mode 100644 index 00000000000..d199f47a60d --- /dev/null +++ b/packages/ubnt_unifi/data_stream/logs/_dev/test/pipeline/test-logs-filestream-cef.json @@ -0,0 +1,610 @@ +{ + "events": [ + { + "@timestamp": "2025-07-05T04:01:40.124Z", + "source": { + "ip": "192.168.7.84" + }, + "ecs": { + "version": "8.0.0" + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields", + "forwarded", + "ubnt-unifi-cef" + ], + "observer": { + "name": "unifi-r1", + "vendor": "Ubiquiti", + "product": "UniFi Network", + "version": "9.3.33" + }, + "agent": { + "type": "filebeat", + "version": "9.0.3", + "ephemeral_id": "01234567-89ab-cdef-4ec2-b38f-e99536291b74", + "id": "01234567-89ab-cdef-4f5d-a1b8-6ed60bda2132", + "name": "elastic-agent-01.internal.example.com" + }, + "log": { + "offset": 33374, + "file": { + "fingerprint": "4971336f945694d052c56620d03d45fc31bdbba970ed13f50e2e10bfc2f46eeb", + "path": "/home/some.admin/tmp/ubnt_unifi-samples-filestream.log", + "device_id": "64773", + "inode": "25962857" + }, + "syslog": { + "hostname": "unifi.example.com" + } + }, + "event": { + "code": "544", + "severity": 1, + "original": "Jul 5 04:01:40 unifi.example.com 2025-07-05T04: 01:40.124Z unifi-r1 CEF:0|Ubiquiti|UniFi Network|9.3.33|544|Admin Accessed UniFi Network|1|UNIFIcategory=System UNIFIsubCategory=Admin UNIFIhost=unifi-r1 UNIFIaccessMethod=web UNIFIadmin=Some Admin src=192.168.7.84 msg=Some Admin accessed UniFi Network using the web. Source IP: 192.168.7.84" + }, + "input": { + "type": "filestream" + }, + "message": "Some Admin accessed UniFi Network using the web. Source IP: 192.168.7.84", + "cef": { + "name": "Admin Accessed UniFi Network", + "severity": "1", + "extensions": { + "sourceAddress": "192.168.7.84", + "message": "Some Admin accessed UniFi Network using the web. Source IP: 192.168.7.84", + "UNIFIcategory": "System", + "UNIFIsubCategory": "Admin", + "UNIFIhost": "unifi-r1", + "UNIFIaccessMethod": "web", + "UNIFIadmin": "Some Admin" + }, + "version": "0", + "device": { + "vendor": "Ubiquiti", + "product": "UniFi Network", + "version": "9.3.33", + "event_class_id": "544" + } + } + }, + { + "@timestamp": "2025-07-05T04:21:57.873Z", + "log": { + "syslog": { + "hostname": "unifi.example.com" + }, + "file": { + "fingerprint": "4971336f945694d052c56620d03d45fc31bdbba970ed13f50e2e10bfc2f46eeb", + "path": "/home/some.admin/tmp/ubnt_unifi-samples-filestream.log", + "device_id": "64773", + "inode": "25962857" + }, + "offset": 149394 + }, + "event": { + "original": "Jul 5 04:21:57 unifi.example.com 2025-07-05T04: 21:57.873Z unifi-r1 CEF:0|Ubiquiti|UniFi Network|9.3.33|549|Admin Removed Config|3|UNIFIcategory=System UNIFIsubCategory=Admin UNIFIhost=unifi-r1 UNIFIsettingsChanges=logging: UNIFIaccessMethod=web UNIFIsettingsSection=FIREWALL_POLICY UNIFIsettingsEntry=[WAN_CUSTOM2]Block Invalid Traffic UNIFIadmin=Some Admin src=192.168.7.84 msg=Some Admin removed [WAN_CUSTOM2]Block Invalid Traffic Firewall Policy. Source IP: 192.168.7.84", + "code": "549", + "severity": 3 + }, + "input": { + "type": "filestream" + }, + "message": "Some Admin removed [WAN_CUSTOM2]Block Invalid Traffic Firewall Policy. Source IP: 192.168.7.84", + "cef": { + "version": "0", + "device": { + "vendor": "Ubiquiti", + "product": "UniFi Network", + "version": "9.3.33", + "event_class_id": "549" + }, + "name": "Admin Removed Config", + "severity": "3", + "extensions": { + "UNIFIadmin": "Some Admin", + "UNIFIsettingsEntry": "[WAN_CUSTOM2]Block Invalid Traffic", + "UNIFIsettingsChanges": "logging: ", + "UNIFIaccessMethod": "web", + "message": "Some Admin removed [WAN_CUSTOM2]Block Invalid Traffic Firewall Policy. Source IP: 192.168.7.84", + "UNIFIsubCategory": "Admin", + "UNIFIsettingsSection": "FIREWALL_POLICY", + "sourceAddress": "192.168.7.84", + "UNIFIcategory": "System", + "UNIFIhost": "unifi-r1" + } + }, + "ecs": { + "version": "8.0.0" + }, + "agent": { + "id": "01234567-89ab-cdef-4f5d-a1b8-6ed60bda2132", + "name": "elastic-agent-01.internal.example.com", + "type": "filebeat", + "version": "9.0.3", + "ephemeral_id": "01234567-89ab-cdef-4ec2-b38f-e99536291b74" + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields", + "forwarded", + "ubnt-unifi-cef" + ], + "observer": { + "name": "unifi-r1", + "vendor": "Ubiquiti", + "product": "UniFi Network", + "version": "9.3.33" + }, + "source": { + "ip": "192.168.7.84" + } + }, + { + "@timestamp": "2025-07-05T04:24:50.922Z", + "source": { + "ip": "192.168.7.84" + }, + "ecs": { + "version": "8.0.0" + }, + "event": { + "original": "Jul 5 04:24:50 unifi.example.com 2025-07-05T04: 24:50.922Z unifi-r1 CEF:0|Ubiquiti|UniFi Network|9.3.33|545|Admin Created New Config|2|cnt=3 UNIFIcategory=System UNIFIsubCategory=Admin UNIFIhost=unifi-r1 UNIFIsettingsChanges=group_members[32400]: 32400; name: Plex; group_type: port-group UNIFIaccessMethod=web UNIFIsettingsSection=FIREWALL_GROUP UNIFIsettingsEntry=Plex UNIFIadmin=Some Admin src=192.168.7.84 msg=Some Admin created Plex Firewall Group. Source IP: 192.168.7.84", + "code": "545", + "severity": 2 + }, + "log": { + "offset": 182100, + "file": { + "path": "/home/some.admin/tmp/ubnt_unifi-samples-filestream.log", + "device_id": "64773", + "inode": "25962857", + "fingerprint": "4971336f945694d052c56620d03d45fc31bdbba970ed13f50e2e10bfc2f46eeb" + }, + "syslog": { + "hostname": "unifi.example.com" + } + }, + "message": "Some Admin created Plex Firewall Group. Source IP: 192.168.7.84", + "agent": { + "type": "filebeat", + "version": "9.0.3", + "ephemeral_id": "01234567-89ab-cdef-4ec2-b38f-e99536291b74", + "id": "01234567-89ab-cdef-4f5d-a1b8-6ed60bda2132", + "name": "elastic-agent-01.internal.example.com" + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields", + "forwarded", + "ubnt-unifi-cef" + ], + "input": { + "type": "filestream" + }, + "cef": { + "version": "0", + "device": { + "vendor": "Ubiquiti", + "product": "UniFi Network", + "version": "9.3.33", + "event_class_id": "545" + }, + "name": "Admin Created New Config", + "severity": "2", + "extensions": { + "UNIFIcategory": "System", + "UNIFIhost": "unifi-r1", + "UNIFIsettingsChanges": "group_members[32400]: 32400; name: Plex; group_type: port-group", + "UNIFIsettingsSection": "FIREWALL_GROUP", + "UNIFIsubCategory": "Admin", + "UNIFIsettingsEntry": "Plex", + "message": "Some Admin created Plex Firewall Group. Source IP: 192.168.7.84", + "UNIFIaccessMethod": "web", + "UNIFIadmin": "Some Admin", + "baseEventCount": 3, + "sourceAddress": "192.168.7.84" + } + }, + "observer": { + "name": "unifi-r1", + "vendor": "Ubiquiti", + "product": "UniFi Network", + "version": "9.3.33" + } + }, + { + "@timestamp": "2025-07-05T04:28:12.739Z", + "observer": { + "product": "UniFi Network", + "version": "9.3.33", + "name": "unifi-r1", + "vendor": "Ubiquiti" + }, + "source": { + "ip": "192.168.7.84" + }, + "ecs": { + "version": "8.0.0" + }, + "event": { + "original": "Jul 5 04:28:12 unifi.example.com 2025-07-05T04: 28:12.739Z unifi-r1 CEF:0|Ubiquiti|UniFi Network|9.3.33|545|Admin Created New Config|2|cnt=25 UNIFIcategory=System UNIFIsubCategory=Admin UNIFIhost=unifi-r1 UNIFIsettingsChanges=icmp_v6_typename: ANY; destination.match_opposite_ports: false; icmp_typename: ANY; destination.zone_id: 67763563a85b7750082d50e3; destination._class: DESTINATION_ANY; schedule.all_day: false; action: ALLOW; connection_state_type: ALL; source.zone_id: 67763563a85b7750082d50e6; source._class: SOURCE_ANY; match_ip_sec: false; match_opposite_protocol: false; schedule.mode: ALWAYS; name: Allow VPN to Internal - Web; create_allow_respond: true; source.matching_target: ANY; index: 10005; source.match_opposite_ports: false; logging: true; destination.port_matching_type: OBJECT; source.port_matching_type: ANY; ip_version: BOTH; protocol: tcp; destination.port_group_id: 6868a9c9c086eb1906c49862; destination.matching_target: ANY UNIFIaccessMethod=web UNIFIsettingsSection=FIREWALL_POLICY UNIFIsettingsEntry=Allow VPN to Internal - Web UNIFIadmin=Some Admin src=192.168.7.84 msg=Some Admin created Allow VPN to Internal - Web Firewall Policy. Source IP: 192.168.7.84", + "code": "545", + "severity": 2 + }, + "log": { + "offset": 249046, + "file": { + "path": "/home/some.admin/tmp/ubnt_unifi-samples-filestream.log", + "device_id": "64773", + "inode": "25962857", + "fingerprint": "4971336f945694d052c56620d03d45fc31bdbba970ed13f50e2e10bfc2f46eeb" + }, + "syslog": { + "hostname": "unifi.example.com" + } + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields", + "forwarded", + "ubnt-unifi-cef" + ], + "message": "Some Admin created Allow VPN to Internal - Web Firewall Policy. Source IP: 192.168.7.84", + "cef": { + "version": "0", + "device": { + "vendor": "Ubiquiti", + "product": "UniFi Network", + "version": "9.3.33", + "event_class_id": "545" + }, + "name": "Admin Created New Config", + "severity": "2", + "extensions": { + "sourceAddress": "192.168.7.84", + "UNIFIcategory": "System", + "UNIFIsettingsSection": "FIREWALL_POLICY", + "message": "Some Admin created Allow VPN to Internal - Web Firewall Policy. Source IP: 192.168.7.84", + "UNIFIsettingsChanges": "icmp_v6_typename: ANY; destination.match_opposite_ports: false; icmp_typename: ANY; destination.zone_id: 67763563a85b7750082d50e3; destination._class: DESTINATION_ANY; schedule.all_day: false; action: ALLOW; connection_state_type: ALL; source.zone_id: 67763563a85b7750082d50e6; source._class: SOURCE_ANY; match_ip_sec: false; match_opposite_protocol: false; schedule.mode: ALWAYS; name: Allow VPN to Internal - Web; create_allow_respond: true; source.matching_target: ANY; index: 10005; source.match_opposite_ports: false; logging: true; destination.port_matching_type: OBJECT; source.port_matching_type: ANY; ip_version: BOTH; protocol: tcp; destination.port_group_id: 6868a9c9c086eb1906c49862; destination.matching_target: ANY", + "UNIFIadmin": "Some Admin", + "UNIFIsubCategory": "Admin", + "UNIFIhost": "unifi-r1", + "baseEventCount": 25, + "UNIFIaccessMethod": "web", + "UNIFIsettingsEntry": "Allow VPN to Internal - Web" + } + }, + "agent": { + "version": "9.0.3", + "ephemeral_id": "01234567-89ab-cdef-4ec2-b38f-e99536291b74", + "id": "01234567-89ab-cdef-4f5d-a1b8-6ed60bda2132", + "name": "elastic-agent-01.internal.example.com", + "type": "filebeat" + }, + "input": { + "type": "filestream" + } + }, + { + "@timestamp": "2025-07-05T04:29:04.222Z", + "log": { + "syslog": { + "hostname": "unifi.example.com" + }, + "offset": 288074, + "file": { + "path": "/home/some.admin/tmp/ubnt_unifi-samples-filestream.log", + "device_id": "64773", + "inode": "25962857", + "fingerprint": "4971336f945694d052c56620d03d45fc31bdbba970ed13f50e2e10bfc2f46eeb" + } + }, + "event": { + "severity": 6, + "original": "Jul 5 04:29:04 unifi.example.com 2025-07-05T04: 29:04.222Z unifi-r1 CEF:0|Ubiquiti|UniFi Network|9.3.33|202|Honeypot Triggered|6|UNIFIcategory=Security UNIFIsubCategory=Honeypot UNIFIhost=unifi-r1 UNIFIdeviceMac=01:23:45:89:c2:43 UNIFIdeviceName=unifi-r1 UNIFIdeviceModel=UniFi Dream Machine PRO SE UNIFIdeviceIp=10.238.202.175 UNIFIdeviceVersion=4.3.5 UNIFIclientAlias=NUC UNIFIclientHostname=nuc UNIFIclientMac=01:23:45:9c:98:4c msg=Honeypot triggered by NUC.", + "code": "202" + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields", + "forwarded", + "ubnt-unifi-cef" + ], + "input": { + "type": "filestream" + }, + "cef": { + "version": "0", + "device": { + "event_class_id": "202", + "vendor": "Ubiquiti", + "product": "UniFi Network", + "version": "9.3.33" + }, + "name": "Honeypot Triggered", + "severity": "6", + "extensions": { + "UNIFIdeviceModel": "UniFi Dream Machine PRO SE", + "UNIFIclientMac": "01:23:45:9c:98:4c", + "UNIFIdeviceIp": "10.238.202.175", + "UNIFIdeviceName": "unifi-r1", + "UNIFIhost": "unifi-r1", + "UNIFIdeviceMac": "01:23:45:89:c2:43", + "UNIFIdeviceVersion": "4.3.5", + "UNIFIclientAlias": "NUC", + "UNIFIclientHostname": "nuc", + "message": "Honeypot triggered by NUC.", + "UNIFIcategory": "Security", + "UNIFIsubCategory": "Honeypot" + } + }, + "message": "Honeypot triggered by NUC.", + "observer": { + "version": "9.3.33", + "name": "unifi-r1", + "vendor": "Ubiquiti", + "product": "UniFi Network" + }, + "ecs": { + "version": "8.0.0" + }, + "agent": { + "name": "elastic-agent-01.internal.example.com", + "type": "filebeat", + "version": "9.0.3", + "ephemeral_id": "01234567-89ab-cdef-4ec2-b38f-e99536291b74", + "id": "01234567-89ab-cdef-4f5d-a1b8-6ed60bda2132" + } + }, + { + "@timestamp": "2025-07-05T04:29:36.878Z", + "destination": { + "ip": "10.238.202.175", + "port": 8000 + }, + "ecs": { + "version": "8.0.0" + }, + "log": { + "offset": 298711, + "file": { + "inode": "25962857", + "fingerprint": "4971336f945694d052c56620d03d45fc31bdbba970ed13f50e2e10bfc2f46eeb", + "path": "/home/some.admin/tmp/ubnt_unifi-samples-filestream.log", + "device_id": "64773" + }, + "syslog": { + "hostname": "unifi.example.com" + } + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields", + "forwarded", + "ubnt-unifi-cef" + ], + "cef": { + "version": "0", + "device": { + "vendor": "Ubiquiti", + "product": "UniFi Network", + "version": "9.3.33", + "event_class_id": "201" + }, + "name": "Threat Detected and Blocked", + "severity": "9", + "extensions": { + "sourcePort": 60700, + "sourceAddress": "10.238.202.175", + "UNIFIdeviceIp": "10.238.202.175", + "transportProtocol": "TCP", + "UNIFIipsSignature": "ET SCAN Possible Nmap User-Agent Observed", + "message": "A network intrusion attempt from 10.238.202.175 to 10.238.202.175 has been detected and blocked.", + "UNIFIdeviceName": "unifi-r1", + "UNIFIhost": "unifi-r1", + "UNIFIdeviceMac": "01:23:45:89:c2:43", + "UNIFIrisk": "high", + "UNIFIipsSessionId": "255132502100797", + "destinationPort": 8000, + "UNIFIsubCategory": "Intrusion Prevention", + "UNIFIipsSignatureId": "2024364", + "UNIFIcategory": "Security", + "UNIFIdeviceModel": "UniFi Dream Machine PRO SE", + "destinationAddress": "10.238.202.175", + "UNIFIdeviceVersion": "4.3.5" + } + }, + "observer": { + "name": "unifi-r1", + "vendor": "Ubiquiti", + "product": "UniFi Network", + "version": "9.3.33" + }, + "source": { + "port": 60700, + "ip": "10.238.202.175" + }, + "network": { + "transport": "tcp" + }, + "agent": { + "id": "01234567-89ab-cdef-4f5d-a1b8-6ed60bda2132", + "name": "elastic-agent-01.internal.example.com", + "type": "filebeat", + "version": "9.0.3", + "ephemeral_id": "01234567-89ab-cdef-4ec2-b38f-e99536291b74" + }, + "event": { + "original": "Jul 5 04:29:36 unifi.example.com 2025-07-05T04: 29:36.878Z unifi-r1 CEF:0|Ubiquiti|UniFi Network|9.3.33|201|Threat Detected and Blocked|9|proto=TCP src=10.238.202.175 spt=60700 dst=10.238.202.175 dpt=8000 UNIFIcategory=Security UNIFIsubCategory=Intrusion Prevention UNIFIhost=unifi-r1 UNIFIdeviceMac=01:23:45:89:c2:43 UNIFIdeviceName=unifi-r1 UNIFIdeviceModel=UniFi Dream Machine PRO SE UNIFIdeviceIp=10.238.202.175 UNIFIdeviceVersion=4.3.5 UNIFIrisk=high UNIFIipsSessionId=255132502100797 UNIFIipsSignature=ET SCAN Possible Nmap User-Agent Observed UNIFIipsSignatureId=2024364 msg=A network intrusion attempt from 10.238.202.175 to 10.238.202.175 has been detected and blocked.", + "code": "201", + "severity": 9 + }, + "input": { + "type": "filestream" + }, + "message": "A network intrusion attempt from 10.238.202.175 to 10.238.202.175 has been detected and blocked." + }, + { + "@timestamp": "2025-07-15T22:24:19.421Z", + "message": "Pixel 7a connected to ohai!secure on unifi-ap1. Connection Info: Ch. 44 (5 GHz, 80 MHz), -77 dBm. IP: 10.238.202.175", + "observer": { + "version": "9.3.43", + "vendor": "Ubiquiti", + "product": "UniFi Network" + }, + "source": { + "user": { + "name": "some.admin" + } + }, + "agent": { + "name": "elastic-agent.internal", + "type": "filebeat", + "version": "9.0.3", + "ephemeral_id": "01234567-89ab-cdef-4483-a9ff-b46996c08296", + "id": "01234567-89ab-cdef-4806-a514-ea1feb145b64" + }, + "ecs": { + "version": "8.0.0" + }, + "log": { + "offset": 17599, + "file": { + "path": "/Users/some.admin/tmp/ubnt_unifi-samples-filestream.log", + "device_id": "16777231", + "inode": "252511402", + "fingerprint": "774aac143e417dff96abf0423c14ff696ebbdcaf59cf3195b29b13fe764d02d5" + }, + "syslog": { + "hostname": "unifi-r1" + } + }, + "input": { + "type": "filestream" + }, + "cef": { + "extensions": { + "UNIFIconnectedToDeviceIp": "10.238.202.175", + "UNIFIclientAlias": "Pixel 7a", + "UNIFIconnectedToDeviceName": "unifi-ap1", + "UNIFIconnectedToDeviceVersion": "8.0.49", + "UNIFIwifiChannelWidth": "80", + "UNIFInetworkName": "default", + "UNIFIclientMac": "01:23:45:12:f6:c1", + "UNIFIsubCategory": "WiFi", + "UNIFIclientHostname": "Pixel-7a", + "UNIFIcategory": "Monitoring", + "UNIFIhost": "unifi-r1", + "UNIFIclientIp": "10.238.202.175", + "UNIFIwifiBand": "na", + "UNIFIauthMethod": "wpaeap", + "UNIFIWiFiRssi": "-77", + "message": "Pixel 7a connected to ohai!secure on unifi-ap1. Connection Info: Ch. 44 (5 GHz, 80 MHz), -77 dBm. IP: 10.238.202.175", + "UNIFIconnectedToDeviceModel": "U7-Pro", + "UNIFInetworkSubnet": "10.238.202.175/24", + "UNIFIconnectedToDeviceMac": "01:23:45:8b:a1:71", + "UNIFIwifiChannel": "44", + "sourceUserName": "some.admin", + "UNIFIwifiName": "ohai!secure", + "UNIFInetworkVlan": "1" + }, + "version": "0", + "device": { + "product": "UniFi Network", + "version": "9.3.43", + "event_class_id": "400", + "vendor": "Ubiquiti" + }, + "name": "WiFi Client Connected", + "severity": "1" + }, + "event": { + "original": "Jul 15 22:24:19 2025-07-15T22:24:19.421Z unifi-r1 CEF:0|Ubiquiti|UniFi Network|9.3.43|400|WiFi Client Connected|1|UNIFIcategory=Monitoring UNIFIsubCategory=WiFi UNIFIhost=unifi-r1 UNIFIconnectedToDeviceName=unifi-ap1 UNIFIconnectedToDeviceIp=10.238.202.175 UNIFIconnectedToDeviceMac=01:23:45:8b:a1:71 UNIFIconnectedToDeviceModel=U7-Pro UNIFIconnectedToDeviceVersion=8.0.49 suser=some.admin UNIFIclientAlias=Pixel 7a UNIFIclientHostname=Pixel-7a UNIFIclientIp=10.238.202.175 UNIFIclientMac=01:23:45:12:f6:c1 UNIFIwifiChannel=44 UNIFIwifiChannelWidth=80 UNIFIwifiName=ohai!secure UNIFIwifiBand=na UNIFIauthMethod=wpaeap UNIFIWiFiRssi=-77 UNIFInetworkName=default UNIFInetworkSubnet=10.238.202.175/24 UNIFInetworkVlan=1 msg=Pixel 7a connected to ohai!secure on unifi-ap1. Connection Info: Ch. 44 (5 GHz, 80 MHz), -77 dBm. IP: 10.238.202.175", + "code": "400", + "severity": 1 + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields", + "forwarded", + "ubnt-unifi-cef" + ] + }, + { + "@timestamp": "2025-07-15T21:00:39.255Z", + "input": { + "type": "filestream" + }, + "cef": { + "device": { + "product": "UniFi Network", + "version": "9.3.43", + "event_class_id": "512", + "vendor": "Ubiquiti" + }, + "name": "Device Offline", + "severity": "8", + "extensions": { + "UNIFIdeviceIp": "10.238.202.175", + "UNIFIconnectedToDeviceVersion": "2.1.8", + "UNIFIreference": "https://help.ui.com/hc/en-us/articles/7258465146519", + "UNIFIconnectedToDevicePort": "8", + "UNIFIdeviceVersion": "6.7.17", + "UNIFIdeviceModel": "U6-Lite", + "UNIFIconnectedToDeviceModel": "USW-Flex-2.5G-8", + "UNIFIdeviceMac": "01:23:45:48:b9:34", + "message": "unifi-ap4 is offline.", + "UNIFIhost": "unifi-r1", + "UNIFIconnectedToDeviceName": "unifi-l2s2", + "UNIFIconnectedToDeviceIp": "10.238.202.175", + "UNIFIsubCategory": "Devices", + "UNIFIdeviceName": "unifi-ap4", + "UNIFIconnectedToDeviceMac": "01:23:45:4e:c3:a3", + "UNIFIcategory": "System" + }, + "version": "0" + }, + "ecs": { + "version": "8.0.0" + }, + "message": "unifi-ap4 is offline.", + "observer": { + "vendor": "Ubiquiti", + "product": "UniFi Network", + "version": "9.3.43" + }, + "agent": { + "id": "01234567-89ab-cdef-4806-a514-ea1feb145b64", + "name": "elastic-agent.internal", + "type": "filebeat", + "version": "9.0.3", + "ephemeral_id": "01234567-89ab-cdef-4483-a9ff-b46996c08296" + }, + "log": { + "file": { + "path": "/Users/some.admin/tmp/ubnt_unifi-samples-filestream.log", + "device_id": "16777231", + "inode": "252511402", + "fingerprint": "774aac143e417dff96abf0423c14ff696ebbdcaf59cf3195b29b13fe764d02d5" + }, + "offset": 18437, + "syslog": { + "hostname": "unifi-r1" + } + }, + "event": { + "original": "Jul 15 21:00:39 2025-07-15T21:00:39.255Z unifi-r1 CEF:0|Ubiquiti|UniFi Network|9.3.43|512|Device Offline|8|UNIFIcategory=System UNIFIsubCategory=Devices UNIFIhost=unifi-r1 UNIFIreference=https://help.ui.com/hc/en-us/articles/7258465146519 UNIFIdeviceMac=01:23:45:48:b9:34 UNIFIdeviceName=unifi-ap4 UNIFIdeviceModel=U6-Lite UNIFIdeviceIp=10.238.202.175 UNIFIdeviceVersion=6.7.17 UNIFIconnectedToDeviceName=unifi-l2s2 UNIFIconnectedToDevicePort=8 UNIFIconnectedToDeviceIp=10.238.202.175 UNIFIconnectedToDeviceMac=01:23:45:4e:c3:a3 UNIFIconnectedToDeviceModel=USW-Flex-2.5G-8 UNIFIconnectedToDeviceVersion=2.1.8 msg=unifi-ap4 is offline.", + "code": "512", + "severity": 8 + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields", + "forwarded", + "ubnt-unifi-cef" + ] + } + ] +} \ No newline at end of file diff --git a/packages/ubnt_unifi/data_stream/logs/_dev/test/pipeline/test-logs-filestream-cef.json-expected.json b/packages/ubnt_unifi/data_stream/logs/_dev/test/pipeline/test-logs-filestream-cef.json-expected.json new file mode 100644 index 00000000000..58fa6dae716 --- /dev/null +++ b/packages/ubnt_unifi/data_stream/logs/_dev/test/pipeline/test-logs-filestream-cef.json-expected.json @@ -0,0 +1,853 @@ +{ + "expected": [ + { + "@timestamp": "2025-07-05T04:01:40.124Z", + "agent": { + "ephemeral_id": "01234567-89ab-cdef-4ec2-b38f-e99536291b74", + "id": "01234567-89ab-cdef-4f5d-a1b8-6ed60bda2132", + "name": "elastic-agent-01.internal.example.com", + "type": "filebeat", + "version": "9.0.3" + }, + "cef": { + "device": { + "event_class_id": "544", + "product": "UniFi Network", + "vendor": "Ubiquiti", + "version": "9.3.33" + }, + "extensions": { + "UNIFIaccessMethod": "web", + "UNIFIadmin": "Some Admin", + "UNIFIcategory": "System", + "UNIFIhost": "unifi-r1", + "UNIFIsubCategory": "Admin", + "message": "Some Admin accessed UniFi Network using the web. Source IP: 192.168.7.84", + "sourceAddress": "192.168.7.84" + }, + "name": "Admin Accessed UniFi Network", + "severity": "1", + "version": "0" + }, + "ecs": { + "version": "8.17.0" + }, + "event": { + "action": "Admin Accessed UniFi Network", + "category": [ + "host", + "network" + ], + "code": "544", + "kind": "event", + "original": "Jul 5 04:01:40 unifi.example.com 2025-07-05T04: 01:40.124Z unifi-r1 CEF:0|Ubiquiti|UniFi Network|9.3.33|544|Admin Accessed UniFi Network|1|UNIFIcategory=System UNIFIsubCategory=Admin UNIFIhost=unifi-r1 UNIFIaccessMethod=web UNIFIadmin=Some Admin src=192.168.7.84 msg=Some Admin accessed UniFi Network using the web. Source IP: 192.168.7.84", + "severity": 1, + "type": [ + "access", + "info" + ] + }, + "host": { + "hostname": "unifi-r1" + }, + "input": { + "type": "filestream" + }, + "log": { + "file": { + "device_id": "64773", + "fingerprint": "4971336f945694d052c56620d03d45fc31bdbba970ed13f50e2e10bfc2f46eeb", + "inode": "25962857", + "path": "/home/some.admin/tmp/ubnt_unifi-samples-filestream.log" + }, + "level": "1", + "offset": 33374, + "syslog": { + "hostname": "unifi.example.com" + } + }, + "message": "Some Admin accessed UniFi Network using the web. Source IP: 192.168.7.84", + "observer": { + "hostname": "unifi.example.com", + "name": "unifi-r1", + "product": "UniFi Network", + "vendor": "Ubiquiti", + "version": "9.3.33" + }, + "related": { + "hosts": [ + "unifi.example.com" + ], + "ip": [ + "192.168.7.84" + ] + }, + "source": { + "address": "192.168.7.84", + "ip": "192.168.7.84" + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields", + "forwarded", + "ubnt-unifi-cef" + ], + "user": { + "full_name": "Some Admin" + } + }, + { + "@timestamp": "2025-07-05T04:21:57.873Z", + "agent": { + "ephemeral_id": "01234567-89ab-cdef-4ec2-b38f-e99536291b74", + "id": "01234567-89ab-cdef-4f5d-a1b8-6ed60bda2132", + "name": "elastic-agent-01.internal.example.com", + "type": "filebeat", + "version": "9.0.3" + }, + "cef": { + "device": { + "event_class_id": "549", + "product": "UniFi Network", + "vendor": "Ubiquiti", + "version": "9.3.33" + }, + "extensions": { + "UNIFIaccessMethod": "web", + "UNIFIadmin": "Some Admin", + "UNIFIcategory": "System", + "UNIFIhost": "unifi-r1", + "UNIFIsettingsChanges": "logging: ", + "UNIFIsettingsEntry": "[WAN_CUSTOM2]Block Invalid Traffic", + "UNIFIsettingsSection": "FIREWALL_POLICY", + "UNIFIsubCategory": "Admin", + "message": "Some Admin removed [WAN_CUSTOM2]Block Invalid Traffic Firewall Policy. Source IP: 192.168.7.84", + "sourceAddress": "192.168.7.84" + }, + "name": "Admin Removed Config", + "severity": "3", + "version": "0" + }, + "ecs": { + "version": "8.17.0" + }, + "event": { + "action": "Admin Removed Config", + "category": [ + "host", + "configuration", + "network" + ], + "code": "549", + "kind": "event", + "original": "Jul 5 04:21:57 unifi.example.com 2025-07-05T04: 21:57.873Z unifi-r1 CEF:0|Ubiquiti|UniFi Network|9.3.33|549|Admin Removed Config|3|UNIFIcategory=System UNIFIsubCategory=Admin UNIFIhost=unifi-r1 UNIFIsettingsChanges=logging: UNIFIaccessMethod=web UNIFIsettingsSection=FIREWALL_POLICY UNIFIsettingsEntry=[WAN_CUSTOM2]Block Invalid Traffic UNIFIadmin=Some Admin src=192.168.7.84 msg=Some Admin removed [WAN_CUSTOM2]Block Invalid Traffic Firewall Policy. Source IP: 192.168.7.84", + "severity": 3, + "type": [ + "change", + "info" + ] + }, + "host": { + "hostname": "unifi-r1" + }, + "input": { + "type": "filestream" + }, + "log": { + "file": { + "device_id": "64773", + "fingerprint": "4971336f945694d052c56620d03d45fc31bdbba970ed13f50e2e10bfc2f46eeb", + "inode": "25962857", + "path": "/home/some.admin/tmp/ubnt_unifi-samples-filestream.log" + }, + "level": "3", + "offset": 149394, + "syslog": { + "hostname": "unifi.example.com" + } + }, + "message": "Some Admin removed [WAN_CUSTOM2]Block Invalid Traffic Firewall Policy. Source IP: 192.168.7.84", + "observer": { + "hostname": "unifi.example.com", + "name": "unifi-r1", + "product": "UniFi Network", + "vendor": "Ubiquiti", + "version": "9.3.33" + }, + "related": { + "hosts": [ + "unifi.example.com" + ], + "ip": [ + "192.168.7.84" + ] + }, + "source": { + "address": "192.168.7.84", + "ip": "192.168.7.84" + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields", + "forwarded", + "ubnt-unifi-cef" + ], + "user": { + "full_name": "Some Admin" + } + }, + { + "@timestamp": "2025-07-05T04:24:50.922Z", + "agent": { + "ephemeral_id": "01234567-89ab-cdef-4ec2-b38f-e99536291b74", + "id": "01234567-89ab-cdef-4f5d-a1b8-6ed60bda2132", + "name": "elastic-agent-01.internal.example.com", + "type": "filebeat", + "version": "9.0.3" + }, + "cef": { + "device": { + "event_class_id": "545", + "product": "UniFi Network", + "vendor": "Ubiquiti", + "version": "9.3.33" + }, + "extensions": { + "UNIFIaccessMethod": "web", + "UNIFIadmin": "Some Admin", + "UNIFIcategory": "System", + "UNIFIhost": "unifi-r1", + "UNIFIsettingsChanges": "group_members[32400]: 32400; name: Plex; group_type: port-group", + "UNIFIsettingsEntry": "Plex", + "UNIFIsettingsSection": "FIREWALL_GROUP", + "UNIFIsubCategory": "Admin", + "baseEventCount": 3, + "message": "Some Admin created Plex Firewall Group. Source IP: 192.168.7.84", + "sourceAddress": "192.168.7.84" + }, + "name": "Admin Created New Config", + "severity": "2", + "version": "0" + }, + "ecs": { + "version": "8.17.0" + }, + "event": { + "action": "Admin Created New Config", + "category": [ + "host", + "configuration", + "network" + ], + "code": "545", + "kind": "event", + "original": "Jul 5 04:24:50 unifi.example.com 2025-07-05T04: 24:50.922Z unifi-r1 CEF:0|Ubiquiti|UniFi Network|9.3.33|545|Admin Created New Config|2|cnt=3 UNIFIcategory=System UNIFIsubCategory=Admin UNIFIhost=unifi-r1 UNIFIsettingsChanges=group_members[32400]: 32400; name: Plex; group_type: port-group UNIFIaccessMethod=web UNIFIsettingsSection=FIREWALL_GROUP UNIFIsettingsEntry=Plex UNIFIadmin=Some Admin src=192.168.7.84 msg=Some Admin created Plex Firewall Group. Source IP: 192.168.7.84", + "severity": 2, + "type": [ + "change", + "info" + ] + }, + "host": { + "hostname": "unifi-r1" + }, + "input": { + "type": "filestream" + }, + "log": { + "file": { + "device_id": "64773", + "fingerprint": "4971336f945694d052c56620d03d45fc31bdbba970ed13f50e2e10bfc2f46eeb", + "inode": "25962857", + "path": "/home/some.admin/tmp/ubnt_unifi-samples-filestream.log" + }, + "level": "2", + "offset": 182100, + "syslog": { + "hostname": "unifi.example.com" + } + }, + "message": "Some Admin created Plex Firewall Group. Source IP: 192.168.7.84", + "observer": { + "hostname": "unifi.example.com", + "name": "unifi-r1", + "product": "UniFi Network", + "vendor": "Ubiquiti", + "version": "9.3.33" + }, + "related": { + "hosts": [ + "unifi.example.com" + ], + "ip": [ + "192.168.7.84" + ] + }, + "source": { + "address": "192.168.7.84", + "ip": "192.168.7.84" + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields", + "forwarded", + "ubnt-unifi-cef" + ], + "user": { + "full_name": "Some Admin" + } + }, + { + "@timestamp": "2025-07-05T04:28:12.739Z", + "agent": { + "ephemeral_id": "01234567-89ab-cdef-4ec2-b38f-e99536291b74", + "id": "01234567-89ab-cdef-4f5d-a1b8-6ed60bda2132", + "name": "elastic-agent-01.internal.example.com", + "type": "filebeat", + "version": "9.0.3" + }, + "cef": { + "device": { + "event_class_id": "545", + "product": "UniFi Network", + "vendor": "Ubiquiti", + "version": "9.3.33" + }, + "extensions": { + "UNIFIaccessMethod": "web", + "UNIFIadmin": "Some Admin", + "UNIFIcategory": "System", + "UNIFIhost": "unifi-r1", + "UNIFIsettingsChanges": "icmp_v6_typename: ANY; destination.match_opposite_ports: false; icmp_typename: ANY; destination.zone_id: 67763563a85b7750082d50e3; destination._class: DESTINATION_ANY; schedule.all_day: false; action: ALLOW; connection_state_type: ALL; source.zone_id: 67763563a85b7750082d50e6; source._class: SOURCE_ANY; match_ip_sec: false; match_opposite_protocol: false; schedule.mode: ALWAYS; name: Allow VPN to Internal - Web; create_allow_respond: true; source.matching_target: ANY; index: 10005; source.match_opposite_ports: false; logging: true; destination.port_matching_type: OBJECT; source.port_matching_type: ANY; ip_version: BOTH; protocol: tcp; destination.port_group_id: 6868a9c9c086eb1906c49862; destination.matching_target: ANY", + "UNIFIsettingsEntry": "Allow VPN to Internal - Web", + "UNIFIsettingsSection": "FIREWALL_POLICY", + "UNIFIsubCategory": "Admin", + "baseEventCount": 25, + "message": "Some Admin created Allow VPN to Internal - Web Firewall Policy. Source IP: 192.168.7.84", + "sourceAddress": "192.168.7.84" + }, + "name": "Admin Created New Config", + "severity": "2", + "version": "0" + }, + "ecs": { + "version": "8.17.0" + }, + "event": { + "action": "Admin Created New Config", + "category": [ + "host", + "configuration", + "network" + ], + "code": "545", + "kind": "event", + "original": "Jul 5 04:28:12 unifi.example.com 2025-07-05T04: 28:12.739Z unifi-r1 CEF:0|Ubiquiti|UniFi Network|9.3.33|545|Admin Created New Config|2|cnt=25 UNIFIcategory=System UNIFIsubCategory=Admin UNIFIhost=unifi-r1 UNIFIsettingsChanges=icmp_v6_typename: ANY; destination.match_opposite_ports: false; icmp_typename: ANY; destination.zone_id: 67763563a85b7750082d50e3; destination._class: DESTINATION_ANY; schedule.all_day: false; action: ALLOW; connection_state_type: ALL; source.zone_id: 67763563a85b7750082d50e6; source._class: SOURCE_ANY; match_ip_sec: false; match_opposite_protocol: false; schedule.mode: ALWAYS; name: Allow VPN to Internal - Web; create_allow_respond: true; source.matching_target: ANY; index: 10005; source.match_opposite_ports: false; logging: true; destination.port_matching_type: OBJECT; source.port_matching_type: ANY; ip_version: BOTH; protocol: tcp; destination.port_group_id: 6868a9c9c086eb1906c49862; destination.matching_target: ANY UNIFIaccessMethod=web UNIFIsettingsSection=FIREWALL_POLICY UNIFIsettingsEntry=Allow VPN to Internal - Web UNIFIadmin=Some Admin src=192.168.7.84 msg=Some Admin created Allow VPN to Internal - Web Firewall Policy. Source IP: 192.168.7.84", + "severity": 2, + "type": [ + "change", + "info" + ] + }, + "host": { + "hostname": "unifi-r1" + }, + "input": { + "type": "filestream" + }, + "log": { + "file": { + "device_id": "64773", + "fingerprint": "4971336f945694d052c56620d03d45fc31bdbba970ed13f50e2e10bfc2f46eeb", + "inode": "25962857", + "path": "/home/some.admin/tmp/ubnt_unifi-samples-filestream.log" + }, + "level": "2", + "offset": 249046, + "syslog": { + "hostname": "unifi.example.com" + } + }, + "message": "Some Admin created Allow VPN to Internal - Web Firewall Policy. Source IP: 192.168.7.84", + "observer": { + "hostname": "unifi.example.com", + "name": "unifi-r1", + "product": "UniFi Network", + "vendor": "Ubiquiti", + "version": "9.3.33" + }, + "related": { + "hosts": [ + "unifi.example.com" + ], + "ip": [ + "192.168.7.84" + ] + }, + "source": { + "address": "192.168.7.84", + "ip": "192.168.7.84" + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields", + "forwarded", + "ubnt-unifi-cef" + ], + "user": { + "full_name": "Some Admin" + } + }, + { + "@timestamp": "2025-07-05T04:29:04.222Z", + "agent": { + "ephemeral_id": "01234567-89ab-cdef-4ec2-b38f-e99536291b74", + "id": "01234567-89ab-cdef-4f5d-a1b8-6ed60bda2132", + "name": "elastic-agent-01.internal.example.com", + "type": "filebeat", + "version": "9.0.3" + }, + "cef": { + "device": { + "event_class_id": "202", + "product": "UniFi Network", + "vendor": "Ubiquiti", + "version": "9.3.33" + }, + "extensions": { + "UNIFIcategory": "Security", + "UNIFIclientAlias": "NUC", + "UNIFIclientHostname": "nuc", + "UNIFIclientMac": "01:23:45:9c:98:4c", + "UNIFIdeviceIp": "10.238.202.175", + "UNIFIdeviceMac": "01:23:45:89:c2:43", + "UNIFIdeviceModel": "UniFi Dream Machine PRO SE", + "UNIFIdeviceName": "unifi-r1", + "UNIFIdeviceVersion": "4.3.5", + "UNIFIhost": "unifi-r1", + "UNIFIsubCategory": "Honeypot", + "message": "Honeypot triggered by NUC." + }, + "name": "Honeypot Triggered", + "severity": "6", + "version": "0" + }, + "client": { + "domain": "nuc", + "mac": "01-23-45-9C-98-4C" + }, + "ecs": { + "version": "8.17.0" + }, + "event": { + "action": "Honeypot Triggered", + "category": [ + "network", + "intrusion_detection" + ], + "code": "202", + "kind": "alert", + "original": "Jul 5 04:29:04 unifi.example.com 2025-07-05T04: 29:04.222Z unifi-r1 CEF:0|Ubiquiti|UniFi Network|9.3.33|202|Honeypot Triggered|6|UNIFIcategory=Security UNIFIsubCategory=Honeypot UNIFIhost=unifi-r1 UNIFIdeviceMac=01:23:45:89:c2:43 UNIFIdeviceName=unifi-r1 UNIFIdeviceModel=UniFi Dream Machine PRO SE UNIFIdeviceIp=10.238.202.175 UNIFIdeviceVersion=4.3.5 UNIFIclientAlias=NUC UNIFIclientHostname=nuc UNIFIclientMac=01:23:45:9c:98:4c msg=Honeypot triggered by NUC.", + "severity": 6, + "type": [ + "access", + "info" + ] + }, + "host": { + "hostname": "unifi-r1" + }, + "input": { + "type": "filestream" + }, + "log": { + "file": { + "device_id": "64773", + "fingerprint": "4971336f945694d052c56620d03d45fc31bdbba970ed13f50e2e10bfc2f46eeb", + "inode": "25962857", + "path": "/home/some.admin/tmp/ubnt_unifi-samples-filestream.log" + }, + "level": "6", + "offset": 288074, + "syslog": { + "hostname": "unifi.example.com" + } + }, + "message": "Honeypot triggered by NUC.", + "observer": { + "hostname": "unifi.example.com", + "name": "unifi-r1", + "product": "UniFi Network", + "vendor": "Ubiquiti", + "version": "9.3.33" + }, + "related": { + "hosts": [ + "unifi.example.com", + "nuc", + "unifi-r1" + ] + }, + "server": { + "domain": "unifi-r1", + "mac": "01-23-45-89-C2-43" + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields", + "forwarded", + "ubnt-unifi-cef" + ] + }, + { + "@timestamp": "2025-07-05T04:29:36.878Z", + "agent": { + "ephemeral_id": "01234567-89ab-cdef-4ec2-b38f-e99536291b74", + "id": "01234567-89ab-cdef-4f5d-a1b8-6ed60bda2132", + "name": "elastic-agent-01.internal.example.com", + "type": "filebeat", + "version": "9.0.3" + }, + "cef": { + "device": { + "event_class_id": "201", + "product": "UniFi Network", + "vendor": "Ubiquiti", + "version": "9.3.33" + }, + "extensions": { + "UNIFIcategory": "Security", + "UNIFIdeviceIp": "10.238.202.175", + "UNIFIdeviceMac": "01:23:45:89:c2:43", + "UNIFIdeviceModel": "UniFi Dream Machine PRO SE", + "UNIFIdeviceName": "unifi-r1", + "UNIFIdeviceVersion": "4.3.5", + "UNIFIhost": "unifi-r1", + "UNIFIipsSessionId": "255132502100797", + "UNIFIipsSignature": "ET SCAN Possible Nmap User-Agent Observed", + "UNIFIipsSignatureId": "2024364", + "UNIFIrisk": "high", + "UNIFIsubCategory": "Intrusion Prevention", + "destinationAddress": "10.238.202.175", + "destinationPort": 8000, + "message": "A network intrusion attempt from 10.238.202.175 to 10.238.202.175 has been detected and blocked.", + "sourceAddress": "10.238.202.175", + "sourcePort": 60700, + "transportProtocol": "TCP" + }, + "name": "Threat Detected and Blocked", + "severity": "9", + "version": "0" + }, + "destination": { + "address": "10.238.202.175", + "ip": "10.238.202.175", + "port": 8000 + }, + "ecs": { + "version": "8.17.0" + }, + "event": { + "action": "Threat Detected and Blocked", + "category": [ + "network", + "intrusion_detection" + ], + "code": "201", + "kind": "alert", + "original": "Jul 5 04:29:36 unifi.example.com 2025-07-05T04: 29:36.878Z unifi-r1 CEF:0|Ubiquiti|UniFi Network|9.3.33|201|Threat Detected and Blocked|9|proto=TCP src=10.238.202.175 spt=60700 dst=10.238.202.175 dpt=8000 UNIFIcategory=Security UNIFIsubCategory=Intrusion Prevention UNIFIhost=unifi-r1 UNIFIdeviceMac=01:23:45:89:c2:43 UNIFIdeviceName=unifi-r1 UNIFIdeviceModel=UniFi Dream Machine PRO SE UNIFIdeviceIp=10.238.202.175 UNIFIdeviceVersion=4.3.5 UNIFIrisk=high UNIFIipsSessionId=255132502100797 UNIFIipsSignature=ET SCAN Possible Nmap User-Agent Observed UNIFIipsSignatureId=2024364 msg=A network intrusion attempt from 10.238.202.175 to 10.238.202.175 has been detected and blocked.", + "reason": "ET SCAN Possible Nmap User-Agent Observed", + "severity": 9, + "type": [ + "denied", + "info" + ] + }, + "host": { + "hostname": "unifi-r1" + }, + "input": { + "type": "filestream" + }, + "log": { + "file": { + "device_id": "64773", + "fingerprint": "4971336f945694d052c56620d03d45fc31bdbba970ed13f50e2e10bfc2f46eeb", + "inode": "25962857", + "path": "/home/some.admin/tmp/ubnt_unifi-samples-filestream.log" + }, + "level": "9", + "offset": 298711, + "syslog": { + "hostname": "unifi.example.com" + } + }, + "message": "A network intrusion attempt from 10.238.202.175 to 10.238.202.175 has been detected and blocked.", + "network": { + "transport": "tcp" + }, + "observer": { + "hostname": "unifi.example.com", + "name": "unifi-r1", + "product": "UniFi Network", + "vendor": "Ubiquiti", + "version": "9.3.33" + }, + "related": { + "hosts": [ + "unifi.example.com" + ], + "ip": [ + "10.238.202.175" + ] + }, + "risk": { + "static_level": "High", + "static_score": 75, + "static_score_norm": 75 + }, + "source": { + "address": "10.238.202.175", + "ip": "10.238.202.175", + "port": 60700 + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields", + "forwarded", + "ubnt-unifi-cef" + ] + }, + { + "@timestamp": "2025-07-15T22:24:19.421Z", + "agent": { + "ephemeral_id": "01234567-89ab-cdef-4483-a9ff-b46996c08296", + "id": "01234567-89ab-cdef-4806-a514-ea1feb145b64", + "name": "elastic-agent.internal", + "type": "filebeat", + "version": "9.0.3" + }, + "cef": { + "device": { + "event_class_id": "400", + "product": "UniFi Network", + "vendor": "Ubiquiti", + "version": "9.3.43" + }, + "extensions": { + "UNIFIWiFiRssi": "-77", + "UNIFIauthMethod": "wpaeap", + "UNIFIcategory": "Monitoring", + "UNIFIclientAlias": "Pixel 7a", + "UNIFIclientHostname": "Pixel-7a", + "UNIFIclientIp": "10.238.202.175", + "UNIFIclientMac": "01:23:45:12:f6:c1", + "UNIFIconnectedToDeviceIp": "10.238.202.175", + "UNIFIconnectedToDeviceMac": "01:23:45:8b:a1:71", + "UNIFIconnectedToDeviceModel": "U7-Pro", + "UNIFIconnectedToDeviceName": "unifi-ap1", + "UNIFIconnectedToDeviceVersion": "8.0.49", + "UNIFIhost": "unifi-r1", + "UNIFInetworkName": "default", + "UNIFInetworkSubnet": "10.238.202.175/24", + "UNIFInetworkVlan": "1", + "UNIFIsubCategory": "WiFi", + "UNIFIwifiBand": "na", + "UNIFIwifiChannel": "44", + "UNIFIwifiChannelWidth": "80", + "UNIFIwifiName": "ohai!secure", + "message": "Pixel 7a connected to ohai!secure on unifi-ap1. Connection Info: Ch. 44 (5 GHz, 80 MHz), -77 dBm. IP: 10.238.202.175", + "sourceUserName": "some.admin" + }, + "name": "WiFi Client Connected", + "severity": "1", + "version": "0" + }, + "client": { + "address": "10.238.202.175", + "domain": "Pixel-7a", + "ip": "10.238.202.175", + "mac": "01-23-45-12-F6-C1" + }, + "ecs": { + "version": "8.17.0" + }, + "event": { + "action": "WiFi Client Connected", + "category": [ + "network" + ], + "code": "400", + "kind": "event", + "original": "Jul 15 22:24:19 2025-07-15T22:24:19.421Z unifi-r1 CEF:0|Ubiquiti|UniFi Network|9.3.43|400|WiFi Client Connected|1|UNIFIcategory=Monitoring UNIFIsubCategory=WiFi UNIFIhost=unifi-r1 UNIFIconnectedToDeviceName=unifi-ap1 UNIFIconnectedToDeviceIp=10.238.202.175 UNIFIconnectedToDeviceMac=01:23:45:8b:a1:71 UNIFIconnectedToDeviceModel=U7-Pro UNIFIconnectedToDeviceVersion=8.0.49 suser=some.admin UNIFIclientAlias=Pixel 7a UNIFIclientHostname=Pixel-7a UNIFIclientIp=10.238.202.175 UNIFIclientMac=01:23:45:12:f6:c1 UNIFIwifiChannel=44 UNIFIwifiChannelWidth=80 UNIFIwifiName=ohai!secure UNIFIwifiBand=na UNIFIauthMethod=wpaeap UNIFIWiFiRssi=-77 UNIFInetworkName=default UNIFInetworkSubnet=10.238.202.175/24 UNIFInetworkVlan=1 msg=Pixel 7a connected to ohai!secure on unifi-ap1. Connection Info: Ch. 44 (5 GHz, 80 MHz), -77 dBm. IP: 10.238.202.175", + "severity": 1, + "type": [ + "access", + "connection", + "start", + "info" + ] + }, + "host": { + "hostname": "unifi-r1" + }, + "input": { + "type": "filestream" + }, + "log": { + "file": { + "device_id": "16777231", + "fingerprint": "774aac143e417dff96abf0423c14ff696ebbdcaf59cf3195b29b13fe764d02d5", + "inode": "252511402", + "path": "/Users/some.admin/tmp/ubnt_unifi-samples-filestream.log" + }, + "level": "1", + "offset": 17599, + "syslog": { + "hostname": "unifi-r1" + } + }, + "message": "Pixel 7a connected to ohai!secure on unifi-ap1. Connection Info: Ch. 44 (5 GHz, 80 MHz), -77 dBm. IP: 10.238.202.175", + "observer": { + "hostname": "unifi-r1", + "product": "UniFi Network", + "vendor": "Ubiquiti", + "version": "9.3.43" + }, + "related": { + "hosts": [ + "unifi-r1", + "Pixel-7a", + "unifi-ap1" + ], + "user": [ + "some.admin" + ] + }, + "server": { + "address": "10.238.202.175", + "domain": "unifi-ap1", + "ip": "10.238.202.175", + "mac": "01-23-45-8B-A1-71" + }, + "source": { + "user": { + "name": "some.admin" + } + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields", + "forwarded", + "ubnt-unifi-cef" + ] + }, + { + "@timestamp": "2025-07-15T21:00:39.255Z", + "agent": { + "ephemeral_id": "01234567-89ab-cdef-4483-a9ff-b46996c08296", + "id": "01234567-89ab-cdef-4806-a514-ea1feb145b64", + "name": "elastic-agent.internal", + "type": "filebeat", + "version": "9.0.3" + }, + "cef": { + "device": { + "event_class_id": "512", + "product": "UniFi Network", + "vendor": "Ubiquiti", + "version": "9.3.43" + }, + "extensions": { + "UNIFIcategory": "System", + "UNIFIconnectedToDeviceIp": "10.238.202.175", + "UNIFIconnectedToDeviceMac": "01:23:45:4e:c3:a3", + "UNIFIconnectedToDeviceModel": "USW-Flex-2.5G-8", + "UNIFIconnectedToDeviceName": "unifi-l2s2", + "UNIFIconnectedToDevicePort": "8", + "UNIFIconnectedToDeviceVersion": "2.1.8", + "UNIFIdeviceIp": "10.238.202.175", + "UNIFIdeviceMac": "01:23:45:48:b9:34", + "UNIFIdeviceModel": "U6-Lite", + "UNIFIdeviceName": "unifi-ap4", + "UNIFIdeviceVersion": "6.7.17", + "UNIFIhost": "unifi-r1", + "UNIFIreference": "https://help.ui.com/hc/en-us/articles/7258465146519", + "UNIFIsubCategory": "Devices", + "message": "unifi-ap4 is offline." + }, + "name": "Device Offline", + "severity": "8", + "version": "0" + }, + "client": { + "address": "10.238.202.175", + "domain": "unifi-ap4", + "ip": "10.238.202.175", + "mac": "01-23-45-48-B9-34" + }, + "ecs": { + "version": "8.17.0" + }, + "event": { + "action": "Device Offline", + "category": [ + "network" + ], + "code": "512", + "kind": "event", + "original": "Jul 15 21:00:39 2025-07-15T21:00:39.255Z unifi-r1 CEF:0|Ubiquiti|UniFi Network|9.3.43|512|Device Offline|8|UNIFIcategory=System UNIFIsubCategory=Devices UNIFIhost=unifi-r1 UNIFIreference=https://help.ui.com/hc/en-us/articles/7258465146519 UNIFIdeviceMac=01:23:45:48:b9:34 UNIFIdeviceName=unifi-ap4 UNIFIdeviceModel=U6-Lite UNIFIdeviceIp=10.238.202.175 UNIFIdeviceVersion=6.7.17 UNIFIconnectedToDeviceName=unifi-l2s2 UNIFIconnectedToDevicePort=8 UNIFIconnectedToDeviceIp=10.238.202.175 UNIFIconnectedToDeviceMac=01:23:45:4e:c3:a3 UNIFIconnectedToDeviceModel=USW-Flex-2.5G-8 UNIFIconnectedToDeviceVersion=2.1.8 msg=unifi-ap4 is offline.", + "severity": 8, + "type": [ + "info" + ], + "url": "https://help.ui.com/hc/en-us/articles/7258465146519" + }, + "host": { + "hostname": "unifi-r1" + }, + "input": { + "type": "filestream" + }, + "log": { + "file": { + "device_id": "16777231", + "fingerprint": "774aac143e417dff96abf0423c14ff696ebbdcaf59cf3195b29b13fe764d02d5", + "inode": "252511402", + "path": "/Users/some.admin/tmp/ubnt_unifi-samples-filestream.log" + }, + "level": "8", + "offset": 18437, + "syslog": { + "hostname": "unifi-r1" + } + }, + "message": "unifi-ap4 is offline.", + "observer": { + "hostname": "unifi-r1", + "product": "UniFi Network", + "vendor": "Ubiquiti", + "version": "9.3.43" + }, + "related": { + "hosts": [ + "unifi-r1", + "unifi-ap4", + "unifi-l2s2" + ] + }, + "server": { + "address": "10.238.202.175", + "domain": "unifi-l2s2", + "ip": "10.238.202.175", + "mac": "01-23-45-4E-C3-A3" + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields", + "forwarded", + "ubnt-unifi-cef" + ] + } + ] +} diff --git a/packages/ubnt_unifi/data_stream/logs/_dev/test/pipeline/test-logs-filestream-other.json b/packages/ubnt_unifi/data_stream/logs/_dev/test/pipeline/test-logs-filestream-other.json new file mode 100644 index 00000000000..f0403a50f38 --- /dev/null +++ b/packages/ubnt_unifi/data_stream/logs/_dev/test/pipeline/test-logs-filestream-other.json @@ -0,0 +1,519 @@ +{ + "events": [ + { + "@timestamp": "2025-07-05T04:29:55.000Z", + "agent": { + "ephemeral_id": "01234567-89ab-cdef-4ec2-b38f-e99536291b74", + "id": "01234567-89ab-cdef-4f5d-a1b8-6ed60bda2132", + "name": "elastic-agent-01.internal.example.com", + "type": "filebeat", + "version": "9.0.3" + }, + "ecs": { + "version": "8.0.0" + }, + "log": { + "offset": 314553, + "file": { + "inode": "25962857", + "fingerprint": "4971336f945694d052c56620d03d45fc31bdbba970ed13f50e2e10bfc2f46eeb", + "path": "/home/some.admin/tmp/ubnt_unifi-samples-filestream.log", + "device_id": "64773" + }, + "syslog": { + "hostname": "unifi-ap3", + "appname": "0123456789aa,U6-Lite-6.7.17+15512" + } + }, + "event": { + "original": "Jul 5 14:29:55 unifi-ap3 0123456789aa,U6-Lite-6.7.17+15512: stahtd: stahtd[2786]: [STA-TRACKER].stahtd_dump_event(): {\"message_type\":\"STA_ASSOC_TRACKER\",\"mac\":\"01:23:45:01:f2:b8\",\"vap\":\"ra0\",\"event_type\":\"sta_leave\",\"assoc_status\":\"0\",\"event_id\":\"0\",\"sta_dc_reason\":\"sta left\",\"disassoc_reason\":\"8\"}" + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields", + "forwarded", + "ubnt-unifi-other" + ], + "input": { + "type": "filestream" + }, + "message": "stahtd: stahtd[2786]: [STA-TRACKER].stahtd_dump_event(): {\"message_type\":\"STA_ASSOC_TRACKER\",\"mac\":\"01:23:45:01:f2:b8\",\"vap\":\"ra0\",\"event_type\":\"sta_leave\",\"assoc_status\":\"0\",\"event_id\":\"0\",\"sta_dc_reason\":\"sta left\",\"disassoc_reason\":\"8\"}" + }, + { + "@timestamp": "2025-07-05T04:30:40.000Z", + "ecs": { + "version": "8.0.0" + }, + "agent": { + "name": "elastic-agent-01.internal.example.com", + "type": "filebeat", + "version": "9.0.3", + "ephemeral_id": "01234567-89ab-cdef-4ec2-b38f-e99536291b74", + "id": "01234567-89ab-cdef-4f5d-a1b8-6ed60bda2132" + }, + "log": { + "file": { + "path": "/home/some.admin/tmp/ubnt_unifi-samples-filestream.log", + "device_id": "64773", + "inode": "25962857", + "fingerprint": "4971336f945694d052c56620d03d45fc31bdbba970ed13f50e2e10bfc2f46eeb" + }, + "offset": 329808, + "syslog": { + "hostname": "unifi-r1" + } + }, + "event": { + "original": "Jul 5 14:30:40 unifi-r1 unifi-r1 utmdaemon[1574169]: [INFO] utm_start_honeypot:1160: UTM Honeypot Service: Starting Thread" + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields", + "forwarded", + "ubnt-unifi-other" + ], + "input": { + "type": "filestream" + }, + "message": "utmdaemon[1574169]: [INFO] utm_start_honeypot:1160: UTM Honeypot Service: Starting Thread" + }, + { + "@timestamp": "2025-07-05T04:30:40.000Z", + "message": "ubios-udapi-server[1574169]: [INFO] utm_start_honeypot:1160: UTM Honeypot Service: Starting Thread", + "ecs": { + "version": "8.0.0" + }, + "agent": { + "type": "filebeat", + "version": "9.0.3", + "ephemeral_id": "01234567-89ab-cdef-4ec2-b38f-e99536291b74", + "id": "01234567-89ab-cdef-4f5d-a1b8-6ed60bda2132", + "name": "elastic-agent-01.internal.example.com" + }, + "log": { + "offset": 329934, + "file": { + "path": "/home/some.admin/tmp/ubnt_unifi-samples-filestream.log", + "device_id": "64773", + "inode": "25962857", + "fingerprint": "4971336f945694d052c56620d03d45fc31bdbba970ed13f50e2e10bfc2f46eeb" + }, + "syslog": { + "hostname": "unifi-r1" + } + }, + "event": { + "original": "Jul 5 14:30:40 unifi-r1 unifi-r1 ubios-udapi-server[1574169]: [INFO] utm_start_honeypot:1160: UTM Honeypot Service: Starting Thread" + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields", + "forwarded", + "ubnt-unifi-other" + ], + "input": { + "type": "filestream" + } + }, + { + "@timestamp": "2025-07-05T04:32:04.000Z", + "ecs": { + "version": "8.0.0" + }, + "agent": { + "version": "9.0.3", + "ephemeral_id": "01234567-89ab-cdef-4ec2-b38f-e99536291b74", + "id": "01234567-89ab-cdef-4f5d-a1b8-6ed60bda2132", + "name": "elastic-agent-01.internal.example.com", + "type": "filebeat" + }, + "log": { + "offset": 338961, + "file": { + "path": "/home/some.admin/tmp/ubnt_unifi-samples-filestream.log", + "device_id": "64773", + "inode": "25962857", + "fingerprint": "4971336f945694d052c56620d03d45fc31bdbba970ed13f50e2e10bfc2f46eeb" + }, + "syslog": { + "hostname": "unifi-ap3", + "appname": "0123456789aa,U6-Lite-6.7.17+15512" + } + }, + "event": { + "original": "Jul 5 14:32:04 unifi-ap3 0123456789aa,U6-Lite-6.7.17+15512: libubnt[15024]: mcad[15024]: wireless_agg_stats.log_sta_anomalies(): bssid=01:23:45:28:ab:59 radio=ra0 vap=ra2 sta=01:23:45:1f:be:e0 satisfaction_now=72 anomalies=tcp_latency" + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields", + "forwarded", + "ubnt-unifi-other" + ], + "input": { + "type": "filestream" + }, + "message": "libubnt[15024]: mcad[15024]: wireless_agg_stats.log_sta_anomalies(): bssid=01:23:45:28:ab:59 radio=ra0 vap=ra2 sta=01:23:45:1f:be:e0 satisfaction_now=72 anomalies=tcp_latency" + }, + { + "@timestamp": "2025-07-16T11:39:08.000Z", + "ecs": { + "version": "8.0.0" + }, + "agent": { + "version": "9.0.3", + "ephemeral_id": "a050b3ef-166f-4483-a9ff-b46996c08296", + "id": "cda862cd-99d5-4806-a514-ea1feb145b64", + "name": "elastic-agent.internal", + "type": "filebeat" + }, + "log": { + "offset": 19073, + "file": { + "path": "/Users/some.user/tmp/ubnt_unifi-samples-filestream.log", + "device_id": "16777231", + "inode": "252511402", + "fingerprint": "774aac143e417dff96abf0423c14ff696ebbdcaf59cf3195b29b13fe764d02d5" + }, + "syslog": { + "priority": 30, + "facility": { + "code": 3, + "name": "system" + }, + "severity": { + "code": 6, + "name": "Informational" + }, + "appname": "0123456789aa,U6-Lite-6.7.17+15512", + "hostname": "unifi-ap3" + } + }, + "event": { + "original": "<30>Jul 16 21:39:08 unifi-ap3 78455848ab58,U6-Lite-6.7.17+15512: mcad: mcad[26341]: wireless_agg_stats.log_sta_anomalies(): bssid=01:23:45:28:ab:59 radio=ra0 vap=ra2 sta=01:23:45:1f:be:e0 satisfaction_now=56 anomalies=tcp_latency" + }, + "tags": [ + "forwarded", + "ubnt-unifi-other" + ], + "input": { + "type": "filestream" + }, + "message": "mcad: mcad[26341]: wireless_agg_stats.log_sta_anomalies(): bssid=7a:45:58:28:ab:59 radio=ra0 vap=ra2 sta=a0:a3:b3:1f:be:e0 satisfaction_now=56 anomalies=tcp_latency" + }, + { + "@timestamp": "2025-07-16T11:37:20.000Z", + "agent": { + "type": "filebeat", + "version": "9.0.3", + "ephemeral_id": "a050b3ef-166f-4483-a9ff-b46996c08296", + "id": "cda862cd-99d5-4806-a514-ea1feb145b64", + "name": "elastic-agent.internal" + }, + "ecs": { + "version": "8.0.0" + }, + "event": { + "original": "<14>Jul 16 21:37:20 unifi-r1 unifi-r1 linkcheck[1583]: speedtest.ui_speedtest_log_results(): {\n \"url\": \"http://119.18.32.1:8069\",\n \"latitude\": -28.068000793457031,\n \"longitude\": 153.452392578125,\n \"provider\": \"Aussie Broadband\",\n \"providerUrl\": \"https://www.aussiebroadband.com.au\",\n \"city\": \"Brisbane\",\n \"country\": \"Australia\",\n \"countryCode\": \"AU\",\n \"speedMbps\": 1000\n}\n" + }, + "log": { + "offset": 19717, + "file": { + "path": "/Users/some.user/tmp/ubnt_unifi-samples-filestream.log", + "device_id": "16777231", + "inode": "252672383", + "fingerprint": "774aac143e417dff96abf0423c14ff696ebbdcaf59cf3195b29b13fe764d02d5" + }, + "syslog": { + "facility": { + "name": "user-level", + "code": 1 + }, + "severity": { + "code": 6, + "name": "Informational" + }, + "hostname": "unifi-r1", + "priority": 14 + } + }, + "tags": [ + "forwarded", + "ubnt-unifi-other" + ], + "input": { + "type": "filestream" + }, + "message": "linkcheck[1583]: speedtest.ui_speedtest_log_results(): {\n \"url\": \"http://119.18.32.1:8069\",\n \"latitude\": -28.068000793457031,\n \"longitude\": 153.452392578125,\n \"provider\": \"Aussie Broadband\",\n \"providerUrl\": \"https://www.aussiebroadband.com.au\",\n \"city\": \"Brisbane\",\n \"country\": \"Australia\",\n \"countryCode\": \"AU\",\n \"speedMbps\": 1000\n}\n" + }, + { + "@timestamp": "2025-07-16T11:37:20.000Z", + "input": { + "type": "filestream" + }, + "message": "linkcheck[1583]: speedtest.ui_speedtest_log_results(): resultUrl: https://wifiman.com/?result=UUID_WOULD_BE_HERE", + "agent": { + "ephemeral_id": "a050b3ef-166f-4483-a9ff-b46996c08296", + "id": "cda862cd-99d5-4806-a514-ea1feb145b64", + "name": "elastic-agent.internal", + "type": "filebeat", + "version": "9.0.3" + }, + "ecs": { + "version": "8.0.0" + }, + "log": { + "offset": 20142, + "file": { + "path": "/Users/some.user/tmp/ubnt_unifi-samples-filestream.log", + "device_id": "16777231", + "inode": "252681175", + "fingerprint": "774aac143e417dff96abf0423c14ff696ebbdcaf59cf3195b29b13fe764d02d5" + }, + "syslog": { + "priority": 14, + "facility": { + "code": 1, + "name": "user-level" + }, + "severity": { + "code": 6, + "name": "Informational" + }, + "hostname": "unifi-r1" + } + }, + "event": { + "original": "<14>Jul 16 21:37:20 unifi-r1 unifi-r1 linkcheck[1583]: speedtest.ui_speedtest_log_results(): resultUrl: https://wifiman.com/?result=UUID_WOULD_BE_HERE" + }, + "tags": [ + "forwarded", + "ubnt-unifi-other" + ] + }, + { + "@timestamp": "2025-07-16T12:12:41.000Z", + "message": "earlyoom[1581]: mem avail: 689 of 3946 MiB (17.46%), swap free: 6136 of 7167 MiB (85.61%)", + "ecs": { + "version": "8.0.0" + }, + "agent": { + "type": "filebeat", + "version": "9.0.3", + "ephemeral_id": "a050b3ef-166f-4483-a9ff-b46996c08296", + "id": "cda862cd-99d5-4806-a514-ea1feb145b64", + "name": "elastic-agent.internal" + }, + "log": { + "file": { + "inode": "252682608", + "fingerprint": "774aac143e417dff96abf0423c14ff696ebbdcaf59cf3195b29b13fe764d02d5", + "path": "/Users/some.user/tmp/ubnt_unifi-samples-filestream.log", + "device_id": "16777231" + }, + "offset": 20313, + "syslog": { + "hostname": "unifi-r1", + "priority": 30, + "facility": { + "code": 3, + "name": "system" + }, + "severity": { + "code": 6, + "name": "Informational" + } + } + }, + "event": { + "original": "<30>Jul 16 22:12:41 unifi-r1 unifi-r1 earlyoom[1581]: mem avail: 689 of 3946 MiB (17.46%), swap free: 6136 of 7167 MiB (85.61%)" + }, + "tags": [ + "forwarded", + "ubnt-unifi-other" + ], + "input": { + "type": "filestream" + } + }, + { + "@timestamp": "2025-07-16T12:09:43.000Z", + "input": { + "type": "filestream" + }, + "message": "kernel: al_mod_eth_mac_stats_get: MAC statistics not supported in this mode RGMII (0)", + "ecs": { + "version": "8.0.0" + }, + "agent": { + "name": "elastic-agent.internal", + "type": "filebeat", + "version": "9.0.3", + "ephemeral_id": "a050b3ef-166f-4483-a9ff-b46996c08296", + "id": "cda862cd-99d5-4806-a514-ea1feb145b64" + }, + "log": { + "offset": 20446, + "file": { + "path": "/Users/some.user/tmp/ubnt_unifi-samples-filestream.log", + "device_id": "16777231", + "inode": "252683406", + "fingerprint": "774aac143e417dff96abf0423c14ff696ebbdcaf59cf3195b29b13fe764d02d5" + }, + "syslog": { + "hostname": "unifi-r1", + "priority": 3, + "facility": { + "code": 0, + "name": "kernel" + }, + "severity": { + "code": 3, + "name": "Error" + } + } + }, + "event": { + "original": "<3>Jul 16 22:09:43 unifi-r1 unifi-r1 kernel: al_mod_eth_mac_stats_get: MAC statistics not supported in this mode RGMII (0)" + }, + "tags": [ + "forwarded", + "ubnt-unifi-other" + ] + }, + { + "@timestamp": "2025-07-16T12:34:29.000Z", + "event": { + "original": "<30>Jul 16 22:34:29 unifi-r1 unifi-r1 dnsmasq-dhcp[3408207]: DHCPACK(br3) 192.168.0.18 01:23:45:54:f8:be L535" + }, + "tags": [ + "forwarded", + "ubnt-unifi-other" + ], + "input": { + "type": "filestream" + }, + "message": "dnsmasq-dhcp[3408207]: DHCPACK(br3) 192.168.0.18 01:23:45:54:f8:be L535", + "agent": { + "type": "filebeat", + "version": "9.0.3", + "ephemeral_id": "a050b3ef-166f-4483-a9ff-b46996c08296", + "id": "cda862cd-99d5-4806-a514-ea1feb145b64", + "name": "elastic-agent.internal" + }, + "ecs": { + "version": "8.0.0" + }, + "log": { + "file": { + "fingerprint": "774aac143e417dff96abf0423c14ff696ebbdcaf59cf3195b29b13fe764d02d5", + "path": "/Users/some.user/tmp/ubnt_unifi-samples-filestream.log", + "device_id": "16777231", + "inode": "252689772" + }, + "offset": 20571, + "syslog": { + "priority": 30, + "facility": { + "name": "system", + "code": 3 + }, + "severity": { + "code": 6, + "name": "Informational" + }, + "hostname": "unifi-r1" + } + } + }, + { + "@timestamp": "2025-07-16T12:34:29.000Z", + "event": { + "original": "<30>Jul 16 22:34:29 unifi-r1 unifi-r1 dnsmasq-dhcp[3408207]: DHCPREQUEST(br3) 192.168.0.18 01:23:45:54:f8:be" + }, + "log": { + "offset": 20684, + "file": { + "inode": "252689772", + "fingerprint": "774aac143e417dff96abf0423c14ff696ebbdcaf59cf3195b29b13fe764d02d5", + "path": "/Users/some.user/tmp/ubnt_unifi-samples-filestream.log", + "device_id": "16777231" + }, + "syslog": { + "priority": 30, + "facility": { + "code": 3, + "name": "system" + }, + "severity": { + "code": 6, + "name": "Informational" + }, + "hostname": "unifi-r1" + } + }, + "tags": [ + "forwarded", + "ubnt-unifi-other" + ], + "input": { + "type": "filestream" + }, + "message": "dnsmasq-dhcp[3408207]: DHCPREQUEST(br3) 192.168.0.18 01:23:45:54:f8:be", + "ecs": { + "version": "8.0.0" + }, + "agent": { + "version": "9.0.3", + "ephemeral_id": "a050b3ef-166f-4483-a9ff-b46996c08296", + "id": "cda862cd-99d5-4806-a514-ea1feb145b64", + "name": "elastic-agent.internal", + "type": "filebeat" + } + }, + { + "@timestamp": "2025-07-16T11:37:20.000Z", + "message": "linkcheck[1583]: linkcheck.run_speedtest(): Completed: Downlink 269.000 Mbps, Uplink 23.000 Mbps", + "ecs": { + "version": "8.0.0" + }, + "agent": { + "ephemeral_id": "a050b3ef-166f-4483-a9ff-b46996c08296", + "id": "cda862cd-99d5-4806-a514-ea1feb145b64", + "name": "office-monitor-left.home", + "type": "filebeat", + "version": "9.0.3" + }, + "log": { + "offset": 20796, + "file": { + "inode": "252705548", + "fingerprint": "774aac143e417dff96abf0423c14ff696ebbdcaf59cf3195b29b13fe764d02d5", + "path": "/Users/colin.stubbs/tmp/ubnt_unifi-samples-filestream.log", + "device_id": "16777231" + }, + "syslog": { + "priority": 14, + "facility": { + "name": "user-level", + "code": 1 + }, + "severity": { + "code": 6, + "name": "Informational" + }, + "hostname": "bimini-r1" + } + }, + "event": { + "original": "<14>Jul 16 21:37:20 bimini-r1 bimini-r1 linkcheck[1583]: linkcheck.run_speedtest(): Completed: Downlink 269.000 Mbps, Uplink 23.000 Mbps" + }, + "tags": [ + "forwarded", + "ubnt-unifi-other" + ], + "input": { + "type": "filestream" + } + } + ] +} diff --git a/packages/ubnt_unifi/data_stream/logs/_dev/test/pipeline/test-logs-filestream-other.json-expected.json b/packages/ubnt_unifi/data_stream/logs/_dev/test/pipeline/test-logs-filestream-other.json-expected.json new file mode 100644 index 00000000000..1689ddf87b3 --- /dev/null +++ b/packages/ubnt_unifi/data_stream/logs/_dev/test/pipeline/test-logs-filestream-other.json-expected.json @@ -0,0 +1,839 @@ +{ + "expected": [ + { + "@timestamp": "2025-07-05T04:29:55.000Z", + "agent": { + "ephemeral_id": "01234567-89ab-cdef-4ec2-b38f-e99536291b74", + "id": "01234567-89ab-cdef-4f5d-a1b8-6ed60bda2132", + "name": "elastic-agent-01.internal.example.com", + "type": "filebeat", + "version": "9.0.3" + }, + "ecs": { + "version": "8.17.0" + }, + "event": { + "kind": "event", + "original": "Jul 5 14:29:55 unifi-ap3 0123456789aa,U6-Lite-6.7.17+15512: stahtd: stahtd[2786]: [STA-TRACKER].stahtd_dump_event(): {\"message_type\":\"STA_ASSOC_TRACKER\",\"mac\":\"01:23:45:01:f2:b8\",\"vap\":\"ra0\",\"event_type\":\"sta_leave\",\"assoc_status\":\"0\",\"event_id\":\"0\",\"sta_dc_reason\":\"sta left\",\"disassoc_reason\":\"8\"}", + "type": [ + "info" + ] + }, + "input": { + "type": "filestream" + }, + "log": { + "file": { + "device_id": "64773", + "fingerprint": "4971336f945694d052c56620d03d45fc31bdbba970ed13f50e2e10bfc2f46eeb", + "inode": "25962857", + "path": "/home/some.admin/tmp/ubnt_unifi-samples-filestream.log" + }, + "offset": 314553, + "syslog": { + "appname": "0123456789aa,U6-Lite-6.7.17+15512", + "hostname": "unifi-ap3" + } + }, + "message": "{\"message_type\":\"STA_ASSOC_TRACKER\",\"mac\":\"01:23:45:01:f2:b8\",\"vap\":\"ra0\",\"event_type\":\"sta_leave\",\"assoc_status\":\"0\",\"event_id\":\"0\",\"sta_dc_reason\":\"sta left\",\"disassoc_reason\":\"8\"}", + "observer": { + "hostname": "unifi-ap3", + "product": "U6-Lite", + "serial_number": "0123456789aa", + "vendor": "Ubiquiti", + "version": "6.7.17+15512" + }, + "process": { + "name": "stahtd", + "pid": 2786 + }, + "related": { + "hosts": [ + "unifi-ap3" + ] + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields", + "forwarded", + "ubnt-unifi-other" + ], + "ubnt": { + "unifi": { + "stahtd": { + "dump": { + "assoc_status": "0", + "disassoc_reason": "8", + "event_id": "0", + "event_type": "sta_leave", + "mac": "01:23:45:01:f2:b8", + "message_type": "STA_ASSOC_TRACKER", + "sta_dc_reason": "sta left", + "vap": "ra0" + } + } + } + } + }, + { + "@timestamp": "2025-07-05T04:30:40.000Z", + "agent": { + "ephemeral_id": "01234567-89ab-cdef-4ec2-b38f-e99536291b74", + "id": "01234567-89ab-cdef-4f5d-a1b8-6ed60bda2132", + "name": "elastic-agent-01.internal.example.com", + "type": "filebeat", + "version": "9.0.3" + }, + "ecs": { + "version": "8.17.0" + }, + "event": { + "kind": "event", + "original": "Jul 5 14:30:40 unifi-r1 unifi-r1 utmdaemon[1574169]: [INFO] utm_start_honeypot:1160: UTM Honeypot Service: Starting Thread", + "type": [ + "info" + ] + }, + "input": { + "type": "filestream" + }, + "log": { + "file": { + "device_id": "64773", + "fingerprint": "4971336f945694d052c56620d03d45fc31bdbba970ed13f50e2e10bfc2f46eeb", + "inode": "25962857", + "path": "/home/some.admin/tmp/ubnt_unifi-samples-filestream.log" + }, + "offset": 329808, + "syslog": { + "hostname": "unifi-r1" + } + }, + "message": "[INFO] utm_start_honeypot:1160: UTM Honeypot Service: Starting Thread", + "observer": { + "hostname": "unifi-r1", + "product": "UniFi", + "vendor": "Ubiquiti" + }, + "process": { + "name": "utmdaemon", + "pid": 1574169 + }, + "related": { + "hosts": [ + "unifi-r1" + ] + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields", + "forwarded", + "ubnt-unifi-other" + ] + }, + { + "@timestamp": "2025-07-05T04:30:40.000Z", + "agent": { + "ephemeral_id": "01234567-89ab-cdef-4ec2-b38f-e99536291b74", + "id": "01234567-89ab-cdef-4f5d-a1b8-6ed60bda2132", + "name": "elastic-agent-01.internal.example.com", + "type": "filebeat", + "version": "9.0.3" + }, + "ecs": { + "version": "8.17.0" + }, + "event": { + "kind": "event", + "original": "Jul 5 14:30:40 unifi-r1 unifi-r1 ubios-udapi-server[1574169]: [INFO] utm_start_honeypot:1160: UTM Honeypot Service: Starting Thread", + "type": [ + "info" + ] + }, + "input": { + "type": "filestream" + }, + "log": { + "file": { + "device_id": "64773", + "fingerprint": "4971336f945694d052c56620d03d45fc31bdbba970ed13f50e2e10bfc2f46eeb", + "inode": "25962857", + "path": "/home/some.admin/tmp/ubnt_unifi-samples-filestream.log" + }, + "offset": 329934, + "syslog": { + "hostname": "unifi-r1" + } + }, + "message": "[INFO] utm_start_honeypot:1160: UTM Honeypot Service: Starting Thread", + "observer": { + "hostname": "unifi-r1", + "product": "UniFi", + "vendor": "Ubiquiti" + }, + "process": { + "name": "ubios-udapi-server", + "pid": 1574169 + }, + "related": { + "hosts": [ + "unifi-r1" + ] + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields", + "forwarded", + "ubnt-unifi-other" + ] + }, + { + "@timestamp": "2025-07-05T04:32:04.000Z", + "agent": { + "ephemeral_id": "01234567-89ab-cdef-4ec2-b38f-e99536291b74", + "id": "01234567-89ab-cdef-4f5d-a1b8-6ed60bda2132", + "name": "elastic-agent-01.internal.example.com", + "type": "filebeat", + "version": "9.0.3" + }, + "ecs": { + "version": "8.17.0" + }, + "event": { + "kind": "event", + "original": "Jul 5 14:32:04 unifi-ap3 0123456789aa,U6-Lite-6.7.17+15512: libubnt[15024]: mcad[15024]: wireless_agg_stats.log_sta_anomalies(): bssid=01:23:45:28:ab:59 radio=ra0 vap=ra2 sta=01:23:45:1f:be:e0 satisfaction_now=72 anomalies=tcp_latency", + "type": [ + "info" + ] + }, + "input": { + "type": "filestream" + }, + "log": { + "file": { + "device_id": "64773", + "fingerprint": "4971336f945694d052c56620d03d45fc31bdbba970ed13f50e2e10bfc2f46eeb", + "inode": "25962857", + "path": "/home/some.admin/tmp/ubnt_unifi-samples-filestream.log" + }, + "offset": 338961, + "syslog": { + "appname": "0123456789aa,U6-Lite-6.7.17+15512", + "hostname": "unifi-ap3" + } + }, + "message": "wireless_agg_stats.log_sta_anomalies(): bssid=01:23:45:28:ab:59 radio=ra0 vap=ra2 sta=01:23:45:1f:be:e0 satisfaction_now=72 anomalies=tcp_latency", + "observer": { + "hostname": "unifi-ap3", + "product": "U6-Lite", + "serial_number": "0123456789aa", + "vendor": "Ubiquiti", + "version": "6.7.17+15512" + }, + "process": { + "name": "mcad", + "parent": { + "name": "libubnt", + "pid": 15024 + }, + "pid": 15024 + }, + "related": { + "hosts": [ + "unifi-ap3" + ] + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields", + "forwarded", + "ubnt-unifi-other" + ] + }, + { + "@timestamp": "2025-07-16T11:39:08.000Z", + "agent": { + "ephemeral_id": "a050b3ef-166f-4483-a9ff-b46996c08296", + "id": "cda862cd-99d5-4806-a514-ea1feb145b64", + "name": "elastic-agent.internal", + "type": "filebeat", + "version": "9.0.3" + }, + "ecs": { + "version": "8.17.0" + }, + "event": { + "kind": "event", + "type": [ + "info" + ] + }, + "input": { + "type": "filestream" + }, + "log": { + "file": { + "device_id": "16777231", + "fingerprint": "774aac143e417dff96abf0423c14ff696ebbdcaf59cf3195b29b13fe764d02d5", + "inode": "252511402", + "path": "/Users/some.user/tmp/ubnt_unifi-samples-filestream.log" + }, + "offset": 19073, + "syslog": { + "appname": "0123456789aa,U6-Lite-6.7.17+15512", + "facility": { + "code": 3, + "name": "system" + }, + "hostname": "unifi-ap3", + "priority": 30, + "severity": { + "code": 6, + "name": "Informational" + } + } + }, + "observer": { + "hostname": "unifi-ap3", + "product": "U6-Lite", + "serial_number": "0123456789aa", + "vendor": "Ubiquiti", + "version": "6.7.17+15512" + }, + "process": { + "name": "mcad", + "pid": 26341 + }, + "related": { + "hosts": [ + "unifi-ap3" + ] + }, + "tags": [ + "forwarded", + "ubnt-unifi-other" + ], + "ubnt": { + "unifi": { + "mcad": { + "wireless_agg_stats": { + "log_sta_anomalies": { + "anomalies": "tcp_latency", + "bssid": "7a:45:58:28:ab:59", + "radio": "ra0", + "satisfaction_now": "56", + "sta": "a0:a3:b3:1f:be:e0", + "vap": "ra2" + } + } + } + } + } + }, + { + "@timestamp": "2025-07-16T11:37:20.000Z", + "agent": { + "ephemeral_id": "a050b3ef-166f-4483-a9ff-b46996c08296", + "id": "cda862cd-99d5-4806-a514-ea1feb145b64", + "name": "elastic-agent.internal", + "type": "filebeat", + "version": "9.0.3" + }, + "ecs": { + "version": "8.17.0" + }, + "event": { + "kind": "event", + "type": [ + "info" + ] + }, + "input": { + "type": "filestream" + }, + "log": { + "file": { + "device_id": "16777231", + "fingerprint": "774aac143e417dff96abf0423c14ff696ebbdcaf59cf3195b29b13fe764d02d5", + "inode": "252672383", + "path": "/Users/some.user/tmp/ubnt_unifi-samples-filestream.log" + }, + "offset": 19717, + "syslog": { + "facility": { + "code": 1, + "name": "user-level" + }, + "hostname": "unifi-r1", + "priority": 14, + "severity": { + "code": 6, + "name": "Informational" + } + } + }, + "observer": { + "hostname": "unifi-r1", + "product": "UniFi", + "vendor": "Ubiquiti" + }, + "process": { + "name": "linkcheck", + "pid": 1583 + }, + "related": { + "hosts": [ + "unifi-r1" + ] + }, + "tags": [ + "forwarded", + "ubnt-unifi-other" + ], + "ubnt": { + "unifi": { + "linkcheck": { + "city": "Brisbane", + "country": "Australia", + "countryCode": "AU", + "latitude": -28.06800079345703, + "longitude": 153.452392578125, + "provider": "Aussie Broadband", + "providerUrl": "https://www.aussiebroadband.com.au", + "speedMbps": 1000, + "url": "http://119.18.32.1:8069" + } + } + } + }, + { + "@timestamp": "2025-07-16T11:37:20.000Z", + "agent": { + "ephemeral_id": "a050b3ef-166f-4483-a9ff-b46996c08296", + "id": "cda862cd-99d5-4806-a514-ea1feb145b64", + "name": "elastic-agent.internal", + "type": "filebeat", + "version": "9.0.3" + }, + "ecs": { + "version": "8.17.0" + }, + "event": { + "kind": "event", + "type": [ + "info" + ] + }, + "input": { + "type": "filestream" + }, + "log": { + "file": { + "device_id": "16777231", + "fingerprint": "774aac143e417dff96abf0423c14ff696ebbdcaf59cf3195b29b13fe764d02d5", + "inode": "252681175", + "path": "/Users/some.user/tmp/ubnt_unifi-samples-filestream.log" + }, + "offset": 20142, + "syslog": { + "facility": { + "code": 1, + "name": "user-level" + }, + "hostname": "unifi-r1", + "priority": 14, + "severity": { + "code": 6, + "name": "Informational" + } + } + }, + "observer": { + "hostname": "unifi-r1", + "product": "UniFi", + "vendor": "Ubiquiti" + }, + "process": { + "name": "linkcheck", + "pid": 1583 + }, + "related": { + "hosts": [ + "unifi-r1" + ] + }, + "tags": [ + "forwarded", + "ubnt-unifi-other" + ], + "ubnt": { + "unifi": { + "linkcheck": { + "function": "speedtest.ui_speedtest_log_results", + "resultUrl": "https://wifiman.com/?result=UUID_WOULD_BE_HERE" + } + } + } + }, + { + "@timestamp": "2025-07-16T12:12:41.000Z", + "agent": { + "ephemeral_id": "a050b3ef-166f-4483-a9ff-b46996c08296", + "id": "cda862cd-99d5-4806-a514-ea1feb145b64", + "name": "elastic-agent.internal", + "type": "filebeat", + "version": "9.0.3" + }, + "ecs": { + "version": "8.17.0" + }, + "event": { + "kind": "event", + "type": [ + "info" + ] + }, + "input": { + "type": "filestream" + }, + "log": { + "file": { + "device_id": "16777231", + "fingerprint": "774aac143e417dff96abf0423c14ff696ebbdcaf59cf3195b29b13fe764d02d5", + "inode": "252682608", + "path": "/Users/some.user/tmp/ubnt_unifi-samples-filestream.log" + }, + "offset": 20313, + "syslog": { + "facility": { + "code": 3, + "name": "system" + }, + "hostname": "unifi-r1", + "priority": 30, + "severity": { + "code": 6, + "name": "Informational" + } + } + }, + "observer": { + "hostname": "unifi-r1", + "product": "UniFi", + "vendor": "Ubiquiti" + }, + "process": { + "name": "earlyoom", + "pid": 1581 + }, + "related": { + "hosts": [ + "unifi-r1" + ] + }, + "tags": [ + "forwarded", + "ubnt-unifi-other" + ], + "ubnt": { + "unifi": { + "earlyoom": { + "memory": { + "total": 3946, + "used": 689, + "used_pct": 17.46 + }, + "swap": { + "total": 7167, + "used": 6136, + "used_pct": 85.61 + } + } + } + } + }, + { + "@timestamp": "2025-07-16T12:09:43.000Z", + "agent": { + "ephemeral_id": "a050b3ef-166f-4483-a9ff-b46996c08296", + "id": "cda862cd-99d5-4806-a514-ea1feb145b64", + "name": "elastic-agent.internal", + "type": "filebeat", + "version": "9.0.3" + }, + "ecs": { + "version": "8.17.0" + }, + "event": { + "kind": "event", + "type": [ + "info" + ] + }, + "input": { + "type": "filestream" + }, + "log": { + "file": { + "device_id": "16777231", + "fingerprint": "774aac143e417dff96abf0423c14ff696ebbdcaf59cf3195b29b13fe764d02d5", + "inode": "252683406", + "path": "/Users/some.user/tmp/ubnt_unifi-samples-filestream.log" + }, + "offset": 20446, + "syslog": { + "facility": { + "code": 0, + "name": "kernel" + }, + "hostname": "unifi-r1", + "priority": 3, + "severity": { + "code": 3, + "name": "Error" + } + } + }, + "message": "MAC statistics not supported in this mode RGMII (0)", + "observer": { + "hostname": "unifi-r1", + "product": "UniFi", + "vendor": "Ubiquiti" + }, + "process": { + "name": "al_mod_eth_mac_stats_get", + "parent": { + "name": "kernel" + } + }, + "related": { + "hosts": [ + "unifi-r1" + ] + }, + "tags": [ + "forwarded", + "ubnt-unifi-other" + ] + }, + { + "@timestamp": "2025-07-16T12:34:29.000Z", + "agent": { + "ephemeral_id": "a050b3ef-166f-4483-a9ff-b46996c08296", + "id": "cda862cd-99d5-4806-a514-ea1feb145b64", + "name": "elastic-agent.internal", + "type": "filebeat", + "version": "9.0.3" + }, + "ecs": { + "version": "8.17.0" + }, + "event": { + "kind": "event", + "type": [ + "info" + ] + }, + "input": { + "type": "filestream" + }, + "log": { + "file": { + "device_id": "16777231", + "fingerprint": "774aac143e417dff96abf0423c14ff696ebbdcaf59cf3195b29b13fe764d02d5", + "inode": "252689772", + "path": "/Users/some.user/tmp/ubnt_unifi-samples-filestream.log" + }, + "offset": 20571, + "syslog": { + "facility": { + "code": 3, + "name": "system" + }, + "hostname": "unifi-r1", + "priority": 30, + "severity": { + "code": 6, + "name": "Informational" + } + } + }, + "observer": { + "hostname": "unifi-r1", + "product": "UniFi", + "vendor": "Ubiquiti" + }, + "process": { + "name": "dnsmasq-dhcp", + "pid": 3408207 + }, + "related": { + "hosts": [ + "unifi-r1" + ] + }, + "tags": [ + "forwarded", + "ubnt-unifi-other" + ], + "ubnt": { + "unifi": { + "dhcp": { + "interface": "br3", + "ip": "192.168.0.18", + "mac": "01:23:45:54:f8:be", + "message": "ACK", + "name": "L535" + } + } + } + }, + { + "@timestamp": "2025-07-16T12:34:29.000Z", + "agent": { + "ephemeral_id": "a050b3ef-166f-4483-a9ff-b46996c08296", + "id": "cda862cd-99d5-4806-a514-ea1feb145b64", + "name": "elastic-agent.internal", + "type": "filebeat", + "version": "9.0.3" + }, + "ecs": { + "version": "8.17.0" + }, + "event": { + "kind": "event", + "type": [ + "info" + ] + }, + "input": { + "type": "filestream" + }, + "log": { + "file": { + "device_id": "16777231", + "fingerprint": "774aac143e417dff96abf0423c14ff696ebbdcaf59cf3195b29b13fe764d02d5", + "inode": "252689772", + "path": "/Users/some.user/tmp/ubnt_unifi-samples-filestream.log" + }, + "offset": 20684, + "syslog": { + "facility": { + "code": 3, + "name": "system" + }, + "hostname": "unifi-r1", + "priority": 30, + "severity": { + "code": 6, + "name": "Informational" + } + } + }, + "observer": { + "hostname": "unifi-r1", + "product": "UniFi", + "vendor": "Ubiquiti" + }, + "process": { + "name": "dnsmasq-dhcp", + "pid": 3408207 + }, + "related": { + "hosts": [ + "unifi-r1" + ] + }, + "tags": [ + "forwarded", + "ubnt-unifi-other" + ], + "ubnt": { + "unifi": { + "dhcp": { + "interface": "br3", + "ip": "192.168.0.18", + "mac": "01:23:45:54:f8:be", + "message": "REQUEST" + } + } + } + }, + { + "@timestamp": "2025-07-16T11:37:20.000Z", + "agent": { + "ephemeral_id": "a050b3ef-166f-4483-a9ff-b46996c08296", + "id": "cda862cd-99d5-4806-a514-ea1feb145b64", + "name": "office-monitor-left.home", + "type": "filebeat", + "version": "9.0.3" + }, + "ecs": { + "version": "8.17.0" + }, + "event": { + "kind": "event", + "type": [ + "info" + ] + }, + "input": { + "type": "filestream" + }, + "log": { + "file": { + "device_id": "16777231", + "fingerprint": "774aac143e417dff96abf0423c14ff696ebbdcaf59cf3195b29b13fe764d02d5", + "inode": "252705548", + "path": "/Users/colin.stubbs/tmp/ubnt_unifi-samples-filestream.log" + }, + "offset": 20796, + "syslog": { + "facility": { + "code": 1, + "name": "user-level" + }, + "hostname": "bimini-r1", + "priority": 14, + "severity": { + "code": 6, + "name": "Informational" + } + } + }, + "observer": { + "hostname": "bimini-r1", + "product": "UniFi", + "vendor": "Ubiquiti" + }, + "process": { + "name": "linkcheck", + "pid": 1583 + }, + "related": { + "hosts": [ + "bimini-r1" + ] + }, + "tags": [ + "forwarded", + "ubnt-unifi-other" + ], + "ubnt": { + "unifi": { + "linkcheck": { + "downlink": { + "rate": "Mbps", + "speed": 269.0 + }, + "function": "linkcheck.run_speedtest", + "uplink": { + "rate": "Mbps", + "speed": 23.0 + } + } + } + } + } + ] +} diff --git a/packages/ubnt_unifi/data_stream/logs/_dev/test/pipeline/test-logs-udp-syslog-firewall.json b/packages/ubnt_unifi/data_stream/logs/_dev/test/pipeline/test-logs-udp-syslog-firewall.json new file mode 100644 index 00000000000..32d8e6f0d14 --- /dev/null +++ b/packages/ubnt_unifi/data_stream/logs/_dev/test/pipeline/test-logs-udp-syslog-firewall.json @@ -0,0 +1,188 @@ +{ + "events": [ + { + "@timestamp": "2025-07-05T03:31:11.000Z", + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields", + "forwarded", + "ubnt-unifi-iptables" + ], + "input": { + "type": "udp" + }, + "message": "[VPN_LAN-D-10004] DESCR=\"Block VPN to Internal - Other\" IN=wgsrv1 OUT=br3 MAC= SRC=192.168.63.120 DST=10.95.75.174 LEN=64 TOS=00 PREC=0x00 TTL=253 ID=0 DF PROTO=TCP SPT=51523 DPT=80 SEQ=3925309497 ACK=0 WINDOW=65535 SYN URGP=0 MARK=1a0000 ", + "agent": { + "type": "filebeat", + "version": "9.0.3", + "ephemeral_id": "01234567-89ab-cdef-4ec2-b38f-e99536291b74", + "id": "01234567-89ab-cdef-4f5d-a1b8-6ed60bda2132", + "name": "elastic-agent-01.internal.example.com" + }, + "ecs": { + "version": "8.0.0" + }, + "event": { + "original": "<13>Jul 5 13:31:11 unifi-r1 [VPN_LAN-D-10004] DESCR=\"Block VPN to Internal - Other\" IN=wgsrv1 OUT=br3 MAC= SRC=192.168.63.120 DST=10.95.75.174 LEN=64 TOS=00 PREC=0x00 TTL=253 ID=0 DF PROTO=TCP SPT=51523 DPT=80 SEQ=3925309497 ACK=0 WINDOW=65535 SYN URGP=0 MARK=1a0000 \n" + }, + "log": { + "source": { + "address": "10.95.75.174:48352" + }, + "syslog": { + "severity": { + "code": 5, + "name": "Notice" + }, + "hostname": "unifi-r1", + "priority": 13, + "facility": { + "code": 1, + "name": "user-level" + } + } + } + }, + { + "@timestamp": "2025-07-07T12:45:52.000Z", + "ecs": { + "version": "8.0.0" + }, + "log": { + "file": { + "fingerprint": "3461751eb885f70d840f2ba889207880634eec70cb3ec35d32d932180b9cb4ab", + "path": "/Users/some.admin/tmp/ubnt_unifi-samples-filestream.log", + "device_id": "16777229", + "inode": "250494616" + }, + "syslog": { + "priority": 13, + "facility": { + "code": 1, + "name": "user-level" + }, + "severity": { + "code": 5, + "name": "Notice" + }, + "hostname": "unifi-r1" + }, + "offset": 5355 + }, + "event": { + "original": "<13>Jul 7 22:45:52 unifi-r1 [LOCAL_CUSTOM2-A-2147483647] DESCR=\\\"[LOCAL_CUSTOM2]Allow All T\\\" IN= OUT=br999 MAC= SRC=192.168.63.120 DST=255.255.255.255 LEN=32 TOS=00 PREC=0x00 TTL=64 ID=5692 DF PROTO=UDP SPT=45148 DPT=10001 LEN=12 UID=0 GID=0 MARK=1a0000 " + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields", + "forwarded", + "ubnt-unifi-iptables" + ], + "input": { + "type": "filestream" + }, + "message": "[LOCAL_CUSTOM2-A-2147483647] DESCR=\\\"[LOCAL_CUSTOM2]Allow All T\\\" IN= OUT=br999 MAC= SRC=192.168.63.120 DST=255.255.255.255 LEN=32 TOS=00 PREC=0x00 TTL=64 ID=5692 DF PROTO=UDP SPT=45148 DPT=10001 LEN=12 UID=0 GID=0 MARK=1a0000 ", + "agent": { + "ephemeral_id": "01234567-89ab-cdef-48f6-9e97-a152a19d90f5", + "id": "01234567-89ab-cdef-4ca6-af23-6caf5f262f1c", + "name": "elastic-agent.internal", + "type": "filebeat", + "version": "9.0.3" + } + }, + { + "@timestamp": "2025-07-08T03:32:04.000Z", + "agent": { + "name": "elastic-agent.internal", + "type": "filebeat", + "version": "9.0.3", + "ephemeral_id": "01234567-89ab-cdef-48f6-9e97-a152a19d90f5", + "id": "01234567-89ab-cdef-4ca6-af23-6caf5f262f1c" + }, + "ecs": { + "version": "8.0.0" + }, + "log": { + "offset": 5955, + "file": { + "fingerprint": "3461751eb885f70d840f2ba889207880634eec70cb3ec35d32d932180b9cb4ab", + "path": "/Users/some.admin/tmp/ubnt_unifi-samples-filestream.log", + "device_id": "16777229", + "inode": "250494616" + }, + "syslog": { + "priority": 13, + "facility": { + "code": 1, + "name": "user-level" + }, + "severity": { + "code": 5, + "name": "Notice" + }, + "hostname": "unifi-r1" + } + }, + "event": { + "original": "<13>Jul 8 13:32:04 unifi-r1 [WAN_LAN-D-2147483647] DESCR=\\\"[WAN_LAN]Block All Traffic\\\" IN=eth7 OUT=br5 MAC=d0:21:f9:89:c2:4a:ea:04:fe:a2:00:82:86:dd:60:05:85:34 SRC=2a02:cf40:5b54:f902:3f0e:1126:568a:75d9 DST=2a02:cf40:5503:de01:3dba:d6d6:5949:f36e LEN=56 TC=0 HOPLIMIT=246 FLOWLBL=361780 PROTO=ICMPv6 TYPE=128 CODE=0 ID=1 SEQ=1 MARK=1a0000 " + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields", + "forwarded", + "ubnt-unifi-iptables" + ], + "input": { + "type": "filestream" + }, + "message": "[WAN_LAN-D-2147483647] DESCR=\\\"[WAN_LAN]Block All Traffic\\\" IN=eth7 OUT=br5 MAC=d0:21:f9:89:c2:4a:ea:04:fe:a2:00:82:86:dd:60:05:85:34 SRC=2a02:cf40:5b54:f902:3f0e:1126:568a:75d9 DST=2a02:cf40:5503:de01:3dba:d6d6:5949:f36e LEN=56 TC=0 HOPLIMIT=246 FLOWLBL=361780 PROTO=ICMPv6 TYPE=128 CODE=0 ID=1 SEQ=1 MARK=1a0000 " + }, + { + "@timestamp": "2025-07-08T01:06:32.000Z", + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields", + "forwarded", + "ubnt-unifi-iptables" + ], + "input": { + "type": "filestream" + }, + "message": "[LAN_VPN-A-2147483647] DESCR=\"[LAN_VPN]Allow All Traffic\" IN=br2 OUT=wgsrv1 MAC=d0:21:f9:89:c2:4b:58:47:ca:7c:4d:01:08:00 SRC=10.95.75.174 DST=192.168.63.120 LEN=28 TOS=00 PREC=0x00 TTL=2 ID=60827 DF PROTO=ICMP TYPE=8 CODE=0 ID=50973 SEQ=60615 MARK=1a0000 ", + "ecs": { + "version": "8.0.0" + }, + "agent": { + "ephemeral_id": "01234567-89ab-cdef-48f6-9e97-a152a19d90f5", + "id": "01234567-89ab-cdef-4ca6-af23-6caf5f262f1c", + "name": "elastic-agent.internal", + "type": "filebeat", + "version": "9.0.3" + }, + "log": { + "offset": 6300, + "file": { + "path": "/Users/some.admin/tmp/ubnt_unifi-samples-filestream.log", + "device_id": "16777229", + "inode": "250494616", + "fingerprint": "3461751eb885f70d840f2ba889207880634eec70cb3ec35d32d932180b9cb4ab" + }, + "syslog": { + "priority": 13, + "facility": { + "code": 1, + "name": "user-level" + }, + "severity": { + "name": "Notice", + "code": 5 + }, + "hostname": "unifi-r1" + } + }, + "event": { + "original": "<13>Jul 8 11:06:32 unifi-r1 [LAN_VPN-A-2147483647] DESCR=\"[LAN_VPN]Allow All Traffic\" IN=br2 OUT=wgsrv1 MAC=d0:21:f9:89:c2:4b:58:47:ca:7c:4d:01:08:00 SRC=192.168.0.32 DST=192.168.10.2 LEN=28 TOS=00 PREC=0x00 TTL=2 ID=60827 DF PROTO=ICMP TYPE=8 CODE=0 ID=50973 SEQ=60615 MARK=1a0000 " + } + } + ] +} \ No newline at end of file diff --git a/packages/ubnt_unifi/data_stream/logs/_dev/test/pipeline/test-logs-udp-syslog-firewall.json-expected.json b/packages/ubnt_unifi/data_stream/logs/_dev/test/pipeline/test-logs-udp-syslog-firewall.json-expected.json new file mode 100644 index 00000000000..08a8ed4e6b3 --- /dev/null +++ b/packages/ubnt_unifi/data_stream/logs/_dev/test/pipeline/test-logs-udp-syslog-firewall.json-expected.json @@ -0,0 +1,524 @@ +{ + "expected": [ + { + "@timestamp": "2025-07-05T03:31:11.000Z", + "agent": { + "ephemeral_id": "01234567-89ab-cdef-4ec2-b38f-e99536291b74", + "id": "01234567-89ab-cdef-4f5d-a1b8-6ed60bda2132", + "name": "elastic-agent-01.internal.example.com", + "type": "filebeat", + "version": "9.0.3" + }, + "destination": { + "address": "10.95.75.174", + "ip": "10.95.75.174", + "port": 80 + }, + "ecs": { + "version": "8.17.0" + }, + "event": { + "action": "drop", + "category": [ + "network" + ], + "kind": "event", + "original": "<13>Jul 5 13:31:11 unifi-r1 [VPN_LAN-D-10004] DESCR=\"Block VPN to Internal - Other\" IN=wgsrv1 OUT=br3 MAC= SRC=192.168.63.120 DST=10.95.75.174 LEN=64 TOS=00 PREC=0x00 TTL=253 ID=0 DF PROTO=TCP SPT=51523 DPT=80 SEQ=3925309497 ACK=0 WINDOW=65535 SYN URGP=0 MARK=1a0000 \n", + "type": [ + "denied", + "connection" + ] + }, + "input": { + "type": "udp" + }, + "iptables": { + "fragment_flags": [ + "DF" + ], + "id": 0, + "length": 64, + "mark": "1a0000", + "precedence_bits": 0, + "tcp": { + "ack": 0, + "flags": [ + "SYN" + ], + "seq": 3925309497, + "urgp": "0", + "window": 65535 + }, + "tos": 0, + "ttl": 253, + "ubiquiti": { + "rule_description": "Block VPN to Internal - Other", + "rule_name": "VPN_LAN-D-10004", + "rule_number": "10004", + "rule_set": "VPN_LAN" + } + }, + "log": { + "source": { + "address": "10.95.75.174:48352" + }, + "syslog": { + "facility": { + "code": 1, + "name": "user-level" + }, + "hostname": "unifi-r1", + "priority": 13, + "severity": { + "code": 5, + "name": "Notice" + } + } + }, + "message": "[VPN_LAN-D-10004] DESCR=\"Block VPN to Internal - Other\" IN=wgsrv1 OUT=br3 MAC= SRC=192.168.63.120 DST=10.95.75.174 LEN=64 TOS=00 PREC=0x00 TTL=253 ID=0 DF PROTO=TCP SPT=51523 DPT=80 SEQ=3925309497 ACK=0 WINDOW=65535 SYN URGP=0 MARK=1a0000 ", + "network": { + "community_id": "1:BtTl5uUCVWA6Vgql8suoqSnHL0M=", + "direction": "internal", + "transport": "tcp" + }, + "observer": { + "egress": { + "interface": { + "name": "br3" + }, + "zone": "LAN" + }, + "hostname": "unifi-r1", + "ingress": { + "interface": { + "name": "wgsrv1" + }, + "zone": "VPN" + }, + "ip": [ + "10.95.75.174" + ], + "product": "UniFi", + "vendor": "Ubiquiti" + }, + "related": { + "hosts": [ + "unifi-r1" + ], + "ip": [ + "10.95.75.174", + "192.168.63.120" + ] + }, + "rule": { + "description": "Block VPN to Internal - Other", + "id": "10004", + "name": "VPN_LAN-D-10004", + "ruleset": "VPN_LAN" + }, + "source": { + "address": "192.168.63.120", + "ip": "192.168.63.120", + "port": 51523 + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields", + "forwarded", + "ubnt-unifi-iptables" + ] + }, + { + "@timestamp": "2025-07-07T12:45:52.000Z", + "agent": { + "ephemeral_id": "01234567-89ab-cdef-48f6-9e97-a152a19d90f5", + "id": "01234567-89ab-cdef-4ca6-af23-6caf5f262f1c", + "name": "elastic-agent.internal", + "type": "filebeat", + "version": "9.0.3" + }, + "destination": { + "address": "255.255.255.255", + "ip": "255.255.255.255", + "port": 10001 + }, + "ecs": { + "version": "8.17.0" + }, + "event": { + "action": "accept", + "category": [ + "network" + ], + "kind": "event", + "original": "<13>Jul 7 22:45:52 unifi-r1 [LOCAL_CUSTOM2-A-2147483647] DESCR=\\\"[LOCAL_CUSTOM2]Allow All T\\\" IN= OUT=br999 MAC= SRC=192.168.63.120 DST=255.255.255.255 LEN=32 TOS=00 PREC=0x00 TTL=64 ID=5692 DF PROTO=UDP SPT=45148 DPT=10001 LEN=12 UID=0 GID=0 MARK=1a0000 ", + "type": [ + "allowed", + "connection" + ] + }, + "input": { + "type": "filestream" + }, + "iptables": { + "fragment_flags": [ + "DF" + ], + "gid": "0", + "id": 5692, + "length": 32, + "mark": "1a0000", + "precedence_bits": 0, + "tos": 0, + "ttl": 64, + "ubiquiti": { + "rule_description": "[LOCAL_CUSTOM2]Allow All T", + "rule_name": "LOCAL_CUSTOM2-A-2147483647", + "rule_number": "2147483647", + "rule_set": "LOCAL_CUSTOM2" + }, + "udp": { + "length": 12 + }, + "uid": "0" + }, + "log": { + "file": { + "device_id": "16777229", + "fingerprint": "3461751eb885f70d840f2ba889207880634eec70cb3ec35d32d932180b9cb4ab", + "inode": "250494616", + "path": "/Users/some.admin/tmp/ubnt_unifi-samples-filestream.log" + }, + "offset": 5355, + "syslog": { + "facility": { + "code": 1, + "name": "user-level" + }, + "hostname": "unifi-r1", + "priority": 13, + "severity": { + "code": 5, + "name": "Notice" + } + } + }, + "message": "[LOCAL_CUSTOM2-A-2147483647] DESCR=\\\"[LOCAL_CUSTOM2]Allow All T\\\" IN= OUT=br999 MAC= SRC=192.168.63.120 DST=255.255.255.255 LEN=32 TOS=00 PREC=0x00 TTL=64 ID=5692 DF PROTO=UDP SPT=45148 DPT=10001 LEN=12 UID=0 GID=0 MARK=1a0000 ", + "network": { + "community_id": "1:C1gJ2CvGaCbcQ1s+qz5ReCCYi9c=", + "direction": "outbound", + "transport": "udp" + }, + "observer": { + "egress": { + "interface": { + "name": "br999" + }, + "zone": "CUSTOM2" + }, + "hostname": "unifi-r1", + "ingress": { + "zone": "LOCAL" + }, + "product": "UniFi", + "vendor": "Ubiquiti" + }, + "related": { + "hosts": [ + "unifi-r1" + ], + "ip": [ + "192.168.63.120", + "255.255.255.255" + ] + }, + "rule": { + "description": "[LOCAL_CUSTOM2]Allow All T", + "id": "2147483647", + "name": "LOCAL_CUSTOM2-A-2147483647", + "ruleset": "LOCAL_CUSTOM2" + }, + "source": { + "address": "192.168.63.120", + "ip": "192.168.63.120", + "port": 45148 + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields", + "forwarded", + "ubnt-unifi-iptables" + ] + }, + { + "@timestamp": "2025-07-08T03:32:04.000Z", + "agent": { + "ephemeral_id": "01234567-89ab-cdef-48f6-9e97-a152a19d90f5", + "id": "01234567-89ab-cdef-4ca6-af23-6caf5f262f1c", + "name": "elastic-agent.internal", + "type": "filebeat", + "version": "9.0.3" + }, + "destination": { + "address": "2a02:cf40:5503:de01:3dba:d6d6:5949:f36e", + "geo": { + "continent_name": "Europe", + "country_iso_code": "NO", + "country_name": "Norway", + "location": { + "lat": 62.0, + "lon": 10.0 + } + }, + "ip": "2a02:cf40:5503:de01:3dba:d6d6:5949:f36e", + "mac": "D0-21-F9-89-C2-4A" + }, + "ecs": { + "version": "8.17.0" + }, + "event": { + "action": "drop", + "category": [ + "network" + ], + "kind": "event", + "original": "<13>Jul 8 13:32:04 unifi-r1 [WAN_LAN-D-2147483647] DESCR=\\\"[WAN_LAN]Block All Traffic\\\" IN=eth7 OUT=br5 MAC=d0:21:f9:89:c2:4a:ea:04:fe:a2:00:82:86:dd:60:05:85:34 SRC=2a02:cf40:5b54:f902:3f0e:1126:568a:75d9 DST=2a02:cf40:5503:de01:3dba:d6d6:5949:f36e LEN=56 TC=0 HOPLIMIT=246 FLOWLBL=361780 PROTO=ICMPv6 TYPE=128 CODE=0 ID=1 SEQ=1 MARK=1a0000 ", + "type": [ + "denied", + "connection" + ] + }, + "input": { + "type": "filestream" + }, + "iptables": { + "ether_type": 34525, + "flow_label": 361780, + "icmp": { + "code": 0, + "id": 1, + "seq": 1, + "type": 128 + }, + "length": 56, + "mark": "1a0000", + "tos": 0, + "ttl": 246, + "ubiquiti": { + "rule_description": "[WAN_LAN]Block All Traffic", + "rule_name": "WAN_LAN-D-2147483647", + "rule_number": "2147483647", + "rule_set": "WAN_LAN" + } + }, + "log": { + "file": { + "device_id": "16777229", + "fingerprint": "3461751eb885f70d840f2ba889207880634eec70cb3ec35d32d932180b9cb4ab", + "inode": "250494616", + "path": "/Users/some.admin/tmp/ubnt_unifi-samples-filestream.log" + }, + "offset": 5955, + "syslog": { + "facility": { + "code": 1, + "name": "user-level" + }, + "hostname": "unifi-r1", + "priority": 13, + "severity": { + "code": 5, + "name": "Notice" + } + } + }, + "message": "[WAN_LAN-D-2147483647] DESCR=\\\"[WAN_LAN]Block All Traffic\\\" IN=eth7 OUT=br5 MAC=d0:21:f9:89:c2:4a:ea:04:fe:a2:00:82:86:dd:60:05:85:34 SRC=2a02:cf40:5b54:f902:3f0e:1126:568a:75d9 DST=2a02:cf40:5503:de01:3dba:d6d6:5949:f36e LEN=56 TC=0 HOPLIMIT=246 FLOWLBL=361780 PROTO=ICMPv6 TYPE=128 CODE=0 ID=1 SEQ=1 MARK=1a0000 ", + "network": { + "community_id": "1:OXHwYPwR8WKez+Gdfp5RvztfUCw=", + "direction": "external", + "transport": "ipv6-icmp", + "type": "ipv6" + }, + "observer": { + "egress": { + "interface": { + "name": "br5" + }, + "zone": "LAN" + }, + "hostname": "unifi-r1", + "ingress": { + "interface": { + "name": "eth7" + }, + "zone": "WAN" + }, + "product": "UniFi", + "vendor": "Ubiquiti" + }, + "related": { + "hosts": [ + "unifi-r1" + ], + "ip": [ + "2a02:cf40:5b54:f902:3f0e:1126:568a:75d9", + "2a02:cf40:5503:de01:3dba:d6d6:5949:f36e" + ] + }, + "rule": { + "description": "[WAN_LAN]Block All Traffic", + "id": "2147483647", + "name": "WAN_LAN-D-2147483647", + "ruleset": "WAN_LAN" + }, + "source": { + "address": "2a02:cf40:5b54:f902:3f0e:1126:568a:75d9", + "geo": { + "continent_name": "Europe", + "country_iso_code": "NO", + "country_name": "Norway", + "location": { + "lat": 62.0, + "lon": 10.0 + } + }, + "ip": "2a02:cf40:5b54:f902:3f0e:1126:568a:75d9", + "mac": "EA-04-FE-A2-00-82" + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields", + "forwarded", + "ubnt-unifi-iptables" + ] + }, + { + "@timestamp": "2025-07-08T01:06:32.000Z", + "agent": { + "ephemeral_id": "01234567-89ab-cdef-48f6-9e97-a152a19d90f5", + "id": "01234567-89ab-cdef-4ca6-af23-6caf5f262f1c", + "name": "elastic-agent.internal", + "type": "filebeat", + "version": "9.0.3" + }, + "destination": { + "address": "192.168.10.2", + "ip": "192.168.10.2", + "mac": "D0-21-F9-89-C2-4B" + }, + "ecs": { + "version": "8.17.0" + }, + "event": { + "action": "accept", + "category": [ + "network" + ], + "kind": "event", + "original": "<13>Jul 8 11:06:32 unifi-r1 [LAN_VPN-A-2147483647] DESCR=\"[LAN_VPN]Allow All Traffic\" IN=br2 OUT=wgsrv1 MAC=d0:21:f9:89:c2:4b:58:47:ca:7c:4d:01:08:00 SRC=192.168.0.32 DST=192.168.10.2 LEN=28 TOS=00 PREC=0x00 TTL=2 ID=60827 DF PROTO=ICMP TYPE=8 CODE=0 ID=50973 SEQ=60615 MARK=1a0000 ", + "type": [ + "allowed", + "connection" + ] + }, + "input": { + "type": "filestream" + }, + "iptables": { + "ether_type": 2048, + "fragment_flags": [ + "DF" + ], + "icmp": { + "code": 0, + "id": 50973, + "seq": 60615, + "type": 8 + }, + "id": 60827, + "length": 28, + "mark": "1a0000", + "precedence_bits": 0, + "tos": 0, + "ttl": 2, + "ubiquiti": { + "rule_description": "[LAN_VPN]Allow All Traffic", + "rule_name": "LAN_VPN-A-2147483647", + "rule_number": "2147483647", + "rule_set": "LAN_VPN" + } + }, + "log": { + "file": { + "device_id": "16777229", + "fingerprint": "3461751eb885f70d840f2ba889207880634eec70cb3ec35d32d932180b9cb4ab", + "inode": "250494616", + "path": "/Users/some.admin/tmp/ubnt_unifi-samples-filestream.log" + }, + "offset": 6300, + "syslog": { + "facility": { + "code": 1, + "name": "user-level" + }, + "hostname": "unifi-r1", + "priority": 13, + "severity": { + "code": 5, + "name": "Notice" + } + } + }, + "message": "[LAN_VPN-A-2147483647] DESCR=\"[LAN_VPN]Allow All Traffic\" IN=br2 OUT=wgsrv1 MAC=d0:21:f9:89:c2:4b:58:47:ca:7c:4d:01:08:00 SRC=10.95.75.174 DST=192.168.63.120 LEN=28 TOS=00 PREC=0x00 TTL=2 ID=60827 DF PROTO=ICMP TYPE=8 CODE=0 ID=50973 SEQ=60615 MARK=1a0000 ", + "network": { + "community_id": "1:B6C44SyDDqpCk+R5K/pYArwZZFU=", + "direction": "internal", + "transport": "icmp", + "type": "ipv4" + }, + "observer": { + "egress": { + "interface": { + "name": "wgsrv1" + }, + "zone": "VPN" + }, + "hostname": "unifi-r1", + "ingress": { + "interface": { + "name": "br2" + }, + "zone": "LAN" + }, + "product": "UniFi", + "vendor": "Ubiquiti" + }, + "related": { + "hosts": [ + "unifi-r1" + ], + "ip": [ + "192.168.0.32", + "192.168.10.2" + ] + }, + "rule": { + "description": "[LAN_VPN]Allow All Traffic", + "id": "2147483647", + "name": "LAN_VPN-A-2147483647", + "ruleset": "LAN_VPN" + }, + "source": { + "address": "192.168.0.32", + "ip": "192.168.0.32", + "mac": "58-47-CA-7C-4D-01" + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields", + "forwarded", + "ubnt-unifi-iptables" + ] + } + ] +} diff --git a/packages/ubnt_unifi/data_stream/logs/_dev/test/system/test-filestream-config.yml b/packages/ubnt_unifi/data_stream/logs/_dev/test/system/test-filestream-config.yml new file mode 100644 index 00000000000..c803fb6d4d6 --- /dev/null +++ b/packages/ubnt_unifi/data_stream/logs/_dev/test/system/test-filestream-config.yml @@ -0,0 +1,10 @@ +service: test-filestream +input: filestream +data_stream: + vars: + paths: + - "{{SERVICE_LOGS_DIR}}/logs-filestream-*.log" + preserve_original_event: true + preserve_duplicate_custom_fields: true +assert: + hit_count: 28 diff --git a/packages/ubnt_unifi/data_stream/logs/_dev/test/system/test-udp-syslog-config.yml b/packages/ubnt_unifi/data_stream/logs/_dev/test/system/test-udp-syslog-config.yml new file mode 100644 index 00000000000..7115526f24b --- /dev/null +++ b/packages/ubnt_unifi/data_stream/logs/_dev/test/system/test-udp-syslog-config.yml @@ -0,0 +1,11 @@ +service: test-udp-syslog +service_notify_signal: SIGHUP +input: udp +data_stream: + vars: + listen_address: 0.0.0.0 + listen_port: 9514 + preserve_original_event: true + preserve_duplicate_custom_fields: true +assert: + hit_count: 12 diff --git a/packages/ubnt_unifi/data_stream/logs/agent/stream/filestream.yml.hbs b/packages/ubnt_unifi/data_stream/logs/agent/stream/filestream.yml.hbs new file mode 100644 index 00000000000..b546cc2b78f --- /dev/null +++ b/packages/ubnt_unifi/data_stream/logs/agent/stream/filestream.yml.hbs @@ -0,0 +1,196 @@ +paths: +{{#each paths as |path|}} +- {{path}} +{{/each}} +prospector.scanner.exclude_files: ['\.gz$'] +tags: +{{#if preserve_original_event}} +- preserve_original_event +{{/if}} +{{#if preserve_duplicate_custom_fields}} +- preserve_duplicate_custom_fields +{{/if}} +{{#each tags as |tag i|}} +- {{tag}} +{{/each}} +{{#contains "forwarded" tags}} +publisher_pipeline.disable_host: true +{{/contains}} +processors: +- rename: + fields: + - {from: "message", to: "event.original"} +- if: + regexp: + event.original: "^<[0-9]+>" + then: + - syslog: # useful for facility/severity etc... we expect the user to force the timezone if they want/need to. + {{syslog_options}} +- script: + lang: javascript + id: parse_out_message + source: > + function process(event) { + var message = event.Get("event.original").replace(/^<[0-9]+>/g,''); + if (!message) { + return; + } + + // The many flavours of non-syslog CEF being sent to a syslog destination... + // Jul 3 11:56:52 hostname CEF: 0|Ubiquiti|UniFi OS|4.3.5| + // Jul 03 10:03:35 2025-07-03T10:03:35.913Z hostname CEF:0|Ubiquiti|UniFi Network|9.3.29| + // Jul 3 01:56:54 hostname.fqdn 2025-07-03T01: 56:54.222Z hostname CEF:0|Ubiquiti|UniFi Network|9.3.29| + + // Syslog message variants observed, the syslog parser should already have down something with these... + // <30>Jul 4 09:01:45 hostname-ap SERIALNUMBER,U6-Lite-6.7.17+15512: %{MESSAGE}% + // <27>hostname-switch SERIALNUMBER,USW-Flex-2.5G-8-2.1.8.971: %{MESSAGE}% + // <4>Jul 4 09:02:12 hostname-ap SERIALNUMBER,U7-Pro-8.0.19+16619: %{MESSAGE}% + // <14>Jul 4 09:00:35 hostname-router hostname-router %{MESSAGE}% + + // rsyslog produced files, traditional format, we should also assume that RFC3339 format timestamps may be used. + // Jul 3 11:56:52 hostname CEF: 0|Ubiquiti|UniFi OS|4.3.5| + // Jul 3 10:03:35 2025-07-03T10:03:35.913Z hostname CEF:0|Ubiquiti|UniFi Network|9.3.29| + // Jul 3 01:56:54 hostname.fqdn 2025-07-03T01: 56:54.222Z hostname CEF:0|Ubiquiti|UniFi Network|9.3.29| + // Jul 4 09:01:45 hostname-ap SERIALNUMBER,U6-Lite-6.7.17+15512: %{MESSAGE}% + // Jul 3 11:56:36 hostname-switch SERIALNUMBER,USW-Flex-2.5G-8-2.1.8.971: %{MESSAGE}% + // Jul 4 09:02:12 hostname-ap SERIALNUMBER,U7-Pro-8.0.19+16619: %{MESSAGE}% + // Jul 4 09:00:35 hostname-router hostname-router %{MESSAGE}% + + var parts = message.match(/^(([a-zA-Z0-9: ]+|[0-9\:\-\.TZ]+) (\S+)|([a-zA-Z0-9: ]+|[0-9\:\-\.TZ]+) ([0-9\:\-\.TZ ]+) (\S+)|([a-zA-Z0-9: ]+|[0-9\:\-\.TZ]+) (\S+) ([0-9\:\-\.TZ ]+) (\S+)) (CEF:.*)$/m); + // 12 parts total + + if (parts && parts.length === 12) { + event.Tag("ubnt-unifi-cef"); + event.Put("message", parts[11]); + + if (parts[2] != null) { + event.Put("_tmp.timestamp1", parts[2]); + event.Put("log.syslog.hostname", parts[3]); + } else if (parts[4] != null) { + event.Put("_tmp.timestamp1", parts[4]); + event.Put("_tmp.timestamp2", parts[5].replace(/ /g,'')); + event.Put("log.syslog.hostname", parts[6]); + } else if (parts[7] != null) { + event.Put("_tmp.timestamp1", parts[7]); + event.Put("log.syslog.hostname", parts[8]); + event.Put("_tmp.timestamp2", parts[9].replace(/ /g,'')); + event.Put("observer.name", parts[10]); + } + } else { // did not match CEF specific regex, try more generic variants + parts = message.match(/^(([a-zA-Z0-9: ]+|[0-9\:\-\.TZ]+) (\S+) \3|([a-zA-Z0-9: ]+|[0-9\:\-\.TZ]+) (\S+) (\S+):|([a-zA-Z0-9: ]+|[0-9\:\-\.TZ]+) (\S+)|(\S+) (\S+):|(\S+)) (.*)$/m); + // 13 parts total + + if (message.match(/ DESCR=\\*"/)) { + event.Tag("ubnt-unifi-iptables"); + } else { + event.Tag("ubnt-unifi-other"); + } + + if (parts && parts.length === 13) { + event.Put("message", parts[12]); + + if (parts[2] != null) { + event.Put("_tmp.timestamp1", parts[2]); + event.Put("log.syslog.hostname", parts[3]); + } else if (parts[4] != null) { + event.Put("_tmp.timestamp1", parts[4]); + event.Put("log.syslog.hostname", parts[5]); + event.Put("log.syslog.appname", parts[6]); + } else if (parts[7] != null) { + event.Put("_tmp.timestamp1", parts[7]); + event.Put("log.syslog.hostname", parts[8]); + } else if (parts[9] != null) { + event.Put("log.syslog.hostname", parts[9]); + event.Put("log.syslog.appname", parts[10]); + } else if (parts[11] != null) { + event.Put("log.syslog.hostname", parts[11]); + } + } + } + } +- timestamp: # use first instance of a timestamp if available, does not indicate zone however so force use of our default timezone if set, otherwise local will be assumed + field: _tmp.timestamp1 + ignore_missing: true + ignore_failure: false + layouts: + - 'Jan 2 15:04:05' + - 'Jan 2 15:04:05' + - 'Jan 02 15:04:05' + - '2006-01-02T15:04:05.000Z' + test: + - 'Mar 9 14:49:36' + - 'Mar 9 14:49:36' + - 'Mar 09 14:49:36' + - '2025-07-03T06:24:37.921Z' +{{#if default_timezone}} + timezone: {{default_timezone}} +{{/if}} +- timestamp: # override the timestamp with the second instance if available, as this will be in theory the more accurate one + field: _tmp.timestamp2 + ignore_missing: true + ignore_failure: false + layouts: + - '2006-01-02T15:04:05.000Z' + test: + - '2025-07-03T06:24:37.921Z' +- drop_fields: + fields: + - _tmp + ignore_missing: true +{{#if drop_cef}} +- drop_event: # drop before we attempt to fix and parse CEF so we don't waste our time... + when: + contains: + tags: ubnt-unifi-cef +{{/if}} +{{^if drop_cef}} +- if: + contains: + tags: ubnt-unifi-cef + then: + - replace: + description: "Fixes malformed CEF that Unifi can produce whereby a space will exist before the CEF version indicator which filebeats cef parser will not understand" + fields: + - {field: message, pattern: ' CEF:[\s\t]+([0-9]+)', replacement: ' CEF:$1'} + ignore_missing: true + fail_on_error: false + - replace: + description: "Fixes malformed CEF that Unifi can produce whereby a trailing | may be missing which filebeats cef parser requires to exist" + fields: + - {field: message, pattern: '\|Medium$', replacement: '|Medium|'} + ignore_missing: true + fail_on_error: false + - replace: + description: "Fixes malformed CEF that Unifi can produce whereby a mandatory cef field will be missing, we simply insert an extra | to create an empty mandatory field which should be acceptable to filebeat" + fields: + - {field: message, pattern: '(CEF\:0\|[^\|]*\|[^\|]*\|[^\|]*\|)([^\|]*\|[^\|]*\|msg=)', replacement: '$1|$2'} + ignore_missing: true + fail_on_error: false + - decode_cef: + field: message + ignore_missing: true + ignore_empty_values: true + ignore_failure: true +{{#if default_timezone}} + timezone: {{default_timezone}} +{{/if}} +{{/if}} +{{#if drop_iptables}} +- drop_event: + when: + contains: + tags: ubnt-unifi-iptables +{{/if}} +{{#if drop_noise}} +- drop_event: + when: + not: + or: + - contains: + tags: ubnt-unifi-cef + - contains: + tags: ubnt-unifi-iptables +{{/if}} +{{#if processors}} +{{processors}} +{{/if}} diff --git a/packages/ubnt_unifi/data_stream/logs/agent/stream/udp-syslog.yml.hbs b/packages/ubnt_unifi/data_stream/logs/agent/stream/udp-syslog.yml.hbs new file mode 100644 index 00000000000..df4bf615436 --- /dev/null +++ b/packages/ubnt_unifi/data_stream/logs/agent/stream/udp-syslog.yml.hbs @@ -0,0 +1,192 @@ +host: {{listen_address}}:{{listen_port}} +tags: +{{#if preserve_original_event}} +- preserve_original_event +{{/if}} +{{#if preserve_duplicate_custom_fields}} +- preserve_duplicate_custom_fields +{{/if}} +{{#each tags as |tag i|}} +- {{tag}} +{{/each}} +{{#contains "forwarded" tags}} +publisher_pipeline.disable_host: true +{{/contains}} +processors: +- rename: + fields: + - {from: "message", to: "event.original"} +- if: + regexp: + event.original: "^<[0-9]+>" + then: + - syslog: # useful for facility/severity etc... we expect the user to force the timezone if they want/need to. + {{syslog_options}} +- script: + lang: javascript + id: parse_out_message + source: > + function process(event) { + var message = event.Get("event.original").replace(/^<[0-9]+>/g,''); + if (!message) { + return; + } + + // The many flavours of non-syslog CEF being sent to a syslog destination... + // Jul 3 11:56:52 hostname CEF: 0|Ubiquiti|UniFi OS|4.3.5| + // Jul 03 10:03:35 2025-07-03T10:03:35.913Z hostname CEF:0|Ubiquiti|UniFi Network|9.3.29| + // Jul 3 01:56:54 hostname.fqdn 2025-07-03T01: 56:54.222Z hostname CEF:0|Ubiquiti|UniFi Network|9.3.29| + + // Syslog message variants observed, the syslog parser should already have down something with these... + // <30>Jul 4 09:01:45 hostname-ap SERIALNUMBER,U6-Lite-6.7.17+15512: %{MESSAGE}% + // <27>hostname-switch SERIALNUMBER,USW-Flex-2.5G-8-2.1.8.971: %{MESSAGE}% + // <4>Jul 4 09:02:12 hostname-ap SERIALNUMBER,U7-Pro-8.0.19+16619: %{MESSAGE}% + // <14>Jul 4 09:00:35 hostname-router hostname-router %{MESSAGE}% + + // rsyslog produced files, traditional format, we should also assume that RFC3339 format timestamps may be used. + // Jul 3 11:56:52 hostname CEF: 0|Ubiquiti|UniFi OS|4.3.5| + // Jul 3 10:03:35 2025-07-03T10:03:35.913Z hostname CEF:0|Ubiquiti|UniFi Network|9.3.29| + // Jul 3 01:56:54 hostname.fqdn 2025-07-03T01: 56:54.222Z hostname CEF:0|Ubiquiti|UniFi Network|9.3.29| + // Jul 4 09:01:45 hostname-ap SERIALNUMBER,U6-Lite-6.7.17+15512: %{MESSAGE}% + // Jul 3 11:56:36 hostname-switch SERIALNUMBER,USW-Flex-2.5G-8-2.1.8.971: %{MESSAGE}% + // Jul 4 09:02:12 hostname-ap SERIALNUMBER,U7-Pro-8.0.19+16619: %{MESSAGE}% + // Jul 4 09:00:35 hostname-router hostname-router %{MESSAGE}% + + var parts = message.match(/^(([a-zA-Z0-9: ]+|[0-9\:\-\.TZ]+) (\S+)|([a-zA-Z0-9: ]+|[0-9\:\-\.TZ]+) ([0-9\:\-\.TZ ]+) (\S+)|([a-zA-Z0-9: ]+|[0-9\:\-\.TZ]+) (\S+) ([0-9\:\-\.TZ ]+) (\S+)) (CEF:.*)$/m); + // 12 parts total + + if (parts && parts.length === 12) { + event.Tag("ubnt-unifi-cef"); + event.Put("message", parts[11]); + + if (parts[2] != null) { + event.Put("_tmp.timestamp1", parts[2]); + event.Put("log.syslog.hostname", parts[3]); + } else if (parts[4] != null) { + event.Put("_tmp.timestamp1", parts[4]); + event.Put("_tmp.timestamp2", parts[5].replace(/ /g,'')); + event.Put("log.syslog.hostname", parts[6]); + } else if (parts[7] != null) { + event.Put("_tmp.timestamp1", parts[7]); + event.Put("log.syslog.hostname", parts[8]); + event.Put("_tmp.timestamp2", parts[9].replace(/ /g,'')); + event.Put("observer.name", parts[10]); + } + } else { // did not match CEF specific regex, try more generic variants + parts = message.match(/^(([a-zA-Z0-9: ]+|[0-9\:\-\.TZ]+) (\S+) \3|([a-zA-Z0-9: ]+|[0-9\:\-\.TZ]+) (\S+) (\S+):|([a-zA-Z0-9: ]+|[0-9\:\-\.TZ]+) (\S+)|(\S+) (\S+):|(\S+)) (.*)$/m); + // 13 parts total + + if (message.match(/ DESCR=\\*"/)) { + event.Tag("ubnt-unifi-iptables"); + } else { + event.Tag("ubnt-unifi-other"); + } + + if (parts && parts.length === 13) { + event.Put("message", parts[12]); + + if (parts[2] != null) { + event.Put("_tmp.timestamp1", parts[2]); + event.Put("log.syslog.hostname", parts[3]); + } else if (parts[4] != null) { + event.Put("_tmp.timestamp1", parts[4]); + event.Put("log.syslog.hostname", parts[5]); + event.Put("log.syslog.appname", parts[6]); + } else if (parts[7] != null) { + event.Put("_tmp.timestamp1", parts[7]); + event.Put("log.syslog.hostname", parts[8]); + } else if (parts[9] != null) { + event.Put("log.syslog.hostname", parts[9]); + event.Put("log.syslog.appname", parts[10]); + } else if (parts[11] != null) { + event.Put("log.syslog.hostname", parts[11]); + } + } + } + } +- timestamp: # use first instance of a timestamp if available, does not indicate zone however so force use of our default timezone if set, otherwise local will be assumed + field: _tmp.timestamp1 + ignore_missing: true + ignore_failure: false + layouts: + - 'Jan 2 15:04:05' + - 'Jan 2 15:04:05' + - 'Jan 02 15:04:05' + - '2006-01-02T15:04:05.000Z' + test: + - 'Mar 9 14:49:36' + - 'Mar 9 14:49:36' + - 'Mar 09 14:49:36' + - '2025-07-03T06:24:37.921Z' +{{#if default_timezone}} + timezone: {{default_timezone}} +{{/if}} +- timestamp: # override the timestamp with the second instance if available, as this will be in theory the more accurate one + field: _tmp.timestamp2 + ignore_missing: true + ignore_failure: false + layouts: + - '2006-01-02T15:04:05.000Z' + test: + - '2025-07-03T06:24:37.921Z' +- drop_fields: + fields: + - _tmp + ignore_missing: true +{{#if drop_cef}} +- drop_event: # drop before we attempt to fix and parse CEF so we don't waste our time... + when: + contains: + tags: ubnt-unifi-cef +{{/if}} +{{^if drop_cef}} +- if: + contains: + tags: ubnt-unifi-cef + then: + - replace: + description: "Fixes malformed CEF that Unifi can produce whereby a space will exist before the CEF version indicator which filebeats cef parser will not understand" + fields: + - {field: message, pattern: ' CEF:[\s\t]+([0-9]+)', replacement: ' CEF:$1'} + ignore_missing: true + fail_on_error: false + - replace: + description: "Fixes malformed CEF that Unifi can produce whereby a trailing | may be missing which filebeats cef parser requires to exist" + fields: + - {field: message, pattern: '\|Medium$', replacement: '|Medium|'} + ignore_missing: true + fail_on_error: false + - replace: + description: "Fixes malformed CEF that Unifi can produce whereby a mandatory cef field will be missing, we simply insert an extra | to create an empty mandatory field which should be acceptable to filebeat" + fields: + - {field: message, pattern: '(CEF\:0\|[^\|]*\|[^\|]*\|[^\|]*\|)([^\|]*\|[^\|]*\|msg=)', replacement: '$1|$2'} + ignore_missing: true + fail_on_error: false + - decode_cef: + field: message + ignore_missing: true + ignore_empty_values: true + ignore_failure: true +{{#if default_timezone}} + timezone: {{default_timezone}} +{{/if}} +{{/if}} +{{#if drop_iptables}} +- drop_event: + when: + contains: + tags: ubnt-unifi-iptables +{{/if}} +{{#if drop_noise}} +- drop_event: + when: + not: + or: + - contains: + tags: ubnt-unifi-cef + - contains: + tags: ubnt-unifi-iptables +{{/if}} +{{#if processors}} +{{processors}} +{{/if}} diff --git a/packages/ubnt_unifi/data_stream/logs/elasticsearch/ingest_pipeline/cef.yml b/packages/ubnt_unifi/data_stream/logs/elasticsearch/ingest_pipeline/cef.yml new file mode 100644 index 00000000000..9615c00d12e --- /dev/null +++ b/packages/ubnt_unifi/data_stream/logs/elasticsearch/ingest_pipeline/cef.yml @@ -0,0 +1,320 @@ +--- +description: Pipeline for processing Ubiquiti UniFi "CEF" logs +processors: + +- set: + field: log.level + copy_from: cef.severity + ignore_empty_value: true + override: false + tag: set_log_level_from_cef_severity + +######################### +### Event Information ### +######################### + +- script: + description: "High level default assignments based on subCategory" + if: ctx.cef?.extensions?.UNIFIsubCategory != null + lang: painless + params: + "Admin": {"kind": "event", "type": ["info"], "category": ["host","network"]} + "Intrusion Prevention": {"kind": "alert", "type": ["info"], "category": ["network","intrusion_detection"]} + "Honeypot": {"kind": "alert", "type": ["access","info"], "category": ["network","intrusion_detection"]} + "WiFi": {"kind": "event", "type": ["info"], "category": ["network"]} + "Wired": {"kind": "event", "type": ["info"], "category": ["network"]} + "Devices": {"kind": "event", "type": ["info"], "category": ["network"]} + source: > + def schemaId = ctx.cef.extensions.UNIFIsubCategory.toString(); + def schema = params[schemaId]; + if (schema != null) { + if (ctx.event == null) { + ctx.event = new HashMap(); + } + ctx.event.kind = schema.kind; + ctx.event.type = schema.type; + ctx.event.category = schema.category; + } + tag: set_event_kind_type_category_script_from_cef_unifisubcategory + +- set: + field: event.action + copy_from: cef.name + ignore_empty_value: true + +- script: + description: "More specific assignments based on action description" + if: ctx.event?.action != null + lang: painless + params: + "Admin Accessed UniFi Network": {"type": ["access","info"], "category": ["host","network"]} + "Admin Made Config Changes": {"type": ["change","info"], "category": ["host","configuration","network"]} + "Admin Created New Config": {"type": ["change","info"], "category": ["host","configuration","network"]} + "Admin Removed Config": {"type": ["change","info"], "category": ["host","configuration","network"]} + "WiFi Client Connected": {"type": ["access","connection","start","info"], "category": ["network"]} + "WiFi Client Disconnected": {"type": ["access","connection","end","info"], "category": ["network"]} + "WiFi Client Roamed": {"type": ["access","connection","change","info"], "category": ["network"]} + "AP Channel Change": {"type": ["change","info"], "category": ["network"]} + "Device Offline": {"type": ["info"], "category": ["network"]} + "Device Reconnected": {"type": ["access","info"], "category": ["network"]} + "Device Updated": {"type": ["change","info"], "category": ["network"]} + "Multiple Devices Reconnected": {"type": ["connection","info"], "category": ["network"]} + "Wired Client Connected": {"type": ["connection","start","info"], "category": ["network"]} + "Wired Client Disconnected": {"type": ["connection","end","info"], "category": ["network"]} + "VPN Client Connected": {"type": ["connection","start","info"], "category": ["network"]} + "VPN Client Disconnected": {"type": ["connection","end","info"], "category": ["network"]} + "Threat Detected and Blocked": {"type": ["denied","info"], "category": ["network","intrusion_detection"]} + "Honeypot Triggered": {"type": ["access","info"], "category": ["network","intrusion_detection"]} + source: > + def schemaId = ctx.event.action.toString(); + def schema = params[schemaId]; + if (schema != null) { + if (ctx.event == null) { + ctx.event = new HashMap(); + } + ctx.event.type = schema.type; + ctx.event.category = schema.category; + } + tag: set_event_type_category_script_from_cef_event_action + +# if there's changes of some kind adjust event.type to include "change" instead of "access" +# this may have been superceded by the above, but I'm not sure I've captured all common cef.name/event.action's as yet... +- set: + if: ctx.cef?.extensions?.UNIFIsettingsChanges != null && ctx.cef.extensions.UNIFIsettingsChanges != "" + field: event.type + value: ["change","info"] + tag: set_event_type_to_change_info_from_cef_unifisettingschanges + +- set: + if: (ctx.cef?.extensions?.UNIFIsubCategory == "Honeypot" || ctx.cef?.extensions?.UNIFIsubCategory == "Intrusion Prevention") && ctx.cef.extensions.UNIFIipsSignature != null + field: event.reason + value: "{{{cef.extensions.UNIFIipsSignature}}}" + tag: set_event_reason_from_cef_unifisignature + +- script: + if: ctx.cef?.extensions?.UNIFIrisk != null + lang: painless + params: + "low": {"static_level": "Low", "static_score": 25.0, "static_score_norm": 25.0} + "medium": {"static_level": "Medium", "static_score": 50.0, "static_score_norm": 50.0} + "high": {"static_level": "High", "static_score": 75.0, "static_score_norm": 75.0} + source: > + def schemaId = ctx.cef.extensions.UNIFIrisk.toString(); + def schema = params[schemaId]; + if (schema != null) { + ctx.risk = schema; + } + tag: set_risk_from_cef_unifirisk + +- set: + field: event.url + copy_from: cef.extensions.UNIFIreference + ignore_empty_value: true + override: false + tag: set_event_url_from_cef_unifireference + +# fix taken from existing cef integration. May not be relevant to Ubiquiti UniFi? +- convert: + field: event.id + ignore_missing: true + tag: convert event.id + type: string + +######################### +### Host Information ### +######################### + +- set: + field: host.hostname + copy_from: cef.extensions.UNIFIhost + ignore_empty_value: true + override: false + tag: set_host_hostname_from_cef_unifihost + +######################## +### User Information ### +######################## + +- set: + field: user.full_name + copy_from: cef.extensions.UNIFIadmin + ignore_empty_value: true + override: false + tag: set_user_full_name_from_cef_unifiadmin + +########################## +### Client/Server Information ### +########################## + +# The situations in which these fields may be set are highly variable :-( +# cef.extensions.UNIFIclient* + cef.extensions.UNIFIdevice* == client -> server +# cef.extensions.UNIFIclient* + cef.extensions.UNIFIconnectedToDevice* == client -> server +# cef.extensions.UNIFIdevice* + cef.extensions.UNIFIconnectedToDevice* == client -> server + +- set: + field: client.address + copy_from: cef.extensions.UNIFIclientIp + ignore_empty_value: true + override: false + tag: set_client_address_from_cef_unificlientip + +- set: + field: client.mac + copy_from: cef.extensions.UNIFIclientMac + ignore_empty_value: true + override: false + tag: set_client_mac_from_cef_unificlientmac + +- set: + field: client.domain + copy_from: cef.extensions.UNIFIclientHostname + ignore_empty_value: true + override: false + tag: set_client_domain_from_cef_unificlienthostname + +- set: + if: ctx.cef?.extensions?.UNIFIclientIp == null && ctx.cef?.extensions?.UNIFIconnectedToDeviceIp != null + field: client.address + copy_from: cef.extensions.UNIFIdeviceIp + ignore_empty_value: true + override: false + tag: set_client_address_from_cef_unifideviceip + +- set: + if: ctx.cef?.extensions?.UNIFIclientMac == null && ctx.cef?.extensions?.UNIFIconnectedToDeviceMac != null + field: client.mac + copy_from: cef.extensions.UNIFIdeviceMac + ignore_empty_value: true + override: false + tag: set_client_mac_from_cef_unifidevicemac + +- set: + if: ctx.cef?.extensions?.UNIFIclientHostname == null && ctx.cef?.extensions?.UNIFIconnectedToDeviceName != null + field: client.domain + copy_from: cef.extensions.UNIFIdeviceName + ignore_empty_value: true + override: false + tag: set_client_domain_from_cef_unifidevicename + +- set: + if: ctx.cef?.extensions?.UNIFIclientIp != null && ctx.cef?.extensions?.UNIFIconnectedToDeviceIp == null + field: server.address + copy_from: cef.extensions.UNIFIdeviceIp + ignore_empty_value: true + override: false + tag: set_server_address_from_cef_unifideviceip + +- set: + if: ctx.cef?.extensions?.UNIFIclientMac != null && ctx.cef?.extensions?.UNIFIconnectedToDeviceMac == null + field: server.mac + copy_from: cef.extensions.UNIFIdeviceMac + ignore_empty_value: true + override: false + tag: set_server_mac_from_cef_unifidevicemac + +- set: + if: ctx.cef?.extensions?.UNIFIclientHostname != null && ctx.cef?.extensions?.UNIFIconnectedToDeviceName == null + field: server.domain + copy_from: cef.extensions.UNIFIdeviceName + ignore_empty_value: true + override: false + tag: set_server_domain_from_cef_unifidevicename + +- set: + field: server.address + copy_from: cef.extensions.UNIFIconnectedToDeviceIp + ignore_empty_value: true + override: false + tag: set_server_address_from_cef_unificonnectedtodeviceip + +- set: + field: server.mac + copy_from: cef.extensions.UNIFIconnectedToDeviceMac + ignore_empty_value: true + override: false + tag: set_server_mac_from_cef_unificonnectedtodevicemac + +- set: + field: server.domain + copy_from: cef.extensions.UNIFIconnectedToDeviceName + ignore_empty_value: true + override: false + tag: set_server_domain_from_cef_unificonnectedtodevicename + +########################### +### Source IP Inference ### +########################### + +- set: + field: source.address + copy_from: cef.extensions.admin_ip + ignore_empty_value: true + override: false + tag: set_source_address_from_cef_admin_ip + +################################ +### Destination IP Inference ### +################################ + +- convert: + description: "Use log.syslog.hostname as destination.ip, if source.ip already exists, as in some cases UniFi is logging an event about itself and source, but it doesn't provide itself as a destination yet includes it's IP in a way that winds up in log.syslog.hostname. Ignore failures in case it's not IP and rather a hostname." + if: ctx.source?.address != null || ctx.source?.ip != null + field: log.syslog.hostname + target_field: destination.address + type: ip + ignore_missing: true + ignore_failure: true + +############### +### Cleanup ### +############### + +- remove: + if: ctx?.tags == null || !(ctx.tags.contains('preserve_duplicate_custom_fields')) + description: "Remove ECS mapped fields, duplicated data, and those that have no meaningful value to the events" + tag: remove_fields_if_not_preserve_duplicate_custom_fields + field: + - cef.device.event_class_id # event.code + - cef.device.product # observer.product + - cef.device.vendor # observer.vendor + - cef.device.version # observer.version + - cef.extensions.UNIFIadmin # user.full_name + - cef.extensions.UNIFIclientIp # client.address + - cef.extensions.UNIFIclientMac # client.mac + - cef.extensions.UNIFIclientHostname # client.domain + - cef.extensions.UNIFIconnectedToDeviceIp # server.address + - cef.extensions.UNIFIconnectedToDeviceMac # server.mac + - cef.extensions.UNIFIconnectedToDeviceName # server.domain + - cef.extensions.UNIFIdeviceIp # client.address + - cef.extensions.UNIFIdeviceMac # client.mac + - cef.extensions.UNIFIdeviceName # client.domain + - cef.extensions.UNIFIhost # host.hostname + - cef.extensions.UNIFIreference # event.url + - cef.extensions.UNIFIrisk # risk.level + - cef.extensions.admin_ip # source.address + - cef.extensions.message # message + - cef.extensions.signature_type # rule.ruleset + - cef.extensions.signature_id # rule.id + - cef.extensions.sourceAddress # source.address + - cef.extensions.sourceUserName # source.user.name + - cef.name # event.action + - cef.severity # log.level / event.severity + - cef.version # not useful + ignore_missing: true + +###################### +## Failure Handling ## +###################### + +on_failure: + - remove: + field: + - _tmp + ignore_missing: true + - append: + field: error.message + value: |- + Processor "{{{ _ingest.on_failure_processor_type }}}" with tag "{{{ _ingest.on_failure_processor_tag }}}" in pipeline "{{{ _ingest.on_failure_pipeline }}}" failed with message "{{{ _ingest.on_failure_message }}}" + - set: + field: event.kind + value: pipeline_error diff --git a/packages/ubnt_unifi/data_stream/logs/elasticsearch/ingest_pipeline/default.yml b/packages/ubnt_unifi/data_stream/logs/elasticsearch/ingest_pipeline/default.yml new file mode 100644 index 00000000000..07277e9d968 --- /dev/null +++ b/packages/ubnt_unifi/data_stream/logs/elasticsearch/ingest_pipeline/default.yml @@ -0,0 +1,428 @@ +--- +description: Pipeline for processing Ubiquiti Unifi Network logs +processors: +- set: + field: ecs.version + value: '8.17.0' + +- set: + field: event.type + value: [info] + +- set: + field: event.kind + value: event + +### Additional Observer Fields ### + +- set: + if: ctx.observer?.vendor == null + field: observer.vendor + value: Ubiquiti + override: false + +- set: + if: ctx.observer?.product == null + field: observer.product + value: UniFi + override: false + +- set: + if: ctx.observer?.hostname == null + field: observer.hostname + copy_from: log.syslog.hostname + ignore_empty_value: true + override: false + +### "CEF", as produced by various UniFi OS and UniFi application components... ### + +- pipeline: + if: ctx.tags != null && ctx.tags.contains("ubnt-unifi-cef") + description: "Pass the event to dedicated pipeline for processing CEF logs" + name: '{{ IngestPipeline "cef" }}' + +### iptables from UniFi routers... ### + +- pipeline: + if: ctx.tags != null && ctx.tags.contains("ubnt-unifi-iptables") + description: "Pass the event to dedicated pipeline for processing iptables logs" + name: '{{ IngestPipeline "iptables" }}' + +### other, e.g. *nix style generic syslog stuff that spews forth from Ubiquiti equipment... ### + +- pipeline: + if: ctx.tags == null || (!(ctx.tags.contains("ubnt-unifi-cef")) && !(ctx.tags.contains("ubnt-unifi-iptables"))) + description: "Pass the event to dedicated pipeline for processing all other log types" + name: '{{ IngestPipeline "other" }}' + +### common parsing that will(/may?) be applicable to some logs independent of type... ### + +- grok: + description: Parse Ubiquiti UniFi equipment serial number, model and version information if sent via syslog appname tag. + field: log.syslog.appname + ignore_missing: true + ignore_failure: true + patterns: + - '%{UBNT_UNIFI_SERIAL:observer.serial_number},%{DATA:observer.product}-%{UBNT_UNIFI_VERSION:observer.version}' + pattern_definitions: + UBNT_UNIFI_VERSION: '[0-9\.\+]{3,}' + UBNT_UNIFI_SERIAL: '[0-9a-f]{12,}' + +### fix MAC addresses ### + +# Format source.mac address. +- gsub: + field: source.mac + pattern: '[-:.]' + replacement: '' + ignore_missing: true +- gsub: + field: source.mac + pattern: '(..)(?!$)' + replacement: '$1-' + ignore_missing: true +- uppercase: + field: source.mac + ignore_missing: true + +# Format client.mac address. +- gsub: + field: client.mac + pattern: '[-:.]' + replacement: '' + ignore_missing: true +- gsub: + field: client.mac + pattern: '(..)(?!$)' + replacement: '$1-' + ignore_missing: true +- uppercase: + field: client.mac + ignore_missing: true + +# Format destination.mac address. +- gsub: + field: destination.mac + pattern: '[-:.]' + replacement: '' + ignore_missing: true +- gsub: + field: destination.mac + pattern: '(..)(?!$)' + replacement: '$1-' + ignore_missing: true +- uppercase: + field: destination.mac + ignore_missing: true + +# Format server.mac address. +- gsub: + field: server.mac + pattern: '[-:.]' + replacement: '' + ignore_missing: true +- gsub: + field: server.mac + pattern: '(..)(?!$)' + replacement: '$1-' + ignore_missing: true +- uppercase: + field: server.mac + ignore_missing: true + +### Observer info inference attempts if observer fields are still missing ### + +- grok: + description: "Extract observer.ip from log.source.address if available" + if: ctx.observer?.ip == null + field: log.source.address + patterns: + - '%{IP:_tmp.observer_ip}:%{POSINT}' + - '%{IP:_tmp.observer_ip}' + ignore_missing: true + ignore_failure: true + +- convert: + description: "Extract observer.ip from log.syslog.hostname if IP in place as syslog hostname" + if: ctx.observer?.ip == null + field: log.syslog.hostname + target_field: _tmp.observer_ip + type: ip + ignore_missing: true + ignore_failure: true + +- append: + if: ctx._tmp?.observer_ip != null + field: observer.ip + value: '{{{_tmp.observer_ip}}}' + allow_duplicates: false + media_type: text/plain + +- append: + if: ctx._tmp?.observer_ip != null + field: related.ip + value: '{{{_tmp.observer_ip}}}' + allow_duplicates: false + media_type: text/plain + +- set: + description: "Use syslog hostname as observer hostname if not already set and not an IP address" + if: ctx.observer?.hostname == null && ctx.log?.syslog?.hostname != ctx.observer?.ip + field: observer.hostname + copy_from: log.syslog.hostname + ignore_empty_value: true + +### Relateds ### + +- append: + if: ctx?.source?.ip != null && ctx?.source?.ip != '' + field: related.ip + allow_duplicates: false + value: '{{{source.ip}}}' + media_type: text/plain + +- append: + if: ctx?.source?.nat?.ip != null && ctx?.source?.nat?.ip != '' + field: related.ip + allow_duplicates: false + value: '{{{source.nat.ip}}}' + media_type: text/plain + +- append: + if: ctx?.destination?.ip != null && ctx?.destination?.ip != '' + field: related.ip + allow_duplicates: false + value: '{{{destination.ip}}}' + media_type: text/plain + +- append: + if: ctx?.destination?.nat?.ip != null && ctx?.destination?.nat?.ip != '' + field: related.ip + allow_duplicates: false + value: '{{{destination.nat.ip}}}' + media_type: text/plain + +- append: + if: ctx?.client?.ip != null && ctx?.client?.ip != '' + field: related.ip + allow_duplicates: false + value: '{{{client.ip}}}' + media_type: text/plain + +- append: + if: ctx?.server?.ip != null && ctx?.server?.ip != '' + field: related.ip + allow_duplicates: false + value: '{{{server.ip}}}' + media_type: text/plain + +- append: + if: ctx?.destination?.user?.name != null + field: related.user + value: '{{{destination.user.name}}}' + media_type: text/plain + +- append: + if: ctx?.source?.user?.name != null && ctx?.source?.user?.name != '' + field: related.user + allow_duplicates: false + value: '{{{source.user.name}}}' + media_type: text/plain + +- append: + if: ctx?.observer?.hostname != null && ctx?.observer?.hostname != '' + field: related.hosts + allow_duplicates: false + value: '{{{observer.hostname}}}' + media_type: text/plain + +- append: + if: ctx?.source?.domain != null && ctx?.source?.domain != '' + field: related.hosts + allow_duplicates: false + value: '{{{source.domain}}}' + media_type: text/plain + +- append: + if: ctx?.destination?.domain != null && ctx?.destination?.domain != '' + field: related.hosts + allow_duplicates: false + value: '{{{destination.domain}}}' + media_type: text/plain + +- append: + if: ctx?.client?.domain != null && ctx?.client?.domain != '' + field: related.hosts + allow_duplicates: false + value: '{{{client.domain}}}' + media_type: text/plain + +- append: + if: ctx?.server?.domain != null && ctx?.server?.domain != '' + field: related.hosts + allow_duplicates: false + value: '{{{server.domain}}}' + media_type: text/plain + +### IP Conversion from address to ip if successful, we assume other pipelines have set the .address field and left the conversion to us here ### + +- convert: + if: ctx.source?.ip == null + field: source.address + target_field: source.ip + type: ip + ignore_missing: true + +- convert: + if: ctx.destination?.ip == null + field: destination.address + target_field: destination.ip + type: ip + ignore_missing: true + +- convert: + if: ctx.client?.ip == null + field: client.address + target_field: client.ip + type: ip + ignore_missing: true + +- convert: + if: ctx.server?.ip == null + field: server.address + target_field: server.ip + type: ip + ignore_missing: true + +- set: + if: ctx.source?.address == null + field: source.address + value: '{{{source.ip}}}' + ignore_empty_value: true + +- set: + if: ctx.destination?.address == null + field: destination.address + value: '{{{destination.ip}}}' + ignore_empty_value: true + +- set: + if: ctx.client?.address == null + field: client.address + value: '{{{client.ip}}}' + ignore_empty_value: true + +- set: + if: ctx.server?.address == null + field: server.address + value: '{{{server.ip}}}' + ignore_empty_value: true + +### GeoIP Enrichment ### + +- geoip: + description: "Enrich event with source IP GeoIP data" + field: source.ip + target_field: source.geo + ignore_missing: true + +- geoip: + description: "Enrich event with source IP GeoIP ASN contextual data" + field: source.ip + target_field: source.as + database_file: GeoLite2-ASN.mmdb + properties: + - asn + - organization_name + ignore_missing: true + +- rename: + field: source.as.asn + target_field: source.as.number + ignore_missing: true + +- rename: + field: source.as.organization_name + target_field: source.as.organization.name + ignore_missing: true + +- geoip: + description: "Enrich event with source IP GeoIP data" + field: destination.ip + target_field: destination.geo + ignore_missing: true + +- geoip: + description: "Enrich event with destination IP GeoIP ASN contextual data" + field: destination.ip + target_field: destination.as + database_file: GeoLite2-ASN.mmdb + properties: + - asn + - organization_name + ignore_missing: true + +- rename: + field: destination.as.asn + target_field: destination.as.number + ignore_missing: true + +- rename: + field: destination.as.organization_name + target_field: destination.as.organization.name + ignore_missing: true + +######################################################################################################## +## Clean Up generic fields and values - the other pipelines *should* have cleaned up after themselves ## +######################################################################################################## + +- remove: + if: ctx?.tags == null || !(ctx.tags.contains('preserve_duplicate_custom_fields')) + description: "Remove ECS mapped fields, duplicated data, and those that have no meaningful value to the events" + tag: remove_fields_if_not_preserve_duplicate_custom_fields + field: + - event.original + ignore_failure: true + ignore_missing: true + +- script: + description: Drops null/empty values recursively to minimise event size + lang: painless + source: | + boolean dropEmptyFields(Object object) { + if (object == null || object == '') { + return true; + } else if (object instanceof Map) { + ((Map) object).values().removeIf(value -> dropEmptyFields(value)); + return (((Map) object).size() == 0); + } else if (object instanceof List) { + ((List) object).removeIf(value -> dropEmptyFields(value)); + return (((List) object).length == 0); + } + return false; + } + dropEmptyFields(ctx); + +- remove: + description: "Remove temporary and other unnecessary fields" + field: + - _tmp + - _config + ignore_missing: true + +###################### +## Failure Handling ## +###################### + +on_failure: + - remove: + field: + - _tmp + ignore_missing: true + - append: + field: error.message + value: |- + Processor "{{{ _ingest.on_failure_processor_type }}}" with tag "{{{ _ingest.on_failure_processor_tag }}}" in pipeline "{{{ _ingest.on_failure_pipeline }}}" failed with message "{{{ _ingest.on_failure_message }}}" + - set: + field: event.kind + value: pipeline_error diff --git a/packages/ubnt_unifi/data_stream/logs/elasticsearch/ingest_pipeline/iptables.yml b/packages/ubnt_unifi/data_stream/logs/elasticsearch/ingest_pipeline/iptables.yml new file mode 100644 index 00000000000..d20146feb3e --- /dev/null +++ b/packages/ubnt_unifi/data_stream/logs/elasticsearch/ingest_pipeline/iptables.yml @@ -0,0 +1,273 @@ +--- +description: Pipeline for processing Ubiquiti Unifi Network Firewall logs. Heavily based on the original iptables integration pipeline yet slightly different because Ubiquiti. +processors: + +# Examples, +# [VPN_LAN-D-10004] DESCR=\"Block VPN to Internal - Other\" IN=wgsrv1 OUT=br3 MAC= SRC=192.168.0.167 DST=192.168.1.2 LEN=64 TOS=00 PREC=0x00 TTL=253 ID=0 DF PROTO=TCP SPT=51523 DPT=80 SEQ=3925309497 ACK=0 WINDOW=65535 SYN URGP=0 MARK=1a0000 +# [LOCAL_CUSTOM2-A-2147483647] DESCR=\\\"[LOCAL_CUSTOM2]Allow All T\\\" IN= OUT=br999 MAC= SRC=192.168.0.1 DST=255.255.255.255 LEN=32 TOS=00 PREC=0x00 TTL=64 ID=5692 DF PROTO=UDP SPT=45148 DPT=10001 LEN=12 UID=0 GID=0 MARK=1a0000 +# [WAN_LAN-D-2147483647] DESCR=\\\"[WAN_LAN]Block All Traffic\\\" IN=eth7 OUT=br5 MAC=d0:21:f9:89:c2:4a:ea:04:fe:a2:00:82:86:dd:60:05:85:34 SRC=2a02:cf40:5b54:f902:3f0e:1126:568a:75d9 DST=2a02:cf40:5503:de01:3dba:d6d6:5949:f36e LEN=56 TC=0 HOPLIMIT=246 FLOWLBL=361780 PROTO=ICMPv6 TYPE=128 CODE=0 ID=1 SEQ=1 MARK=1a0000 +# [LAN_VPN-A-2147483647] DESCR=\"[LAN_VPN]Allow All Traffic\" IN=br2 OUT=wgsrv1 MAC=d0:21:f9:89:c2:4b:58:47:ca:7c:4d:01:08:00 SRC=192.168.1.32 DST=192.168.0.2 LEN=28 TOS=00 PREC=0x00 TTL=2 ID=60827 DF PROTO=ICMP TYPE=8 CODE=0 ID=50973 SEQ=60615 MARK=1a0000 + +- grok: + field: event.original + patterns: + - '%{GREEDYDATA}%{UBIQUITI_UNIFI_LOGS}%{GREEDYDATA}' + - '%{GREEDYDATA}%{IPTABLES}%{GREEDYDATA}' + - '%{GREEDYDATA}%{UBIQUITI_UNIFI_LOGS}' + - '%{GREEDYDATA}%{IPTABLES}' + pattern_definitions: + ECS_SYSLOG_PRI: '<%{NONNEGINT:log.syslog.priority>' + IPTABLES_HOSTNAME: '%{HOSTNAME:observer.name}%{SPACE}(%{NOTSPACE}%{SPACE})?kernel:' + IPTABLES_ACTION: '(:?%{WORD:event.action}:|%{IPTABLES_HOSTNAME}%{SPACE}iptables%{SPACE}%{WORD:event.action}|%{IPTABLES_HOSTNAME})' + UNSIGNED_INT: '[0-9]+' + ETHTYPE: (?:[A-Fa-f0-9]{2}):(?:[A-Fa-f0-9]{2}) + ETHTYPE_DISCARD: (?::[A-Fa-f0-9]{2})* + NETFILTERMAC: (?:%{MAC:destination.mac}:%{MAC:source.mac}:%{ETHTYPE:iptables.ether_type}?%{ETHTYPE_DISCARD}|%{MAC:destination.mac}%{ETHTYPE_DISCARD}:%{ETHTYPE:iptables.ether_type}|[^\s]*?) + IPTABLES_ETHERNET: '(?:IN=%{DATA:observer.ingress.interface.name})? (?:OUT=%{DATA:observer.egress.interface.name})? (?:MAC=%{NETFILTERMAC})?' + IPTABLES_PORT_PAIR: SPT=%{UNSIGNED_INT:source.port:int} DPT=%{UNSIGNED_INT:destination.port:int} + IPTABLES_TCP_FLAGS: (CWR |ECE |URG |ACK |PSH |RST |SYN |FIN )* + IPTABLES_TCP_SEQ: SEQ=%{UNSIGNED_INT:iptables.tcp.seq:long} ACK=%{UNSIGNED_INT:iptables.tcp.ack:long} + IPTABLES_TCP_URGP: URGP=%{DATA:iptables.tcp.urgp} + IPTABLES_TCP_RESERVED_BITS: RES=0x%{BASE16NUM:iptables.tcp_reserved_bits} + IPTABLES_TCP_WINDOW: WINDOW=%{UNSIGNED_INT:iptables.tcp.window:int} + IPTABLES_MARK: MARK=%{NOTSPACE:iptables.mark} + IPTABLES_TCP_DETAILS: (?:%{IPTABLES_TCP_SEQ} )?%{IPTABLES_TCP_WINDOW} (?:IPTABLES_TCP_RESERVED_BITS} )?%{IPTABLES_TCP_FLAGS:iptables.tcp.flags}(?:%{IPTABLES_TCP_URGP})? + IPTABLES_INCOMPLETE_PACKET: INCOMPLETE \[%{UNSIGNED_INT:iptables.incomplete_bytes:int} bytes\] + IPTABLES_UDP_DETAILS: (?:LEN=%{UNSIGNED_INT:iptables.udp.length:int})? + IPTABLES_ICMP_EXTRA_ECHO: ID=%{UNSIGNED_INT:iptables.icmp.id:int} SEQ=%{UNSIGNED_INT:iptables.icmp.seq:long} + IPTABLES_ICMP_EXTRA_PARAM: PARAMETER=%{UNSIGNED_INT:iptables.icmp.parameter:int} + IPTABLES_ICMP_EXTRA_REDIRECT: GATEWAY=%{IP:iptables.icmp.redirect} + IPTABLES_ICMP_EXTRA: ( (?:%{IPTABLES_ICMP_EXTRA_ECHO}|%{IPTABLES_ICMP_EXTRA_PARAM}|%{IPTABLES_ICMP_EXTRA_REDIRECT}))* + IPTABLES_ICMP_DETAILS: TYPE=%{UNSIGNED_INT:iptables.icmp.type:int} CODE=%{UNSIGNED_INT:iptables.icmp.code:int}((%{IPTABLES_INCOMPLETE_PACKET})|%{IPTABLES_ICMP_EXTRA}) + IPTABLES_COMMON_EXTRAS: ((UID=%{NOTSPACE:iptables.uid} )?(GID=%{NOTSPACE:iptables.gid} ))?(%{IPTABLES_MARK} )? + IPTABLES_PROTOCOL: PROTO=(?[a-zA-Z0-9]+) + IPTABLES_IP_PAYLOAD: '%{IPTABLES_PROTOCOL}( %{IPTABLES_PORT_PAIR})?( (%{IPTABLES_TCP_DETAILS}|%{IPTABLES_UDP_DETAILS}|%{IPTABLES_ICMP_DETAILS}|%{IPTABLES_INCOMPLETE_PACKET}) )?(?:%{IPTABLES_COMMON_EXTRAS})?' + IPTABLES_IP_FRAGFLAG: ((?<= )(CE|DF|MF))* + IPTABLES_IP_START: 'SRC=%{IPV4:source.ip} DST=%{IPV4:destination.ip} LEN=%{UNSIGNED_INT:iptables.length:int} TOS=(?:0x)?%{BASE16NUM:iptables.tos} PREC=0x%{BASE16NUM:iptables.precedence_bits} TTL=%{UNSIGNED_INT:iptables.ttl:int} ID=%{UNSIGNED_INT:iptables.id:int}(?: %{IPTABLES_IP_FRAGFLAG:iptables.fragment_flags})?(?: FRAG: %{UNSIGNED_INT:iptables.fragment_offset:int})?' + IPTABLES_IP: '%{IPTABLES_IP_START} %{IPTABLES_IP_PAYLOAD}' + IPTABLES_IPV6_START: SRC=%{IPV6:source.ip} DST=%{IPV6:destination.ip} LEN=%{UNSIGNED_INT:iptables.length:int} TC=%{UNSIGNED_INT:iptables.tos} HOPLIMIT=%{UNSIGNED_INT:iptables.ttl:int} FLOWLBL=%{UNSIGNED_INT:iptables.flow_label:int} + IPTABLES_IPV6: '%{IPTABLES_IPV6_START} %{IPTABLES_IP_PAYLOAD}' + IPTABLES: '%{IPTABLES_ETHERNET} (:?%{IPTABLES_IP}|%{IPTABLES_IPV6})' + UBIQUITI_FIELD: '[^-\]]*' + UBIQUITI_RULESET_NAME: '[^\]]*' + UBIQUITI_UNIFI_RULE_NAME: '%{UBIQUITI_RULESET_NAME:iptables.ubiquiti.rule_set}-%{UBIQUITI_FIELD:event.action}-%{UBIQUITI_FIELD:iptables.ubiquiti.rule_number}' + UBIQUITI_UNIFI_LOGS: '\[%{UBIQUITI_UNIFI_RULE_NAME:iptables.ubiquiti.rule_name}\]%{SPACE}DESCR=\\*"*%{DATA:iptables.ubiquiti.rule_description}\\*"*%{SPACE}%{IPTABLES}' + +- grok: + field: iptables.ubiquiti.rule_set + ignore_missing: true + ignore_failure: true + patterns: + - '%{UBIQUITI_FIELD:observer.ingress.zone}-%{UBIQUITI_FIELD:observer.egress.zone}' + - '%{UBIQUITI_FIELD:observer.ingress.zone}_%{UBIQUITI_FIELD:observer.egress.zone}' + pattern_definitions: + UBIQUITI_FIELD: '[^-]*' + +- rename: + description: Rename network.transport to network.iana_number if it is a number. + if: ctx.network?.iana_number == null && ctx.network.transport != null && ctx.network.transport.chars().allMatch(Character::isDigit) + field: network.transport + target_field: network.iana_number + ignore_missing: true + +- lowercase: + field: network.transport + ignore_missing: true + +- lowercase: + field: event.action + ignore_missing: true + +- script: + description: Enrich event with ECS fields. + lang: painless + params: + mappings: + - source: + object: iptables + key: ether_type + destination: + object: network + key: type + map: + 08:00: ipv4 + 86:dd: ipv6 + - source: + object: event + key: action + destination: + object: event + key: action + map: + d: drop + a: accept + r: reject + - source: + object: event + key: action + destination: + object: event + key: type + map: + drop: denied + accept: allowed + deny: denied + drop_input: denied + reject: denied + - source: + object: network + key: transport + destination: + object: network + key: transport + map: + icmpv6: ipv6-icmp + source: >- + for (action in params.mappings) { + def src = ctx[action.source.object]; + if (src != null) { + Map map = action.map; + String key = src[action.source.key]; + String mapping = map[key]; + if (mapping != null) { + Map dst = ctx[action.destination.object]; + if (dst == null) { + dst = new HashMap(); + ctx[action.destination.object] = dst; + } + dst[action.destination.key] = mapping; + } + } + } + +- community_id: + ignore_missing: true + ignore_failure: true + icmp_type: iptables.icmp.type + icmp_code: iptables.icmp.code + +- script: + description: Convert bit fields to numbers. + lang: painless + params: + hex_fields_to_convert: + - ether_type + - tos + - precedence_bits + - tcp_reserved_bits + source: >- + def iptables = ctx['iptables']; + if (iptables != null) { + for (key in params.hex_fields_to_convert) { + long value = 0; + def field = iptables[key]; + if (field == null) continue; + char[] hex = field.toLowerCase().toCharArray(); + for (chr in hex) { + long v = -1; + if (chr >= (char) 'a' && chr <= (char) 'f') v = (long) chr - (char) 'a' + 10; + else if (chr >= (char) '0' && chr <= (char) '9') v = (long) chr - (char) '0'; + if (v >= 0) { + value = value * 16 + v; + } + iptables[key] = value; + } + } + } +- set: + field: event.kind + value: event + +- append: + field: event.category + value: network + +- append: + field: event.type + value: connection + if: ctx?.source?.ip != null && ctx?.destination?.ip != null + +- rename: + field: iptables.tcp_reserved_bits + target_field: iptables.tcp.reserved_bits + ignore_missing: true + +- split: + field: iptables.tcp.flags + separator: "\\s+" + ignore_missing: true + +- split: + field: iptables.fragment_flags + separator: "\\s+" + ignore_missing: true + +- set: + if: ctx?.iptables?.ubiquiti?.rule_number != null + field: rule.id + copy_from: iptables.ubiquiti.rule_number + +- set: + if: ctx?.iptables?.ubiquiti?.rule_set != null + field: rule.ruleset + copy_from: iptables.ubiquiti.rule_set + +- set: + if: ctx?.iptables?.ubiquiti?.rule_name != null + field: rule.name + copy_from: iptables.ubiquiti.rule_name + +- set: + if: ctx?.iptables?.ubiquiti?.rule_description != null + field: rule.description + copy_from: iptables.ubiquiti.rule_description + +- network_direction: + description: Use default internal networks to determine network direction + if: ctx._config?.internal_networks == null + tag: network_direction + ignore_missing: true + internal_networks: + - loopback + - private + - unspecified + +- network_direction: + description: Use supplied field of internal networks to determine network direction + if: ctx._config?.internal_networks != null + tag: network_direction + ignore_missing: true + internal_networks_field: _config.internal_networks + +######################################### +## Clean Up "iptables" specific fields ## +######################################### + +- remove: + if: ctx?.tags == null || !(ctx.tags.contains('preserve_duplicate_custom_fields')) + description: "Remove ECS mapped fields, duplicated data, and those that have no meaningful value to the events" + tag: remove_fields_if_not_preserve_duplicate_custom_fields + field: + - message # parsed to various fields if we arrived here successfully... + - iptables.ubiquiti.rule_number # rule.id + - iptables.ubiquiti.rule_set # rule.ruleset + - iptables.ubiquiti.rule_name # rule.name + - iptables.ubiquiti.rule_description # rule.description + ignore_missing: true + +###################### +## Failure Handling ## +###################### + +on_failure: + - remove: + field: + - _tmp + ignore_missing: true + - append: + field: error.message + value: |- + Processor "{{{ _ingest.on_failure_processor_type }}}" with tag "{{{ _ingest.on_failure_processor_tag }}}" in pipeline "{{{ _ingest.on_failure_pipeline }}}" failed with message "{{{ _ingest.on_failure_message }}}" + - set: + field: event.kind + value: pipeline_error diff --git a/packages/ubnt_unifi/data_stream/logs/elasticsearch/ingest_pipeline/other.yml b/packages/ubnt_unifi/data_stream/logs/elasticsearch/ingest_pipeline/other.yml new file mode 100644 index 00000000000..6e0f9b83b9c --- /dev/null +++ b/packages/ubnt_unifi/data_stream/logs/elasticsearch/ingest_pipeline/other.yml @@ -0,0 +1,110 @@ +--- +description: Pipeline for processing Ubiquiti Unifi Network "other" logs +processors: + +- append: + description: Add a tag to the event so we can identify it as a UniFi "other" log + field: tags + value: ubnt-unifi-other + allow_duplicates: false + +- grok: + description: Grok fields from more generic syslog messages so we can try and do more useful things with them later on + if: ctx?.tags == null || !(ctx.tags.contains('ubnt-unifi-cef') || ctx.tags.contains('ubnt-unifi-iptables')) + field: message + ignore_missing: true + ignore_failure: true + patterns: + - 'stahtd: %{PROCESS_NAME:process.name}\[%{POSINT:process.pid:int}\]: \[STA-TRACKER\].stahtd_dump_event\(\): %{GREEDYDATA:message}' + - 'mcad: %{PROCESS_NAME:process.name}\[%{POSINT:process.pid:int}\]: wireless_agg_stats.log_sta_anomalies\(\): %{GREEDYDATA:message}' + - '%{PROCESS_NAME:process.name}\[%{POSINT:process.pid:int}\]: %{DATA:ubnt.unifi.linkcheck.function}\(\): resultUrl: %{GREEDYDATA:ubnt.unifi.linkcheck.resultUrl}' + - '%{PROCESS_NAME:process.name}\[%{POSINT:process.pid:int}\]: %{DATA:ubnt.unifi.linkcheck.function}\(\): Completed: Downlink %{NUMBER:ubnt.unifi.linkcheck.downlink.speed:float} %{WORD:ubnt.unifi.linkcheck.downlink.rate}, Uplink %{NUMBER:ubnt.unifi.linkcheck.uplink.speed:float} %{WORD:ubnt.unifi.linkcheck.uplink.rate}' + - '%{PROCESS_NAME:process.name}\[%{POSINT:process.pid:int}\]: (?speedtest\.ui_speedtest_log_results)\(\): (?(.|\r|\n)*)' + - '%{PROCESS_NAME:process.name}\[%{POSINT:process.pid:int}\]: mem avail:%{SPACE}%{POSINT:ubnt.unifi.earlyoom.memory.used:int}%{SPACE}of%{SPACE}%{POSINT:ubnt.unifi.earlyoom.memory.total:int}%{SPACE}MiB%{SPACE}\(%{NUMBER:ubnt.unifi.earlyoom.memory.used_pct:float}%\),%{SPACE}swap free:%{SPACE}%{POSINT:ubnt.unifi.earlyoom.swap.used:int}%{SPACE}of%{SPACE}%{POSINT:ubnt.unifi.earlyoom.swap.total:int}%{SPACE}MiB%{SPACE}\(%{NUMBER:ubnt.unifi.earlyoom.swap.used_pct:float}%\)%{GREEDYDATA}' + - '%{PROCESS_NAME:process.name}\[%{POSINT:process.pid:int}\]: DHCP%{DATA:ubnt.unifi.dhcp.message}\(%{DATA:ubnt.unifi.dhcp.interface}\) %{IP:ubnt.unifi.dhcp.ip} %{MAC:ubnt.unifi.dhcp.mac} %{GREEDYDATA:ubnt.unifi.dhcp.name}' + - '%{PROCESS_NAME:process.name}\[%{POSINT:process.pid:int}\]: DHCP%{DATA:ubnt.unifi.dhcp.message}\(%{DATA:ubnt.unifi.dhcp.interface}\) %{IP:ubnt.unifi.dhcp.ip} %{MAC:ubnt.unifi.dhcp.mac}' + - '%{PROCESS_NAME:process.parent.name}\[%{POSINT:process.parent.pid:int}\]: %{PROCESS_NAME:process.name}\[%{POSINT:process.pid:int}\]: %{GREEDYDATA:message}' + - '%{PROCESS_NAME:process.parent.name}\[%{POSINT:process.parent.pid:int}\]: %{PROCESS_NAME:process.name}: %{GREEDYDATA:message}' + - '%{PROCESS_NAME:process.name}\[%{POSINT:process.pid:int}\]: %{GREEDYDATA:message}' + - '%{PROCESS_NAME:process.parent.name}: %{PROCESS_NAME:process.name}\[%{POSINT:process.pid:int}\]: %{GREEDYDATA:message}' + - '%{PROCESS_NAME:process.parent.name}: %{PROCESS_NAME:process.name}: %{GREEDYDATA:message}' + - '%{PROCESS_NAME:process.name}: %{GREEDYDATA:message}' + pattern_definitions: + PROCESS_NAME: "[^:\\[\\]\ ]+" + +- json: + description: Attempt to parse JSON format stahtd event dump + if: ctx.process?.name == "stahtd" && ctx.message != null + field: message + target_field: ubnt.unifi.stahtd.dump + ignore_failure: true + +- kv: + description: Attempt to parse KVP mcad wireless_agg_stats.log_sta_anomalies output + if: ctx.process?.name == "mcad" && ctx.message != null + field: message + ignore_missing: true + field_split: ' ' + value_split: '=' + trim_key: '\s\t' + trim_value: '\s\t' + target_field: ubnt.unifi.mcad.wireless_agg_stats.log_sta_anomalies + ignore_failure: true + +- json: + description: Attempt to parse JSON format linkcheck output + if: ctx.process?.name == "linkcheck" && ctx.message != null + field: message + target_field: ubnt.unifi.linkcheck + ignore_failure: true + +###################################### +## Clean Up "other" specific fields ## +###################################### + +- remove: + description: Remove JSON text message if we just parsed it successfully + if: (ctx?.tags == null || !(ctx.tags.contains('preserve_duplicate_custom_fields'))) && ctx.process?.name == "stahtd" && ctx.ubnt?.unifi?.stahtd?.dump != null + field: message + ignore_missing: true + +- remove: + description: Remove JSON text message if we just parsed it successfully + if: (ctx?.tags == null || !(ctx.tags.contains('preserve_duplicate_custom_fields'))) && ctx.process?.name == "mcad" && ctx.ubnt?.unifi?.mcad?.wireless_agg_stats?.log_sta_anomalies != null + field: message + ignore_missing: true + +- remove: + description: Remove JSON text message if we just parsed it successfully + if: (ctx?.tags == null || !(ctx.tags.contains('preserve_duplicate_custom_fields'))) && ctx.process?.name == "linkcheck" && ctx.ubnt?.unifi?.linkcheck != null + field: message + ignore_missing: true + +- remove: + description: Remove message if we extracted the earlyoom values + if: (ctx?.tags == null || !(ctx.tags.contains('preserve_duplicate_custom_fields'))) && ctx.process?.name == "earlyoom" && ctx.ubnt?.unifi?.earlyoom != null + field: message + ignore_missing: true + +- remove: + description: Remove message if we extracted DHCP info + if: (ctx?.tags == null || !(ctx.tags.contains('preserve_duplicate_custom_fields'))) && ctx.process?.name == "dnsmasq-dhcp" && ctx.ubnt?.unifi?.dhcp != null + field: message + ignore_missing: true + +###################### +## Failure Handling ## +###################### + +on_failure: + - remove: + field: + - _tmp + ignore_missing: true + - append: + field: error.message + value: |- + Processor "{{{ _ingest.on_failure_processor_type }}}" with tag "{{{ _ingest.on_failure_processor_tag }}}" in pipeline "{{{ _ingest.on_failure_pipeline }}}" failed with message "{{{ _ingest.on_failure_message }}}" + - set: + field: event.kind + value: pipeline_error diff --git a/packages/ubnt_unifi/data_stream/logs/fields/agent.yml b/packages/ubnt_unifi/data_stream/logs/fields/agent.yml new file mode 100644 index 00000000000..fe1ab7f7bb4 --- /dev/null +++ b/packages/ubnt_unifi/data_stream/logs/fields/agent.yml @@ -0,0 +1,18 @@ +- name: input.type + type: keyword + description: Input type. +- name: log.file + type: group + fields: + - name: device_id + type: keyword + description: ID of the device containing the filesystem where the file resides. + - name: fingerprint + type: keyword + description: The sha256 fingerprint identity of the file when fingerprinting is enabled. + - name: inode + type: keyword + description: Inode number of the log file. +- name: log.offset + type: long + description: Offset of the entry in the log file. diff --git a/packages/ubnt_unifi/data_stream/logs/fields/base-fields.yml b/packages/ubnt_unifi/data_stream/logs/fields/base-fields.yml new file mode 100644 index 00000000000..db64cf2be0f --- /dev/null +++ b/packages/ubnt_unifi/data_stream/logs/fields/base-fields.yml @@ -0,0 +1,14 @@ +- name: data_stream.type + type: constant_keyword + description: Data stream type. +- name: data_stream.dataset + type: constant_keyword + description: Data stream dataset. +- name: data_stream.namespace + type: constant_keyword + description: Data stream namespace. +- name: '@timestamp' + type: date + description: Event timestamp. +- name: log.source.address + type: keyword diff --git a/packages/ubnt_unifi/data_stream/logs/fields/fields.yml b/packages/ubnt_unifi/data_stream/logs/fields/fields.yml new file mode 100644 index 00000000000..a860b237d5f --- /dev/null +++ b/packages/ubnt_unifi/data_stream/logs/fields/fields.yml @@ -0,0 +1,363 @@ +- name: cef + type: group + fields: + - name: device + type: group + fields: + - name: event_class_id + type: keyword + - name: product + type: keyword + - name: vendor + type: keyword + - name: version + type: keyword + - name: extensions + type: group + fields: + - name: baseEventCount + type: long + - name: destinationAddress + type: keyword + - name: destinationPort + type: integer + - name: fixed_ap_enabled + type: keyword + - name: fixed_ip + type: keyword + - name: local_dns_record_enabled + type: keyword + - name: message + type: keyword + - name: name + type: keyword + - name: note + type: keyword + - name: sourceAddress + type: keyword + - name: sourcePort + type: integer + - name: sourceUserName + type: keyword + - name: transportProtocol + type: keyword + - name: UNIFIaccessMethod + type: keyword + - name: UNIFIadmin + type: keyword + - name: UNIFIauthMethod + type: keyword + - name: UNIFIcategory + type: keyword + - name: UNIFIclientAlias + type: keyword + - name: UNIFIclientHostname + type: keyword + - name: UNIFIclientIp + type: keyword + - name: UNIFIclientMac + type: keyword + - name: UNIFIconnectedToDeviceIp + type: keyword + - name: UNIFIconnectedToDeviceMac + type: keyword + - name: UNIFIconnectedToDeviceModel + type: keyword + - name: UNIFIconnectedToDeviceName + type: keyword + - name: UNIFIconnectedToDeviceVersion + type: keyword + - name: UNIFIdeviceIp + type: keyword + - name: UNIFIdeviceMac + type: keyword + - name: UNIFIdeviceModel + type: keyword + - name: UNIFIdeviceName + type: keyword + - name: UNIFIdeviceVersion + type: keyword + - name: UNIFIduration + type: keyword + - name: UNIFIhost + type: keyword + - name: UNIFIipsSessionId + type: keyword + - name: UNIFIipsSignature + type: keyword + - name: UNIFIipsSignatureId + type: keyword + - name: UNIFIlastConnectedToDeviceIp + type: keyword + - name: UNIFIlastConnectedToDeviceMac + type: keyword + - name: UNIFIlastConnectedToDeviceModel + type: keyword + - name: UNIFIlastConnectedToDeviceName + type: keyword + - name: UNIFIconnectedToDevicePort + type: keyword + - name: UNIFIlastConnectedToDeviceVersion + type: keyword + - name: UNIFIlastConnectedToWiFiRssi + type: keyword + - name: UNIFInetworkName + type: keyword + - name: UNIFInetworkSubnet + type: keyword + - name: UNIFInetworkVlan + type: keyword + - name: UNIFIreference + type: keyword + - name: UNIFIrisk + type: keyword + - name: UNIFIsettingsChanges + type: text + - name: UNIFIsettingsEntry + type: keyword + - name: UNIFIsettingsSection + type: keyword + - name: UNIFIsubCategory + type: keyword + - name: UNIFIusageDown + type: keyword + - name: UNIFIusageUp + type: keyword + - name: UNIFIwifiAirtimeUtilization + type: keyword + - name: UNIFIwifiBand + type: keyword + - name: UNIFIwifiChannel + type: keyword + - name: UNIFIwifiChannelWidth + type: keyword + - name: UNIFIwifiInterference + type: keyword + - name: UNIFIwifiName + type: keyword + - name: UNIFIWiFiRssi + type: keyword + - name: use_fixedip + type: keyword + - name: virtual_network_override_enabled + type: keyword + - name: name + type: keyword + - name: severity + type: keyword + - name: version + type: keyword +- name: ubnt + type: group + fields: + - name: unifi + type: group + fields: + - name: dhcp + type: group + fields: + - name: interface + type: keyword + description: The interface name associated with the DHCP event + example: br3 + - name: ip + type: keyword + description: The IP address associated to the source + example: 10.255.253.18 + - name: mac + type: keyword + description: The MAC address associated to the source + example: dc:62:79:54:f8:be + - name: message + type: keyword + description: The message associated to the DHCP event + example: REQUEST + - name: name + type: keyword + description: The name of the device associated to the DHCP event + example: L535 + - name: earlyoom + type: group + fields: + - name: memory + type: group + fields: + - name: total + type: integer + description: The total amount of memory available + example: 1024 + - name: used + type: integer + description: The amount of memory used + example: 512 + - name: used_pct + type: float + description: The percentage of memory used + example: 50.0 + - name: swap + type: group + fields: + - name: total + type: integer + description: The total amount of swap available + example: 1024 + - name: used + type: integer + description: The amount of swap used + example: 512 + - name: used_pct + type: float + description: The percentage of swap used + example: 50.0 + - name: linkcheck + type: group + fields: + - name: city + type: keyword + description: The city associated with the link check + example: San Francisco + - name: country + type: keyword + description: The country associated with the link check + example: United States + - name: countryCode + type: keyword + description: The country code associated with the link check + example: US + - name: latitude + type: float + description: The latitude associated with the link check + example: 37.774929 + - name: longitude + type: float + description: The longitude associated with the link check + example: -122.419416 + - name: provider + type: keyword + description: The provider associated with the link check + example: Comcast + - name: providerUrl + type: keyword + description: The URL associated with the provider + example: https://www.comcast.com + - name: resultUrl + type: keyword + description: The WifiMan speed test result URL + example: https://wifiman.com/?result=UUID + - name: speedMbps + type: float + description: The speed in Mbps associated with the link check + example: 100.0 + - name: url + type: keyword + description: The URL associated with the link check + example: http://119.18.32.1:8069" + - name: downlink + type: group + fields: + - name: rate + type: keyword + description: The rate associated with the downlink + example: Mbps + - name: speed + type: float + description: The speed associated with the downlink + example: 100.0 + - name: uplink + type: group + fields: + - name: rate + type: keyword + description: The rate associated with the uplink + example: Mbps + - name: speed + type: float + description: The speed associated with the uplink + example: 100.0 + - name: function + type: keyword + description: The function associated with the link check + example: speedtest.ui_speedtest_log_results + - name: mcad + type: group + fields: + - name: wireless_agg_stats + type: group + fields: + - name: log_sta_anomalies + type: group + fields: + - name: anomalies + type: keyword + description: The anomalies associated with the wireless aggregate stats + example: "STA_ANOMALY_BSSID_CHANGE" + - name: bssid + type: keyword + description: The BSSID associated with the wireless aggregate stats + example: "00:11:22:33:44:55" + - name: radio + type: keyword + description: The radio associated with the wireless aggregate stats + example: "radio0" + - name: satisfaction_now + type: keyword + description: The satisfaction now associated with the wireless aggregate stats + example: 0.95 + - name: sta + type: keyword + description: The STA associated with the wireless aggregate stats + example: "00:11:22:33:44:55" + - name: vap + type: keyword + description: The VAP associated with the wireless aggregate stats + example: "vap0" + - name: stahtd + type: group + fields: + - name: dump + type: group + fields: + - name: arp_reply_gw_seen + type: keyword + - name: assoc_delta + type: keyword + - name: assoc_status + type: keyword + - name: auth_delta + type: keyword + - name: auth_failures + type: keyword + - name: auth_ts + type: keyword + - name: avg_rssi + type: keyword + - name: disassoc_reason + type: keyword + - name: dns_resp_seen + type: keyword + - name: dns_responses + type: keyword + - name: dns_timeouts + type: keyword + - name: event_id + type: keyword + - name: event_type + type: keyword + - name: ip_assign_type + type: keyword + - name: ip_delta + type: keyword + - name: mac + type: keyword + - name: message_type + type: keyword + - name: query_* + type: keyword + - name: sta_dc_reason + type: keyword + - name: traffic_delta + type: keyword + - name: vap + type: keyword + - name: wpa_auth_delta + type: keyword diff --git a/packages/ubnt_unifi/data_stream/logs/fields/iptables.yml b/packages/ubnt_unifi/data_stream/logs/fields/iptables.yml new file mode 100644 index 00000000000..d7a3761581a --- /dev/null +++ b/packages/ubnt_unifi/data_stream/logs/fields/iptables.yml @@ -0,0 +1,148 @@ +- name: iptables.ether_type + type: long + description: | + Value of the ethernet type field identifying the network layer protocol. +- name: iptables.flow_label + type: integer + description: | + IPv6 flow label. +- name: iptables.fragment_flags + type: keyword + description: | + IP fragment flags. A combination of CE, DF and MF. +- name: iptables.fragment_offset + type: long + description: | + Offset of the current IP fragment. +- name: iptables.gid + type: keyword + description: | + GID associated with the packet. + example: 0 +- name: iptables.icmp + type: group + fields: + - name: code + type: long + description: | + ICMP code. + - name: id + type: long + description: | + ICMP ID. + - name: parameter + type: long + description: | + ICMP parameter. + - name: redirect + type: ip + description: | + ICMP redirect address. + - name: seq + type: long + description: | + ICMP sequence number. + - name: type + type: long + description: | + ICMP type. +- name: iptables.id + type: long + description: | + Packet identifier. +- name: iptables.incomplete_bytes + type: long + description: | + Number of incomplete bytes. +- name: iptables.input_device + type: keyword + description: | + Device that received the packet. +- name: iptables.mark + type: keyword + description: | + MARK= +- name: iptables.precedence_bits + type: short + description: | + IP precedence bits. +- name: iptables.tos + type: long + description: | + IP Type of Service field. +- name: iptables.length + type: long + description: | + Packet length. +- name: iptables.output_device + type: keyword + description: | + Device that output the packet. +- name: iptables.tcp + type: group + fields: + - name: ack + type: long + description: | + TCP Acknowledgment number. + - name: flags + type: keyword + description: | + TCP flags. + - name: reserved_bits + type: short + description: | + TCP reserved bits. + - name: seq + type: long + description: | + TCP sequence number. + - name: urgp + type: keyword + description: | + URGP= + - name: window + type: long + description: | + Advertised TCP window size. +- name: iptables.ttl + type: integer + description: | + Time To Live field. +- name: iptables.udp + type: group + fields: + - name: length + type: long + description: | + Length of the UDP header and payload. +- name: iptables.ubiquiti + type: group + fields: + - name: rule_description + type: keyword + description: | + Description of the rule. + - name: rule_name + type: keyword + description: | + Name of the rule. + - name: input_zone + type: keyword + description: | + Input zone. + - name: output_zone + type: keyword + description: | + Output zone. + - name: rule_number + type: keyword + description: The rule number within the rule set. + - name: rule_set + type: keyword + description: The rule set name. +- name: iptables.uid + type: keyword + description: | + UID associated with the packet. + example: 0 diff --git a/packages/ubnt_unifi/data_stream/logs/manifest.yml b/packages/ubnt_unifi/data_stream/logs/manifest.yml new file mode 100644 index 00000000000..89b66b69727 --- /dev/null +++ b/packages/ubnt_unifi/data_stream/logs/manifest.yml @@ -0,0 +1,209 @@ +title: "Ubiquiti Unifi" +type: logs +streams: + - input: filestream + template_path: filestream.yml.hbs + title: Ubiquiti Unifi Log Files + description: Collect Ubiquiti Unifi logs via files + enabled: false + vars: + - name: paths + type: text + title: Paths + multi: true + required: true + show_user: true + default: + - /var/log/ubnt-*.log + - name: drop_noise + required: true + show_user: true + title: Drop generic + description: Drops generic events, e.g. any message that does not include CEF or iptables event content. + type: bool + multi: false + default: false + - name: drop_cef + required: true + show_user: true + title: Drop CEF + description: Drops events that include CEF format event content. + type: bool + multi: false + default: false + - name: drop_iptables + required: true + show_user: true + title: Drop iptables + description: Drops events that include iptables format event content. + type: bool + multi: false + default: false + - name: tags + type: text + title: Tags + description: Tags to include in the published event + required: false + multi: true + show_user: true + default: + - forwarded + - name: preserve_original_event + required: true + show_user: true + title: Preserve original event + description: Preserves a raw copy of the original event, added to the field `event.original`. + type: bool + multi: false + default: false + - name: preserve_duplicate_custom_fields + required: true + show_user: false + title: Preserve duplicate custom fields + description: Preserve custom fields for all ECS mappings. + type: bool + multi: false + default: false + - name: syslog_options + type: yaml + title: Syslog Configuration + description: i.e. field, format, time zone, etc. See [Syslog](https://www.elastic.co/guide/en/beats/filebeat/current/syslog.html) for details. + multi: false + required: false + show_user: false + default: | + field: event.original + overwrite_keys: false + ignore_failure: true + format: auto + #timezone: Local + - name: default_timezone + type: text + title: Default Timezone + description: tzdata format timezone to utilise when manual timezone override is needed. Only needed if some CEF events are sent in non-RFC syslog compliant format. + multi: false + required: false + show_user: false + default: UTC + - name: processors + type: yaml + title: Processors + multi: false + required: false + show_user: false + description: | + Processors are used to reduce the number of fields in the exported event or to enhance the event with metadata. This executes in the agent before the logs are parsed. See [Processors](https://www.elastic.co/guide/en/beats/filebeat/current/filtering-and-enhancing-data.html) for details. + - input: udp + template_path: udp-syslog.yml.hbs + title: Ubiquiti Unifi Syslog + description: Collect Ubiquiti Unifi logs via UDP syslog + enabled: true + vars: + - name: listen_address + type: text + title: Listen Address + description: | + Bind address for the listener. Use 0.0.0.0 to listen on all interfaces. + required: true + show_user: true + default: localhost + - name: listen_port + type: text + title: Listen port + description: | + Bind port for the listener. + required: true + show_user: true + default: 5514 + - name: drop_noise + required: true + show_user: true + title: Drop generic + description: Drops generic events, e.g. any message that does not include CEF or iptables event content. + type: bool + multi: false + default: false + - name: drop_cef + required: true + show_user: true + title: Drop CEF + description: Drops events that include CEF format event content. + type: bool + multi: false + default: false + - name: drop_iptables + required: true + show_user: true + title: Drop iptables + description: Drops events that include iptables format event content. + type: bool + multi: false + default: false + - name: tags + type: text + title: Tags + description: Tags to include in the published event + required: false + multi: true + show_user: true + default: + - forwarded + - name: preserve_original_event + required: true + show_user: true + title: Preserve original event + description: Preserves a raw copy of the original event, added to the field `event.original`. + type: bool + multi: false + default: false + - name: preserve_duplicate_custom_fields + required: true + show_user: false + title: Preserve duplicate custom fields + description: Preserve custom fields for all ECS mappings. + type: bool + multi: false + default: false + - name: udp_options + type: yaml + title: Custom UDP Options + multi: false + required: false + show_user: false + default: | + #read_buffer: 100MiB + #max_message_size: 50KiB + #timeout: 300s + - name: syslog_options + type: yaml + title: Syslog Configuration + description: i.e. field, format, time zone, etc. See [Syslog](https://www.elastic.co/guide/en/beats/filebeat/current/syslog.html) for details. + multi: false + required: false + show_user: false + default: | + field: event.original + overwrite_keys: false + ignore_failure: true + format: auto + #timezone: Local + - name: default_timezone + type: text + title: Default Timezone + description: tzdata format timezone to utilise when manual timezone override is needed. Only needed if some CEF events are sent in non-RFC syslog compliant format. + multi: false + required: false + show_user: false + default: UTC + - name: processors + type: yaml + title: Processors + multi: false + required: false + show_user: false + description: | + Processors are used to reduce the number of fields in the exported event or to enhance the event with metadata. This executes in the agent before the logs are parsed. See [Processors](https://www.elastic.co/guide/en/beats/filebeat/current/filtering-and-enhancing-data.html) for details. +elasticsearch: + index_template: + mappings: + subobjects: false diff --git a/packages/ubnt_unifi/data_stream/logs/sample_event.json b/packages/ubnt_unifi/data_stream/logs/sample_event.json new file mode 100644 index 00000000000..a91b4fbef94 --- /dev/null +++ b/packages/ubnt_unifi/data_stream/logs/sample_event.json @@ -0,0 +1,109 @@ +{ + "@timestamp": "2025-07-05T04:29:36.878Z", + "agent": { + "ephemeral_id": "11a9cafc-a45e-4ec2-b38f-e99536291b74", + "id": "131ef900-601b-4f5d-a1b8-6ed60bda2132", + "name": "ubnt-unifi-logs", + "type": "filebeat", + "version": "9.0.3" + }, + "cef": { + "device": { + "event_class_id": "201", + "product": "UniFi Network", + "vendor": "Ubiquiti", + "version": "9.3.33" + }, + "extensions": { + "UNIFIcategory": "Security", + "UNIFIdeviceIp": "192.168.0.1", + "UNIFIdeviceMac": "01:23:45:67:89:0a", + "UNIFIdeviceModel": "UniFi Dream Machine PRO SE", + "UNIFIdeviceName": "udm-pro-se", + "UNIFIdeviceVersion": "4.3.5", + "UNIFIhost": "udm-pro-se", + "UNIFIipsSessionId": "255132502100797", + "UNIFIipsSignature": "ET SCAN Possible Nmap User-Agent Observed", + "UNIFIipsSignatureId": "2024364", + "UNIFIrisk": "high", + "UNIFIsubCategory": "Intrusion Prevention", + "destinationAddress": "192.168.0.2", + "destinationPort": 8000, + "message": "A network intrusion attempt from 192.168.0.16 to 192.168.0.2 has been detected and blocked.", + "sourceAddress": "192.168.0.16", + "sourcePort": 60700, + "transportProtocol": "TCP" + }, + "name": "Threat Detected and Blocked", + "severity": "9", + "version": "0" + }, + "destination": { + "ip": "192.168.0.2", + "port": 8000 + }, + "ecs": { + "version": "8.17.0" + }, + "event": { + "action": "Threat Detected and Blocked", + "category": [ + "network", + "intrusion_detection" + ], + "code": "201", + "kind": "alert", + "original": "Jul 5 04:29:36 udm-pro-se.localnet 2025-07-05T04: 29:36.878Z udm-pro-se CEF:0|Ubiquiti|UniFi Network|9.3.33|201|Threat Detected and Blocked|9|proto=TCP src=192.168.0.16 spt=60700 dst=192.168.0.2 dpt=8000 UNIFIcategory=Security UNIFIsubCategory=Intrusion Prevention UNIFIhost=udm-pro-se UNIFIdeviceMac=01:23:45:67:89:0a UNIFIdeviceName=udm-pro-se UNIFIdeviceModel=UniFi Dream Machine PRO SE UNIFIdeviceIp=192.168.0.1 UNIFIdeviceVersion=4.3.5 UNIFIrisk=high UNIFIipsSessionId=255132502100797 UNIFIipsSignature=ET SCAN Possible Nmap User-Agent Observed UNIFIipsSignatureId=2024364 msg=A network intrusion attempt from 192.168.0.16 to 192.168.0.2 has been detected and blocked.", + "reason": "A network intrusion attempt from 192.168.0.16 to 192.168.0.2 has been detected and blocked.", + "severity": 9, + "type": [ + "info" + ] + }, + "host": { + "hostname": "udm-pro-se" + }, + "input": { + "type": "filestream" + }, + "log": { + "file": { + "device_id": "64773", + "fingerprint": "4971336f945694d052c56620d03d45fc31bdbba970ed13f50e2e10bfc2f46eeb", + "inode": "25962857", + "path": "/var/log/remote/udm-pro-se.log" + }, + "level": "9", + "offset": 298711, + "syslog": { + "hostname": "udm-pro-se" + } + }, + "message": "A network intrusion attempt from 192.168.0.16 to 192.168.0.2 has been detected and blocked.", + "network": { + "transport": "tcp" + }, + "observer": { + "hostname": "udm-pro-se.localnet", + "name": "udm-pro-se", + "product": "UniFi Network", + "vendor": "Ubiquiti", + "version": "9.3.33" + }, + "related": { + "ip": [ + "192.168.0.16", + "192.168.0.2" + ] + }, + "source": { + "ip": "192.168.0.16", + "port": 60700 + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields", + "forwarded", + "ubnt-unifi-cef" + ] +} \ No newline at end of file diff --git a/packages/ubnt_unifi/data_stream/webhooks/_dev/test/pipeline/test-logs-webhooks.json b/packages/ubnt_unifi/data_stream/webhooks/_dev/test/pipeline/test-logs-webhooks.json new file mode 100644 index 00000000000..c7ebb37224e --- /dev/null +++ b/packages/ubnt_unifi/data_stream/webhooks/_dev/test/pipeline/test-logs-webhooks.json @@ -0,0 +1,187 @@ +{ + "events": [ + { + "@timestamp": "2025-07-03T02:35:27.367Z", + "agent": { + "ephemeral_id": "b94ccb9f-4c9f-45db-a695-f8d8c7ff3127", + "id": "0f02a820-19e9-4f66-bd46-87f9b72fda41", + "name": "ec-cs-soc-exposed-agent", + "type": "filebeat", + "version": "9.0.3" + }, + "data_stream": { + "dataset": "ubnt_unifi.webhooks", + "namespace": "default", + "type": "logs" + }, + "ecs": { + "version": "8.0.0" + }, + "elastic_agent": { + "id": "0f02a820-19e9-4f66-bd46-87f9b72fda41", + "snapshot": false, + "version": "9.0.3" + }, + "event": { + "agent_id_status": "verified", + "dataset": "ubnt_unifi.webhooks", + "ingested": "2025-07-03T02:35:36Z", + "original": "{\"alarm\":{\"conditions\":[{\"condition\":{\"source\":\"person\",\"type\":\"is\"}},{\"condition\":{\"source\":\"vehicle\",\"type\":\"is\"}},{\"condition\":{\"source\":\"package\",\"type\":\"is\"}},{\"condition\":{\"source\":\"animal\",\"type\":\"is\"}}],\"eventLocalLink\":\"https://192.168.0.1/protect/events/event/6865ec6d025ca803e42878ca\",\"eventPath\":\"/protect/events/event/6865ec6d025ca803e42878ca\",\"name\":\"Elastic - Objects - All\",\"sources\":[{\"device\":\"28704E178669\",\"type\":\"include\"},{\"device\":\"F4E2C677123A\",\"type\":\"include\"},{\"device\":\"28704E17931C\",\"type\":\"include\"},{\"device\":\"28704E19BD42\",\"type\":\"include\"},{\"device\":\"28704E19C21C\",\"type\":\"include\"}],\"triggers\":[{\"device\":\"28704E17931C\",\"eventId\":\"6865ec6d025ca803e42878ca\",\"key\":\"person\",\"timestamp\":1751510125629,\"zones\":{\"line\":[],\"loiter\":[],\"zone\":[1]}}]},\"timestamp\":1751510126224}" + }, + "input": { + "type": "http_endpoint" + }, + "json": { + "alarm": { + "conditions": [ + { + "condition": { + "source": "person", + "type": "is" + } + }, + { + "condition": { + "source": "vehicle", + "type": "is" + } + }, + { + "condition": { + "source": "package", + "type": "is" + } + }, + { + "condition": { + "source": "animal", + "type": "is" + } + } + ], + "eventLocalLink": "https://192.168.0.1/protect/events/event/6865ec6d025ca803e42878ca", + "eventPath": "/protect/events/event/6865ec6d025ca803e42878ca", + "name": "Elastic - Objects - All", + "sources": [ + { + "type": "include", + "device": "28704E178669" + }, + { + "type": "include", + "device": "F4E2C677123A" + }, + { + "type": "include", + "device": "28704E17931C" + }, + { + "type": "include", + "device": "28704E19BD42" + }, + { + "type": "include", + "device": "28704E19C21C" + } + ], + "triggers": [ + { + "eventId": "6865ec6d025ca803e42878ca", + "zones": { + "loiter": [], + "zone": [ + 1 + ], + "line": [] + }, + "device": "28704E17931C", + "key": "person", + "timestamp": 1751510125629 + } + ] + }, + "timestamp": "2025-07-03T02:35:26.224Z" + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields", + "forwarded", + "ubnt-unifi-webhook" + ] + }, + { + "@timestamp": "2025-07-03T03:37:07.350Z", + "json": { + "events": [ + { + "id": "event.threat_detected", + "alert_key": "THREAT_BLOCKED_V3", + "alert_id": "6865fae1897bb377dc0fa3e7", + "scope": { + "site_id": "67762da9ac8a666067034163" + } + } + ] + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields", + "forwarded", + "ubnt-unifi-webhook" + ], + "input": { + "type": "http_endpoint" + }, + "ecs": { + "version": "8.0.0" + }, + "agent": { + "version": "9.0.3", + "ephemeral_id": "5de3e243-c6ff-4e1c-934c-1f1be16969e0", + "id": "59d583ab-1ad7-4b7e-a14d-2b26e3748101", + "name": "elastic-agent-01.internal.example.com", + "type": "filebeat" + }, + "event": { + "original": "{\"events\":[{\"alert_id\":\"6865fae1897bb377dc0fa3e7\",\"alert_key\":\"THREAT_BLOCKED_V3\",\"id\":\"event.threat_detected\",\"scope\":{\"site_id\":\"67762da9ac8a666067034163\"}}]}" + } + }, + { + "@timestamp": "2025-07-03T03:36:38.820Z", + "agent": { + "version": "9.0.3", + "ephemeral_id": "5de3e243-c6ff-4e1c-934c-1f1be16969e0", + "id": "59d583ab-1ad7-4b7e-a14d-2b26e3748101", + "name": "elastic-agent-01.internal.example.com", + "type": "filebeat" + }, + "ecs": { + "version": "8.0.0" + }, + "json": { + "events": [ + { + "alert_key": "HONEYPOT_HIT_DETECTED_KNOWN_CLIENT", + "alert_id": "6865fac5897bb377dc0fa32c", + "scope": { + "site_id": "67762da9ac8a666067034163" + }, + "id": "event.honeypot_triggered" + } + ] + }, + "event": { + "original": "{\"events\":[{\"alert_id\":\"6865fac5897bb377dc0fa32c\",\"alert_key\":\"HONEYPOT_HIT_DETECTED_KNOWN_CLIENT\",\"id\":\"event.honeypot_triggered\",\"scope\":{\"site_id\":\"67762da9ac8a666067034163\"}}]}" + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields", + "forwarded", + "ubnt-unifi-webhook" + ], + "input": { + "type": "http_endpoint" + } + } + ] +} \ No newline at end of file diff --git a/packages/ubnt_unifi/data_stream/webhooks/_dev/test/pipeline/test-logs-webhooks.json-expected.json b/packages/ubnt_unifi/data_stream/webhooks/_dev/test/pipeline/test-logs-webhooks.json-expected.json new file mode 100644 index 00000000000..a2e589e68a5 --- /dev/null +++ b/packages/ubnt_unifi/data_stream/webhooks/_dev/test/pipeline/test-logs-webhooks.json-expected.json @@ -0,0 +1,221 @@ +{ + "expected": [ + { + "@timestamp": "2025-07-03T02:35:27.367Z", + "agent": { + "ephemeral_id": "b94ccb9f-4c9f-45db-a695-f8d8c7ff3127", + "id": "0f02a820-19e9-4f66-bd46-87f9b72fda41", + "name": "ec-cs-soc-exposed-agent", + "type": "filebeat", + "version": "9.0.3" + }, + "data_stream": { + "dataset": "ubnt_unifi.webhooks", + "namespace": "default", + "type": "logs" + }, + "ecs": { + "version": "8.17.0" + }, + "elastic_agent": { + "id": "0f02a820-19e9-4f66-bd46-87f9b72fda41", + "snapshot": false, + "version": "9.0.3" + }, + "event": { + "agent_id_status": "verified", + "dataset": "ubnt_unifi.webhooks", + "ingested": "2025-07-03T02:35:36Z", + "kind": "event", + "original": "{\"alarm\":{\"conditions\":[{\"condition\":{\"source\":\"person\",\"type\":\"is\"}},{\"condition\":{\"source\":\"vehicle\",\"type\":\"is\"}},{\"condition\":{\"source\":\"package\",\"type\":\"is\"}},{\"condition\":{\"source\":\"animal\",\"type\":\"is\"}}],\"eventLocalLink\":\"https://192.168.0.1/protect/events/event/6865ec6d025ca803e42878ca\",\"eventPath\":\"/protect/events/event/6865ec6d025ca803e42878ca\",\"name\":\"Elastic - Objects - All\",\"sources\":[{\"device\":\"28704E178669\",\"type\":\"include\"},{\"device\":\"F4E2C677123A\",\"type\":\"include\"},{\"device\":\"28704E17931C\",\"type\":\"include\"},{\"device\":\"28704E19BD42\",\"type\":\"include\"},{\"device\":\"28704E19C21C\",\"type\":\"include\"}],\"triggers\":[{\"device\":\"28704E17931C\",\"eventId\":\"6865ec6d025ca803e42878ca\",\"key\":\"person\",\"timestamp\":1751510125629,\"zones\":{\"line\":[],\"loiter\":[],\"zone\":[1]}}]},\"timestamp\":1751510126224}", + "type": [ + "info" + ] + }, + "input": { + "type": "http_endpoint" + }, + "observer": { + "product": "UniFi Alarm Manager", + "vendor": "Ubiquiti" + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields", + "forwarded", + "ubnt-unifi-webhook" + ], + "ubnt": { + "unifi": { + "webhook": { + "alarm": { + "conditions": [ + { + "condition": { + "source": "person", + "type": "is" + } + }, + { + "condition": { + "source": "vehicle", + "type": "is" + } + }, + { + "condition": { + "source": "package", + "type": "is" + } + }, + { + "condition": { + "source": "animal", + "type": "is" + } + } + ], + "eventLocalLink": "https://192.168.0.1/protect/events/event/6865ec6d025ca803e42878ca", + "eventPath": "/protect/events/event/6865ec6d025ca803e42878ca", + "name": "Elastic - Objects - All", + "sources": [ + { + "device": "28704E178669", + "type": "include" + }, + { + "device": "F4E2C677123A", + "type": "include" + }, + { + "device": "28704E17931C", + "type": "include" + }, + { + "device": "28704E19BD42", + "type": "include" + }, + { + "device": "28704E19C21C", + "type": "include" + } + ], + "triggers": [ + { + "device": "28704E17931C", + "eventId": "6865ec6d025ca803e42878ca", + "key": "person", + "timestamp": 1751510125629, + "zones": { + "zone": [ + 1 + ] + } + } + ] + }, + "timestamp": "2025-07-03T02:35:26.224Z" + } + } + } + }, + { + "@timestamp": "2025-07-03T03:37:07.350Z", + "agent": { + "ephemeral_id": "5de3e243-c6ff-4e1c-934c-1f1be16969e0", + "id": "59d583ab-1ad7-4b7e-a14d-2b26e3748101", + "name": "elastic-agent-01.internal.example.com", + "type": "filebeat", + "version": "9.0.3" + }, + "ecs": { + "version": "8.17.0" + }, + "event": { + "kind": "event", + "original": "{\"events\":[{\"alert_id\":\"6865fae1897bb377dc0fa3e7\",\"alert_key\":\"THREAT_BLOCKED_V3\",\"id\":\"event.threat_detected\",\"scope\":{\"site_id\":\"67762da9ac8a666067034163\"}}]}", + "type": [ + "info" + ] + }, + "input": { + "type": "http_endpoint" + }, + "observer": { + "product": "UniFi Alarm Manager", + "vendor": "Ubiquiti" + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields", + "forwarded", + "ubnt-unifi-webhook" + ], + "ubnt": { + "unifi": { + "webhook": { + "events": [ + { + "alert_id": "6865fae1897bb377dc0fa3e7", + "alert_key": "THREAT_BLOCKED_V3", + "id": "event.threat_detected", + "scope": { + "site_id": "67762da9ac8a666067034163" + } + } + ] + } + } + } + }, + { + "@timestamp": "2025-07-03T03:36:38.820Z", + "agent": { + "ephemeral_id": "5de3e243-c6ff-4e1c-934c-1f1be16969e0", + "id": "59d583ab-1ad7-4b7e-a14d-2b26e3748101", + "name": "elastic-agent-01.internal.example.com", + "type": "filebeat", + "version": "9.0.3" + }, + "ecs": { + "version": "8.17.0" + }, + "event": { + "kind": "event", + "original": "{\"events\":[{\"alert_id\":\"6865fac5897bb377dc0fa32c\",\"alert_key\":\"HONEYPOT_HIT_DETECTED_KNOWN_CLIENT\",\"id\":\"event.honeypot_triggered\",\"scope\":{\"site_id\":\"67762da9ac8a666067034163\"}}]}", + "type": [ + "info" + ] + }, + "input": { + "type": "http_endpoint" + }, + "observer": { + "product": "UniFi Alarm Manager", + "vendor": "Ubiquiti" + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields", + "forwarded", + "ubnt-unifi-webhook" + ], + "ubnt": { + "unifi": { + "webhook": { + "events": [ + { + "alert_id": "6865fac5897bb377dc0fa32c", + "alert_key": "HONEYPOT_HIT_DETECTED_KNOWN_CLIENT", + "id": "event.honeypot_triggered", + "scope": { + "site_id": "67762da9ac8a666067034163" + } + } + ] + } + } + } + } + ] +} diff --git a/packages/ubnt_unifi/data_stream/webhooks/_dev/test/system/test-http_endpoint-config.yml b/packages/ubnt_unifi/data_stream/webhooks/_dev/test/system/test-http_endpoint-config.yml new file mode 100644 index 00000000000..5f206ed61ec --- /dev/null +++ b/packages/ubnt_unifi/data_stream/webhooks/_dev/test/system/test-http_endpoint-config.yml @@ -0,0 +1,11 @@ +service: test-http_endpoint +service_notify_signal: SIGHUP +input: http_endpoint +data_stream: + vars: + listen_address: 0.0.0.0 + listen_port: 10002 + preserve_original_event: true + preserve_duplicate_custom_fields: true +assert: + hit_count: 6 diff --git a/packages/ubnt_unifi/data_stream/webhooks/agent/stream/http_endpoint.yml.hbs b/packages/ubnt_unifi/data_stream/webhooks/agent/stream/http_endpoint.yml.hbs new file mode 100644 index 00000000000..3e7cf1e4ca6 --- /dev/null +++ b/packages/ubnt_unifi/data_stream/webhooks/agent/stream/http_endpoint.yml.hbs @@ -0,0 +1,34 @@ +listen_address: {{listen_address}} +listen_port: {{listen_port}} +{{#if ssl}} +ssl: {{ssl}} +{{/if}} +{{#if url}} +url: {{url}} +{{/if}} +{{#if secret_header}} +secret.header: {{secret_header}} +{{/if}} +{{#if secret_value}} +secret.value: {{secret_value}} +{{/if}} +{{#if preserve_original_event}} +preserve_original_event: true +{{/if}} +tags: +{{#if preserve_original_event}} + - preserve_original_event +{{/if}} +{{#if preserve_duplicate_custom_fields}} + - preserve_duplicate_custom_fields +{{/if}} +{{#each tags as |tag|}} + - {{tag}} +{{/each}} +{{#contains "forwarded" tags}} +publisher_pipeline.disable_host: true +{{/contains}} +{{#if processors}} +processors: +{{processors}} +{{/if}} \ No newline at end of file diff --git a/packages/ubnt_unifi/data_stream/webhooks/elasticsearch/ingest_pipeline/default.yml b/packages/ubnt_unifi/data_stream/webhooks/elasticsearch/ingest_pipeline/default.yml new file mode 100644 index 00000000000..24c7a8a2e08 --- /dev/null +++ b/packages/ubnt_unifi/data_stream/webhooks/elasticsearch/ingest_pipeline/default.yml @@ -0,0 +1,71 @@ +--- +description: Pipeline for processing Ubiquiti UniFi Alarm Manager webhook events +processors: +- set: + field: ecs.version + value: '8.17.0' + +- set: + field: event.type + value: [info] + +- set: + field: event.kind + value: event + +### Add some basic observer fields so we can co-exist with ubnt_unifi.logs at a basic level ### + +- set: + if: ctx.observer?.vendor == null + field: observer.vendor + value: Ubiquiti + +- set: + if: ctx.observer?.product == null + field: observer.product + value: UniFi Alarm Manager + +- rename: + description: The entire webhook payload should have been parsed into a JSON object, we do nothing more than rename this at this point. + field: json + target_field: ubnt.unifi.webhook + ignore_missing: true + +############## +## Clean Up ## +############## + +- script: + description: Drops null/empty values recursively to minimise event size + lang: painless + source: | + boolean dropEmptyFields(Object object) { + if (object == null || object == '') { + return true; + } else if (object instanceof Map) { + ((Map) object).values().removeIf(value -> dropEmptyFields(value)); + return (((Map) object).size() == 0); + } else if (object instanceof List) { + ((List) object).removeIf(value -> dropEmptyFields(value)); + return (((List) object).length == 0); + } + return false; + } + dropEmptyFields(ctx); + +###################### +## Failure Handling ## +###################### + +on_failure: + - remove: + field: + - _tmp + ignore_missing: true + - append: + field: error.message + value: |- + Processor "{{{ _ingest.on_failure_processor_type }}}" with tag "{{{ _ingest.on_failure_processor_tag }}}" in pipeline "{{{ _ingest.on_failure_pipeline }}}" failed with message "{{{ _ingest.on_failure_message }}}" + - set: + field: event.kind + value: pipeline_error diff --git a/packages/ubnt_unifi/data_stream/webhooks/fields/agent.yml b/packages/ubnt_unifi/data_stream/webhooks/fields/agent.yml new file mode 100644 index 00000000000..b4f84cf84a4 --- /dev/null +++ b/packages/ubnt_unifi/data_stream/webhooks/fields/agent.yml @@ -0,0 +1,3 @@ +- name: input.type + type: keyword + description: Input type. diff --git a/packages/ubnt_unifi/data_stream/webhooks/fields/base-fields.yml b/packages/ubnt_unifi/data_stream/webhooks/fields/base-fields.yml new file mode 100644 index 00000000000..7c798f4534c --- /dev/null +++ b/packages/ubnt_unifi/data_stream/webhooks/fields/base-fields.yml @@ -0,0 +1,12 @@ +- name: data_stream.type + type: constant_keyword + description: Data stream type. +- name: data_stream.dataset + type: constant_keyword + description: Data stream dataset. +- name: data_stream.namespace + type: constant_keyword + description: Data stream namespace. +- name: '@timestamp' + type: date + description: Event timestamp. diff --git a/packages/ubnt_unifi/data_stream/webhooks/fields/fields.yml b/packages/ubnt_unifi/data_stream/webhooks/fields/fields.yml new file mode 100644 index 00000000000..735af20073f --- /dev/null +++ b/packages/ubnt_unifi/data_stream/webhooks/fields/fields.yml @@ -0,0 +1,71 @@ +- name: ubnt + type: group + fields: + - name: unifi + type: group + fields: + - name: webhook + type: group + fields: + - name: events + type: group + fields: + - name: id + type: keyword + - name: alert_key + type: keyword + - name: alert_id + type: keyword + - name: scope + type: group + fields: + - name: site_id + type: keyword + - name: client_device_id + type: keyword + - name: alarm + type: group + fields: + - name: conditions + type: group + fields: + - name: condition + type: group + fields: + - name: source + type: keyword + - name: type + type: keyword + - name: value + type: keyword + - name: eventLocalLink + type: keyword + - name: eventPath + type: keyword + - name: name + type: keyword + - name: sources + type: group + fields: + - name: device + type: keyword + - name: type + type: keyword + - name: triggers + type: group + fields: + - name: device + type: keyword + - name: eventId + type: keyword + - name: key + type: keyword + - name: timestamp + type: date + - name: zones + type: group + fields: + - name: zone + type: float + - name: timestamp + type: date diff --git a/packages/ubnt_unifi/data_stream/webhooks/manifest.yml b/packages/ubnt_unifi/data_stream/webhooks/manifest.yml new file mode 100644 index 00000000000..0519554f32e --- /dev/null +++ b/packages/ubnt_unifi/data_stream/webhooks/manifest.yml @@ -0,0 +1,109 @@ +title: "Ubiquiti Unifi" +type: logs +streams: + - input: http_endpoint + template_path: http_endpoint.yml.hbs + title: Ubiquiti Unifi Webhooks + description: Collect Ubiquiti Unifi events via webhooks + enabled: false + vars: + - name: listen_address + type: text + title: Listen Address + description: The bind address to listen for http endpoint connections. Set to `0.0.0.0` to bind to all available interfaces. + multi: false + required: true + show_user: true + default: localhost + - name: listen_port + type: integer + title: Listen Port + description: The port number the listener binds to. + multi: false + required: true + show_user: true + default: 8080 + - name: url + type: text + title: URL + description: This option specifies which URL path to accept requests on. Defaults to /. + multi: false + required: true + show_user: true + default: / + - name: secret_header + type: text + title: Secret Header + description: The header to check for a specific value specified by `secret.value`. + required: false + show_user: false + secret: false + - name: secret_value + type: password + title: Secret Value + description: The secret stored in the header name specified by `secret.header`. + required: false + show_user: false + secret: true + - name: ssl + type: yaml + title: SSL Configuration + description: i.e. certificate_authorities, supported_protocols, verification_mode etc. + multi: false + required: false + show_user: false + default: | + #certificate_authorities: + # - | + # -----BEGIN CERTIFICATE----- + # MIIDCjCCAfKgAwIBAgITJ706Mu2wJlKckpIvkWxEHvEyijANBgkqhkiG9w0BAQsF + # ADAUMRIwEAYDVQQDDAlsb2NhbGhvc3QwIBcNMTkwNzIyMTkyOTA0WhgPMjExOTA2 + # MjgxOTI5MDRaMBQxEjAQBgNVBAMMCWxvY2FsaG9zdDCCASIwDQYJKoZIhvcNAQEB + # BQADggEPADCCAQoCggEBANce58Y/JykI58iyOXpxGfw0/gMvF0hUQAcUrSMxEO6n + # fZRA49b4OV4SwWmA3395uL2eB2NB8y8qdQ9muXUdPBWE4l9rMZ6gmfu90N5B5uEl + # 94NcfBfYOKi1fJQ9i7WKhTjlRkMCgBkWPkUokvBZFRt8RtF7zI77BSEorHGQCk9t + # /D7BS0GJyfVEhftbWcFEAG3VRcoMhF7kUzYwp+qESoriFRYLeDWv68ZOvG7eoWnP + # PsvZStEVEimjvK5NSESEQa9xWyJOmlOKXhkdymtcUd/nXnx6UTCFgnkgzSdTWV41 + # CI6B6aJ9svCTI2QuoIq2HxX/ix7OvW1huVmcyHVxyUECAwEAAaNTMFEwHQYDVR0O + # BBYEFPwN1OceFGm9v6ux8G+DZ3TUDYxqMB8GA1UdIwQYMBaAFPwN1OceFGm9v6ux + # 8G+DZ3TUDYxqMA8GA1UdEwEB/wQFMAMBAf8wDQYJKoZIhvcNAQELBQADggEBAG5D + # 874A4YI7YUwOVsVAdbWtgp1d0zKcPRR+r2OdSbTAV5/gcS3jgBJ3i1BN34JuDVFw + # 3DeJSYT3nxy2Y56lLnxDeF8CUTUtVQx3CuGkRg1ouGAHpO/6OqOhwLLorEmxi7tA + # H2O8mtT0poX5AnOAhzVy7QW0D/k4WaoLyckM5hUa6RtvgvLxOwA0U+VGurCDoctu + # 8F4QOgTAWyh8EZIwaKCliFRSynDpv3JTUwtfZkxo6K6nce1RhCWFAsMvDZL8Dgc0 + # yvgJ38BRsFOtkRuAGSf6ZUwTO8JJRRIFnpUzXflAnGivK9M13D5GEQMmIl6U9Pvk + # sxSmbIUfc2SGJGCJD4I= + # -----END CERTIFICATE----- + - name: tags + type: text + title: Tags + multi: true + required: true + show_user: false + default: + - forwarded + - ubnt-unifi-webhook + - name: preserve_original_event + required: true + show_user: true + title: Preserve original event + description: Preserves a raw copy of the original event, added to the field `event.original`. + type: bool + multi: false + default: false + - name: preserve_duplicate_custom_fields + required: true + show_user: false + title: Preserve duplicate custom fields + description: Preserve custom fields for all ECS mappings. + type: bool + multi: false + default: false + - name: processors + type: yaml + title: Processors + multi: false + required: false + show_user: false + description: >- + Processors are used to reduce the number of fields in the exported event or to enhance the event with metadata. This executes in the agent before the logs are parsed. See [Processors](https://www.elastic.co/guide/en/beats/filebeat/current/filtering-and-enhancing-data.html) for details. diff --git a/packages/ubnt_unifi/data_stream/webhooks/sample_event.json b/packages/ubnt_unifi/data_stream/webhooks/sample_event.json new file mode 100644 index 00000000000..0edad9251eb --- /dev/null +++ b/packages/ubnt_unifi/data_stream/webhooks/sample_event.json @@ -0,0 +1,126 @@ +{ + "@timestamp": "2025-07-03T05:50:30.055Z", + "agent": { + "ephemeral_id": "87edeb32-f8de-4062-9375-1a1df8ca8c2f", + "id": "af27c668-8f78-4a24-b180-745789969744", + "name": "elastic-agent-30728", + "type": "filebeat", + "version": "8.18.2" + }, + "data_stream": { + "dataset": "ubnt_unifi.webhooks", + "namespace": "29060", + "type": "logs" + }, + "ecs": { + "version": "8.17.0" + }, + "elastic_agent": { + "id": "af27c668-8f78-4a24-b180-745789969744", + "snapshot": false, + "version": "8.18.2" + }, + "event": { + "agent_id_status": "verified", + "dataset": "ubnt_unifi.webhooks", + "ingested": "2025-07-03T05:50:31Z", + "kind": "event", + "original": "{\"alarm\":{\"conditions\":[{\"condition\":{\"source\":\"device_issue\",\"type\":\"is\"}},{\"condition\":{\"source\":\"device_adoption_state_changed\",\"type\":\"is\"}},{\"condition\":{\"source\":\"device_discovery\",\"type\":\"is\"}},{\"condition\":{\"source\":\"admin_access\",\"type\":\"is\"}},{\"condition\":{\"source\":\"admin_recording_clips_manipulations\",\"type\":\"is\"}},{\"condition\":{\"source\":\"admin_geolocation\",\"type\":\"is\"}},{\"condition\":{\"source\":\"admin_settings_change\",\"type\":\"is\"}},{\"condition\":{\"source\":\"device_update_status_change\",\"type\":\"is\"}},{\"condition\":{\"source\":\"camera_utilization_limit\",\"type\":\"is\"}},{\"condition\":{\"source\":\"application_issue\",\"type\":\"is\"}}],\"name\":\"Elastic - System - All\",\"sources\":[],\"triggers\":[{\"device\":\"nvr\",\"eventId\":\"6865498302c5a803e4234efe\",\"key\":\"admin_access\",\"timestamp\":1751468419711}]},\"timestamp\":1751468420734}", + "type": [ + "info" + ] + }, + "input": { + "type": "http_endpoint" + }, + "observer": { + "product": "Unifi", + "vendor": "Ubiquiti" + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields", + "forwarded", + "ubnt-unifi-webhook" + ], + "ubnt": { + "unifi": { + "webhook": { + "alarm": { + "conditions": [ + { + "condition": { + "source": "device_issue", + "type": "is" + } + }, + { + "condition": { + "source": "device_adoption_state_changed", + "type": "is" + } + }, + { + "condition": { + "source": "device_discovery", + "type": "is" + } + }, + { + "condition": { + "source": "admin_access", + "type": "is" + } + }, + { + "condition": { + "source": "admin_recording_clips_manipulations", + "type": "is" + } + }, + { + "condition": { + "source": "admin_geolocation", + "type": "is" + } + }, + { + "condition": { + "source": "admin_settings_change", + "type": "is" + } + }, + { + "condition": { + "source": "device_update_status_change", + "type": "is" + } + }, + { + "condition": { + "source": "camera_utilization_limit", + "type": "is" + } + }, + { + "condition": { + "source": "application_issue", + "type": "is" + } + } + ], + "name": "Elastic - System - All", + "triggers": [ + { + "device": "nvr", + "eventId": "6865498302c5a803e4234efe", + "key": "admin_access", + "timestamp": 1751468419711 + } + ] + }, + "timestamp": 1751468420734 + } + } + } +} diff --git a/packages/ubnt_unifi/docs/README.md b/packages/ubnt_unifi/docs/README.md new file mode 100644 index 00000000000..9177e6a0f67 --- /dev/null +++ b/packages/ubnt_unifi/docs/README.md @@ -0,0 +1,552 @@ +# Ubiquiti UniFi + +This integration is for [Ubiquiti UniFi](https://ui.com) equipment event logs. The package processes events collected from Ubiquiti Unifi devices. + +## Data Streams + +The Ubiquiti UniFi integration collects the following event types: + +- **logs**, Logs produced via UDP syslog from a Unifi controller, application or device. + +This includes CEF logs, iptables firewall logs, and other Unix/Linux style syslog messages that may be produced. + +You can use Elastic Agent to read files of logs if you already have a syslog aggregation system that is already collecting UniFi syslog output. Or alternatively you can configure your UniFi systems to log directly to a UDP listener on an Elastic Agent. + +- **webhooks**, Events produced by Unifi Alarm Manager as webhooks, aka. HTTP POST's with a JSON body. + +The Ubiquiti UniFi Alarm Manager and webhook based alarms are very new features and the content currently included in the body of a webhook is highly variable in terms of quality and field completeness. + +## Related Integrations + +**NOTE**: Ubiquiti UniFi now supports NetFlow based traffic logging. If network flow visibility is desired you can and should utilise the existing Elastic [Netflow](https://www.elastic.co/docs/reference/integrations/netflow) integration using NetFlow Version 9 to collect flow records from your Ubiquiti UniFi equipment. Refer to [https://community.ui.com/releases](https://community.ui.com/releases) for further documentation regarding NetFlow support and configuration instructions. + +**NOTE**: Ubiquiti UniFi produces iptables "style" firewall logs with a slightly different format to the firewall logs previously produced by other Ubiquiti systems. You do not need to, and should not, install or utilise existing Ubiquiti support within the [iptables](https://www.elastic.co/docs/reference/integrations/iptables) integration as it will not work for firewall logs produced by UniFi systems. You should utilise this integration to collect Ubiquiti UniFi firewall logs independently of other non-UniFi Ubiquiti equipment. + +**NOTE**: Ubiquiti UniFi components produce iptables style firewall logs, *some* CEF format logs for configuration activity and events on UniFi consoles and within applications, as well as some common *nix style logs. While at times these are sent with a syslog prefix at other times they are not sent with a syslog prefix. At present not all CEF logs produced by UniFi components are conformant to the Common Event Format (CEF) specification. You do not need to, and should not, attempt to utilise the existing Elastic [CEF](https://www.elastic.co/docs/reference/integrations/cef) integration to process Ubiquiti UniFi logs in any way. This Ubiquiti UniFi integration includes Elastic Agent beat level content fixes for the format problems that are often produced by Ubiquiti UniFi components at present. + +## Requirements + +For `logs` based event collection Elastic Agent *MUST* be utilised due to the pre-processing and filtering that occurs at the agent level. For example CEF parsing is completed by the Elastic Agent, as this is the only component that natively supports CEF parsing, when logs are first received from the network or read from file. A number of content fixes are applied. + +If `logs` are received/aggregated or otherwise handled by something else and delivered to Elasticsearch for indexing, without passing thru an Elastic Agent, you should replicate the Elastic Agent behaviour, including content fixes, CEF parsing, as well as appropriate tagging. + +`webhooks` events from the Ubiquiti UniFi Alarm Manager feature/s require no special Elastic Agent based pre-processing and can be delivered to Elasticsearch for indexing via any method that is suitable for your environment; provided you tag the events appropriately. + +For more details and installation instructions, please refer to the [Elastic Agent Installation Guide](https://www.elastic.co/guide/en/fleet/current/elastic-agent-installation.html). + +Your Ubiquiti UniFi infrastructure should consist of: +- Ubiquiti UniFi OS `4.0.0` or higher, if running a Ubquiti Unifi Cloud Gateway or similar appliance. +- Ubiquiti UniFi Applications, e.g. Network, `9.0.0` or higher, either on a Ubquiti Unifi Cloud Gateway or self hosted. + +Refer to [https://community.ui.com/releases](https://community.ui.com/releases) for current release information, upgrade instructions and further documentation. + +**NOTE**: This integration has been tested with Ubiquiti UniFi Cloud Gateways only, self-hosted versions of UniFi applications should work but have not been tested. + +**NOTE**: This integration has only been tested with Ubiquiti UniFi Network and Protect applications at this time. + +### Installing and managing an Elastic Agent: + +There are several options for installing and managing Elastic Agent: + +### Install a Fleet-managed Elastic Agent (recommended): + +With this approach, you install Elastic Agent and use Fleet in Kibana to define, configure, and manage your agents in a central location. We recommend using Fleet management because it makes the management and upgrade of your agents considerably easier. + +### Install Elastic Agent in standalone mode (advanced users): + +With this approach, you install Elastic Agent and manually configure the agent locally on the system where it’s installed. You are responsible for managing and upgrading the agents. This approach is reserved for advanced users only. + +### Install Elastic Agent in a containerized environment: + +You can run Elastic Agent inside a container, either with Fleet Server or standalone. Docker images for all versions of Elastic Agent are available from the Elastic Docker registry, and we provide deployment manifests for running on Kubernetes. + +Please note, there are minimum requirements for running Elastic Agent. For more information, refer to the [Elastic Agent Minimum Requirements](https://www.elastic.co/guide/en/fleet/current/elastic-agent-installation.html#elastic-agent-installation-minimum-requirements). + + +### Enabling the integration in Elastic: + +1. In Kibana navigate to Management > Integrations. +2. In "Search for integrations" top bar, search for `Ubiquiti UniFi`. +3. Select the "Ubiquiti UniFi" integration from the search results. +4. Select "Add Ubiquiti UniFi" to add the integration. +5. Add all the required integration configuration parameters. +6. Select "Save and continue" to save the integration. + +The default syslog based log collection configuration is likely suitable for most environments, e.g. + +![Default Integration Configuration](../img/add-integration-defaults.png) + +### Enabling SIEM integration in Ubiquiti UniFi: + +Logging for UnifiOS and Unifi applications can be configured via, + +1. Login to your Unifi system, navigate to Settings, typically found via the gear icon in the menu bar to the left +2. Click on "Control Plane" in the second level menu to the left of the screen +3. Click on "Integrations" in the third level menu near the top of the screen +4. Select "SIEM Server" next to "Activity Logging (Syslog)" +5. Select Activity Log Categories as appropriate, note that "UniFi OS" categories will be for admin activity and other system events, while "Network" categories can be used to enable traffic logging including logging of traffic that matches the default firewally policy. +6. Enter the IP address and port that your Elastic Agent Ubiquiti UniFi syslog integration listener has been configured to use +7. Optionally click "Send Test Event" and ensure ingest to Elastic is occurring +8. Click "Save" to save the configuration + +Additional logging options may be available via other screens. + +![Control Plane SIEM Integration Configuration](../img/configure-unifi-siem-integration.png) + +## Logs + +### Ubiquiti UniFi Logs + +The `logs` dataset collects Ubiquiti Unifi logs sent via syslog. + +An example event for `logs` looks as following: + +```json +{ + "@timestamp": "2025-07-05T04:29:36.878Z", + "agent": { + "ephemeral_id": "11a9cafc-a45e-4ec2-b38f-e99536291b74", + "id": "131ef900-601b-4f5d-a1b8-6ed60bda2132", + "name": "ubnt-unifi-logs", + "type": "filebeat", + "version": "9.0.3" + }, + "cef": { + "device": { + "event_class_id": "201", + "product": "UniFi Network", + "vendor": "Ubiquiti", + "version": "9.3.33" + }, + "extensions": { + "UNIFIcategory": "Security", + "UNIFIdeviceIp": "192.168.0.1", + "UNIFIdeviceMac": "01:23:45:67:89:0a", + "UNIFIdeviceModel": "UniFi Dream Machine PRO SE", + "UNIFIdeviceName": "udm-pro-se", + "UNIFIdeviceVersion": "4.3.5", + "UNIFIhost": "udm-pro-se", + "UNIFIipsSessionId": "255132502100797", + "UNIFIipsSignature": "ET SCAN Possible Nmap User-Agent Observed", + "UNIFIipsSignatureId": "2024364", + "UNIFIrisk": "high", + "UNIFIsubCategory": "Intrusion Prevention", + "destinationAddress": "192.168.0.2", + "destinationPort": 8000, + "message": "A network intrusion attempt from 192.168.0.16 to 192.168.0.2 has been detected and blocked.", + "sourceAddress": "192.168.0.16", + "sourcePort": 60700, + "transportProtocol": "TCP" + }, + "name": "Threat Detected and Blocked", + "severity": "9", + "version": "0" + }, + "destination": { + "ip": "192.168.0.2", + "port": 8000 + }, + "ecs": { + "version": "8.17.0" + }, + "event": { + "action": "Threat Detected and Blocked", + "category": [ + "network", + "intrusion_detection" + ], + "code": "201", + "kind": "alert", + "original": "Jul 5 04:29:36 udm-pro-se.localnet 2025-07-05T04: 29:36.878Z udm-pro-se CEF:0|Ubiquiti|UniFi Network|9.3.33|201|Threat Detected and Blocked|9|proto=TCP src=192.168.0.16 spt=60700 dst=192.168.0.2 dpt=8000 UNIFIcategory=Security UNIFIsubCategory=Intrusion Prevention UNIFIhost=udm-pro-se UNIFIdeviceMac=01:23:45:67:89:0a UNIFIdeviceName=udm-pro-se UNIFIdeviceModel=UniFi Dream Machine PRO SE UNIFIdeviceIp=192.168.0.1 UNIFIdeviceVersion=4.3.5 UNIFIrisk=high UNIFIipsSessionId=255132502100797 UNIFIipsSignature=ET SCAN Possible Nmap User-Agent Observed UNIFIipsSignatureId=2024364 msg=A network intrusion attempt from 192.168.0.16 to 192.168.0.2 has been detected and blocked.", + "reason": "A network intrusion attempt from 192.168.0.16 to 192.168.0.2 has been detected and blocked.", + "severity": 9, + "type": [ + "info" + ] + }, + "host": { + "hostname": "udm-pro-se" + }, + "input": { + "type": "filestream" + }, + "log": { + "file": { + "device_id": "64773", + "fingerprint": "4971336f945694d052c56620d03d45fc31bdbba970ed13f50e2e10bfc2f46eeb", + "inode": "25962857", + "path": "/var/log/remote/udm-pro-se.log" + }, + "level": "9", + "offset": 298711, + "syslog": { + "hostname": "udm-pro-se" + } + }, + "message": "A network intrusion attempt from 192.168.0.16 to 192.168.0.2 has been detected and blocked.", + "network": { + "transport": "tcp" + }, + "observer": { + "hostname": "udm-pro-se.localnet", + "name": "udm-pro-se", + "product": "UniFi Network", + "vendor": "Ubiquiti", + "version": "9.3.33" + }, + "related": { + "ip": [ + "192.168.0.16", + "192.168.0.2" + ] + }, + "source": { + "ip": "192.168.0.16", + "port": 60700 + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields", + "forwarded", + "ubnt-unifi-cef" + ] +} +``` + +**Exported fields** + +| Field | Description | Type | +|---|---|---| +| @timestamp | Event timestamp. | date | +| cef.device.event_class_id | | keyword | +| cef.device.product | | keyword | +| cef.device.vendor | | keyword | +| cef.device.version | | keyword | +| cef.extensions.UNIFIWiFiRssi | | keyword | +| cef.extensions.UNIFIaccessMethod | | keyword | +| cef.extensions.UNIFIadmin | | keyword | +| cef.extensions.UNIFIauthMethod | | keyword | +| cef.extensions.UNIFIcategory | | keyword | +| cef.extensions.UNIFIclientAlias | | keyword | +| cef.extensions.UNIFIclientHostname | | keyword | +| cef.extensions.UNIFIclientIp | | keyword | +| cef.extensions.UNIFIclientMac | | keyword | +| cef.extensions.UNIFIconnectedToDeviceIp | | keyword | +| cef.extensions.UNIFIconnectedToDeviceMac | | keyword | +| cef.extensions.UNIFIconnectedToDeviceModel | | keyword | +| cef.extensions.UNIFIconnectedToDeviceName | | keyword | +| cef.extensions.UNIFIconnectedToDevicePort | | keyword | +| cef.extensions.UNIFIconnectedToDeviceVersion | | keyword | +| cef.extensions.UNIFIdeviceIp | | keyword | +| cef.extensions.UNIFIdeviceMac | | keyword | +| cef.extensions.UNIFIdeviceModel | | keyword | +| cef.extensions.UNIFIdeviceName | | keyword | +| cef.extensions.UNIFIdeviceVersion | | keyword | +| cef.extensions.UNIFIduration | | keyword | +| cef.extensions.UNIFIhost | | keyword | +| cef.extensions.UNIFIipsSessionId | | keyword | +| cef.extensions.UNIFIipsSignature | | keyword | +| cef.extensions.UNIFIipsSignatureId | | keyword | +| cef.extensions.UNIFIlastConnectedToDeviceIp | | keyword | +| cef.extensions.UNIFIlastConnectedToDeviceMac | | keyword | +| cef.extensions.UNIFIlastConnectedToDeviceModel | | keyword | +| cef.extensions.UNIFIlastConnectedToDeviceName | | keyword | +| cef.extensions.UNIFIlastConnectedToDeviceVersion | | keyword | +| cef.extensions.UNIFIlastConnectedToWiFiRssi | | keyword | +| cef.extensions.UNIFInetworkName | | keyword | +| cef.extensions.UNIFInetworkSubnet | | keyword | +| cef.extensions.UNIFInetworkVlan | | keyword | +| cef.extensions.UNIFIreference | | keyword | +| cef.extensions.UNIFIrisk | | keyword | +| cef.extensions.UNIFIsettingsChanges | | text | +| cef.extensions.UNIFIsettingsEntry | | keyword | +| cef.extensions.UNIFIsettingsSection | | keyword | +| cef.extensions.UNIFIsubCategory | | keyword | +| cef.extensions.UNIFIusageDown | | keyword | +| cef.extensions.UNIFIusageUp | | keyword | +| cef.extensions.UNIFIwifiAirtimeUtilization | | keyword | +| cef.extensions.UNIFIwifiBand | | keyword | +| cef.extensions.UNIFIwifiChannel | | keyword | +| cef.extensions.UNIFIwifiChannelWidth | | keyword | +| cef.extensions.UNIFIwifiInterference | | keyword | +| cef.extensions.UNIFIwifiName | | keyword | +| cef.extensions.baseEventCount | | long | +| cef.extensions.destinationAddress | | keyword | +| cef.extensions.destinationPort | | integer | +| cef.extensions.fixed_ap_enabled | | keyword | +| cef.extensions.fixed_ip | | keyword | +| cef.extensions.local_dns_record_enabled | | keyword | +| cef.extensions.message | | keyword | +| cef.extensions.name | | keyword | +| cef.extensions.note | | keyword | +| cef.extensions.sourceAddress | | keyword | +| cef.extensions.sourcePort | | integer | +| cef.extensions.sourceUserName | | keyword | +| cef.extensions.transportProtocol | | keyword | +| cef.extensions.use_fixedip | | keyword | +| cef.extensions.virtual_network_override_enabled | | keyword | +| cef.name | | keyword | +| cef.severity | | keyword | +| cef.version | | keyword | +| data_stream.dataset | Data stream dataset. | constant_keyword | +| data_stream.namespace | Data stream namespace. | constant_keyword | +| data_stream.type | Data stream type. | constant_keyword | +| input.type | Input type. | keyword | +| iptables.ether_type | Value of the ethernet type field identifying the network layer protocol. | long | +| iptables.flow_label | IPv6 flow label. | integer | +| iptables.fragment_flags | IP fragment flags. A combination of CE, DF and MF. | keyword | +| iptables.fragment_offset | Offset of the current IP fragment. | long | +| iptables.gid | GID associated with the packet. | keyword | +| iptables.icmp.code | ICMP code. | long | +| iptables.icmp.id | ICMP ID. | long | +| iptables.icmp.parameter | ICMP parameter. | long | +| iptables.icmp.redirect | ICMP redirect address. | ip | +| iptables.icmp.seq | ICMP sequence number. | long | +| iptables.icmp.type | ICMP type. | long | +| iptables.id | Packet identifier. | long | +| iptables.incomplete_bytes | Number of incomplete bytes. | long | +| iptables.input_device | Device that received the packet. | keyword | +| iptables.length | Packet length. | long | +| iptables.mark | MARK= | keyword | +| iptables.output_device | Device that output the packet. | keyword | +| iptables.precedence_bits | IP precedence bits. | short | +| iptables.tcp.ack | TCP Acknowledgment number. | long | +| iptables.tcp.flags | TCP flags. | keyword | +| iptables.tcp.reserved_bits | TCP reserved bits. | short | +| iptables.tcp.seq | TCP sequence number. | long | +| iptables.tcp.urgp | URGP= | keyword | +| iptables.tcp.window | Advertised TCP window size. | long | +| iptables.tos | IP Type of Service field. | long | +| iptables.ttl | Time To Live field. | integer | +| iptables.ubiquiti.input_zone | Input zone. | keyword | +| iptables.ubiquiti.output_zone | Output zone. | keyword | +| iptables.ubiquiti.rule_description | Description of the rule. | keyword | +| iptables.ubiquiti.rule_name | Name of the rule. | keyword | +| iptables.ubiquiti.rule_number | The rule number within the rule set. | keyword | +| iptables.ubiquiti.rule_set | The rule set name. | keyword | +| iptables.udp.length | Length of the UDP header and payload. | long | +| iptables.uid | UID associated with the packet. | keyword | +| log.file.device_id | ID of the device containing the filesystem where the file resides. | keyword | +| log.file.fingerprint | The sha256 fingerprint identity of the file when fingerprinting is enabled. | keyword | +| log.file.inode | Inode number of the log file. | keyword | +| log.offset | Offset of the entry in the log file. | long | +| log.source.address | | keyword | +| ubnt.unifi.dhcp.interface | The interface name associated with the DHCP event | keyword | +| ubnt.unifi.dhcp.ip | The IP address associated to the source | keyword | +| ubnt.unifi.dhcp.mac | The MAC address associated to the source | keyword | +| ubnt.unifi.dhcp.message | The message associated to the DHCP event | keyword | +| ubnt.unifi.dhcp.name | The name of the device associated to the DHCP event | keyword | +| ubnt.unifi.earlyoom.memory.total | The total amount of memory available | integer | +| ubnt.unifi.earlyoom.memory.used | The amount of memory used | integer | +| ubnt.unifi.earlyoom.memory.used_pct | The percentage of memory used | float | +| ubnt.unifi.earlyoom.swap.total | The total amount of swap available | integer | +| ubnt.unifi.earlyoom.swap.used | The amount of swap used | integer | +| ubnt.unifi.earlyoom.swap.used_pct | The percentage of swap used | float | +| ubnt.unifi.linkcheck.city | The city associated with the link check | keyword | +| ubnt.unifi.linkcheck.country | The country associated with the link check | keyword | +| ubnt.unifi.linkcheck.countryCode | The country code associated with the link check | keyword | +| ubnt.unifi.linkcheck.downlink.rate | The rate associated with the downlink | keyword | +| ubnt.unifi.linkcheck.downlink.speed | The speed associated with the downlink | float | +| ubnt.unifi.linkcheck.function | The function associated with the link check | keyword | +| ubnt.unifi.linkcheck.latitude | The latitude associated with the link check | float | +| ubnt.unifi.linkcheck.longitude | The longitude associated with the link check | float | +| ubnt.unifi.linkcheck.provider | The provider associated with the link check | keyword | +| ubnt.unifi.linkcheck.providerUrl | The URL associated with the provider | keyword | +| ubnt.unifi.linkcheck.resultUrl | The WifiMan speed test result URL | keyword | +| ubnt.unifi.linkcheck.speedMbps | The speed in Mbps associated with the link check | float | +| ubnt.unifi.linkcheck.uplink.rate | The rate associated with the uplink | keyword | +| ubnt.unifi.linkcheck.uplink.speed | The speed associated with the uplink | float | +| ubnt.unifi.linkcheck.url | The URL associated with the link check | keyword | +| ubnt.unifi.mcad.wireless_agg_stats.log_sta_anomalies.anomalies | The anomalies associated with the wireless aggregate stats | keyword | +| ubnt.unifi.mcad.wireless_agg_stats.log_sta_anomalies.bssid | The BSSID associated with the wireless aggregate stats | keyword | +| ubnt.unifi.mcad.wireless_agg_stats.log_sta_anomalies.radio | The radio associated with the wireless aggregate stats | keyword | +| ubnt.unifi.mcad.wireless_agg_stats.log_sta_anomalies.satisfaction_now | The satisfaction now associated with the wireless aggregate stats | keyword | +| ubnt.unifi.mcad.wireless_agg_stats.log_sta_anomalies.sta | The STA associated with the wireless aggregate stats | keyword | +| ubnt.unifi.mcad.wireless_agg_stats.log_sta_anomalies.vap | The VAP associated with the wireless aggregate stats | keyword | +| ubnt.unifi.stahtd.dump.arp_reply_gw_seen | | keyword | +| ubnt.unifi.stahtd.dump.assoc_delta | | keyword | +| ubnt.unifi.stahtd.dump.assoc_status | | keyword | +| ubnt.unifi.stahtd.dump.auth_delta | | keyword | +| ubnt.unifi.stahtd.dump.auth_failures | | keyword | +| ubnt.unifi.stahtd.dump.auth_ts | | keyword | +| ubnt.unifi.stahtd.dump.avg_rssi | | keyword | +| ubnt.unifi.stahtd.dump.disassoc_reason | | keyword | +| ubnt.unifi.stahtd.dump.dns_resp_seen | | keyword | +| ubnt.unifi.stahtd.dump.dns_responses | | keyword | +| ubnt.unifi.stahtd.dump.dns_timeouts | | keyword | +| ubnt.unifi.stahtd.dump.event_id | | keyword | +| ubnt.unifi.stahtd.dump.event_type | | keyword | +| ubnt.unifi.stahtd.dump.ip_assign_type | | keyword | +| ubnt.unifi.stahtd.dump.ip_delta | | keyword | +| ubnt.unifi.stahtd.dump.mac | | keyword | +| ubnt.unifi.stahtd.dump.message_type | | keyword | +| ubnt.unifi.stahtd.dump.query_\* | | keyword | +| ubnt.unifi.stahtd.dump.sta_dc_reason | | keyword | +| ubnt.unifi.stahtd.dump.traffic_delta | | keyword | +| ubnt.unifi.stahtd.dump.vap | | keyword | +| ubnt.unifi.stahtd.dump.wpa_auth_delta | | keyword | + + +### Ubiquiti UniFi Webhooks + +The `webhooks` dataset collects Ubiquiti Unifi events producted by Alarm Manager configurations which send alarms as HTTP POST requests with a JSON body. + +An example event for `webhooks` looks as following: + +```json +{ + "@timestamp": "2025-07-03T05:50:30.055Z", + "agent": { + "ephemeral_id": "87edeb32-f8de-4062-9375-1a1df8ca8c2f", + "id": "af27c668-8f78-4a24-b180-745789969744", + "name": "elastic-agent-30728", + "type": "filebeat", + "version": "8.18.2" + }, + "data_stream": { + "dataset": "ubnt_unifi.webhooks", + "namespace": "29060", + "type": "logs" + }, + "ecs": { + "version": "8.17.0" + }, + "elastic_agent": { + "id": "af27c668-8f78-4a24-b180-745789969744", + "snapshot": false, + "version": "8.18.2" + }, + "event": { + "agent_id_status": "verified", + "dataset": "ubnt_unifi.webhooks", + "ingested": "2025-07-03T05:50:31Z", + "kind": "event", + "original": "{\"alarm\":{\"conditions\":[{\"condition\":{\"source\":\"device_issue\",\"type\":\"is\"}},{\"condition\":{\"source\":\"device_adoption_state_changed\",\"type\":\"is\"}},{\"condition\":{\"source\":\"device_discovery\",\"type\":\"is\"}},{\"condition\":{\"source\":\"admin_access\",\"type\":\"is\"}},{\"condition\":{\"source\":\"admin_recording_clips_manipulations\",\"type\":\"is\"}},{\"condition\":{\"source\":\"admin_geolocation\",\"type\":\"is\"}},{\"condition\":{\"source\":\"admin_settings_change\",\"type\":\"is\"}},{\"condition\":{\"source\":\"device_update_status_change\",\"type\":\"is\"}},{\"condition\":{\"source\":\"camera_utilization_limit\",\"type\":\"is\"}},{\"condition\":{\"source\":\"application_issue\",\"type\":\"is\"}}],\"name\":\"Elastic - System - All\",\"sources\":[],\"triggers\":[{\"device\":\"nvr\",\"eventId\":\"6865498302c5a803e4234efe\",\"key\":\"admin_access\",\"timestamp\":1751468419711}]},\"timestamp\":1751468420734}", + "type": [ + "info" + ] + }, + "input": { + "type": "http_endpoint" + }, + "observer": { + "product": "Unifi", + "vendor": "Ubiquiti" + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields", + "forwarded", + "ubnt-unifi-webhook" + ], + "ubnt": { + "unifi": { + "webhook": { + "alarm": { + "conditions": [ + { + "condition": { + "source": "device_issue", + "type": "is" + } + }, + { + "condition": { + "source": "device_adoption_state_changed", + "type": "is" + } + }, + { + "condition": { + "source": "device_discovery", + "type": "is" + } + }, + { + "condition": { + "source": "admin_access", + "type": "is" + } + }, + { + "condition": { + "source": "admin_recording_clips_manipulations", + "type": "is" + } + }, + { + "condition": { + "source": "admin_geolocation", + "type": "is" + } + }, + { + "condition": { + "source": "admin_settings_change", + "type": "is" + } + }, + { + "condition": { + "source": "device_update_status_change", + "type": "is" + } + }, + { + "condition": { + "source": "camera_utilization_limit", + "type": "is" + } + }, + { + "condition": { + "source": "application_issue", + "type": "is" + } + } + ], + "name": "Elastic - System - All", + "triggers": [ + { + "device": "nvr", + "eventId": "6865498302c5a803e4234efe", + "key": "admin_access", + "timestamp": 1751468419711 + } + ] + }, + "timestamp": 1751468420734 + } + } + } +} +``` + +**Exported fields** + +| Field | Description | Type | +|---|---|---| +| @timestamp | Event timestamp. | date | +| data_stream.dataset | Data stream dataset. | constant_keyword | +| data_stream.namespace | Data stream namespace. | constant_keyword | +| data_stream.type | Data stream type. | constant_keyword | +| input.type | Input type. | keyword | +| ubnt.unifi.webhook.alarm.conditions.condition.source | | keyword | +| ubnt.unifi.webhook.alarm.conditions.condition.type | | keyword | +| ubnt.unifi.webhook.alarm.conditions.condition.value | | keyword | +| ubnt.unifi.webhook.alarm.eventLocalLink | | keyword | +| ubnt.unifi.webhook.alarm.eventPath | | keyword | +| ubnt.unifi.webhook.alarm.name | | keyword | +| ubnt.unifi.webhook.alarm.sources.device | | keyword | +| ubnt.unifi.webhook.alarm.sources.type | | keyword | +| ubnt.unifi.webhook.alarm.triggers.device | | keyword | +| ubnt.unifi.webhook.alarm.triggers.eventId | | keyword | +| ubnt.unifi.webhook.alarm.triggers.key | | keyword | +| ubnt.unifi.webhook.alarm.triggers.timestamp | | date | +| ubnt.unifi.webhook.alarm.triggers.zones.zone | | float | +| ubnt.unifi.webhook.events.alert_id | | keyword | +| ubnt.unifi.webhook.events.alert_key | | keyword | +| ubnt.unifi.webhook.events.id | | keyword | +| ubnt.unifi.webhook.events.scope.client_device_id | | keyword | +| ubnt.unifi.webhook.events.scope.site_id | | keyword | +| ubnt.unifi.webhook.timestamp | | date | diff --git a/packages/ubnt_unifi/img/add-integration-defaults.png b/packages/ubnt_unifi/img/add-integration-defaults.png new file mode 100644 index 00000000000..77853762709 Binary files /dev/null and b/packages/ubnt_unifi/img/add-integration-defaults.png differ diff --git a/packages/ubnt_unifi/img/configure-unifi-siem-integration.png b/packages/ubnt_unifi/img/configure-unifi-siem-integration.png new file mode 100644 index 00000000000..3716764368d Binary files /dev/null and b/packages/ubnt_unifi/img/configure-unifi-siem-integration.png differ diff --git a/packages/ubnt_unifi/img/dashboard-logs-summary.png b/packages/ubnt_unifi/img/dashboard-logs-summary.png new file mode 100644 index 00000000000..72256c2ee0f Binary files /dev/null and b/packages/ubnt_unifi/img/dashboard-logs-summary.png differ diff --git a/packages/ubnt_unifi/img/u.svg b/packages/ubnt_unifi/img/u.svg new file mode 100644 index 00000000000..4547a05304d --- /dev/null +++ b/packages/ubnt_unifi/img/u.svg @@ -0,0 +1 @@ + \ No newline at end of file diff --git a/packages/ubnt_unifi/kibana/dashboard/ubnt_unifi-caf8fd79-ae35-4850-98b6-92f7758f2ca2.json b/packages/ubnt_unifi/kibana/dashboard/ubnt_unifi-caf8fd79-ae35-4850-98b6-92f7758f2ca2.json new file mode 100644 index 00000000000..812c679d767 --- /dev/null +++ b/packages/ubnt_unifi/kibana/dashboard/ubnt_unifi-caf8fd79-ae35-4850-98b6-92f7758f2ca2.json @@ -0,0 +1,2392 @@ +{ + "attributes": { + "controlGroupInput": { + "chainingSystem": "HIERARCHICAL", + "controlStyle": "oneLine", + "ignoreParentSettingsJSON": { + "ignoreFilters": false, + "ignoreQuery": false, + "ignoreTimerange": false, + "ignoreValidations": false + }, + "panelsJSON": { + "14ff87c8-e889-43af-ad59-d0fa1e7ba7fe": { + "explicitInput": { + "dataViewId": "security-solution-default", + "fieldName": "event.type", + "searchTechnique": "prefix", + "selectedOptions": [], + "sort": { + "by": "_count", + "direction": "desc" + } + }, + "grow": false, + "order": 1, + "type": "optionsListControl", + "width": "medium" + }, + "2b1842a2-09b1-4d52-9e1a-6755ca7b71fb": { + "explicitInput": { + "dataViewId": "security-solution-default", + "exclude": false, + "existsSelected": false, + "fieldName": "observer.hostname", + "searchTechnique": "prefix", + "selectedOptions": [], + "sort": { + "by": "_count", + "direction": "desc" + } + }, + "grow": false, + "order": 6, + "type": "optionsListControl", + "width": "medium" + }, + "4fe1a1ce-9005-48ff-bf0c-1aa6e1796d8e": { + "explicitInput": { + "dataViewId": "security-solution-default", + "fieldName": "observer.version", + "searchTechnique": "prefix", + "selectedOptions": [], + "sort": { + "by": "_count", + "direction": "desc" + } + }, + "grow": false, + "order": 7, + "type": "optionsListControl", + "width": "medium" + }, + "723c30e6-d971-44be-8e08-5f6ad06af04c": { + "explicitInput": { + "dataViewId": "security-solution-default", + "fieldName": "event.kind", + "searchTechnique": "prefix", + "selectedOptions": [], + "sort": { + "by": "_count", + "direction": "desc" + } + }, + "grow": false, + "order": 0, + "type": "optionsListControl", + "width": "medium" + }, + "987596fc-37bc-492c-8041-a768c90c5338": { + "explicitInput": { + "dataViewId": "security-solution-default", + "exclude": true, + "fieldName": "tags", + "searchTechnique": "prefix", + "selectedOptions": [], + "sort": { + "by": "_count", + "direction": "desc" + } + }, + "grow": false, + "order": 4, + "type": "optionsListControl", + "width": "medium" + }, + "9fb8d90a-ed73-4a0b-969c-209a79c0a9a3": { + "explicitInput": { + "dataViewId": "security-solution-default", + "fieldName": "event.category", + "searchTechnique": "prefix", + "selectedOptions": [], + "sort": { + "by": "_count", + "direction": "desc" + } + }, + "grow": false, + "order": 2, + "type": "optionsListControl", + "width": "medium" + }, + "ab7743e3-79a4-4e06-8653-9d50d7dad1d3": { + "explicitInput": { + "dataViewId": "security-solution-default", + "fieldName": "event.action", + "searchTechnique": "prefix", + "selectedOptions": [], + "sort": { + "by": "_count", + "direction": "desc" + } + }, + "grow": false, + "order": 3, + "type": "optionsListControl", + "width": "medium" + }, + "c50d8ab9-c2d5-459a-88e0-13fdfdba4082": { + "explicitInput": { + "dataViewId": "security-solution-default", + "fieldName": "observer.product", + "searchTechnique": "prefix", + "selectedOptions": [], + "sort": { + "by": "_count", + "direction": "desc" + } + }, + "grow": false, + "order": 5, + "type": "optionsListControl", + "width": "medium" + } + }, + "showApplySelections": false + }, + "description": "Summary of events stored in ubnt_unifi.logs data stream.", + "kibanaSavedObjectMeta": { + "searchSourceJSON": { + "filter": [ + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "field": "data_stream.dataset", + "indexRefName": "kibanaSavedObjectMeta.searchSourceJSON.filter[0].meta.index", + "key": "data_stream.dataset", + "negate": false, + "params": { + "query": "ubnt_unifi.logs" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "data_stream.dataset": "ubnt_unifi.logs" + } + } + } + ], + "query": { + "language": "kuery", + "query": "" + } + } + }, + "optionsJSON": { + "hidePanelTitles": false, + "syncColors": false, + "syncCursor": true, + "syncTooltips": false, + "useMargins": true + }, + "panelsJSON": [ + { + "embeddableConfig": { + "attributes": { + "references": [ + { + "id": "security-solution-default", + "name": "indexpattern-datasource-layer-8767f105-d6d4-4ab4-bffd-17c6010302cc", + "type": "index-pattern" + } + ], + "state": { + "adHocDataViews": {}, + "datasourceStates": { + "formBased": { + "currentIndexPatternId": "security-solution-default", + "layers": { + "8767f105-d6d4-4ab4-bffd-17c6010302cc": { + "columnOrder": [ + "635445ad-e319-4f5e-be61-fb51df2018e5", + "c6a40940-7da5-48f2-8ea4-94b450d572bd", + "4d9f01a3-ee9e-4bb8-8e9f-42e37bca97f0" + ], + "columns": { + "4d9f01a3-ee9e-4bb8-8e9f-42e37bca97f0": { + "customLabel": true, + "dataType": "number", + "isBucketed": false, + "label": "Logs", + "operationType": "count", + "params": { + "emptyAsNull": true + }, + "scale": "ratio", + "sourceField": "___records___" + }, + "635445ad-e319-4f5e-be61-fb51df2018e5": { + "dataType": "string", + "isBucketed": true, + "label": "Top 5 values of event.kind", + "operationType": "terms", + "params": { + "exclude": [], + "excludeIsRegex": false, + "include": [], + "includeIsRegex": false, + "missingBucket": false, + "orderBy": { + "columnId": "4d9f01a3-ee9e-4bb8-8e9f-42e37bca97f0", + "type": "column" + }, + "orderDirection": "desc", + "otherBucket": true, + "parentFormat": { + "id": "terms" + }, + "secondaryFields": [], + "size": 5 + }, + "scale": "ordinal", + "sourceField": "event.kind" + }, + "c6a40940-7da5-48f2-8ea4-94b450d572bd": { + "customLabel": true, + "dataType": "date", + "isBucketed": true, + "label": "Logs", + "operationType": "date_histogram", + "params": { + "dropPartials": false, + "includeEmptyRows": true, + "interval": "h" + }, + "scale": "interval", + "sourceField": "@timestamp" + } + }, + "incompleteColumns": {}, + "indexPatternId": "security-solution-default", + "sampling": 1 + } + } + }, + "indexpattern": { + "currentIndexPatternId": "security-solution-default", + "layers": {} + }, + "textBased": { + "indexPatternRefs": [ + { + "id": "security-solution-default", + "timeField": "@timestamp", + "title": ".alerts-security.alerts-default,apm-*-transaction*,auditbeat-*,endgame-*,filebeat-*,logs-*,packetbeat-*,traces-apm*,winlogbeat-*,-*elastic-cloud-logs-*" + } + ], + "layers": {} + } + }, + "filters": [], + "internalReferences": [], + "query": { + "language": "kuery", + "query": "" + }, + "visualization": { + "axisTitlesVisibilitySettings": { + "x": true, + "yLeft": true, + "yRight": true + }, + "fittingFunction": "Linear", + "gridlinesVisibilitySettings": { + "x": true, + "yLeft": true, + "yRight": true + }, + "labelsOrientation": { + "x": 0, + "yLeft": 0, + "yRight": 0 + }, + "layers": [ + { + "accessors": [ + "4d9f01a3-ee9e-4bb8-8e9f-42e37bca97f0" + ], + "colorMapping": { + "assignments": [], + "colorMode": { + "type": "categorical" + }, + "paletteId": "elastic_brand_2023", + "specialAssignments": [ + { + "color": { + "type": "loop" + }, + "rule": { + "type": "other" + }, + "touched": false + } + ] + }, + "layerId": "8767f105-d6d4-4ab4-bffd-17c6010302cc", + "layerType": "data", + "position": "top", + "seriesType": "bar_stacked", + "showGridlines": false, + "splitAccessor": "635445ad-e319-4f5e-be61-fb51df2018e5", + "xAccessor": "c6a40940-7da5-48f2-8ea4-94b450d572bd" + } + ], + "legend": { + "isVisible": true, + "position": "right", + "showSingleSeries": true + }, + "preferredSeriesType": "bar_stacked", + "tickLabelsVisibilitySettings": { + "x": true, + "yLeft": true, + "yRight": true + }, + "valueLabels": "hide" + } + }, + "title": "", + "type": "lens", + "visualizationType": "lnsXY" + }, + "enhancements": { + "dynamicActions": { + "events": [] + } + }, + "filters": [], + "query": { + "language": "kuery", + "query": "" + }, + "syncColors": false, + "syncCursor": true, + "syncTooltips": false + }, + "gridData": { + "h": 8, + "i": "11cd71a2-9686-4bcc-8ced-0a8c9213ad77", + "w": 41, + "x": 7, + "y": 0 + }, + "panelIndex": "11cd71a2-9686-4bcc-8ced-0a8c9213ad77", + "type": "lens" + }, + { + "embeddableConfig": { + "attributes": { + "references": [ + { + "id": "security-solution-default", + "name": "indexpattern-datasource-layer-aeb24e48-cd0e-46d1-a961-d1bfc76513bd", + "type": "index-pattern" + }, + { + "id": "security-solution-default", + "name": "250338b1-479f-40bd-8b61-ff47a1de45fa", + "type": "index-pattern" + } + ], + "state": { + "adHocDataViews": {}, + "datasourceStates": { + "formBased": { + "layers": { + "aeb24e48-cd0e-46d1-a961-d1bfc76513bd": { + "columnOrder": [ + "871d84fb-e14c-45cd-92cc-edd790dd8f63" + ], + "columns": { + "871d84fb-e14c-45cd-92cc-edd790dd8f63": { + "customLabel": true, + "dataType": "number", + "isBucketed": false, + "label": "Logs", + "operationType": "count", + "params": { + "emptyAsNull": true + }, + "scale": "ratio", + "sourceField": "___records___" + } + }, + "incompleteColumns": {}, + "sampling": 1 + } + } + }, + "indexpattern": { + "layers": {} + }, + "textBased": { + "layers": {} + } + }, + "filters": [ + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "field": "data_stream.dataset", + "index": "250338b1-479f-40bd-8b61-ff47a1de45fa", + "key": "data_stream.dataset", + "negate": false, + "params": { + "query": "ubnt_unifi.logs" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "data_stream.dataset": "ubnt_unifi.logs" + } + } + } + ], + "internalReferences": [], + "query": { + "language": "kuery", + "query": "" + }, + "visualization": { + "layerId": "aeb24e48-cd0e-46d1-a961-d1bfc76513bd", + "layerType": "data", + "metricAccessor": "871d84fb-e14c-45cd-92cc-edd790dd8f63" + } + }, + "title": "", + "type": "lens", + "visualizationType": "lnsMetric" + }, + "enhancements": { + "dynamicActions": { + "events": [] + } + }, + "filters": [ + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "field": "data_stream.dataset", + "index": "250338b1-479f-40bd-8b61-ff47a1de45fa", + "key": "data_stream.dataset", + "negate": false, + "params": { + "query": "ubnt_unifi.logs" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "data_stream.dataset": "ubnt_unifi.logs" + } + } + } + ], + "query": { + "language": "kuery", + "query": "" + }, + "syncColors": false, + "syncCursor": true, + "syncTooltips": false + }, + "gridData": { + "h": 4, + "i": "dc177bbb-04fb-4b8a-b3e5-ac0a938e76cf", + "w": 7, + "x": 0, + "y": 0 + }, + "panelIndex": "dc177bbb-04fb-4b8a-b3e5-ac0a938e76cf", + "type": "lens" + }, + { + "embeddableConfig": { + "attributes": { + "references": [ + { + "id": "security-solution-default", + "name": "indexpattern-datasource-layer-aeb24e48-cd0e-46d1-a961-d1bfc76513bd", + "type": "index-pattern" + } + ], + "state": { + "adHocDataViews": {}, + "datasourceStates": { + "formBased": { + "currentIndexPatternId": "security-solution-default", + "layers": { + "aeb24e48-cd0e-46d1-a961-d1bfc76513bd": { + "columnOrder": [ + "871d84fb-e14c-45cd-92cc-edd790dd8f63" + ], + "columns": { + "871d84fb-e14c-45cd-92cc-edd790dd8f63": { + "customLabel": true, + "dataType": "number", + "isBucketed": false, + "label": "Alerts", + "operationType": "count", + "params": { + "emptyAsNull": true + }, + "scale": "ratio", + "sourceField": "___records___" + } + }, + "incompleteColumns": {}, + "indexPatternId": "security-solution-default", + "sampling": 1 + } + } + }, + "indexpattern": { + "currentIndexPatternId": "security-solution-default", + "layers": {} + }, + "textBased": { + "indexPatternRefs": [ + { + "id": "security-solution-default", + "timeField": "@timestamp", + "title": ".alerts-security.alerts-default,apm-*-transaction*,auditbeat-*,endgame-*,filebeat-*,logs-*,packetbeat-*,traces-apm*,winlogbeat-*,-*elastic-cloud-logs-*" + } + ], + "layers": {} + } + }, + "filters": [ + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "field": "data_stream.dataset", + "index": "a164a46e-c0fe-485b-a64d-03cda45e1d0d", + "key": "data_stream.dataset", + "negate": false, + "params": { + "query": "ubnt_unifi.logs" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "data_stream.dataset": "ubnt_unifi.logs" + } + } + }, + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "index": "237f8213-fdc9-4a8d-b0b8-854938e70d5d", + "key": "event.kind", + "negate": false, + "params": { + "query": "alert" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "event.kind": "alert" + } + } + } + ], + "internalReferences": [], + "query": { + "language": "kuery", + "query": "" + }, + "visualization": { + "icon": "bell", + "layerId": "aeb24e48-cd0e-46d1-a961-d1bfc76513bd", + "layerType": "data", + "metricAccessor": "871d84fb-e14c-45cd-92cc-edd790dd8f63" + } + }, + "title": "", + "type": "lens", + "visualizationType": "lnsMetric" + }, + "enhancements": { + "dynamicActions": { + "events": [] + } + }, + "filters": [ + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "field": "data_stream.dataset", + "index": "a164a46e-c0fe-485b-a64d-03cda45e1d0d", + "key": "data_stream.dataset", + "negate": false, + "params": { + "query": "ubnt_unifi.logs" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "data_stream.dataset": "ubnt_unifi.logs" + } + } + }, + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "index": "237f8213-fdc9-4a8d-b0b8-854938e70d5d", + "key": "event.kind", + "negate": false, + "params": { + "query": "alert" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "event.kind": "alert" + } + } + } + ], + "query": { + "language": "kuery", + "query": "" + }, + "syncColors": false, + "syncCursor": true, + "syncTooltips": false + }, + "gridData": { + "h": 4, + "i": "dd3c5e39-829e-4254-8708-af6d35c09a89", + "w": 7, + "x": 0, + "y": 4 + }, + "panelIndex": "dd3c5e39-829e-4254-8708-af6d35c09a89", + "title": "", + "type": "lens" + }, + { + "embeddableConfig": { + "attributes": { + "references": [ + { + "id": "security-solution-default", + "name": "indexpattern-datasource-layer-aeb24e48-cd0e-46d1-a961-d1bfc76513bd", + "type": "index-pattern" + }, + { + "id": "security-solution-default", + "name": "852dda7e-9c15-4f19-b721-00febdc78f51", + "type": "index-pattern" + }, + { + "id": "security-solution-default", + "name": "e0249f2b-0916-4c2f-a976-f958d0d041cf", + "type": "index-pattern" + }, + { + "id": "security-solution-default", + "name": "cc16d42c-ec98-4597-a034-532a946f0442", + "type": "index-pattern" + } + ], + "state": { + "adHocDataViews": {}, + "datasourceStates": { + "formBased": { + "layers": { + "aeb24e48-cd0e-46d1-a961-d1bfc76513bd": { + "columnOrder": [ + "871d84fb-e14c-45cd-92cc-edd790dd8f63" + ], + "columns": { + "871d84fb-e14c-45cd-92cc-edd790dd8f63": { + "customLabel": true, + "dataType": "number", + "isBucketed": false, + "label": "Errors", + "operationType": "count", + "params": { + "emptyAsNull": false + }, + "scale": "ratio", + "sourceField": "___records___" + } + }, + "incompleteColumns": {}, + "sampling": 1 + } + } + }, + "indexpattern": { + "layers": {} + }, + "textBased": { + "layers": {} + } + }, + "filters": [ + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "field": "data_stream.dataset", + "index": "852dda7e-9c15-4f19-b721-00febdc78f51", + "key": "data_stream.dataset", + "negate": false, + "params": { + "query": "ubnt_unifi.logs" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "data_stream.dataset": "ubnt_unifi.logs" + } + } + }, + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "index": "e0249f2b-0916-4c2f-a976-f958d0d041cf", + "key": "event.kind", + "negate": true, + "params": { + "query": "event" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "event.kind": "event" + } + } + }, + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "index": "cc16d42c-ec98-4597-a034-532a946f0442", + "key": "event.kind", + "negate": true, + "params": { + "query": "alert" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "event.kind": "alert" + } + } + } + ], + "internalReferences": [], + "query": { + "language": "kuery", + "query": "" + }, + "visualization": { + "icon": "alert", + "layerId": "aeb24e48-cd0e-46d1-a961-d1bfc76513bd", + "layerType": "data", + "metricAccessor": "871d84fb-e14c-45cd-92cc-edd790dd8f63" + } + }, + "title": "", + "type": "lens", + "visualizationType": "lnsMetric" + }, + "enhancements": { + "dynamicActions": { + "events": [] + } + }, + "filters": [ + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "field": "data_stream.dataset", + "index": "852dda7e-9c15-4f19-b721-00febdc78f51", + "key": "data_stream.dataset", + "negate": false, + "params": { + "query": "ubnt_unifi.logs" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "data_stream.dataset": "ubnt_unifi.logs" + } + } + }, + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "index": "e0249f2b-0916-4c2f-a976-f958d0d041cf", + "key": "event.kind", + "negate": true, + "params": { + "query": "event" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "event.kind": "event" + } + } + }, + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "index": "cc16d42c-ec98-4597-a034-532a946f0442", + "key": "event.kind", + "negate": true, + "params": { + "query": "alert" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "event.kind": "alert" + } + } + } + ], + "query": { + "language": "kuery", + "query": "" + }, + "syncColors": false, + "syncCursor": true, + "syncTooltips": false + }, + "gridData": { + "h": 4, + "i": "cced8313-62a3-49a6-9b05-7cfd019825fa", + "w": 7, + "x": 0, + "y": 8 + }, + "panelIndex": "cced8313-62a3-49a6-9b05-7cfd019825fa", + "title": "", + "type": "lens" + }, + { + "embeddableConfig": { + "attributes": { + "references": [ + { + "id": "security-solution-default", + "name": "indexpattern-datasource-layer-aeb24e48-cd0e-46d1-a961-d1bfc76513bd", + "type": "index-pattern" + }, + { + "id": "security-solution-default", + "name": "cff00a52-67a8-452a-aa84-9736fc93168c", + "type": "index-pattern" + } + ], + "state": { + "adHocDataViews": {}, + "datasourceStates": { + "formBased": { + "layers": { + "aeb24e48-cd0e-46d1-a961-d1bfc76513bd": { + "columnOrder": [ + "871d84fb-e14c-45cd-92cc-edd790dd8f63" + ], + "columns": { + "871d84fb-e14c-45cd-92cc-edd790dd8f63": { + "customLabel": true, + "dataType": "number", + "isBucketed": false, + "label": "Firewall Logs", + "operationType": "count", + "params": { + "emptyAsNull": false + }, + "scale": "ratio", + "sourceField": "network.community_id" + } + }, + "incompleteColumns": {}, + "sampling": 1 + } + } + }, + "indexpattern": { + "layers": {} + }, + "textBased": { + "layers": {} + } + }, + "filters": [ + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "field": "data_stream.dataset", + "index": "cff00a52-67a8-452a-aa84-9736fc93168c", + "key": "data_stream.dataset", + "negate": false, + "params": { + "query": "ubnt_unifi.logs" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "data_stream.dataset": "ubnt_unifi.logs" + } + } + } + ], + "internalReferences": [], + "query": { + "language": "kuery", + "query": "" + }, + "visualization": { + "icon": "empty", + "layerId": "aeb24e48-cd0e-46d1-a961-d1bfc76513bd", + "layerType": "data", + "metricAccessor": "871d84fb-e14c-45cd-92cc-edd790dd8f63" + } + }, + "title": "", + "type": "lens", + "visualizationType": "lnsMetric" + }, + "enhancements": { + "dynamicActions": { + "events": [] + } + }, + "filters": [ + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "field": "data_stream.dataset", + "index": "cff00a52-67a8-452a-aa84-9736fc93168c", + "key": "data_stream.dataset", + "negate": false, + "params": { + "query": "ubnt_unifi.logs" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "data_stream.dataset": "ubnt_unifi.logs" + } + } + } + ], + "query": { + "language": "kuery", + "query": "" + }, + "syncColors": false, + "syncCursor": true, + "syncTooltips": false + }, + "gridData": { + "h": 4, + "i": "7629b0e2-9589-470d-ab17-f677925d7d6e", + "w": 7, + "x": 0, + "y": 12 + }, + "panelIndex": "7629b0e2-9589-470d-ab17-f677925d7d6e", + "title": "", + "type": "lens" + }, + { + "embeddableConfig": { + "attributes": { + "references": [ + { + "id": "security-solution-default", + "name": "indexpattern-datasource-layer-aeb24e48-cd0e-46d1-a961-d1bfc76513bd", + "type": "index-pattern" + } + ], + "state": { + "adHocDataViews": {}, + "datasourceStates": { + "formBased": { + "currentIndexPatternId": "security-solution-default", + "layers": { + "aeb24e48-cd0e-46d1-a961-d1bfc76513bd": { + "columnOrder": [ + "871d84fb-e14c-45cd-92cc-edd790dd8f63" + ], + "columns": { + "871d84fb-e14c-45cd-92cc-edd790dd8f63": { + "customLabel": true, + "dataType": "number", + "isBucketed": false, + "label": "UniFi OS Logs", + "operationType": "count", + "params": { + "emptyAsNull": false + }, + "scale": "ratio", + "sourceField": "___records___" + } + }, + "incompleteColumns": {}, + "indexPatternId": "security-solution-default", + "sampling": 1 + } + } + }, + "indexpattern": { + "currentIndexPatternId": "security-solution-default", + "layers": {} + }, + "textBased": { + "indexPatternRefs": [ + { + "id": "security-solution-default", + "timeField": "@timestamp", + "title": ".alerts-security.alerts-default,apm-*-transaction*,auditbeat-*,endgame-*,filebeat-*,logs-*,packetbeat-*,traces-apm*,winlogbeat-*,-*elastic-cloud-logs-*" + } + ], + "initialContext": null, + "layers": {} + } + }, + "filters": [ + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "field": "data_stream.dataset", + "index": "827edd75-af86-4007-b380-098d050ca7a7", + "key": "data_stream.dataset", + "negate": false, + "params": { + "query": "ubnt_unifi.logs" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "data_stream.dataset": "ubnt_unifi.logs" + } + } + }, + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "index": "a5efe007-8eef-49b7-b3a3-3711f57f0d5a", + "key": "observer.product", + "negate": false, + "params": { + "query": "UniFi OS" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "observer.product": "UniFi OS" + } + } + } + ], + "internalReferences": [], + "query": { + "language": "kuery", + "query": "" + }, + "visualization": { + "icon": "compute", + "layerId": "aeb24e48-cd0e-46d1-a961-d1bfc76513bd", + "layerType": "data", + "metricAccessor": "871d84fb-e14c-45cd-92cc-edd790dd8f63" + } + }, + "title": "", + "type": "lens", + "visualizationType": "lnsMetric" + }, + "enhancements": { + "dynamicActions": { + "events": [] + } + }, + "filters": [ + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "field": "data_stream.dataset", + "index": "827edd75-af86-4007-b380-098d050ca7a7", + "key": "data_stream.dataset", + "negate": false, + "params": { + "query": "ubnt_unifi.logs" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "data_stream.dataset": "ubnt_unifi.logs" + } + } + }, + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "index": "a5efe007-8eef-49b7-b3a3-3711f57f0d5a", + "key": "observer.product", + "negate": false, + "params": { + "query": "UniFi OS" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "observer.product": "UniFi OS" + } + } + } + ], + "query": { + "language": "kuery", + "query": "" + }, + "syncColors": false, + "syncCursor": true, + "syncTooltips": false + }, + "gridData": { + "h": 4, + "i": "e069fc73-f5c9-4e4c-9e5f-1f8f0320bf45", + "w": 7, + "x": 0, + "y": 16 + }, + "panelIndex": "e069fc73-f5c9-4e4c-9e5f-1f8f0320bf45", + "title": "", + "type": "lens" + }, + { + "embeddableConfig": { + "attributes": { + "references": [ + { + "id": "security-solution-default", + "name": "indexpattern-datasource-layer-aeb24e48-cd0e-46d1-a961-d1bfc76513bd", + "type": "index-pattern" + }, + { + "id": "security-solution-default", + "name": "f83809dc-fb2a-457a-a643-6eb1de99a15f", + "type": "index-pattern" + }, + { + "id": "security-solution-default", + "name": "eebf8c97-faba-48dc-8caf-3ef68b3bab5a", + "type": "index-pattern" + }, + { + "id": "security-solution-default", + "name": "5e948081-6323-4896-aab8-4a83c0e4ca46", + "type": "index-pattern" + } + ], + "state": { + "adHocDataViews": {}, + "datasourceStates": { + "formBased": { + "layers": { + "aeb24e48-cd0e-46d1-a961-d1bfc76513bd": { + "columnOrder": [ + "871d84fb-e14c-45cd-92cc-edd790dd8f63" + ], + "columns": { + "871d84fb-e14c-45cd-92cc-edd790dd8f63": { + "customLabel": true, + "dataType": "number", + "isBucketed": false, + "label": "UniFi Network Logs", + "operationType": "count", + "params": { + "emptyAsNull": false + }, + "scale": "ratio", + "sourceField": "observer.product" + } + }, + "incompleteColumns": {}, + "sampling": 1 + } + } + }, + "indexpattern": { + "layers": {} + }, + "textBased": { + "layers": {} + } + }, + "filters": [ + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "field": "data_stream.dataset", + "index": "f83809dc-fb2a-457a-a643-6eb1de99a15f", + "key": "data_stream.dataset", + "negate": false, + "params": { + "query": "ubnt_unifi.logs" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "data_stream.dataset": "ubnt_unifi.logs" + } + } + }, + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "index": "eebf8c97-faba-48dc-8caf-3ef68b3bab5a", + "key": "observer.product", + "negate": false, + "params": { + "query": "UniFi Network" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "observer.product": "UniFi Network" + } + } + }, + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "index": "5e948081-6323-4896-aab8-4a83c0e4ca46", + "key": "tags", + "negate": false, + "params": { + "query": "ubnt-unifi-cef" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "tags": "ubnt-unifi-cef" + } + } + } + ], + "internalReferences": [], + "query": { + "language": "kuery", + "query": "" + }, + "visualization": { + "icon": "empty", + "layerId": "aeb24e48-cd0e-46d1-a961-d1bfc76513bd", + "layerType": "data", + "metricAccessor": "871d84fb-e14c-45cd-92cc-edd790dd8f63" + } + }, + "title": "", + "type": "lens", + "visualizationType": "lnsMetric" + }, + "enhancements": { + "dynamicActions": { + "events": [] + } + }, + "filters": [ + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "field": "data_stream.dataset", + "index": "f83809dc-fb2a-457a-a643-6eb1de99a15f", + "key": "data_stream.dataset", + "negate": false, + "params": { + "query": "ubnt_unifi.logs" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "data_stream.dataset": "ubnt_unifi.logs" + } + } + }, + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "index": "eebf8c97-faba-48dc-8caf-3ef68b3bab5a", + "key": "observer.product", + "negate": false, + "params": { + "query": "UniFi Network" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "observer.product": "UniFi Network" + } + } + }, + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "index": "5e948081-6323-4896-aab8-4a83c0e4ca46", + "key": "tags", + "negate": false, + "params": { + "query": "ubnt-unifi-cef" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "tags": "ubnt-unifi-cef" + } + } + } + ], + "query": { + "language": "kuery", + "query": "" + }, + "syncColors": false, + "syncCursor": true, + "syncTooltips": false + }, + "gridData": { + "h": 4, + "i": "04599915-fee3-49ad-a6ad-d89ca76c4ed5", + "w": 7, + "x": 0, + "y": 20 + }, + "panelIndex": "04599915-fee3-49ad-a6ad-d89ca76c4ed5", + "title": "", + "type": "lens" + }, + { + "embeddableConfig": { + "attributes": { + "references": [ + { + "id": "security-solution-default", + "name": "indexpattern-datasource-layer-aeb24e48-cd0e-46d1-a961-d1bfc76513bd", + "type": "index-pattern" + }, + { + "id": "security-solution-default", + "name": "af967d8d-1fc9-446a-a059-fb01992eb166", + "type": "index-pattern" + }, + { + "id": "security-solution-default", + "name": "c2b5c5a8-5467-48d1-8f07-833e43c8f022", + "type": "index-pattern" + }, + { + "id": "security-solution-default", + "name": "c7f9ac4e-da31-4e49-b772-6497bf07fc2e", + "type": "index-pattern" + } + ], + "state": { + "adHocDataViews": {}, + "datasourceStates": { + "formBased": { + "layers": { + "aeb24e48-cd0e-46d1-a961-d1bfc76513bd": { + "columnOrder": [ + "871d84fb-e14c-45cd-92cc-edd790dd8f63" + ], + "columns": { + "871d84fb-e14c-45cd-92cc-edd790dd8f63": { + "customLabel": true, + "dataType": "number", + "isBucketed": false, + "label": "Other Logs", + "operationType": "count", + "params": { + "emptyAsNull": false + }, + "scale": "ratio", + "sourceField": "@timestamp" + } + }, + "incompleteColumns": {}, + "sampling": 1 + } + } + }, + "indexpattern": { + "layers": {} + }, + "textBased": { + "layers": {} + } + }, + "filters": [ + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "field": "data_stream.dataset", + "index": "af967d8d-1fc9-446a-a059-fb01992eb166", + "key": "data_stream.dataset", + "negate": false, + "params": { + "query": "ubnt_unifi.logs" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "data_stream.dataset": "ubnt_unifi.logs" + } + } + }, + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "index": "c2b5c5a8-5467-48d1-8f07-833e43c8f022", + "key": "observer.product", + "negate": true, + "params": { + "query": "UniFi Network" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "observer.product": "UniFi Network" + } + } + }, + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "index": "c7f9ac4e-da31-4e49-b772-6497bf07fc2e", + "key": "observer.product", + "negate": true, + "params": { + "query": "UniFi OS" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "observer.product": "UniFi OS" + } + } + } + ], + "internalReferences": [], + "query": { + "language": "kuery", + "query": "" + }, + "visualization": { + "icon": "empty", + "layerId": "aeb24e48-cd0e-46d1-a961-d1bfc76513bd", + "layerType": "data", + "metricAccessor": "871d84fb-e14c-45cd-92cc-edd790dd8f63" + } + }, + "title": "", + "type": "lens", + "visualizationType": "lnsMetric" + }, + "enhancements": { + "dynamicActions": { + "events": [] + } + }, + "filters": [ + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "field": "data_stream.dataset", + "index": "af967d8d-1fc9-446a-a059-fb01992eb166", + "key": "data_stream.dataset", + "negate": false, + "params": { + "query": "ubnt_unifi.logs" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "data_stream.dataset": "ubnt_unifi.logs" + } + } + }, + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "index": "c2b5c5a8-5467-48d1-8f07-833e43c8f022", + "key": "observer.product", + "negate": true, + "params": { + "query": "UniFi Network" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "observer.product": "UniFi Network" + } + } + }, + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "index": "c7f9ac4e-da31-4e49-b772-6497bf07fc2e", + "key": "observer.product", + "negate": true, + "params": { + "query": "UniFi OS" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "observer.product": "UniFi OS" + } + } + } + ], + "query": { + "language": "kuery", + "query": "" + }, + "syncColors": false, + "syncCursor": true, + "syncTooltips": false + }, + "gridData": { + "h": 4, + "i": "b0828a80-dfae-4436-ac90-2ac2d91560f4", + "w": 7, + "x": 0, + "y": 24 + }, + "panelIndex": "b0828a80-dfae-4436-ac90-2ac2d91560f4", + "title": "", + "type": "lens" + }, + { + "embeddableConfig": { + "attributes": { + "references": [ + { + "id": "security-solution-default", + "name": "indexpattern-datasource-layer-aeb24e48-cd0e-46d1-a961-d1bfc76513bd", + "type": "index-pattern" + }, + { + "id": "security-solution-default", + "name": "4513e263-11aa-4988-b240-b1389e3642f4", + "type": "index-pattern" + } + ], + "state": { + "adHocDataViews": {}, + "datasourceStates": { + "formBased": { + "layers": { + "aeb24e48-cd0e-46d1-a961-d1bfc76513bd": { + "columnOrder": [ + "ecc1e484-9b91-46f9-8692-3db6ec58b1e3", + "871d84fb-e14c-45cd-92cc-edd790dd8f63" + ], + "columns": { + "871d84fb-e14c-45cd-92cc-edd790dd8f63": { + "customLabel": true, + "dataType": "number", + "isBucketed": false, + "label": "Logs", + "operationType": "count", + "params": { + "emptyAsNull": false + }, + "scale": "ratio", + "sourceField": "___records___" + }, + "ecc1e484-9b91-46f9-8692-3db6ec58b1e3": { + "dataType": "string", + "isBucketed": true, + "label": "Top 20 values of process.name", + "operationType": "terms", + "params": { + "exclude": [], + "excludeIsRegex": false, + "include": [], + "includeIsRegex": false, + "missingBucket": false, + "orderBy": { + "columnId": "871d84fb-e14c-45cd-92cc-edd790dd8f63", + "type": "column" + }, + "orderDirection": "desc", + "otherBucket": true, + "parentFormat": { + "id": "terms" + }, + "size": 20 + }, + "scale": "ordinal", + "sourceField": "process.name" + } + }, + "incompleteColumns": {}, + "sampling": 1 + } + } + }, + "indexpattern": { + "layers": {} + }, + "textBased": { + "layers": {} + } + }, + "filters": [ + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "field": "data_stream.dataset", + "index": "4513e263-11aa-4988-b240-b1389e3642f4", + "key": "data_stream.dataset", + "negate": false, + "params": { + "query": "ubnt_unifi.logs" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "data_stream.dataset": "ubnt_unifi.logs" + } + } + } + ], + "internalReferences": [], + "query": { + "language": "kuery", + "query": "" + }, + "visualization": { + "columns": [ + { + "columnId": "ecc1e484-9b91-46f9-8692-3db6ec58b1e3", + "isTransposed": false + }, + { + "columnId": "871d84fb-e14c-45cd-92cc-edd790dd8f63", + "isTransposed": false + } + ], + "layerId": "aeb24e48-cd0e-46d1-a961-d1bfc76513bd", + "layerType": "data" + } + }, + "title": "", + "type": "lens", + "visualizationType": "lnsDatatable" + }, + "enhancements": { + "dynamicActions": { + "events": [] + } + }, + "filters": [ + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "field": "data_stream.dataset", + "index": "4513e263-11aa-4988-b240-b1389e3642f4", + "key": "data_stream.dataset", + "negate": false, + "params": { + "query": "ubnt_unifi.logs" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "data_stream.dataset": "ubnt_unifi.logs" + } + } + } + ], + "query": { + "language": "kuery", + "query": "" + }, + "syncColors": false, + "syncCursor": true, + "syncTooltips": false + }, + "gridData": { + "h": 20, + "i": "55b60bbc-5a8d-43dd-b08d-5685bf5d97ac", + "w": 18, + "x": 30, + "y": 8 + }, + "panelIndex": "55b60bbc-5a8d-43dd-b08d-5685bf5d97ac", + "title": "Generic Logs by Process Name", + "type": "lens" + }, + { + "embeddableConfig": { + "attributes": { + "references": [ + { + "id": "security-solution-default", + "name": "indexpattern-datasource-layer-aeb24e48-cd0e-46d1-a961-d1bfc76513bd", + "type": "index-pattern" + } + ], + "state": { + "adHocDataViews": {}, + "datasourceStates": { + "formBased": { + "currentIndexPatternId": "security-solution-default", + "layers": { + "aeb24e48-cd0e-46d1-a961-d1bfc76513bd": { + "columnOrder": [ + "5460d3db-d55f-4b46-9313-266c04a2e359", + "07b2015b-b1fd-4fb7-8216-bf74a4f51fb6", + "11d28877-1bff-4d83-8d4e-6fc49b3c977b", + "d78e312a-96c2-41ff-9058-1992790967a5", + "871d84fb-e14c-45cd-92cc-edd790dd8f63" + ], + "columns": { + "07b2015b-b1fd-4fb7-8216-bf74a4f51fb6": { + "customLabel": true, + "dataType": "string", + "isBucketed": true, + "label": "Product / Model", + "operationType": "terms", + "params": { + "exclude": [], + "excludeIsRegex": false, + "include": [], + "includeIsRegex": false, + "missingBucket": true, + "orderBy": { + "columnId": "871d84fb-e14c-45cd-92cc-edd790dd8f63", + "type": "column" + }, + "orderDirection": "desc", + "otherBucket": true, + "parentFormat": { + "id": "terms" + }, + "size": 10 + }, + "scale": "ordinal", + "sourceField": "observer.product" + }, + "11d28877-1bff-4d83-8d4e-6fc49b3c977b": { + "customLabel": true, + "dataType": "string", + "isBucketed": true, + "label": "Version", + "operationType": "terms", + "params": { + "exclude": [], + "excludeIsRegex": false, + "include": [], + "includeIsRegex": false, + "missingBucket": true, + "orderBy": { + "columnId": "871d84fb-e14c-45cd-92cc-edd790dd8f63", + "type": "column" + }, + "orderDirection": "desc", + "otherBucket": true, + "parentFormat": { + "id": "terms" + }, + "size": 10 + }, + "scale": "ordinal", + "sourceField": "observer.version" + }, + "5460d3db-d55f-4b46-9313-266c04a2e359": { + "customLabel": true, + "dataType": "string", + "isBucketed": true, + "label": "Name", + "operationType": "terms", + "params": { + "exclude": [], + "excludeIsRegex": false, + "include": [], + "includeIsRegex": false, + "missingBucket": true, + "orderBy": { + "columnId": "871d84fb-e14c-45cd-92cc-edd790dd8f63", + "type": "column" + }, + "orderDirection": "desc", + "otherBucket": true, + "parentFormat": { + "id": "terms" + }, + "size": 10 + }, + "scale": "ordinal", + "sourceField": "observer.hostname" + }, + "871d84fb-e14c-45cd-92cc-edd790dd8f63": { + "customLabel": true, + "dataType": "number", + "isBucketed": false, + "label": "Logs", + "operationType": "count", + "params": { + "emptyAsNull": false + }, + "scale": "ratio", + "sourceField": "___records___" + }, + "d78e312a-96c2-41ff-9058-1992790967a5": { + "customLabel": true, + "dataType": "string", + "isBucketed": true, + "label": "Serial#", + "operationType": "terms", + "params": { + "exclude": [], + "excludeIsRegex": false, + "include": [], + "includeIsRegex": false, + "missingBucket": true, + "orderBy": { + "columnId": "871d84fb-e14c-45cd-92cc-edd790dd8f63", + "type": "column" + }, + "orderDirection": "desc", + "otherBucket": true, + "parentFormat": { + "id": "terms" + }, + "size": 3 + }, + "scale": "ordinal", + "sourceField": "observer.serial_number" + } + }, + "incompleteColumns": {}, + "indexPatternId": "security-solution-default", + "sampling": 1 + } + } + }, + "indexpattern": { + "currentIndexPatternId": "security-solution-default", + "layers": {} + }, + "textBased": { + "indexPatternRefs": [ + { + "id": "security-solution-default", + "timeField": "@timestamp", + "title": ".alerts-security.alerts-default,apm-*-transaction*,auditbeat-*,endgame-*,filebeat-*,logs-*,packetbeat-*,traces-apm*,winlogbeat-*,-*elastic-cloud-logs-*" + } + ], + "layers": {} + } + }, + "filters": [ + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "field": "data_stream.dataset", + "index": "f1cd805b-bbee-4cb0-b27c-9b87385d585d", + "key": "data_stream.dataset", + "negate": false, + "params": { + "query": "ubnt_unifi.logs" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "data_stream.dataset": "ubnt_unifi.logs" + } + } + } + ], + "internalReferences": [], + "query": { + "language": "kuery", + "query": "" + }, + "visualization": { + "columns": [ + { + "columnId": "5460d3db-d55f-4b46-9313-266c04a2e359", + "isTransposed": false + }, + { + "columnId": "07b2015b-b1fd-4fb7-8216-bf74a4f51fb6", + "isTransposed": false + }, + { + "columnId": "11d28877-1bff-4d83-8d4e-6fc49b3c977b", + "isTransposed": false + }, + { + "columnId": "871d84fb-e14c-45cd-92cc-edd790dd8f63", + "isTransposed": false + }, + { + "columnId": "d78e312a-96c2-41ff-9058-1992790967a5", + "isMetric": false, + "isTransposed": false + } + ], + "layerId": "aeb24e48-cd0e-46d1-a961-d1bfc76513bd", + "layerType": "data", + "sorting": { + "columnId": "871d84fb-e14c-45cd-92cc-edd790dd8f63", + "direction": "desc" + } + } + }, + "title": "", + "type": "lens", + "visualizationType": "lnsDatatable" + }, + "enhancements": { + "dynamicActions": { + "events": [] + } + }, + "filters": [ + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "field": "data_stream.dataset", + "index": "f1cd805b-bbee-4cb0-b27c-9b87385d585d", + "key": "data_stream.dataset", + "negate": false, + "params": { + "query": "ubnt_unifi.logs" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "data_stream.dataset": "ubnt_unifi.logs" + } + } + } + ], + "query": { + "language": "kuery", + "query": "" + }, + "syncColors": false, + "syncCursor": true, + "syncTooltips": false + }, + "gridData": { + "h": 20, + "i": "4f81513d-7730-4274-bb18-41fedb103005", + "w": 23, + "x": 7, + "y": 8 + }, + "panelIndex": "4f81513d-7730-4274-bb18-41fedb103005", + "title": "Logs by Source / Observer", + "type": "lens" + }, + { + "embeddableConfig": { + "description": "", + "enhancements": { + "dynamicActions": { + "events": [] + } + }, + "grid": { + "columns": { + "event.category": { + "width": 173 + }, + "event.kind": { + "width": 130 + }, + "event.type": { + "width": 133 + }, + "observer.hostname": { + "width": 191 + }, + "observer.product": { + "width": 176 + }, + "observer.version": { + "width": 180 + } + } + } + }, + "gridData": { + "h": 22, + "i": "cd97428e-825d-43d3-8341-81a4343ac2e2", + "w": 48, + "x": 0, + "y": 28 + }, + "panelIndex": "cd97428e-825d-43d3-8341-81a4343ac2e2", + "panelRefName": "panel_cd97428e-825d-43d3-8341-81a4343ac2e2", + "title": "[Ubiquiti UniFi] Logs", + "type": "search" + } + ], + "timeRestore": false, + "title": "[Ubiquiti UniFi] Logs Summary", + "version": 3 + }, + "coreMigrationVersion": "8.8.0", + "created_at": "2025-07-16T06:10:25.716Z", + "id": "ubnt_unifi-caf8fd79-ae35-4850-98b6-92f7758f2ca2", + "references": [ + { + "id": "security-solution-default", + "name": "kibanaSavedObjectMeta.searchSourceJSON.filter[0].meta.index", + "type": "index-pattern" + }, + { + "id": "ubnt_unifi-19723f42-25ee-4590-816b-db4b52baf6aa", + "name": "cd97428e-825d-43d3-8341-81a4343ac2e2:panel_cd97428e-825d-43d3-8341-81a4343ac2e2", + "type": "search" + }, + { + "id": "security-solution-default", + "name": "11cd71a2-9686-4bcc-8ced-0a8c9213ad77:indexpattern-datasource-layer-8767f105-d6d4-4ab4-bffd-17c6010302cc", + "type": "index-pattern" + }, + { + "id": "security-solution-default", + "name": "dc177bbb-04fb-4b8a-b3e5-ac0a938e76cf:indexpattern-datasource-layer-aeb24e48-cd0e-46d1-a961-d1bfc76513bd", + "type": "index-pattern" + }, + { + "id": "security-solution-default", + "name": "dc177bbb-04fb-4b8a-b3e5-ac0a938e76cf:250338b1-479f-40bd-8b61-ff47a1de45fa", + "type": "index-pattern" + }, + { + "id": "security-solution-default", + "name": "dd3c5e39-829e-4254-8708-af6d35c09a89:indexpattern-datasource-layer-aeb24e48-cd0e-46d1-a961-d1bfc76513bd", + "type": "index-pattern" + }, + { + "id": "security-solution-default", + "name": "cced8313-62a3-49a6-9b05-7cfd019825fa:indexpattern-datasource-layer-aeb24e48-cd0e-46d1-a961-d1bfc76513bd", + "type": "index-pattern" + }, + { + "id": "security-solution-default", + "name": "cced8313-62a3-49a6-9b05-7cfd019825fa:852dda7e-9c15-4f19-b721-00febdc78f51", + "type": "index-pattern" + }, + { + "id": "security-solution-default", + "name": "cced8313-62a3-49a6-9b05-7cfd019825fa:e0249f2b-0916-4c2f-a976-f958d0d041cf", + "type": "index-pattern" + }, + { + "id": "security-solution-default", + "name": "cced8313-62a3-49a6-9b05-7cfd019825fa:cc16d42c-ec98-4597-a034-532a946f0442", + "type": "index-pattern" + }, + { + "id": "security-solution-default", + "name": "7629b0e2-9589-470d-ab17-f677925d7d6e:indexpattern-datasource-layer-aeb24e48-cd0e-46d1-a961-d1bfc76513bd", + "type": "index-pattern" + }, + { + "id": "security-solution-default", + "name": "7629b0e2-9589-470d-ab17-f677925d7d6e:cff00a52-67a8-452a-aa84-9736fc93168c", + "type": "index-pattern" + }, + { + "id": "security-solution-default", + "name": "e069fc73-f5c9-4e4c-9e5f-1f8f0320bf45:indexpattern-datasource-layer-aeb24e48-cd0e-46d1-a961-d1bfc76513bd", + "type": "index-pattern" + }, + { + "id": "security-solution-default", + "name": "04599915-fee3-49ad-a6ad-d89ca76c4ed5:indexpattern-datasource-layer-aeb24e48-cd0e-46d1-a961-d1bfc76513bd", + "type": "index-pattern" + }, + { + "id": "security-solution-default", + "name": "04599915-fee3-49ad-a6ad-d89ca76c4ed5:f83809dc-fb2a-457a-a643-6eb1de99a15f", + "type": "index-pattern" + }, + { + "id": "security-solution-default", + "name": "04599915-fee3-49ad-a6ad-d89ca76c4ed5:eebf8c97-faba-48dc-8caf-3ef68b3bab5a", + "type": "index-pattern" + }, + { + "id": "security-solution-default", + "name": "04599915-fee3-49ad-a6ad-d89ca76c4ed5:5e948081-6323-4896-aab8-4a83c0e4ca46", + "type": "index-pattern" + }, + { + "id": "security-solution-default", + "name": "b0828a80-dfae-4436-ac90-2ac2d91560f4:indexpattern-datasource-layer-aeb24e48-cd0e-46d1-a961-d1bfc76513bd", + "type": "index-pattern" + }, + { + "id": "security-solution-default", + "name": "b0828a80-dfae-4436-ac90-2ac2d91560f4:af967d8d-1fc9-446a-a059-fb01992eb166", + "type": "index-pattern" + }, + { + "id": "security-solution-default", + "name": "b0828a80-dfae-4436-ac90-2ac2d91560f4:c2b5c5a8-5467-48d1-8f07-833e43c8f022", + "type": "index-pattern" + }, + { + "id": "security-solution-default", + "name": "b0828a80-dfae-4436-ac90-2ac2d91560f4:c7f9ac4e-da31-4e49-b772-6497bf07fc2e", + "type": "index-pattern" + }, + { + "id": "security-solution-default", + "name": "55b60bbc-5a8d-43dd-b08d-5685bf5d97ac:indexpattern-datasource-layer-aeb24e48-cd0e-46d1-a961-d1bfc76513bd", + "type": "index-pattern" + }, + { + "id": "security-solution-default", + "name": "55b60bbc-5a8d-43dd-b08d-5685bf5d97ac:4513e263-11aa-4988-b240-b1389e3642f4", + "type": "index-pattern" + }, + { + "id": "security-solution-default", + "name": "4f81513d-7730-4274-bb18-41fedb103005:indexpattern-datasource-layer-aeb24e48-cd0e-46d1-a961-d1bfc76513bd", + "type": "index-pattern" + }, + { + "id": "security-solution-default", + "name": "controlGroup_723c30e6-d971-44be-8e08-5f6ad06af04c:optionsListDataView", + "type": "index-pattern" + }, + { + "id": "security-solution-default", + "name": "controlGroup_14ff87c8-e889-43af-ad59-d0fa1e7ba7fe:optionsListDataView", + "type": "index-pattern" + }, + { + "id": "security-solution-default", + "name": "controlGroup_9fb8d90a-ed73-4a0b-969c-209a79c0a9a3:optionsListDataView", + "type": "index-pattern" + }, + { + "id": "security-solution-default", + "name": "controlGroup_ab7743e3-79a4-4e06-8653-9d50d7dad1d3:optionsListDataView", + "type": "index-pattern" + }, + { + "id": "security-solution-default", + "name": "controlGroup_987596fc-37bc-492c-8041-a768c90c5338:optionsListDataView", + "type": "index-pattern" + }, + { + "id": "security-solution-default", + "name": "controlGroup_c50d8ab9-c2d5-459a-88e0-13fdfdba4082:optionsListDataView", + "type": "index-pattern" + }, + { + "id": "security-solution-default", + "name": "controlGroup_2b1842a2-09b1-4d52-9e1a-6755ca7b71fb:optionsListDataView", + "type": "index-pattern" + }, + { + "id": "security-solution-default", + "name": "controlGroup_4fe1a1ce-9005-48ff-bf0c-1aa6e1796d8e:optionsListDataView", + "type": "index-pattern" + }, + { + "id": "security-solution-default", + "name": "kibanaSavedObjectMeta.searchSourceJSON.filter[0].meta.index", + "type": "index-pattern" + } + ], + "type": "dashboard", + "typeMigrationVersion": "10.2.0", + "updated_by": "u_mGBROF_q5bmFCATbLXAcCwKa0k8JvONAwSruelyKA5E_0" +} \ No newline at end of file diff --git a/packages/ubnt_unifi/kibana/search/ubnt_unifi-19723f42-25ee-4590-816b-db4b52baf6aa.json b/packages/ubnt_unifi/kibana/search/ubnt_unifi-19723f42-25ee-4590-816b-db4b52baf6aa.json new file mode 100644 index 00000000000..5f44d80ce46 --- /dev/null +++ b/packages/ubnt_unifi/kibana/search/ubnt_unifi-19723f42-25ee-4590-816b-db4b52baf6aa.json @@ -0,0 +1,84 @@ +{ + "attributes": { + "columns": [ + "observer.hostname", + "observer.product", + "observer.version", + "event.kind", + "event.type", + "event.category", + "message" + ], + "description": "", + "grid": { + "columns": { + "observer.hostname": { + "width": 206 + }, + "observer.product": { + "width": 194 + } + } + }, + "hideChart": false, + "isTextBasedQuery": false, + "kibanaSavedObjectMeta": { + "searchSourceJSON": { + "filter": [ + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "field": "data_stream.dataset", + "indexRefName": "kibanaSavedObjectMeta.searchSourceJSON.filter[0].meta.index", + "key": "data_stream.dataset", + "negate": false, + "params": { + "query": "ubnt_unifi.logs" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "data_stream.dataset": "ubnt_unifi.logs" + } + } + } + ], + "indexRefName": "kibanaSavedObjectMeta.searchSourceJSON.index", + "query": { + "language": "kuery", + "query": "" + } + } + }, + "sort": [ + [ + "@timestamp", + "desc" + ] + ], + "timeRestore": false, + "title": "[Ubiquiti UniFi] Logs" + }, + "coreMigrationVersion": "8.8.0", + "created_at": "2025-07-16T06:08:19.185Z", + "id": "ubnt_unifi-19723f42-25ee-4590-816b-db4b52baf6aa", + "references": [ + { + "id": "security-solution-default", + "name": "kibanaSavedObjectMeta.searchSourceJSON.index", + "type": "index-pattern" + }, + { + "id": "security-solution-default", + "name": "kibanaSavedObjectMeta.searchSourceJSON.filter[0].meta.index", + "type": "index-pattern" + } + ], + "type": "search", + "typeMigrationVersion": "10.5.0" +} \ No newline at end of file diff --git a/packages/ubnt_unifi/manifest.yml b/packages/ubnt_unifi/manifest.yml new file mode 100644 index 00000000000..9cab51386b4 --- /dev/null +++ b/packages/ubnt_unifi/manifest.yml @@ -0,0 +1,41 @@ +format_version: 3.3.2 +name: ubnt_unifi +title: "Ubiquiti Unifi" +version: 0.1.6 +description: "Ubiquiti Unifi" +type: integration +categories: + - network + - security +conditions: + kibana: + version: "^8.18.2 || ^9.0.0" + elastic: + subscription: "basic" +screenshots: + - src: /img/dashboard-logs-summary.png + title: Logs Summary + size: 600x600 + type: image/png +icons: + - src: /img/u.svg + title: Ubiquiti Unifi + size: 32x32 + type: image/svg+xml +policy_templates: + - name: logs + title: Ubiquiti Unifi Logs + description: Collect Ubiquiti Unifi logs + inputs: + - type: filestream + title: Collect Ubiquiti Unifi logs from file + description: Collecting Ubiquiti Unifi logs + - type: udp + title: Collect Ubiquiti Unifi logs via syslog + description: Collect Ubiquiti Unifi Syslog via a listening UDP port with Elastic Agent. + - type: http_endpoint + title: Collect Ubiquiti Unifi events via webhooks + description: Collect Ubiquiti Unifi events via webhooks +owner: + github: routedlogic/integrations + type: community diff --git a/packages/ubnt_unifi/validation.yml b/packages/ubnt_unifi/validation.yml new file mode 100644 index 00000000000..7bd3c3fb2aa --- /dev/null +++ b/packages/ubnt_unifi/validation.yml @@ -0,0 +1,3 @@ +errors: + exclude_checks: + - SVR00004 # Added search to dashboard.