Skip to content

Commit 47121d9

Browse files
authored
Add plugin for Datadog (#1585)
* Add plugin for Datadog https://github.com/DataDog/fluent-plugin-datadog Signed-off-by: nbifrye <[email protected]> * make src-all Signed-off-by: nbifrye <[email protected]> * Add my name to README Signed-off-by: nbifrye <[email protected]> --------- Signed-off-by: nbifrye <[email protected]>
1 parent 0b77952 commit 47121d9

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

64 files changed

+1551
-2
lines changed

.github/dependabot.yml

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,13 @@ updates:
2727
open-pull-requests-limit: 0
2828

2929

30+
- package-ecosystem: "bundler"
31+
directory: "/docker-image/v1.19/arm64/debian-datadog"
32+
schedule:
33+
interval: "daily"
34+
open-pull-requests-limit: 0
35+
36+
3037
- package-ecosystem: "bundler"
3138
directory: "/docker-image/v1.19/arm64/debian-elasticsearch7"
3239
schedule:
@@ -146,6 +153,13 @@ updates:
146153
open-pull-requests-limit: 0
147154

148155

156+
- package-ecosystem: "bundler"
157+
directory: "/docker-image/v1.19/debian-datadog"
158+
schedule:
159+
interval: "daily"
160+
open-pull-requests-limit: 0
161+
162+
149163
- package-ecosystem: "bundler"
150164
directory: "/docker-image/v1.19/debian-elasticsearch7"
151165
schedule:

.github/workflows/check-build.yml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@ jobs:
1313
[
1414
"debian-azureblob",
1515
"debian-cloudwatch",
16+
"debian-datadog",
1617
"debian-elasticsearch7",
1718
"debian-elasticsearch8",
1819
"debian-forward",

.github/workflows/docker-build.yml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ jobs:
1515
[
1616
"debian-azureblob",
1717
"debian-cloudwatch",
18+
"debian-datadog",
1819
"debian-elasticsearch7",
1920
"debian-elasticsearch8",
2021
"debian-forward",

Makefile

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,8 @@ X86_IMAGES := \
2424
v1.19/debian-logzio:v1.19.0-debian-logzio-amd64-1.0,v1.19-debian-logzio-amd64-1 \
2525
v1.19/debian-kafka:v1.19.0-debian-kafka-amd64-1.0,v1.19-debian-kafka-amd64-1 \
2626
v1.19/debian-kafka2:v1.19.0-debian-kafka2-amd64-1.0,v1.19-debian-kafka2-amd64-1 \
27-
v1.19/debian-kinesis:v1.19.0-debian-kinesis-amd64-1.0,v1.19-debian-kinesis-amd64-1
27+
v1.19/debian-kinesis:v1.19.0-debian-kinesis-amd64-1.0,v1.19-debian-kinesis-amd64-1 \
28+
v1.19/debian-datadog:v1.19.0-debian-datadog-amd64-1.0,v1.19-debian-datadog-amd64-1
2829

2930
# <Dockerfile>:<version>,<tag1>,<tag2>,...
3031

@@ -45,7 +46,8 @@ ARM64_IMAGES := \
4546
v1.19/arm64/debian-logzio:v1.19.0-debian-logzio-arm64-1.0,v1.19-debian-logzio-arm64-1 \
4647
v1.19/arm64/debian-kafka:v1.19.0-debian-kafka-arm64-1.0,v1.19-debian-kafka-arm64-1 \
4748
v1.19/arm64/debian-kafka2:v1.19.0-debian-kafka2-arm64-1.0,v1.19-debian-kafka2-arm64-1 \
48-
v1.19/arm64/debian-kinesis:v1.19.0-debian-kinesis-arm64-1.0,v1.19-debian-kinesis-arm64-1
49+
v1.19/arm64/debian-kinesis:v1.19.0-debian-kinesis-arm64-1.0,v1.19-debian-kinesis-arm64-1 \
50+
v1.19/arm64/debian-datadog:v1.19.0-debian-datadog-arm64-1.0,v1.19-debian-datadog-arm64-1
4951

5052
ALL_IMAGES := $(X86_IMAGES) $(ARM64_IMAGES)
5153

README.md

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -79,6 +79,9 @@ See also dockerhub tags page: https://hub.docker.com/r/fluent/fluentd-kubernetes
7979
- `Kinesis`
8080
- `docker pull fluent/fluentd-kubernetes-daemonset:v1.19.0-debian-kinesis-1.0`
8181
- `docker pull fluent/fluentd-kubernetes-daemonset:v1.19-debian-kinesis-1`
82+
- `Datadog`
83+
- `docker pull fluent/fluentd-kubernetes-daemonset:v1.19.0-debian-datadog-1.0`
84+
- `docker pull fluent/fluentd-kubernetes-daemonset:v1.19-debian-datadog-1`
8285

8386
##### x86_64 images
8487
- `Azureblob` [Dockerfile](docker-image/v1.19/debian-azureblob/Dockerfile)
@@ -133,6 +136,9 @@ See also dockerhub tags page: https://hub.docker.com/r/fluent/fluentd-kubernetes
133136
- `Kinesis` [Dockerfile](docker-image/v1.19/debian-kinesis/Dockerfile)
134137
- `docker pull fluent/fluentd-kubernetes-daemonset:v1.19.0-debian-kinesis-amd64-1.0`
135138
- `docker pull fluent/fluentd-kubernetes-daemonset:v1.19-debian-kinesis-amd64-1`
139+
- `Datadog` [Dockerfile](docker-image/v1.19/debian-datadog/Dockerfile)
140+
- `docker pull fluent/fluentd-kubernetes-daemonset:v1.19.0-debian-datadog-amd64-1.0`
141+
- `docker pull fluent/fluentd-kubernetes-daemonset:v1.19-debian-datadog-amd64-1`
136142

137143
##### arm64 images
138144
- `Azureblob` [Dockerfile](docker-image/v1.19/arm64/debian-azureblob/Dockerfile)
@@ -187,6 +193,9 @@ See also dockerhub tags page: https://hub.docker.com/r/fluent/fluentd-kubernetes
187193
- `Kinesis` [Dockerfile](docker-image/v1.19/arm64/debian-kinesis/Dockerfile)
188194
- `docker pull fluent/fluentd-kubernetes-daemonset:v1.19.0-debian-kinesis-arm64-1.0`
189195
- `docker pull fluent/fluentd-kubernetes-daemonset:v1.19-debian-kinesis-arm64-1`
196+
- `Datadog` [Dockerfile](docker-image/v1.19/arm64/debian-datadog/Dockerfile)
197+
- `docker pull fluent/fluentd-kubernetes-daemonset:v1.19.0-debian-datadog-arm64-1.0`
198+
- `docker pull fluent/fluentd-kubernetes-daemonset:v1.19-debian-datadog-arm64-1`
190199

191200

192201
You can also use `v1-debian-PLUGIN` tag to refer latest v1 image, e.g. `v1-debian-elasticsearch`. On production, strict tag is better to avoid unexpected update.
@@ -373,6 +382,7 @@ Some images are contributed by users. If you have a problem/question for followi
373382
- Amazon Kinesis : @shiftky
374383
- logz.io : @SaMnCo / @jamielennox
375384
- splunkhec: @FutureSharks
385+
- Datadog : @nbifrye
376386

377387
Currently, we don't accept new destination request without contribution.
378388
See https://github.com/fluent/fluentd-kubernetes-daemonset/issues/293
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
**/*.gitkeep
Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
# AUTOMATICALLY GENERATED
2+
# DO NOT EDIT THIS FILE DIRECTLY, USE /templates/Dockerfile.erb
3+
4+
# For multiarch build on Docker hub automated build.
5+
FROM fluent/fluentd:v1.19.0-debian-arm64
6+
7+
LABEL maintainer="Eduardo Silva <[email protected]>"
8+
USER root
9+
WORKDIR /home/fluent
10+
ENV PATH /fluentd/vendor/bundle/ruby/3.4.0/bin:$PATH
11+
ENV GEM_PATH /fluentd/vendor/bundle/ruby/3.4.0
12+
ENV GEM_HOME /fluentd/vendor/bundle/ruby/3.4.0
13+
# skip runtime bundler installation
14+
ENV FLUENTD_DISABLE_BUNDLER_INJECTION 1
15+
16+
COPY Gemfile* /fluentd/
17+
RUN buildDeps="sudo make gcc g++ libc-dev libffi-dev" \
18+
runtimeDeps="" \
19+
&& apt-get update \
20+
&& apt-get upgrade -y \
21+
&& apt-get install \
22+
-y --no-install-recommends \
23+
$buildDeps $runtimeDeps net-tools \
24+
&& gem install bundler --version 2.4.17 \
25+
&& bundle config silence_root_warning true \
26+
&& bundle install --gemfile=/fluentd/Gemfile --path=/fluentd/vendor/bundle \
27+
&& SUDO_FORCE_REMOVE=yes \
28+
apt-get purge -y --auto-remove \
29+
-o APT::AutoRemove::RecommendsImportant=false \
30+
$buildDeps \
31+
'*-dev' \
32+
&& rm -rf /var/lib/apt/lists/* \
33+
&& gem sources --clear-all \
34+
&& rm -rf /tmp/* /var/tmp/* /usr/lib/ruby/gems/*/cache/*.gem
35+
# Copy configuration files
36+
COPY ./conf/fluent.conf /fluentd/etc/
37+
COPY ./conf/systemd.conf /fluentd/etc/
38+
COPY ./conf/kubernetes.conf /fluentd/etc/
39+
COPY ./conf/prometheus.conf /fluentd/etc/
40+
COPY ./conf/tail_container_parse.conf /fluentd/etc/
41+
COPY ./conf/kubernetes/*.conf /fluentd/etc/kubernetes/
42+
RUN touch /fluentd/etc/disable.conf
43+
44+
# Copy plugins
45+
COPY plugins /fluentd/plugins/
46+
COPY entrypoint.sh /fluentd/entrypoint.sh
47+
48+
# Environment variables
49+
ENV FLUENTD_OPT=""
50+
ENV FLUENTD_CONF="fluent.conf"
51+
# Override LD_PRELOAD in fluentd docker image
52+
# Set "/usr/lib/libjemalloc.so.2" if you want to enable jemalloc explicitly
53+
ENV LD_PRELOAD=""
54+
55+
# Overwrite ENTRYPOINT to run fluentd as root for /var/log / /var/lib
56+
ENTRYPOINT ["tini", "--", "/fluentd/entrypoint.sh"]
Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
# AUTOMATICALLY GENERATED
2+
# DO NOT EDIT THIS FILE DIRECTLY, USE /templates/Gemfile.erb
3+
4+
source "https://rubygems.org"
5+
6+
gem "fluentd", "1.19.0"
7+
gem "oj", "~> 3.16.11"
8+
gem "rexml", "~> 3.4.1"
9+
gem "fluent-plugin-multi-format-parser", "~> 1.0.0"
10+
gem "fluent-plugin-concat", "~> 2.6.0"
11+
gem "fluent-plugin-grok-parser", "~> 2.6.2"
12+
gem "fluent-plugin-prometheus", "~> 2.2.1"
13+
gem 'fluent-plugin-json-in-json-2', ">= 1.0.2"
14+
gem "fluent-plugin-record-modifier", "~> 2.2.0"
15+
gem "fluent-plugin-detect-exceptions", "~> 0.0.16"
16+
gem "fluent-plugin-rewrite-tag-filter", "~> 2.4.0"
17+
gem "fluent-plugin-parser-cri", "~> 0.1.1"
18+
gem "fluent-plugin-datadog", "~> 0.14.4"
19+
gem "fluent-plugin-kubernetes_metadata_filter", "~> 3.8.0"
20+
gem "ffi"
21+
gem "fluent-plugin-systemd", "~> 1.1.1"

0 commit comments

Comments
 (0)