6473 – add otel config (#108)

We are using the otel-collector mainly to be able to collect custom metrics from our Rails services. But it is a service that can be used with any of Check's services.

Because it is relevant to all of our services we don't want to keep it in individual repos, but keep it in the main Check one.

This is only relevant to local development.

References: CV2-6473, CV2-4396
Relates to: meedan/pender#586

Author: vasconsaurus

README.md

@@ -49,6 +49,7 @@ This is a [Docker Compose](https://docs.docker.com/compose/) configuration that
 - Narcissus screenshot service (container `narcissus`) at [http://localhost:8687](http://localhost:8687)
 - Fetch fact-checking service (container `fetch`) at [http://localhost:8687/about](http://localhost:8687/about)
 - Search v2 prototype application (container `search`) at [http://localhost:8001](http://localhost:8001)
+- Otel Collector – you can configure receivers, processors and exporters in the otel-collector-config.yaml
 
 ## Testing
 

docker-compose.yml

@@ -126,6 +126,8 @@ services:
     extra_hosts:
       - "host.docker.internal:host-gateway"
   pender:
+    # env_file:
+    #   - ./pender/.env
     build: pender
     platform: linux/x86_64
     shm_size: 1G
@@ -160,13 +162,13 @@ services:
       SERVER_PORT: 3200
     networks:
       - dev
+  # If you want to run the OpenTelemetry Collector, uncomment the lines below
   # otel-collector:
   #   image: otel/opentelemetry-collector-contrib
   #   volumes:
-  #     - ./pender/config/otel-collector-config.yaml:/etc/otelcol-contrib/config.yaml
-  #   depends_on:
-  #     - pender
+  #     - ./otel-collector-config.yaml:/etc/otelcol-contrib/config.yaml
   #   environment:
+  #     x_honeycomb_team: ${X_HONEYCOMB_TEAM} # Set this env var in your shell before running docker-compose
   #     RAILS_ENV: development
   #     SERVER_PORT: 3200
   #   networks:

otel-collector-config.yaml

@@ -0,0 +1,35 @@
+receivers:
+  prometheus:
+    config:
+      scrape_configs:
+        - job_name: "prometheus"
+          scrape_interval: 15s
+          static_configs:
+            - targets: ["pender:3200"]
+
+processors:
+  memory_limiter:
+    check_interval: 1s
+    limit_percentage: 80
+    spike_limit_percentage: 20
+  batch:
+    timeout: 200ms
+    send_batch_size: 8192
+
+exporters:
+  otlp/metrics:
+    endpoint: "api.honeycomb.io:443" # US instance
+    #endpoint: "api.eu1.honeycomb.io:443" # EU instance
+    headers:
+      "x-honeycomb-team": ${env:x_honeycomb_team} # Honeycomb API KEY
+      "x-honeycomb-dataset": "pender"
+
+service:
+  # telemetry:
+  #   logs:
+  #     level: "debug"
+  pipelines:
+    metrics:
+      receivers: [prometheus]
+      processors: [memory_limiter, batch]
+      exporters: [otlp/metrics]