Influxdb metrics pushing error

[Hacker-web-Vi]

Hello!
I'm trying to push zgs_node metrics to the influxdb bucket, but it doesn't work.
Influxdb is up and running. I tried to upload some metrics manually using the same auth method as provided in the config below and it worked. There is no related error in the log.

zgs_node v0.7.4

#######################################################################
###                      Metrics Options                            ###
#######################################################################

[metrics]

# Whether to enable metrics.
enabled = true

# Interval to output metrics periodically, e.g. "10s", "30s" or "60s".
report_interval = "10s"

# File name to output metrics periodically.
file_report_output = "metrics.csv"

# Influxdb configurations to output metrics periodically.
influxdb_report_host = "http://168.119.10.134:8416"
influxdb_report_db = "0g-test"
influxdb_report_username = "0g-test"
influxdb_report_password = "test-0-0-0"

# Storage node name as a tag.
influxdb_report_node = "0g-test"

In trace logs I was able to find our influxdb host

0g-testnet-validator@trusted-point-1:~/.storage-node$ tail -f /home/0g-testnet-validator/.storage-node/log/zgs.log.$(TZ=UTC date +%Y-%m-%d) | grep 168.119.10.134
2024-11-14T19:47:17.073208Z TRACE hyper::client::pool: take? ("http", 168.119.10.134:8416): expiration = Some(90s)
2024-11-14T19:47:17.073216Z DEBUG hyper::client::pool: reuse idle connection for ("http", 168.119.10.134:8416)
2024-11-14T19:47:17.227662Z TRACE hyper::client::pool: put; add idle connection for ("http", 168.119.10.134:8416)
2024-11-14T19:47:17.227665Z DEBUG hyper::client::pool: pooling idle connection for ("http", 168.119.10.134:8416)
2024-11-14T19:47:31.735756Z TRACE hyper::client::pool: take? ("http", 168.119.10.134:8416): expiration = Some(90s)
2024-11-14T19:47:31.735774Z DEBUG hyper::client::pool: reuse idle connection for ("http", 168.119.10.134:8416)
2024-11-14T19:47:31.898813Z TRACE hyper::client::pool: put; add idle connection for ("http", 168.119.10.134:8416)
2024-11-14T19:47:31.898818Z DEBUG hyper::client::pool: pooling idle connection for ("http", 168.119.10.134:8416)
2024-11-14T19:47:39.209178Z TRACE hyper::client::pool: take? ("http", 168.119.10.134:8416): expiration = Some(90s)

[boqiu]

Basically, file_report_output is used for dev purpose, and need not to configure it.

As for reporting to influxdb, is there any stderr or logs? E.g.

• stderr: Exit metrics reporting due to error
• debug log: failed to write points to influxdb

Moreover, you could check the metrics via RPC:

curl -X POST --data '{"jsonrpc":"2.0","method":"admin_getMetrics","params":["metrics"],"id":1}' -H "Content-Type: application/json" http://127.0.0.1:5679 | jq -C ".result"

There is a metric metrics_report_failures records the number of failures to report metrics to influcdb.

[boqiu]

@Hacker-web-Vi Have your issue resolved?

[Hacker-web-Vi]

@Hacker-web-Vi Have your issue resolved?

Don't think so

#######################################################################
###                      Metrics Options                            ###
#######################################################################

[metrics]

# Whether to enable metrics.
enabled = true

# Interval to output metrics periodically, e.g. "10s", "30s" or "60s".
report_interval = "10s"

# File name to output metrics periodically.
# file_report_output = "metrics.csv"

# Influxdb configurations to output metrics periodically.
influxdb_report_host = "http://168.119.10.134:8416"
influxdb_report_db = "0g-test"
influxdb_report_username = "0g-test"
influxdb_report_password = "12345678"

0g@trusted-point-1:~# curl -s -X POST --data '{"jsonrpc":"2.0","method":"admin_getMetrics","params":["metrics"],"id":1}' -H "Content-Type: application/json" http://127.0.0.1:5679 | jq -C ".result"
{
  "metrics_report_failures": "Counter 3593",
  "metrics_report_time": "Gauge 1116472"
}

And I could not find any related error in either debug/trace/stderr logs. Furthermore, there is no incoming pushes from node's host in influxdb logs too.

What influxdb is this expected to be pushing to? We are trying on v2.7.10. Below python snippet to test db connection and permissions works fine.

from influxdb_client import InfluxDBClient, Point, WritePrecision
from influxdb_client.client.write_api import SYNCHRONOUS
import datetime

url = "http://168.119.10.134:8416"
org = "0g-test"
bucket = "0g-test"
username = "0g-test"
password = "12345678"

client = InfluxDBClient(url=url, org=org, username=username, password=password)

write_api = client.write_api(write_options=SYNCHRONOUS)

point = (
    Point("measurement_name")
    .tag("location", "office")
    .field("temperature", 23.5)
    .time(datetime.datetime.now(datetime.timezone.utc), WritePrecision.NS)
)

write_api.write(bucket=bucket, org=org, record=point)
client.close()

[Hacker-web-Vi]

We just tested it on influxdb V1 instead of V2 and it worked. We can close the issue now.

Thank you!