Skip to content

Abstract Java Kafka client metrics via Producer/ConsumerMetrics#430

Merged
dfakhritdinov merged 9 commits intomasterfrom
df/expose-java-client-metrics
Jul 15, 2024
Merged

Abstract Java Kafka client metrics via Producer/ConsumerMetrics#430
dfakhritdinov merged 9 commits intomasterfrom
df/expose-java-client-metrics

Conversation

@dfakhritdinov
Copy link
Copy Markdown
Contributor

@dfakhritdinov dfakhritdinov commented Jul 11, 2024

The PR addresses issue evolution-gaming/smetrics#237 and based on @Z1kkurat PR #324, thanks for inspiration!

Goal of the change is in exposing Java Kafka client metrics in the same way as native skafka metrics.

Native metrics represented by ProducerMetrics and ConsumerMetrics traits, their implementation require com.evolutiongaming.smetrics.CollectorRegistry. This approach allows decoupling metrics and their collector implementation, which is Prometheus (at least smetrics provides Prometheus support, other implementations also possible).

Java Kafka client metrics can be exposed to Prometheus with help of com.evolutiongaming.skafka.metrics.KafkaMetricsCollector - Prometheus collector was placed in separate module with dependency on Prometheus client. This PR aimed integrate the collector with ProducerMetrics and ConsumerMetrics traits in main skafka module (one without dependency on Prometheus client). For that new methods were introduced in both traits with default empty implementation. Actual implementation is in helper objects ProducerMetricsOf and ConsumerMetricsOf respectively.

Example:

import ConsumerMetricsOf.*

val config: ConsumerConfig = ???
val registry: CollectorRegistry = ???   // Prometheus client
val metrics: ConsumerMetrics[IO] = ???

for {
  metrics   <- metrics.exposeJavaClientMetrics(registry)
  consumerOf = ConsumerOf.apply1(metrics1.some) 
  consumer  <- consumerOf(config)
} yield ???

@coveralls
Copy link
Copy Markdown

coveralls commented Jul 11, 2024

Pull Request Test Coverage Report for Build 9937634721

Details

  • 19 of 26 (73.08%) changed or added relevant lines in 5 files are covered.
  • 4 unchanged lines in 2 files lost coverage.
  • Overall coverage decreased (-0.2%) to 96.88%

Changes Missing Coverage Covered Lines Changed/Added Lines %
modules/metrics/src/main/scala/com/evolutiongaming/skafka/metrics/KafkaMetricsRegistry.scala 0 7 0.0%
Files with Coverage Reduction New Missed Lines %
skafka/src/main/scala/com/evolutiongaming/skafka/consumer/ConsumerLogging.scala 2 92.12%
skafka/src/main/scala/com/evolutiongaming/skafka/producer/ProducerMetrics.scala 2 98.21%
Totals Coverage Status
Change from base Build 9220432565: -0.2%
Covered Lines: 2205
Relevant Lines: 2276

💛 - Coveralls

Denys Fakhritdinov and others added 4 commits July 11, 2024 16:07
…umer/ConsumerMetricsOf.scala

Co-authored-by: Mareks Rampāns <8796159+mr-git@users.noreply.github.com>
…umer/ConsumerMetricsOf.scala

Co-authored-by: Mareks Rampāns <8796159+mr-git@users.noreply.github.com>
@dfakhritdinov dfakhritdinov requested review from Z1kkurat and mr-git July 11, 2024 14:26
@dfakhritdinov dfakhritdinov merged commit 7a7a5d3 into master Jul 15, 2024
@dfakhritdinov dfakhritdinov deleted the df/expose-java-client-metrics branch July 21, 2024 09:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants