Skip to content

Telemetry and metadata to compute time in pipeline #372

@mattmikolay

Description

@mattmikolay

My codebase uses Broadway with BroadwaySQS. I recently made a few changes to begin capturing a metric for the full duration of time a message spends in the pipeline, from when SQSClient.receive_messages/2 is called, to when the message is acknowledged via Acknowledger.ack/3.

To do this, I store a :received_at timestamp in the metadata of every Broadway.Message. For SQS messages specifically, :sent_timestamp and :approximate_first_receive_timestamp are insufficient. (:sent_timestamp is the timestamp of when the message was enqueued, and :approximate_first_receive_timestamp will be stale whenever the message is redelivered.)

Then, in my implementation of Acknowledger.ack/3, I subtract :received_at from the current system time to compute each message’s time delta, and emit a custom Telemetry event.

I imagine other users of Broadway may wish to capture a similar metric, so I’m wondering if it makes sense to add one or both of the following to Broadway itself:

  • A new Telemetry event for acks, dispatched when a group of messages is acknowledged via Acknowledger.ack/3
  • A :received_at (or :created_at?) field in the metadata of all messages, automatically set when a message is created by a producer

This essentially generalizes what I implemented. Thoughts? I’d be happy to put together a PR with these changes.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions