feat(outputs.nats): Allow providing a subject layout

[mgale]

Summary

This PR provides the following feature:

1. Provides a subject_layout option to direct metrics to subjects based on tags, metric name and field.

Checklist

• No AI generated code was used in this PR

Related issues

resolves #16558

[srebhan]

Thanks for your contribution @mgale! Can we please split this PR into two, one for the subject templating and one for the externally managed streams!? Feel free to create a new one and modify this for the other subject...

[mgale]

Thanks for your contribution @mgale! Can we please split this PR into two, one for the subject templating and one for the externally managed streams!? Feel free to create a new one and modify this for the other subject...

I have moved #17072 into:
#17244

I will complete the cleanup of this PR shortly.

[srebhan]

Thanks for the PR @mgale! My question is, why not just treat the subject setting as template instead of introducing another (redundant) setting? We've done the same in e.g. the remote file output...

[mgale]

remote file output

Would you consider changing the subject from a string to a list with this change?

If not, the primary concern with that approach arises when disable_stream_creation = false and the client is responsible for creating the stream. In that case, the stream subject needs to include a wildcard to support the dynamic components—for example, using a pattern like "telegraf.>".

If the subject template is embedded as a single string (e.g., subject = "telegraf.{{ .GetTag "region" }}.{{ .GetTag "host" }}"), we run into a few challenges:
• We would need to ensure that the subject starts with a static prefix, a base that is not part of the dynamic portion.
• We’d have to parse out the static portion of the subject up to the first dynamic tag. It’s unclear how straightforward or reliable that parsing would be, especially across varying templates.

I think this would make it difficult to correctly derive the subject wildcard needed for stream creation.

[srebhan]

@mgale I'm a bit lost here. Why does the first element has to be static if Telegraf is responsible for generating the stream? If we go for subject templates, you have to CreateOrUpdate the stream for every batch in the Write method. That's the cost of dynamically computing the subject. You could cache existing streams to minimize the number of calls to the method though...

[mgale]

@mgale I'm a bit lost here. Why does the first element has to be static if Telegraf is responsible for generating the stream? If we go for subject templates, you have to CreateOrUpdate the stream for every batch in the Write method. That's the cost of dynamically computing the subject. You could cache existing streams to minimize the number of calls to the method though...

You’re right, the first element doesn’t need to be static. I had initially leaned that way for simplicity, but I’ve since refactored the code to better align with dynamic subject handling.

Specifically:
• The code now uses the full subject template directly.
• I have updated the behavior so that only the subjects explicitly listed under outputs.nats.jetstream.subjects are registered to the stream if a dynamic (templated) subject is in use.

This avoids relying on the computed subject during Write to update the stream, which I think is important. I’d prefer to avoid performing stream registration or modification at write time if we can.

Let me know if you’re okay with this approach and constraint. I’m open to alternatives, but I believe this keeps stream creation/update logic clean and predictable.

[srebhan]

Thanks for the update @mgale! Much better but too complex. ;-) Please find my suggestions and comments in the code.

[srebhan]

Thanks @mgale! This looks much better. Some more comments from my side...

[mgale]

@srebhan I believe all feedback has now been addressed. Given the extensive back-and-forth on this PR, I’ve resolved all comments from previous reviews and believe all outstanding items have been covered.
Please feel free to re-open anything I may have missed or raise any remaining concerns. I really appreciate you working through these changes with me.

[srebhan]

Thanks @mgale for your patience! The code looks much cleaner now. Some more comments, especially regarding looping over the metrics only once...

[telegraf-tiger]

Download PR build artifacts for linux_amd64.tar.gz, darwin_arm64.tar.gz, and windows_amd64.zip.
Downloads for additional architectures and packages are available below.

☺️ This pull request doesn't significantly change the Telegraf binary size (less than 1%)

📦 Click here to get additional PR build artifacts

Artifact URLs

. DEB	. RPM	. TAR . GZ	. ZIP
[[amd64.deb aarch64.rpm darwin_amd64.tar.gz windows_amd64.zip] [arm64.deb armel.rpm darwin_arm64.tar.gz windows_arm64.zip] [armel.deb armv6hl.rpm freebsd_amd64.tar.gz windows_i386.zip] [armhf.deb i386.rpm freebsd_armv7.tar.gz ] [i386.deb ppc64le.rpm freebsd_i386.tar.gz ] [mips.deb riscv64.rpm linux_amd64.tar.gz ] [mipsel.deb s390x.rpm linux_arm64.tar.gz ] [ppc64el.deb x86_64.rpm linux_armel.tar.gz ] [riscv64.deb linux_armhf.tar.gz ] [s390x.deb linux_i386.tar.gz ] [ linux_mips.tar.gz ] [ linux_mipsel.tar.gz ] [ linux_ppc64le.tar.gz ] [ linux_riscv64.tar.gz ] [ linux_s390x.tar.gz ]]

[mgale]

@srebhan I believe all comments have been addressed, let me know if there is further work required on this PR.