Change MQTT tuning procedure to a "custom-hiera"-based one #4262

pablomh · 2025-09-16T23:10:24Z

That way the configuration file won't be deleted when foreman-installer is run again and won't be needed to reload or restart services manually.

What changes are you introducing?

Use custom-hiera.yaml to tune MQTT.

Why are you introducing these changes? (Explanation, links to references, issues, etc.)

Change some manual configuration instructions that would be reverted on the next call to foreman-installer.

Anything else to add? (Considerations, potential downsides, alternative solutions you have explored, etc.)

Contributor checklists

I am okay with my commits getting squashed when you merge this PR.
I am familiar with the contributing guidelines.

Please cherry-pick my commits into:

Foreman 3.16/Katello 4.18 (Satellite 6.18)
Foreman 3.15/Katello 4.17
Foreman 3.14/Katello 4.16 (Satellite 6.17; orcharhino 7.4)
Foreman 3.13/Katello 4.15 (EL9 only)
Foreman 3.12/Katello 4.14 (Satellite 6.16; orcharhino 7.2 on EL9 only; orcharhino 7.3)
Foreman 3.11/Katello 4.13 (orcharhino 6.11 on EL8 only; orcharhino 7.0 on EL8+EL9; orcharhino 7.1 with Leapp)
Foreman 3.10/Katello 4.12
Foreman 3.9/Katello 4.11 (Satellite 6.15; orcharhino 6.8/6.9/6.10)
We do not accept PRs for Foreman older than 3.9.

Review checklists

Tech review (performed by an Engineer who did not author the PR; can be skipped if tech review is unnecessary):

The PR documents a recommended, user-friendly path.
The PR removes steps that have been made unnecessary or obsolete.
Any steps introduced or updated in the PR have been tested to confirm that they lead to the documented end result.

Style review (by a Technical Writer who did not author the PR):

The PR conforms with the team's style guidelines.
The PR introduces documentation that describes a user story rather than a product feature.

github-actions · 2025-09-16T23:16:37Z

The PR preview for 963bfd8 is available at theforeman-foreman-documentation-preview-pr-4262.surge.sh

The following output files are affected by this PR:

show diff

show diff as HTML

aneta-petrova · 2025-09-17T07:09:48Z

I'm not sure about this. A few months ago there was a conversation about how we should steer (Satellite) users away from using hiera (#3752 was the result). That this PR adds a step to edit custom-hiera.yaml seems to go against that.

@pablomh Can you share a bit more context? Is using hiera justified here even for Satellite users?

guides/common/modules/proc_increasing-host-limit-for-pull-based-rex-transport.adoc

pablomh · 2025-09-17T07:57:43Z

Sure. The current problem is that the systemd configuration file will be wiped the next time foreman-installer is run.

In our performance testing we tend to run foreman-installer several times (one to make the basic installation, other to run additional settings, another to configure MQTT if needed based on the location of the smart-proxy, another to enable IoP, and so on). In our current (as in until one month ago when I changed it to follow this approach), we had to apply the current method (add the file, reload systemd config and restart the service every time) every time we run foreman-installer either in the main Foreman server or the some smart-capsule.

ekohl

I've always been a bit torn by this. We don't support any content in custom-hiera.yaml and by documenting it we effectively support it. I didn't test it, but I had the impression we didn't manage any drop ins so it wouldn't be pruned. Turns out I was wrong: https://github.com/voxpupuli/puppet-mosquitto/blob/3e88c0a2a447cb5399b4b5a71a864caf7dbff240/manifests/service.pp#L24-L38. So it does make sense in this case.

And then I also had a look at whether that's still needed. eclipse-mosquitto/mosquitto@1f31f14 has recently been merged, but is currently not in any release.

ekohl · 2025-09-17T08:05:31Z

guides/common/modules/proc_increasing-host-limit-for-pull-based-rex-transport.adoc

 [options="nowrap", subs="+quotes,verbatim,attributes"]
 ----
-# {foreman-installer} --foreman-proxy-plugin-remote-execution-script-mode pull-mqtt
+cat >>/etc/foreman-installer/custom-hiera.yaml <<EOF


Typically we don't tell users to run shell commands to add content to files. It's very easy to mess up. In this case they can type cat > /etc/foreman-installer/custom-hiera.yaml and wipe out any contents they had before.

That's why I was careful to write '>>' in case someone copy-pasted it (as I would do). Do you have any recomendation?

Looking at our writers, but I think we typically have some text that comes down to "add the following to /etc/foreman-installer/custom-hiera.yaml".

Another thing to keep in mind: if the user had already added the Hiera key then your command will break. By having a more generic description we put that responsibility on the user.

ekohl · 2025-09-17T08:05:57Z

guides/common/modules/proc_increasing-host-limit-for-pull-based-rex-transport.adoc


 .Procedure
-. Enable pull-based remote execution on your {ProjectServer} or {SmartProxyServer}:
+This example configures the `mosquitto` service on a {ProjectServer} or {SmartProxyServer} to handle 5000 content hosts.


We're moving away from content hosts in favor of just hosts

aneta-petrova · 2025-09-17T08:13:39Z

Thanks for taking the time to explain @pablomh! Looking at the whole guide, I can see that it references the manual config file a few times already so at the very least, adding one more should be okay. Consider my concern addressed :)

pablomh · 2025-09-17T11:43:16Z

In an ideal world, we'd be OK if this was properly implemented, but I had little time to play with it and had to resort to this approach.

ekohl · 2025-09-17T12:22:15Z

In an ideal world, we'd be OK if this was properly implemented, but I had little time to play with it and had to resort to this approach.

A short term workaround that we deploy by default could be to enhance https://github.com/voxpupuli/puppet-mosquitto to gain a parameter. We can then always set that in our Hiera config. That way we don't need any tuning guide section but it just works out of the box for users. AFAIK increasing the maximum number of open files doesn't have a huge downside.

That way the configuration file won't be deleted when `foreman-installer` is run again and won't be needed to reload or restart services manually.

pablomh · 2025-09-17T16:33:08Z

I think I've included all the recommended changes (I don't know how to deal with the GitHub reviews, sorry).

maximiliankolb

Thanks Pablo. I left three more minor suggestions.

maximiliankolb · 2025-09-18T05:42:40Z

guides/common/modules/proc_increasing-host-limit-for-pull-based-rex-transport.adoc

----
-# {foreman-installer} --foreman-proxy-plugin-remote-execution-script-mode pull-mqtt
----
+This example configures the `mosquitto` service on a {ProjectServer} or {SmartProxyServer} to handle up to 5000 hosts.


"{ProjectServer}" can resolve to "orcharhino Server", which would lead to "a orcharhino Server". I have two suggestions: Either ".. on your {SmartProxy}" or "on your {ProjectServer} or {SmartProxyServer}".

maximiliankolb · 2025-09-18T05:44:37Z

guides/common/modules/proc_increasing-host-limit-for-pull-based-rex-transport.adoc

+
+.Prerequisites
+* You have enabled pull-based remove execution on your {SmartProxy}.
 +


Suggested change

+

We don't need a line break between these sentences. Example: https://github.com/theforeman/foreman-documentation/blob/master/guides/common/modules/proc_installing-and-configuring-puppet-agent-during-host-provisioning.adoc?plain=1#L10

maximiliankolb · 2025-09-18T05:44:55Z

guides/common/modules/proc_increasing-host-limit-for-pull-based-rex-transport.adoc

+.Procedure
+. On your {SmartProxy}, set the upper limit of connected hosts for pull-based remote execution in `/etc/foreman-installer/custom-hiera.yaml`:
 +
 [options="nowrap", subs="+quotes,verbatim,attributes"]


Suggested change

[options="nowrap", subs="+quotes,verbatim,attributes"]

[source, yaml, options="nowrap", subs="+quotes,verbatim,attributes"]

pr-processor bot added the Not yet reviewed label Sep 16, 2025

github-actions bot added Needs tech review Requires a review from the technical perspective Needs style review Requires a review from docs style/grammar perspective Needs testing Requires functional testing labels Sep 16, 2025

maximiliankolb requested changes Sep 17, 2025

View reviewed changes

pr-processor bot added Waiting on contributor Requires an action from the author and removed Not yet reviewed labels Sep 17, 2025

ekohl reviewed Sep 17, 2025

View reviewed changes

Change MQTT tuning procedure to a "custom-hiera"-based one

963bfd8

That way the configuration file won't be deleted when `foreman-installer` is run again and won't be needed to reload or restart services manually.

pablomh force-pushed the patch-2 branch from 5d8299e to 963bfd8 Compare September 17, 2025 16:31

pr-processor bot added Needs re-review and removed Waiting on contributor Requires an action from the author labels Sep 17, 2025

maximiliankolb reviewed Sep 18, 2025

View reviewed changes

pr-processor bot removed the Needs re-review label Sep 18, 2025

Lennonka added the Waiting on contributor Requires an action from the author label Oct 2, 2025

	[options="nowrap", subs="+quotes,verbatim,attributes"]
	[source, yaml, options="nowrap", subs="+quotes,verbatim,attributes"]

Change MQTT tuning procedure to a "custom-hiera"-based one #4262

Are you sure you want to change the base?

Change MQTT tuning procedure to a "custom-hiera"-based one #4262

Uh oh!

Conversation

pablomh commented Sep 16, 2025 • edited by maximiliankolb Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changes are you introducing?

Why are you introducing these changes? (Explanation, links to references, issues, etc.)

Anything else to add? (Considerations, potential downsides, alternative solutions you have explored, etc.)

Contributor checklists

Review checklists

Uh oh!

github-actions bot commented Sep 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

aneta-petrova commented Sep 17, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

pablomh commented Sep 17, 2025

Uh oh!

ekohl left a comment

Choose a reason for hiding this comment

Uh oh!

ekohl Sep 17, 2025

Choose a reason for hiding this comment

Uh oh!

pablomh Sep 17, 2025

Choose a reason for hiding this comment

Uh oh!

ekohl Sep 17, 2025

Choose a reason for hiding this comment

Uh oh!

ekohl Sep 17, 2025

Choose a reason for hiding this comment

Uh oh!

aneta-petrova commented Sep 17, 2025

Uh oh!

pablomh commented Sep 17, 2025

Uh oh!

ekohl commented Sep 17, 2025

Uh oh!

pablomh commented Sep 17, 2025

Uh oh!

maximiliankolb left a comment

Choose a reason for hiding this comment

Uh oh!

maximiliankolb Sep 18, 2025

Choose a reason for hiding this comment

Uh oh!

maximiliankolb Sep 18, 2025

Choose a reason for hiding this comment

Uh oh!

maximiliankolb Sep 18, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

pablomh commented Sep 16, 2025 •

edited by maximiliankolb

Loading

github-actions bot commented Sep 16, 2025 •

edited

Loading