Error message while starting icingaweb2 docker container

[alamp1360]

Hi!

From time to time the icingaweb docker container produces the following error message while starting:

[Wed Jul 30 08:43:55.26908997 2025] [docker_entrypoint:info] [pid 1] DOCKERE: Initializing /data as we're the init process
[Wed Jul 30 08:43:55.269177538 2025] [docker_entrypoint:debug] [pid 1] DOCKERE: Creating "/data/etc/icingaweb2/enabledModules"
[Wed Jul 30 08:43:55.269191491 2025] [docker_entrypoint:debug] [pid 1] DOCKERE: Creating "/data/var/lib/icingaweb2"
[Wed Jul 30 08:43:55.269199977 2025] [docker_entrypoint:debug] [pid 1] DOCKERE: Translating env vars to .ini config
[Wed Jul 30 08:43:55.269206109 2025] [docker_entrypoint:info] [pid 1] DOCKERE: Checking database resources used as backends
[Wed Jul 30 08:43:55.269269481 2025] [docker_entrypoint:crit] [pid 1] DOCKERE: **symlink /usr/share/icingaweb2/modules/setup /data/etc/icingaweb2/enabledModules/setup: file exists**

There is another file in the same directory, which, when existing, also produces a similar error message:

[Wed Jul 30 08:41:31.917828434 2025] [docker_entrypoint:crit] [pid 1] DOCKERE: **symlink /entrypoint-db-init /data/etc/icingaweb2/enabledModules/dockerentrypoint: file exists**

The two files are on a mounted directory from the host system (/opt/APPL/ICINGA/IcingaWeb) and indeed exist:

docker-hostsystem # ls -l /opt/APPL/ICINGA/IcingaWeb/etc/icingaweb2/enabledModules/
total 0
lrwxrwxrwx 1 33 tape 38 Apr  4  2024 director -> /usr/share/icingaweb2/modules/director
lrwxrwxrwx 1 33 tape 19 Jul 30 08:32 dockerentrypoint -> /entrypoint-db-init
lrwxrwxrwx 1 33 tape 41 Nov 29  2024 fileshipper -> /usr/share/icingaweb2/modules/fileshipper
lrwxrwxrwx 1 33 tape 38 Apr  4  2024 graphite -> /usr/share/icingaweb2/modules/graphite
lrwxrwxrwx 1 33 tape 38 Apr  4  2024 icingadb -> /usr/share/icingaweb2/modules/icingadb
lrwxrwxrwx 1 33 tape 39 Apr  4  2024 incubator -> /usr/share/icingaweb2/modules/incubator
lrwxrwxrwx 1 33 tape 39 Apr  9  2024 pdfexport -> /usr/share/icingaweb2/modules/pdfexport
lrwxrwxrwx 1 33 tape 39 Apr  9  2024 reporting -> /usr/share/icingaweb2/modules/reporting
lrwxrwxrwx 1 33 tape 35 Jul 30 08:32 setup -> /usr/share/icingaweb2/modules/setup

When i delete the files, startup is successful, but for some strange reason i do not see those files after startup. So i guess they are being deleted later during the startup procedure. And i also guess that this deletion sometimes does not occur, which would explain the issue i am facing.

Would you please consider changing the main.go to first remove the symlink before creating it?
In the meantime: Would it be safe to just check for and delete those two files in the systemd service file?

Greetings!

[alamp1360]

Is there any chance someone will at least take a look at this case?

[yhabteab]

Hi @alamp1360, thanks for reporting!

Apologize for the delay, everyone is fully occupied with higher priority issues.

From time to time the icingaweb docker container produces the following error message while starting:

What does from time to time mean? Do you maybe use different environment variables to start the container? Unless you explicitly enabled these modules with -e icingaweb.enabledModules=setup, I don't see how these could ended up within that directory. Though, the dockerentrypoint module can't even explicitly be enabled by a user, since it's kinda a special module. Did you maybe kill the container with some special signal while the entrypoint is running and results into a partially initialized container/volume?

Would you please consider changing the main.go to first remove the symlink before creating it?

Would be a reasonable fix but I don't get how the container ended up with that broken state in the first place.

In the meantime: Would it be safe to just check for and delete those two files in the systemd service file?

Systemd service file? How? Regarding to the safety, I don't see any problems in removing these simlinks from the /etc/icingaweb2/enabledModules/ directory beforehand.

[alamp1360]

Hi!

Thanks for looking into this case...

I am by no means a go programmer, but i would say those two symlinks are created by the entrypoint program of icingaweb2 docker container, main.go, in the function initDb:

func initDb(passwords map[string]map[string]string) error {
	logf("info", "Checking database resources used as backends")

{
		enMod := path.Join(enModsDir, "dockerentrypoint")
		if errSl := os.Symlink("/entrypoint-db-init", enMod); errSl != nil {
			return errSl
		}

		defer os.Remove(enMod)
	}

	{
		enMod := path.Join(enModsDir, "setup")

		errSl := os.Symlink(path.Join(modsDir, "setup"), enMod)
		if errSl != nil {
			if le, ok := errSl.(*os.LinkError); !ok || !os.IsNotExist(le.Err) {
				return errSl
			}
		}

		if errSl == nil {
			defer os.Remove(enMod)
		}
	}
   --- snip ---

And if i interpret the above code snippet right, those symlinks should be removed with the defer os.Remove(enMod) when the initDB function returns.
So those two modules do not have to be "enabled".

I have no idea how the container ended up in that state, but i have seen the existence of those two files from time to time. Maybe it is happening when the entrypoint program crashes?

However: At the moment the entrypoint program does not check if the two files are existing, so the entrypoint fails. My proposal would be to change the above code to check the files for existence before blindly trying to create them.

While you consider the code change, my other question was if it would be safe to remove those files by adding some code to the systemd service file which is used to start the container. You've already answered to this question positively, so i will implement this workaround...

Greetings!

[jvanderneutstulen]

I have seen this issue when the database is not ready when on icingaweb2 container start up. Adding a depends_on clause seems to help. But icingaweb should always delete those symlinks (setup and dockerentrypoint), or check for existence first, so the container can start properly later.

jul 02 10:10:09 myhostname icingaweb[3937315]: [Thu Jul 2 10:10:09.156183844 2026] [docker_entrypoint:info] [pid 1] DOCKERE: Initializing /data as we're the init process
jul 02 10:10:09 myhostname icingaweb[3937315]: [Thu Jul 2 10:10:09.156922195 2026] [docker_entrypoint:debug] [pid 1] DOCKERE: Creating "/data/etc/icingaweb2/enabledModules"
jul 02 10:10:09 myhostname icingaweb[3937315]: [Thu Jul 2 10:10:09.156947237 2026] [docker_entrypoint:debug] [pid 1] DOCKERE: Creating "/data/var/lib/icingaweb2"
jul 02 10:10:09 myhostname icingaweb[3937315]: [Thu Jul 2 10:10:09.156962252 2026] [docker_entrypoint:debug] [pid 1] DOCKERE: Translating env vars to .ini config
jul 02 10:10:09 myhostname icingaweb[3937315]: [Thu Jul 2 10:10:09.156972354 2026] [docker_entrypoint:info] [pid 1] DOCKERE: Checking database resources used as backends
jul 02 10:10:09 myhostname icingaweb[3937315]: [Thu Jul 2 10:10:09.311031196 2026] [docker_entrypoint:debug] [pid 1] DOCKERE: Checking database resource "icingaweb_db"
jul 02 10:10:09 myhostname icingaweb[3937315]: [Thu Jul 2 08:10:09.000000 2026] [docker_entrypoint:error] [pid 13] DOCKERE: Can't connect to database: SQLSTATE[HY000] [2002] Connection refused

jul 02 10:10:21 myhostname icingaweb[3939252]: [Thu Jul 2 10:10:21.157366433 2026] [docker_entrypoint:info] [pid 1] DOCKERE: Initializing /data as we're the init process
jul 02 10:10:21 myhostname icingaweb[3939252]: [Thu Jul 2 10:10:21.157581668 2026] [docker_entrypoint:debug] [pid 1] DOCKERE: Creating "/data/etc/icingaweb2/enabledModules"
jul 02 10:10:21 myhostname icingaweb[3939252]: [Thu Jul 2 10:10:21.157609406 2026] [docker_entrypoint:debug] [pid 1] DOCKERE: Creating "/data/var/lib/icingaweb2"
jul 02 10:10:21 myhostname icingaweb[3939252]: [Thu Jul 2 10:10:21.157632988 2026] [docker_entrypoint:debug] [pid 1] DOCKERE: Translating env vars to .ini config
jul 02 10:10:21 myhostname icingaweb[3939252]: [Thu Jul 2 10:10:21.157647194 2026] [docker_entrypoint:info] [pid 1] DOCKERE: Checking database resources used as backends
jul 02 10:10:21 myhostname icingaweb[3939252]: [Thu Jul 2 10:10:21.157668552 2026] [docker_entrypoint:crit] [pid 1] DOCKERE: symlink /entrypoint-db-init /data/etc/icingaweb2/enabledModules/dockerentrypoint: file exists