Templating does not work if only /tmp is writable

[lioman]

Describe the bug

If you use nginx templating function the filled template can not be saved if /etc/nginx/conf.d is not writable.

To reproduce

Steps to reproduce the behavior:

1. Build a NGINX Unprivileged Docker image with an default.conf.template in /etc/nginx/templates
2. Deploy the container in environment, where only /tmp is writable
3. See error in log, that template can not be written to /etc/nginx/conf.d

Expected behavior

In a unprivileged environment, I expect, that templates ca be written and used.

[alessfg]

Let me look into this once I have some time and get back to you. It might be a little while (I have a hefty backlog atm) but I'll update the issue when I have had time to dig into this 😄

[asamofal]

@lioman Probable I've faced the same issue:

ERROR: /etc/nginx/templates exists, but /etc/nginx/conf.d is not writable

In my case the reason was in user property for nginx service in docker compose config. As I've changed the UID/GID of the user inside container, permissions of /etc/nginx/ dir (recursively) started mismatch for running user.
/etc/nginx/conf.d dir permissions is:

drwxrwxr-x 1 nginx root 4096 May 16 18:06 conf.d

By default nginx user resolves to 101 UID. So setting the UID to something else breaking the permissions and config can't be created from template.

In my case I've solved the problem by setting GID of the user to 0 (this is root). In this case script that creates a config file has write access to /etc/nginx/conf.d by group permissions.

So if you change UID/GID of the user (in docker compose config or create a new user in your Dockerfile) don't forget to update correspondent permissions for some paths:

• /var/cache/nginx
• /etc/nginx
• /var/log/nginx

[alessfg]

Have you had a chance to try the above steps @lioman?

[JPeer264]

We actually still have the problem. Changing the UID/GID to 0 is not an option as we run on 65532. But to answer the question yes it does work with root.

[MatthiasWinzeler]

we're happy users of this docker image, but noticed that it is incompatible with kubernetes' readonlyRootFilesystem: true (which becomes more and more best practice these days) - the following warnings appear at startup.

/docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
/docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
/docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
10-listen-on-ipv6-by-default.sh: info: can not modify /etc/nginx/conf.d/default.conf (read-only file system?)
/docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
20-envsubst-on-templates.sh: ERROR: /etc/nginx/templates exists, but /etc/nginx/conf.d is not writable
/docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh
/docker-entrypoint.sh: Configuration complete; ready for start up

I'm not sure this can be easily fixed, since the entrypoint scripts relied on the config being writable, which is AFAIK against the philosophy of having readonlyRootFilesystem: true.

[alessfg]

You are correct in that the entrypoint script rely on the config being writable. The goal is to add some QoL config changes that are environment dependent. There might be room to host an image that doesn't do any of those changes but I'll have to think and talk to the maintainers of the upstream Docker NGINX image about it.

[alessfg]

To go back to the previous discussions, there is an option to change the default UID/GID by passing a new value as an ARG at build time(https://github.com/nginxinc/docker-nginx-unprivileged/blob/main/mainline/alpine/Dockerfile#L15-L16).

Setting a new value when the image is being built should change the NGINX UID, but you'll have to rebuild the image vs using the images on Docker hub.

[Meettya]

As work-around solution:

1. Set in Dockerfile

ENV     NGINX_ENVSUBST_OUTPUT_DIR=/tmp/nginx/conf.d
RUN     mkdir -p        /tmp/nginx/conf.d

2. Get processed file as

include /tmp/nginx/conf.d/*.conf;

[monosoul]

A message to passing by strugglers like me:

No need to rebuild the image, you can make it work with just the right mount points. See this docker compose example:
https://gist.github.com/monosoul/e462b03cba5542bfc52f135417a8209d

Kudos to @Meettya for the idea.

[mureco]

we're happy users of this docker image, but noticed that it is incompatible with kubernetes' readonlyRootFilesystem: true (which becomes more and more best practice these days) - the following warnings appear at startup.

/docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
/docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
/docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
10-listen-on-ipv6-by-default.sh: info: can not modify /etc/nginx/conf.d/default.conf (read-only file system?)
/docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
20-envsubst-on-templates.sh: ERROR: /etc/nginx/templates exists, but /etc/nginx/conf.d is not writable
/docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh
/docker-entrypoint.sh: Configuration complete; ready for start up

I'm not sure this can be easily fixed, since the entrypoint scripts relied on the config being writable, which is AFAIK against the philosophy of having readonlyRootFilesystem: true.

I was able to make the entrypoint script 20-envsubst-on-templates.sh write in /etc/nginx/conf.d with a kubernetes deployment and without any modification to the docker image.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: app
  namespace: app
spec:
  template:
    spec:
      containers:
        - name: "app"
          image: "nginxinc/nginx-unprivileged:1.27.4"
          imagePullPolicy: "Always"
          volumeMounts:
            - mountPath: "/tmp"
              name: "tmp"
            - mountPath: "/etc/nginx/templates/default.conf.template"
              name: "default-conf-template"
              subPath: default.conf.template
            - mountPath: /etc/nginx/conf.d/
              name: "nginx-conf-d"
          securityContext:
            allowPrivilegeEscalation: false
            capabilities:
              drop:
                - "ALL"
            readOnlyRootFilesystem: true
            runAsUser: 10001
      securityContext:
        appArmorProfile:
          type: "RuntimeDefault"
        seccompProfile:
          type: "RuntimeDefault"
        runAsNonRoot: true
      volumes:
        - name: "tmp"
          emptyDir: {}
        - name: default-conf-template
          configMap:
            defaultMode: 420
            items:
              - key: default.conf.template
                path: default.conf.template
            name: app-proxy-conf-template
        - name: "nginx-conf-d"
          emptyDir: {}