Skip to content

feat: add scheduled job to rebuild latest release Docker image #208

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

ihor-sviziev
Copy link
Contributor

@ihor-sviziev ihor-sviziev commented May 22, 2025

This pull request adds re-building last release image, this is needed for security reasons (dependencies updates)

@ihor-sviziev ihor-sviziev force-pushed the enable-periodic-image-rebuilds branch from 6b76dc9 to 6bb57a8 Compare May 22, 2025 15:49
@ramunasd
Copy link
Collaborator

Thanks a lot for your efforts. I think release tags should stay immutable. If we change same tag every month then it might lead to unexpected differences. Imagine somebody use release tag in k8s. What if newly launched node will take more recent image, while 2nd replica on old node stays with older version. They both has same tag, but not equal content. Any inconsistency in image content might lead to hard to debug issues.

Another point - latest packages does not mean more security. Not long ago world had xz utils backdoor issue. More conservative approach helped to avoid this threat.

Lastly, our docker image is based on specific nginx unit image, which was not updated for more than 1 year. It has 5 critical, 24 high severity security issues, and very likely it won't get any updates at all.

Having all this in mind, I would avoid having suggested approach.

Maybe it's better to use tag latest for periodic releases? Or create a new tag, something like $version-$increment and only then build a new image.

@ihor-sviziev
Copy link
Contributor Author

ihor-sviziev commented May 26, 2025

@ramunasd, I just checked - it looks like a best practice for docker images to be rebuild time to time.
Example: https://hub.docker.com/_/nginx/tags both 1.27 and 1.28 tags are rebuild 4 days go
image

I'm sure if you would push these images to docker hub - it will show you they have some vulnerabilities. And this what I wanted to fix.

TBH I don't like both options:

  • tag latest for periodic releases (this mean, that it might just update)

This will still not fix the issue with vulnerable dependencies inside these containers.

  • create a new tag, something like $version-$increment and only then build a new image.

This would require additional effort for updating incrementally the images.

What do you think?

@ramunasd
Copy link
Collaborator

nginx 1.27 and 1.28 are aliases to latest bugfix version from corresponding minor series, so that's different

@ihor-sviziev
Copy link
Contributor Author

@ramunasd,
I've reviewed your suggestion and have some thoughts:

  • Let's keep the latest release versions unchanged (e.g., 3.7.0, 3.6.1, etc.)—they should remain immutable.
  • I propose we add additional release tags such as 3, 3.7, and latest, which will be automatically rebuilt every 30 days.

I’m uncertain about our current compatibility policy. Perhaps we could maintain just the 3 and latest tags. I think any backward-incompatible changes should be introduced in version 4.

What are your thoughts?

@ihor-sviziev
Copy link
Contributor Author

@ramunasd please review

@ludofleury
Copy link
Collaborator

Hi, I didn’t review your change, but I was interested by the issue itself.

On my ends, I’m just sharing standard market practice when it comes to security fixes. Even if your software didn’t change but your are releasing update for dependencies security fixes you should retag.

retagging ensure immutability, this is good for caching, audit etc… it’s the recommended way to release from devsecop and SRE..

@ihor-sviziev
Copy link
Contributor Author

ihor-sviziev commented Jun 6, 2025

@ludofleury, the problem came from not having releases for more than 1 year, which means all dependencies are 1 year old.

As I have example in #208 (comment), major and minor nginx version images are being rebuilt from time to time, same in https://hub.docker.com/_/composer.

So this pull request does the same.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants