Skip to content
This repository was archived by the owner on Sep 12, 2025. It is now read-only.

Conversation

@leborchuk
Copy link

In these PR I added to workflow building jammy Docker container.

The main peculiarity of the Debian package is that we need to build the binary in a specific directory and then install it in /usr/local/cloudberry-db. This differs from the package creation process for RPM-based distros, where you simply list what you want to package (*.spec) along with absolute paths. For Ubuntu you should list relative local build paths and then write install script to copy it in /usr/local/cloudberry-db.

So, I added the ${BUILD_DESTINATION} path (with defaults to /usr/ local/cloudberry-db) to all build automation scripts. This has made my PR slightly more dangerous, because if something goes wrong (I don't think I have, but it needs to be thoroughly reviewed), all our builds could suffer.

@leborchuk
Copy link
Author

@edespino Could you please review my PR? I want to add deb packages to cloudberry and this is my first PR where I try to create Docker images.

@tuhaihe
Copy link
Member

tuhaihe commented Mar 26, 2025

Cool @leborchuk, thanks for your great work. I also would like to help review this PR.

@tuhaihe tuhaihe requested review from edespino and tuhaihe March 26, 2025 13:45
@leborchuk
Copy link
Author

I found out that the deb package's dependencies included our internal python packages, so removed them.

Copy link
Member

@tuhaihe tuhaihe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some non-technical comments, FYI. I will try to test this workflow in my environment, and if any issues, I will let you know. Thanks!

@leborchuk
Copy link
Author

I fixed issues, PR can be reviewed

@tuhaihe
Copy link
Member

tuhaihe commented Apr 1, 2025

Hey @leborchuk , an error returned when running docker build -t cloudberry-db-base-env . under the dir images/docker/cbdb/test/ubuntu22.04:

image

Seems some packages cannot be installed from the ubuntu 22.04 software sources:

glibc-locale-source ~> glibc-source
openssh
openssh-clients ~> openssh-client
procps-ng ~> procps
which

@tuhaihe
Copy link
Member

tuhaihe commented Apr 1, 2025

After building and installing Cloudberry in the build image, I run the test for twice with the following command:

PGOPTIONS='-c optimizer=on' make installcheck

Some errors return:

regression-diffs.txt

 +20250401:07:07:21:037248 gpconfig:cdw:gpadmin-[ERROR]:-USER environment variable must be set.
 +USER environment variable must be set.

Should we add the ENV USER=gpadmin to the build Dockerfile? FYI. Will test later.

@edespino
Copy link
Contributor

edespino commented Apr 1, 2025

APT Flags & Repos

Problem: You're using --allow-unauthenticated and Acquire::AllowInsecureRepositories=true. This is not safe for production.

Fix: Unless absolutely required (e.g., for local mirrors), remove these flags. If used, explain the context in a comment.

@edespino
Copy link
Contributor

edespino commented Apr 1, 2025

Duplicate Packages

libssl-dev appears twice in the APT list.

libaprutil1-dev and libuv1-dev are also later reinstalled under gpadmin.

Fix: Install once globally under root and avoid duplication unless version/context differs.

@edespino
Copy link
Contributor

edespino commented Apr 1, 2025

Missing Cleanup

Problem: The APT cache and downloaded files (e.g., .tar.gz, build dirs) are not cleaned up, which bloats the image.

Fix:

RUN apt-get clean && rm -rf /var/lib/apt/lists/*

Also clean /tmp/, ~/xerces-c, Go tarballs, etc., after use.

@edespino
Copy link
Contributor

edespino commented Apr 1, 2025

Timezone hardcoding

ln -snf /usr/share/zoneinfo/Europe/London is hardcoded, even though you have TIMEZONE_VAR.

Fix: Use the variable:

RUN ln -snf /usr/share/zoneinfo/${TIMEZONE_VAR} /etc/localtime && echo "${TIMEZONE_VAR}" > /etc/timezone

@edespino
Copy link
Contributor

edespino commented Apr 1, 2025

Repetitive locale settings

You run locale-gen and update-locale multiple times.

Fix: Consolidate to a single layer.

@edespino
Copy link
Contributor

edespino commented Apr 1, 2025

Have you considered slimming the image using ubuntu:jammy-minimal?

Copy link
Contributor

@edespino edespino left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have submitted several comments already. I have more to review.

Copy link
Member

@yjhjstz yjhjstz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great works~~

@tuhaihe
Copy link
Member

tuhaihe commented Apr 2, 2025

Hi, there is another issue to resolve: the /tmp/init_system.sh cannot be executed when the new image starts up. FYI.

@edespino
Copy link
Contributor

edespino commented Apr 2, 2025

@leborchuk This is great progress — thanks for putting in the effort to bring Ubuntu 22.04 support to Cloudberry!

That said, I’d really prefer to see this rolled out in more focused, logical chunks so each stage is easy to test, review, and validate independently. Something like:

  1. Developer workflow – Introduce base support for Ubuntu 22.04: container build image, compiler config, developer installcheck support. A developer should be able to build and run installcheck against main.
  2. CI build automation – Add a GitHub Actions workflow to build the Ubuntu 22.04 developer container image automatically.
  3. Release engineering workflow – Extend the container image to support .deb package creation. A developer should be able to build and produce a .deb file inside the image.
  4. QA workflow – Introduce a minimal runtime container image (based on Ubuntu 22.04) to install and validate the .deb package. This image should support basic Cloudberry functionality and simulate a QA-style install, including gpadmin setup and dependency resolution.

Splitting the PR this way would make it easier for others to review and test each part without merging everything at once. Would you be open to restructuring the work along these lines?

@leborchuk leborchuk reopened this Jun 16, 2025
@leborchuk
Copy link
Author

Hi, @tuhaihe @edespino I want to reopen PR once again.

I see that no visible commits have been made to https://github.com/edespino/cloudberry/tree/ubuntu-22.04, so I decided to move forward.

I took all the work that had been done by @edespino and set up my own environment for creating Debian packages. Then, I created a Docker environment and Debian package using it (using github actions). I tried to install the Debian package in my development environment - everything worked well. Of course, all packages include PAX storage. I also checked if the simple PAX scripts worked on Ubuntu - there were no errors, everything was great.

It's a first PR where I want to add scripts do cloudberry-devops-release repo.

The plan for future work:

  1. run workflow and upload created ubuntu docker files to registry
  2. create test debian package workflow (manual run) build based on https://github.com/leborchuk/cloudberrydb/blob/AddJammyBuild/.github/workflows/build-deb-cloudberry.yml and check if all works well
  3. Add build & test debian packages to https://github.com/apache/cloudberry/blob/main/.github/workflows/build-cloudberry.yml
  4. Discuss whether we want to be able to run tests using a special tag on a Debian image or not.

Here you can find details about the workflow and produced artifacts:

  1. Dockerfile - https://github.com/leborchuk/cloudberry-devops-release/actions/runs/15562547058/job/43818330748
  2. Debian package - https://github.com/leborchuk/cloudberrydb/actions/runs/15662457461/job/44122150152

@tuhaihe
Copy link
Member

tuhaihe commented Jun 16, 2025

Thanks @leborchuk for helping push this big feature forward again! Just left some comments, FYI. Will help test earlier this week.

@leborchuk
Copy link
Author

Thanks @leborchuk for helping push this big feature forward again! Just left some comments, FYI. Will help test earlier this week.

Thanks, fixed

@leborchuk
Copy link
Author

Tip: add here libssh2-devel after merging #27 (review)

@leborchuk
Copy link
Author

Rebased to the new cloudberry-env file

@tuhaihe
Copy link
Member

tuhaihe commented Jul 24, 2025

Hey @leborchuk, would you like to start planning the migration of this repo to the main one, as discussed here: https://lists.apache.org/thread/p6otyrrnosg8fsbyr6ok7hl8wxpx4ss2? I think it would be better to open a new PR on the main after this repo's migration is complete. Otherwise, it could make the migration more difficult.

How about this?

@leborchuk
Copy link
Author

Hey @leborchuk, would you like to start planning the migration of this repo to the main one, as discussed here: https://lists.apache.org/thread/p6otyrrnosg8fsbyr6ok7hl8wxpx4ss2? I think it would be better to open a new PR on the main after this repo's migration is complete. Otherwise, it could make the migration more difficult.

How about this?

Well, do you think the time has come? Let's start it then) I will store scripts from this PR in my own repo, move this repo and fix building scripts, then open PR again

@tuhaihe
Copy link
Member

tuhaihe commented Jul 28, 2025

Hey @leborchuk, would you like to start planning the migration of this repo to the main one, as discussed here: https://lists.apache.org/thread/p6otyrrnosg8fsbyr6ok7hl8wxpx4ss2? I think it would be better to open a new PR on the main after this repo's migration is complete. Otherwise, it could make the migration more difficult.
How about this?

Well, do you think the time has come? Let's start it then) I will store scripts from this PR in my own repo, move this repo and fix building scripts, then open PR again

I think it's good to start the plan and move forward now. We can have a discussion about it and see how to reorganize these files in the main repo, then we can take actions. Sounds good?

@leborchuk
Copy link
Author

Created apache/cloudberry#1292

@leborchuk
Copy link
Author

I created apache/cloudberry#1359 in the main repo, these repo will be archived soon.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants