Skip to content

ENT-12600: Document and refactor compile-options #1744

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 7 commits into
base: master
Choose a base branch
from
Open

ENT-12600: Document and refactor compile-options #1744

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
29de899
compile-options: Fixed shellcheck warnings
larsewi Jun 10, 2025
b4f1731
compile-options: Remove option to specify embedded DB
larsewi Jun 10, 2025
e338505
compile-options: Document script
larsewi Jun 10, 2025
8ef4526
compile-options: Refactor/document dependencies
larsewi Jun 13, 2025
f92051f
compile-options: Consistent case statement format
larsewi Jun 17, 2025
7a5ab4c
compile-option: Remove auto detect role code
larsewi Jul 24, 2025
61170dd
compile-options: split comments on dots and commas
larsewi Aug 4, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
245 changes: 126 additions & 119 deletions build-scripts/compile-options
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,22 +5,30 @@

# Autodect PROJECT if not set

if [ x"$PROJECT" = x ]
if [ -z "$PROJECT" ]
then
case x"$JOB_NAME" in
*-community-*) PROJECT=community;;
*-enterprise-*) PROJECT=nova;;
*-hub-*) PROJECT=nova;;
*-agent-*) PROJECT=nova;;
*-community-*)
PROJECT=community
;;
*-enterprise-*)
PROJECT=nova
;;
*-hub-*)
PROJECT=nova
;;
*-agent-*)
PROJECT=nova
;;
esac
fi

# If still not set, then either we are running outside Jenkins, or this
# is not a main "build" type job (it could be the bootstrap job).
# If still not set, then either we are running outside Jenkins,
# or this is not a main "build" type job (it could be the bootstrap job).
# Do directory-based auto-detection.
if [ x"$PROJECT" = x ]
if [ -z "$PROJECT" ]
then
if [ -d $BASEDIR/nova ]
if [ -d "$BASEDIR"/nova ]
then
PROJECT=nova
else
Expand All @@ -34,7 +42,7 @@ export PROJECT
# When running manually, you can just export this variable.
# It's a flag: if it's set to 1 - then we use system OpenSSL.
# Otherwise, we build it.
if [ x"$SYSTEM_SSL" = x ]
if [ -z "$SYSTEM_SSL" ]
then
# We don't bundle OpenSSL on RHEL 8 (and newer in the future)
if [ "$OS" = "rhel" ] && expr "$OS_VERSION" ">=" "8" >/dev/null
Expand All @@ -51,17 +59,20 @@ export SYSTEM_SSL

case "$OS_FAMILY" in
aix)
# This flag is needed because AIX defaults to producing .a shared libraries,
# but we need it to make .so libraries. It doesn't work correctly when
# specified in the Makefile, so we specify it in the environment instead.
# This flag is needed because AIX defaults to producing .a shared libraries, but we need it to make .so libraries.
# It doesn't work correctly when specified in the Makefile, so we specify it in the environment instead.
LDFLAGS="-Wl,-brtl"

# AIX needs default RPATH (libpath on AIX) specified as well, otherwise it
# won't find libc.
# AIX needs default RPATH (libpath on AIX) specified as well, otherwise it won't find libc.
LDFLAGS="$LDFLAGS -L$BUILDPREFIX/lib -Wl,-blibpath:$BUILDPREFIX/lib:/usr/lib:/lib"
;;
hpux)
LDFLAGS="-L$BUILDPREFIX/lib -Wl,+b$BUILDPREFIX/lib"

# Use ‘gcc’ when building things on HP-UX
#
# HP-UX ships with ‘cc’ which invokes the ‘HP-UX bundled C compiler’ which lacks some functionality we need to build things.
# Among the other things, it doesn’t know how to link ‘.so’ files directly and it doesn’t recognize them as valid input type.
CC=gcc
export CC
;;
Expand All @@ -82,7 +93,7 @@ case "$OS_FAMILY" in
esac

# When we don't bundle OpenSSL, then we need to pull it from /usr/lib64.
if [ x"$SYSTEM_SSL" = x1 ]
if [ "$SYSTEM_SSL" = 1 ]
then
LDFLAGS="$LDFLAGS -L/usr/lib64"
fi
Expand All @@ -92,115 +103,97 @@ export LDFLAGS
DEB_LDFLAGS_APPEND="$LDFLAGS"
export DEB_LDFLAGS_APPEND

# Embedded DB selection
EMBEDDED_DB="lmdb"
Comment on lines -95 to -96
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we want to keep this option.

Copy link
Contributor Author

@larsewi larsewi Aug 4, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why? It's not really an option. It's a variable holding a hard coded string. You cannot override it without modifying the script.




############### Fill in build dependencies in DEPS variable ################

DEPS=
[ $OS_FAMILY = mingw ] && var_append DEPS "pthreads-w32 libgnurx"

# libgcc_s.so is needed before we compile any other dependency
# on some platforms!
case "$OS_FAMILY" in
solaris|aix) var_append DEPS "libgcc" ;;
esac
# Windows specific dependencies
if [ "$OS_FAMILY" = mingw ]; then
# Win32 does not support pthreads natively.
# The pthreads-w32 project provides a solution to this problem.
var_append DEPS "pthreads-w32"

var_append DEPS "$EMBEDDED_DB pcre2"

if ! [ x"$SYSTEM_SSL" = x1 ]
then
# FIXME: Why do we need zlib?
# ANSWER: Openssl uses it optionally, TODO DISABLE
var_append DEPS "zlib openssl"
# A port of the regex functionality from the glibc Library for use on Windows platforms
var_append DEPS "libgnurx"
fi

# libsasl needed for solaris
# AIX / Solaris specific dependencies
case "$OS_FAMILY" in
solaris|hpux) var_append DEPS "sasl2" ;;
esac
solaris|aix)

# iconv is needed for libxml2 on some platforms
case "$OS_FAMILY" in
aix|solaris) var_append DEPS "libiconv" ;;
# libgcc_s.so is needed before we compile any other dependency on some platforms!
var_append DEPS "libgcc"

# iconv is needed for libxml2 on some platforms
var_append DEPS "libiconv"
;;
esac

var_append DEPS "libxml2 libyaml"
var_append DEPS "diffutils"
var_append DEPS "librsync"
# We use system bundled SSL on RHEL >= 8
if ! [ "$SYSTEM_SSL" = 1 ]
then
# zlib is a compression library witch is a dependency of OpenSSL.
# However, can we remove it (CFE-4013)?
var_append DEPS "zlib"

# LDAP functions in the agent
# and LDAP authentication functionality in Mission Portal
case "$PROJECT" in
nova) var_append DEPS "openldap"
esac
# Toolkit for TLS
var_append DEPS "openssl"
fi

case "$PROJECT" in
nova) var_append DEPS "leech"
# Solaris / HP-UX specific dependencies
case "$OS_FAMILY" in
solaris|hpux)
# Generic library that implements the Simple Authentication and Security Layer (SASL) framework
var_append DEPS "sasl2"
;;
esac

# libacl & libattr - not for the exotics (linux only?)
# Common dependencies
var_append DEPS "libxml2" # Library for parsing XML
var_append DEPS "libyaml" # Library for parsing YAML
var_append DEPS "diffutils" # Library for comparing files
var_append DEPS "librsync" # Library for synchronization of files
var_append DEPS "lmdb" # Library for key-value store, used extensively by
# CFEngine components (cf-agent, etc)
var_append DEPS "pcre2" # Library for compiling/matching regex

# Enterprise only dependencies
if [ "$PROJECT" = nova ]; then
# openldap is used for storing accounts, passwords and group memberships
# - Agent has LDAP related policy functions
# - Mission Portal uses LDAP for authentication functionality
var_append DEPS "openldap"

# leech is used for efficient synchronization of tabular data.
# - cf-hub requests state changes from cf-serverd
# - state changes are recorded by cf-agent
var_append DEPS "leech"
fi

# Non-exotics dependencies
case "$OS_FAMILY" in
hpux|aix|solaris|freebsd|mingw) ;;
*) var_append DEPS "libattr libacl" ;;
esac
hpux|aix|solaris|freebsd|mingw)
;;
*)
# Library for managing Extended Attributes (xattrs) on filesystems
var_append DEPS "libattr"

# ROLE
# We can autodetect if we are building a hub or an agent, but
# that is suboptimal since the list might change at any time or
# we might want to build an agent package for a platform that is a hub
# or viceversa. By default we assume the old behavior but if the
# environment variable EXPLICIT_ROLE is defined, then we change our behavior
# and build according to the role specified by it.
# POSIX Access Control Lists (ACLs) on filesystems
var_append DEPS "libacl"
;;
esac

case "$EXPLICIT_ROLE" in
agent) ROLE=agent;;
hub) ROLE=hub;;
agent)
ROLE=agent
;;
hub)
ROLE=hub
;;
*)
# Not running under Jenkins?
if [ x"$JENKINS_SERVER_COOKIE" = x ]
then
case "$PROJECT-$ARCH-$OS-${OS_VERSION}" in
community-*) ROLE=agent;;
# We do not support 32 bits hubs anymore
nova-i386-*-*) ROLE=agent;;
nova-s390*-*-*) ROLE=agent;;
nova-*-centos-*) ROLE=hub;;
nova-*-debian-*) ROLE=hub;;
nova-*-opensuse-*) ROLE=hub;;
nova-*-rhel-*) ROLE=hub;;
nova-*-sles-*) ROLE=hub;;
nova-*-ubuntu-*) ROLE=hub;;
nova-*-mingw-*) ROLE=agent;;
nova-*) ROLE=agent;;
*)
echo "Unknown project: $PROJECT"
exit 42
;;
esac
echo "Autodetected $ROLE role based on missing Jenkins label and OS."

else # we are running under Jenkins

# The "label" variable is set in multi-matrix jobs;
# thus it is *not set* in the bootstrap job.
case x"$label" in
x)
echo "label is not set, assuming that this is not the main build job; Setting ROLE=agent."
ROLE=agent
;;
*_HUB_*)
echo "Autodetected hub role based on '_HUB_' in Jenkins label."
ROLE=hub
;;
*)
echo "Autodetected agent role based on missing '_HUB_' in Jenkins label."
ROLE=agent
;;
esac
fi
echo "$(basename "$0"): Error: Bad EXPLICIT_ROLE '$EXPLICIT_ROLE': defaulting to 'agent'"
ROLE=agent
;;
esac

Expand All @@ -212,32 +205,46 @@ export ROLE
case "$ROLE" in
# HUB-ONLY dependencies
hub)
var_append DEPS "libcurl-hub"
var_append DEPS "nghttp2 libexpat apr apr-util apache git rsync"
var_append DEPS "postgresql php"
# Note that we make a separate curl package for the hub.
# This is because the hub will include the curl binary,
# but we don't want that for the clients.
var_append DEPS "libcurl-hub" # Provides API for performing network requests
var_append DEPS "nghttp2" # Provides implementation of the HTTP/2 protocol.
var_append DEPS "libexpat" # Provides stream-oriented XML parser
var_append DEPS "apr apr-util" # Provides a common interface to underlying OS features (used by Apache)
var_append DEPS "apache" # Provides HTTP server
var_append DEPS "git" # Provides a version control system
var_append DEPS "rsync" # Binary to efficiently synchronize files
var_append DEPS "postgresql" # Relational database
var_append DEPS "php" # Scripting language for web development
;;
# AGENT-ONLY dependencies
agent)
var_append DEPS "libcurl"
var_append DEPS "libcurl" # Provides API for performing network requests
;;
esac

# Make sure init.d script and systemd service is used where needed.
#
# We install in all Linux platforms, because it does no harm on platforms without systemd,
# and it has the advantage of working out of the box on platforms that adopt systemd later.
if [ "$OS_FAMILY" = linux ]; then
WITH_SYSTEMD=yes
else
WITH_SYSTEMD=no
fi

# systemd is not a build dependency, but should we generate
# unit files for it?
case "$OS_FAMILY" in
linux) WITH_SYSTEMD=yes;;
*) WITH_SYSTEMD=no;;
esac


# Determine whether or not to run tests
case "$OS_FAMILY" in
mingw|freebsd) TESTS=no ;;
*) TESTS=all ;;
mingw|freebsd)
TESTS=no
;;
*)
TESTS=all
;;
esac
export TESTS


# Don't let existing LD_LIBRARY_PATH variables disturb the build.
# Java sets this when Jenkins launches it.
# PS! Solaris shell returns false if the var is already unset.
Expand Down
6 changes: 1 addition & 5 deletions build-scripts/configure
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,12 +15,8 @@ esac

P=$BUILDPREFIX

ARGS="--prefix=$P --libdir=$P/lib --with-workdir=$P --sysconfdir=/etc --with-openssl=$P --with-pcre2=$P --with-librsync=$P --with-init-script"
ARGS="--prefix=$P --libdir=$P/lib --with-workdir=$P --sysconfdir=/etc --with-lmdb=$P --with-openssl=$P --with-pcre2=$P --with-librsync=$P --with-init-script"

if [ $EMBEDDED_DB = lmdb ]
then
var_append ARGS "--with-lmdb=$P"
fi
case "$DEPS" in
*pthreads-w32*) var_append ARGS "--with-pthreads=$P" ;;
esac
Expand Down
3 changes: 0 additions & 3 deletions scripts/deptool.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -236,8 +236,6 @@ def deps_list(self, ref="master"):
"""Returns a sorted list of dependencies for given ref, for example: `["lcov", "libgnurx", "pthreads-w32"]`.
Assumes the proper ref is checked out by `self.buildscripts_repo`.
"""
# TODO: get value of $EMBEDDED_DB from file
embedded_db = "lmdb"
if ref == "3.7.x":
options_file = self.buildscripts_repo.get_file(
"build-scripts/install-dependencies"
Expand All @@ -264,7 +262,6 @@ def deps_list(self, ref="master"):
# in the middle we also do some clean-ups
only_deps = (
" ".join(only_deps)
.replace("$EMBEDDED_DB", embedded_db)
.replace("libgcc ", "")
.split(" ")
)
Expand Down
Loading