Rundeck DockerOps

Help and business engagements

For any requirements you can contact me at [email protected] for business engagements.

Contents

• Contents
• Rundeck Server
• Required local setup
• Secret files
• NOTE about encrypting files - ansible vault
• HOWTO

Rundeck Server

This docker-compose project contains the following services:

• Rundeck server
• postgres db

Rundeck server is based on official Atlassian parent image.

For semi-automated rollout, Ansible is used.

What is Rundeck

Rundeck can be used for monitoring and task scheduling execution and notification agent.

Rundeck is a runbook automation tool that allows teams to automate and manage their operations tasks. It provides a centralized platform where users can create, schedule, and execute workflows (called "jobs") using existing scripts, commands, or tools. Rundeck supports various interfaces, including a web GUI, API, and CLI, making it versatile and accessible. It helps streamline incident management, reduce manual toil, and improve operational efficiency by enabling self-service access to automation processes(1)(2).

Source: (1) Rundeck Introduction. https://docs.rundeck.com/docs/about/introduction.html. (2) What and Why of Rundeck. https://resources.rundeck.com/learning/an-overview-of-rundeck/.

URL Rundeck

Rundeck can be accessed from port 4443: https://rundeck.example.com:4443/

Adding users

Users can be added in docker/secrets/realm.properties as explained at https://docs.rundeck.com/docs/administration/security/authentication.html#propertyfileloginmodule

Required local setup

Your mileage may vary since there are different distros and operating systems. The general idea is to have:

1. Python
2. Ansible
3. Git
4. An editor, such as VS Code with Extensions such as WSL, Ansible, Python, Pylance, YAML
5. pyenv to allow you to run other versions of python and ansible
6. ansible-lint and other packages using the pyenv command

Notes: For python and ansible, in order to run the playbook, it's important to have a compatible version of python and ansible with the version on the server. Sometimes the newer versions of python and ansible won't allow you to run playbooks on the server and will spit out errors.

Cygwin

Ansible has to be installed locally in order to run the playbook. One way to do so on Windows is to use it via cygwin. Install the "ansible" package in your cygwin installation.

WSL Linux

Another way is to request WSL Linux in your Windows environment. This will allow you to install Ubuntu or any other linux environment with a linux/bash terminal you might feel more comfortable.

General recommendations with WSL Linux and Ansible

With WSL Linux I would recommend installing:

• VS Code: https://code.visualstudio.com/
• In WSL Linux terminal install pipx, python3, and git (your mileage may vary if you choose different distro othen than Debian/Ubuntu): sudo apt install git git-lfs git-all python3 pipx -- Source: https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html#pipx-install
• pipx install --include-deps ansible ansible-lint
• Add in your ~/.bashrc:

export ANSIBLE_CONFIG=./ansible.cfg
export PATH="$HOME/.local/bin:$PATH"

You can then install ansible, python, docker and other extensions in VS Code to enable code checks / lint etc.

Before running the playbook - prepare the environment

Working with ansible and older python (3.6.8) has its downsides. You should install compatible versios for ansible, ansible-base, ansible-lint to make everything work as expected. This goes beyong the scope of this README file, but you must prepare your environment to match the environment of the server (i.e. same python version). Otherwise, expect issues while running the ansible playbook.

Prepare WSL Linux for ansible playbook

Install the following in your WSL Linux:

sudo apt-get install -y make build-essential libssl-dev zlib1g-dev libbz2-dev libreadline-dev libsqlite3-dev wget curl llvm libncurses5-dev libncursesw5-dev xz-utils tk-dev libffi-dev liblzma-dev python-openssl git
sudo apt install clang -y
curl https://pyenv.run | bash
pyenv update
pip install --upgrade pip
pyenv rehash
CC=clang pyenv install 3.6.8

Prepare your ~/.bashrc

Prepare your ~/.bashrc file:

export ANSIBLE_CONFIG=./ansible.cfg
export PATH="$HOME/.local/bin:$PATH"
export PATH="$HOME/.pyenv/bin:$PATH"
export PYENV_PYTHON_BUILD_CACHE_PATH="$HOME/.pyenv_cache"
export PYENV_ROOT="$HOME/.pyenv"
#[[ -d $PYENV_ROOT/bin ]] && export PATH="$PYENV_ROOT/bin:$PATH"
eval "$(~/.pyenv/bin/pyenv init -)"

Configure pyenv to install compatible python version

Configure your local system to install compatible python version. In our case we needed to match python version 3.6.8, so we found the compatible package versions of ansible and dependencies that match that version of python:

pyenv virtualenv 3.6.8 py368
pyenv virtualenvs
pyenv rehash
pyenv activate py368

CC=clang pip install --upgrade pip
pyenv rehash
CC=clang pip install wheel
pyenv rehash
CC=clang pip install cryptography==3.4.8
pyenv rehash
CC=clang pip install ansible-base==2.10.1
pyenv rehash
CC=clang pip install ansible==2.10.0
pyenv rehash
CC=clang pip install ansible-lint==4.3.7

That should be it.

Test using:

python --version
ansible --version

Secret files

Secret files cannot be part of the image, as the docker image gets pushed to the docker registry. Moreover, different files might be needed across the stages (DEV, QA, PRD). Thus, all secret files get mounted as a separate volume from the ./secrets folder. The same is valid for SSL certificates and ssl keystores.

NOTE about encrypting files - ansible vault

Each file in the secrets/ and certs/ subfolder needs to be encrypted with ansible-vault prior to committing to git.

Ansible vault also requires a password to encrypt/decrypt prior deploying to ensure safety of credentials.

Protect against unintended commits

To avoid unintended commits of these files, you have to configure your git to look for hooks in the hooks folder of a project.

Execute locally the following command in the root of the project:

git config core.hooksPath hooks

so that a pre-commit hook defined there checks all your files located in any secrets and certs subfolder. If they are not encrypted, the commit will fail.

HOWTO

Run the playbook

ansible-playbook -i inventory/dev.yaml site.yaml -u <YOURUSERHERE> --ask-pass --vault-password-file .ansible/vault-pw

The parameters -u <YOURUSERHERE> --ask-pass can be omitted when you have created your user with ssh key and passwordless sudo enabled.

If you want to login with a specific ssh key (example path to private key file is ~/.ssh/id_ed25519):

ansible-playbook -i inventory/dev.yaml site.yaml -u <YOURUSERHERE> --vault-password-file .ansible/vault-pw --private-key ~/.ssh/id_ed25519

If you want to check the playbook before actually running it use the --check argument:

ansible-playbook --check -i inventory/dev.yaml site.yaml -u <YOURUSERHERE> --vault-password-file .ansible/vault-pw --private-key ~/.ssh/id_ed25519

Working with Ansible Vault

See also Ansible Vault documentation

First of all place the file containing the encryption password to your home directory. This will be the referred in the following commands as --vault-password-file parameter.

Encrypt a file

ansible-vault encrypt --vault-password-file .ansible/vault-pw encrypted.yml

Encrypt all files in folders

for i in $(find ./docker/secrets ./docker/etc -type f); do ansible-vault encrypt --vault-password-file .ansible/vault-pw $i && echo $i; done

Decrypt a file with ansible-vault

ansible-vault decrypt --vault-password-file .ansible/vault-pw encrypted.yml