diff --git a/docs/run-the-playbooks-for-disconnected.md b/docs/run-the-playbooks-for-disconnected.md index 3942fc6c..46ba4b2d 100644 --- a/docs/run-the-playbooks-for-disconnected.md +++ b/docs/run-the-playbooks-for-disconnected.md @@ -68,7 +68,7 @@ configurations are mentioned over here. ### Overview Mirror the ocp platform and other necessary images to the mirror registry. Please run this playbook before you run **6 Create Nodes Playbook** and after -**0 Setup Playbook**. +**5 Setup Bastion**. ### Outcomes @@ -76,7 +76,7 @@ Mirror the ocp platform and other necessary images to the mirror registry. Pleas * Template the mirror pull secret to the mirror host. * Add the ca cert to the mirror host anchors if ca is not trusted. * Mirror the platform images using `oc adm release mirror` if legacy mirroring is enabled. -* Template the image set to mirror host and then mirror it using `oc-mirror` plogin. +* Template the image set to mirror host and then mirror it using `oc-mirror` plugin. * Copy the results on the `oc-mirror` to ansible controller to apply to cluster in future steps. ### Notes diff --git a/docs/set-variables-group-vars.md b/docs/set-variables-group-vars.md index c657d79e..5c99e26e 100644 --- a/docs/set-variables-group-vars.md +++ b/docs/set-variables-group-vars.md @@ -253,6 +253,7 @@ :--- | :--- | :--- **disconnected.enabled** | True or False, to enable disconnected mode | False **disconnected.registry.url** | String containing url of disconnected registry with or without port and without protocol | registry.tt.testing:5000 +**disconnected.registry.ip** | String containing ip of the registry, which will be used for resolving dns | `192.168.151.1` **disconnected.registry.pull_secret** | String containing pull secret of the disconnected registry to be applied on the *cluster*. Make sure to enclose pull_secret in 'single quotes' and it has appropriate pull access. | '{"auths":{"registry.tt..testing:5000":{"auth":"b3Blb...4yQQ==","email":"test.user@example.com"}}}' **disconnected.registry.mirror_pull_ecret** | String containing pull secret to use for mirroring. Contains Red Hat secret and registry pull secret. Make sure to enclose pull_secret in 'single quotes' and must be able to push to mirror registry. | '{"auths":{"cloud.openshift.com":{"auth":"b3Blb...4yQQ==","email":"redhat.user@gmail.com", "registry.tt..testing:5000":...user@example.com"}}}' **disconnected.registry.ca_trusted** | True or False to indicate that mirror registry CA is implicitly trusted or needs to be made trusted on mirror host and cluster. | False @@ -269,11 +270,14 @@ **disconnected.mirroring.legacy.ocp_repo** | The repo part of the repo on the mirror registry where the release image will be pushed | openshift4 **disconnected.mirroring.legacy.ocp_tag** | The tag part of the repo on the mirror registry where the release image will be pushed. Full image would be as below.: disconnected.registry.url/disconnected.mirroring.legacy.ocp_org/disconnected...ocp_repo:disconnected..ocp_tag | v4.13.1 **disconnected.mirroring.oc_mirror.release_image_tag** | The ocp release image tag you want to install the cluster with. Used when legacy platform mirroring is disabled and **disconnected.mirroring.oc_mirror.image_set** contains platform entries. | 4.13.1-multi +**disconnected.mirroring.oc_mirror.release_image_registry** | The Release Image registry like Brew Registry which stores the release images as mirror to **registry.redhat.io** | `brew.registry.redhat.io` **disconnected.mirroring.oc_mirror.oc_mirror_args.continue_on_error** | True or False to give `--continue-on-error` flag to `oc-mirror` | False **disconnected.mirroring.oc_mirror.oc_mirror_args.source_skip_tls** | True or False to give `--source-skip-tls` flag to `oc-mirror` | False **disconnected.mirroring.oc_mirror.post_mirror.mapping.replace.enabled** | True or False to replace values in `mapping.txt` generated by oc-mirror. This also does a manual repush of the images in `mapping.txt`. | False **disconnected.mirroring.oc_mirror.post_mirror.mapping.replace.list** | List of **regexp** and **replace** where every string/regular expression gets replaced by corresponding *replace* value. | regexp: interal-url.com replace: external-url.com **disconnected.mirroring.oc_mirror.image_set** | YAML fields containing a standard `oc-mirror` [image set](https://docs.openshift.com/container-platform/latest/installing/disconnected_install/installing-mirroring-disconnected.html#oc-mirror-creating-image-set-config_installing-mirroring-disconnected) with some minor changes to schema. Differences are documented as needed. Used to generate final image set. | see template +**disconnected.mirroring.oc_mirror.image_set.apiVersion** | The API version of the `ImageSetConfiguration` content. | `mirror.openshift.io/v1alpha2`, `mirror.openshift.io/v2alpha1` +**disconnected.mirroring.oc_mirror.image_set.storageConfig.enabled** | True or False to set whether oc-mirror is v1 or v2. As for v2 storageConfig flag is removed completely | `True` or `False` **disconnected.mirroring.oc_mirror.image_set.storageConfig.registry.enabled** | True or False to use registry storage backend for pushing mirrored content directly to the registry. Currently only this backend is supported.| True **disconnected.mirroring.oc_mirror.image_set.storageConfig.registry.imageURL.org** | The org part of registry imageURL from standard image set. | mirror **disconnected.mirroring.oc_mirror.image_set.storageConfig.registry.imageURL.repo** | The repo part of registry imageURL from standard image set. Final imageURL will be as below: disconnected.registry.url/disconnected.mirroring.oc_mirror.image_set.storageConfig .registry.imageURL.org/disconnected...imageURL.repo | oc-mirror-metadata diff --git a/inventories/default/group_vars/disconnected.yaml b/inventories/default/group_vars/disconnected.yaml index 1fcab73b..ec3ae690 100644 --- a/inventories/default/group_vars/disconnected.yaml +++ b/inventories/default/group_vars/disconnected.yaml @@ -1,11 +1,12 @@ # Warning: currently, the oc-mirror plugin is officially downloadable to amd64 only. disconnected: - enabled: False + enabled: false registry: url: 'registry url' + ip: 'registry reachable ip' pull_secret: '' # this is similar to env.redhat.pull_secret but it will only contain secrets to be applied to the cluster in disconnected mode. mirror_pull_secret: '' # this should contain a pull secret that contains the combination of env.redhat.pull_secret and pull secret with push access to mirror registry for mirroring - ca_trusted: False + ca_trusted: false ca_cert: | -----BEGIN CERTIFICATE----- if ca_trusted is False, then this ca will be added to mirror host anchors as well as to the install config of cluster @@ -23,27 +24,30 @@ disconnected: ocp_download_url: "https://mirror.openshift.com/pub/openshift-v4/multi/clients/ocp/4.13.1/amd64/" ocp_client_tgz: 'openshift-client-linux.tar.gz' # name of the oc binary. Should be a tar.gz file legacy: - platform: False # if true then platform is mirrored in the old way and install config will be patched with the imagecontentsourcepolicy + platform: false # if true then platform is mirrored in the old way and install config will be patched with the imagecontentsourcepolicy ocp_quay_release_image_tag: '4.13.1-s390x' ocp_org: 'ocp4' ocp_repo: 'openshift4' ocp_tag: 'v4.13.1' # platform images will be pushed to {tegistry_url}/{ocp_org}/{ocp_repo}:{ocp_tag} oc_mirror: oc_mirror_args: - continue_on_error: False - source_skip_tls: False + continue_on_error: false + source_skip_tls: false post_mirror: mapping: replace: - enabled: False + enabled: false list: - - regexp: what - replace: with + - regexp: what + replace: with release_image_tag: '4.13.1-multi' + release_image_registry: 'brew.registry.redhat.io' # Update the release registry to use it as mirror registry for release operator images image_set: # this field is a standard image set from oc-mirror documentation. The only exception is the storageConfig which is altered to allow substitution of disconnected.registry.url + apiVersion: mirror.openshift.io/v2alpha1 # update the version with v2alpha1 for oc-mirror v2 plugin and v1alpha2 for v1 plugin storageConfig: + enable: false # enable this flag if oc-mirror v1 plugin is used since storageConfig is removed for v2 plugin registry: - enabled: True # use registry storage backend. Currently only method supproted + enabled: true # use registry storage backend. Currently only method supproted imageURL: # the final value will be {imageURL: disconnected.registry.url/org/repo} org: mirror repo: oc-mirror-metadata @@ -51,21 +55,21 @@ disconnected: mirror: # this field is also atandard from the oc-mirror schema. It will be substituted as is into the final image set. platform: architectures: - - multi # note: while image tags such as `multi-s390x` are also available on quay, we cannot mirror these, so list can only contain pure architecture names such as `s390x` or the multiarch `multi` + - multi # note: while image tags such as `multi-s390x` are also available on quay, we cannot mirror these, so list can only contain pure architecture names such as `s390x` or the multiarch `multi` channels: - - name: stable-4.13 - full: false - minVersion: 4.13.1 - maxVersion: 4.13.1 + - name: stable-4.13 + full: false + minVersion: 4.13.1 + maxVersion: 4.13.1 operators: - catalog: registry.redhat.io/redhat/redhat-operator-index:v4.13 full: false packages: - - name: serverless-operator - channels: - - name: stable - #minVersion: '2.4.1-0' - #maxVersion: '2.4.1-0' + - name: serverless-operator + channels: + - name: stable + # minVersion: '2.4.1-0' + # maxVersion: '2.4.1-0' additionalImages: - - name: registry.redhat.io/ubi8/ubi:latest + - name: registry.redhat.io/ubi8/ubi:latest helm: {} diff --git a/roles/disconnected_check_vars/tasks/main.yaml b/roles/disconnected_check_vars/tasks/main.yaml index fdb75c9f..9843e2a1 100644 --- a/roles/disconnected_check_vars/tasks/main.yaml +++ b/roles/disconnected_check_vars/tasks/main.yaml @@ -3,7 +3,7 @@ - name: 'Check mandatory variables are defined for disconnected' block: - name: "check mandatory variables for disconnected are set" - assert: + ansible.builtin.assert: that: - disconnected.registry.url is defined and disconnected.registry.url != None - disconnected.registry.pull_secret is defined and disconnected.registry.pull_secret != None @@ -17,3 +17,5 @@ - disconnected.mirroring.client_download.ocp_download_url is defined and disconnected.mirroring.client_download.ocp_download_url != None - disconnected.mirroring.client_download.ocp_client_tgz is defined and disconnected.mirroring.client_download.ocp_client_tgz != None - disconnected.mirroring.oc_mirror.image_set is defined and disconnected.mirroring.oc_mirror.image_set != None + - disconnected.mirroring.oc_mirror.image_set.apiVersion is defined and disconnected.mirroring.oc_mirror.image_set.apiVersion != None + - disconnected.mirroring.oc_mirror.image_set.storageConfig.registry.enabled is defined and disconnected.mirroring.oc_mirror.image_set.storageConfig.registry.enabled != None diff --git a/roles/disconnected_mirror_images/tasks/main.yaml b/roles/disconnected_mirror_images/tasks/main.yaml index 2f2b7bc0..c3d185bd 100644 --- a/roles/disconnected_mirror_images/tasks/main.yaml +++ b/roles/disconnected_mirror_images/tasks/main.yaml @@ -1,6 +1,7 @@ --- - name: mirror artifacts in disconnected mode tags: mirror_artifacts + when: disconnected.enabled block: - name: Setting Up nameserver on mirror host to resolve DNS ansible.builtin.template: @@ -11,26 +12,26 @@ mode: "644" - name: Get user home directory tags: mirror_artifacts - shell: > + ansible.builtin.shell: > getent passwd {{ ansible_user }} | awk -F: '{ print $6 }' changed_when: false register: user_home - name: Delete OCP download directory for idempotency. tags: mirror_artifacts become: true - file: - path: "{{ user_home.stdout }}/ocpinst" + ansible.builtin.file: + path: "{{ user_home.stdout }}/ocpinst_disconnected" state: absent - name: Create OCP download directory tags: mirror_artifacts - file: - path: "{{ user_home.stdout }}/ocpinst" + ansible.builtin.file: + path: "{{ user_home.stdout }}/ocpinst_disconnected" state: directory - name: Unzip OCP client and oc-mirror tags: mirror_artifacts ansible.builtin.unarchive: src: "{{ item }}" - dest: "{{ user_home.stdout }}/ocpinst/" + dest: "{{ user_home.stdout }}/ocpinst_disconnected/" remote_src: yes loop: - "{{ disconnected.mirroring.client_download.ocp_download_url }}{{ disconnected.mirroring.client_download.ocp_client_tgz }}" @@ -39,7 +40,7 @@ tags: mirror_artifacts become: true ansible.builtin.copy: - src: "{{ user_home.stdout }}/ocpinst/{{ item }}" + src: "{{ user_home.stdout }}/ocpinst_disconnected/{{ item }}" dest: /usr/sbin/{{ item }} owner: root group: root @@ -56,7 +57,7 @@ register: home_docker - name: Create directory {{ user_home.stdout }}/.docker tags: mirror_artifacts - file: + ansible.builtin.file: path: "{{ user_home.stdout }}/.docker" state: directory when: not home_docker.stat.exists @@ -100,59 +101,104 @@ tags: mirror_artifacts ansible.builtin.template: src: imageset.yaml.j2 - dest: "{{ user_home.stdout }}/ocpinst/imageset.yaml" - - name: mirror the imageset to registry + dest: "{{ user_home.stdout }}/ocpinst_disconnected/imageset.yaml" + - name: mirror the imageset to registry using oc-mirror v1 + tags: mirror_artifacts + block: + - name: mirror the imageset to registry for oc-mirror plugin v1 + ansible.builtin.shell: | + set -o pipefail + oc mirror --config {{ user_home.stdout }}/ocpinst_disconnected/imageset.yaml docker://{{ disconnected.registry.url }} --ignore-history{{ ' --continue-on-error' if disconnected.mirroring.oc_mirror.oc_mirror_args.continue_on_error }}{{ ' --source-skip-tls' if disconnected.mirroring.oc_mirror.oc_mirror_args.source_skip_tls }} + register: cmd_oc_mirror + args: + chdir: "{{ user_home.stdout }}/ocpinst_disconnected" + - name: print output of imageset mirroring + ansible.builtin.debug: + var: cmd_oc_mirror + when: disconnected.mirroring.oc_mirror.image_set.apiVersion == "mirror.openshift.io/v1alpha2" + - name: mirror the imageset to registry using oc-mirror v2 tags: mirror_artifacts + when: disconnected.mirroring.oc_mirror.image_set.apiVersion == "mirror.openshift.io/v2alpha1" block: - - name: mirror the imageset to registry + - name: pre-tasks if post_mirror is enabled + when: disconnected.mirroring.oc_mirror.post_mirror.mapping.replace.enabled + block: + - name: Check if directory {{ user_home.stdout }}/.config exists + ansible.builtin.stat: + path: "{{ user_home.stdout }}/.config/containers" + register: container_registry_home + - name: Create directory {{ user_home.stdout }}/.config/containers + ansible.builtin.file: + path: "{{ user_home.stdout }}/.config/containers" + state: directory + when: not container_registry_home.stat.exists + - name: Prepare registries.conf for oc-mirror mirroring + ansible.builtin.template: + src: registries.conf.j2 + dest: "{{ user_home.stdout }}/.config/containers/registries.conf" + - name: dry run the imageset to create mapping.txt file for oc-mirror plugin v2 ansible.builtin.shell: | set -o pipefail - oc mirror --config {{ user_home.stdout }}/ocpinst/imageset.yaml docker://{{ disconnected.registry.url}} --ignore-history{{ ' --continue-on-error' if disconnected.mirroring.oc_mirror.oc_mirror_args.continue_on_error == True }} \ - {{ ' --source-skip-tls' if disconnected.mirroring.oc_mirror.oc_mirror_args.source_skip_tls == True }} - # ignore-history set by default for idempotency + oc mirror --v2 --config {{ user_home.stdout }}/ocpinst_disconnected/imageset.yaml --workspace file://{{ user_home.stdout }}/ocpinst_disconnected docker://{{ disconnected.registry.url }} --dry-run + args: + chdir: "{{ user_home.stdout }}/ocpinst_disconnected" + - name: mirror the imageset to registry for oc-mirror plugin v2 + ansible.builtin.shell: | + set -o pipefail + oc mirror --v2 --config {{ user_home.stdout }}/ocpinst_disconnected/imageset.yaml --workspace file://{{ user_home.stdout }}/ocpinst_disconnected docker://{{ disconnected.registry.url }} register: cmd_oc_mirror args: - chdir: "{{ user_home.stdout }}/ocpinst" + chdir: "{{ user_home.stdout }}/ocpinst_disconnected" - name: print output of imageset mirroring ansible.builtin.debug: var: cmd_oc_mirror - - name: Get the results directory from oc mirror operation + - name: Get the results directory from oc mirror v1 plugin operation tags: mirror_artifacts ansible.builtin.shell: | set -o pipefail - ls {{ user_home.stdout }}/ocpinst/oc-mirror-workspace/ | grep results + ls {{ user_home.stdout }}/ocpinst_disconnected/oc-mirror-workspace/ | grep results register: oc_mirror_results_dir_name - - name: perform post mirror actions if requested + when: disconnected.mirroring.oc_mirror.image_set.apiVersion == "mirror.openshift.io/v1alpha2" + - name: perform post mirror actions if requested for oc-mirror v1 plugin tags: mirror_artifacts + when: + - disconnected.mirroring.oc_mirror.post_mirror.mapping.replace.enabled + - disconnected.mirroring.oc_mirror.image_set.apiVersion == "mirror.openshift.io/v1alpha2" block: - name: create a copy of the mappings file ansible.builtin.copy: - src: "{{ user_home.stdout }}/ocpinst/oc-mirror-workspace/{{ oc_mirror_results_dir_name.stdout }}/mapping.txt" - dest: "{{ user_home.stdout }}/ocpinst/oc-mirror-workspace/{{ oc_mirror_results_dir_name.stdout }}/post_mapping.txt" + src: "{{ user_home.stdout }}/ocpinst_disconnected/oc-mirror-workspace/{{ oc_mirror_results_dir_name.stdout }}/mapping.txt" + dest: "{{ user_home.stdout }}/ocpinst_disconnected/oc-mirror-workspace/{{ oc_mirror_results_dir_name.stdout }}/post_mapping.txt" mode: preserve - remote_src: True + remote_src: true - name: replace provided values in the mappings ansible.builtin.replace: - path: "{{ user_home.stdout }}/ocpinst/oc-mirror-workspace/{{ oc_mirror_results_dir_name.stdout }}/post_mapping.txt" + path: "{{ user_home.stdout }}/ocpinst_disconnected/oc-mirror-workspace/{{ oc_mirror_results_dir_name.stdout }}/post_mapping.txt" regexp: "{{ item.regexp }}" replace: "{{ item.replace }}" with_items: "{{ disconnected.mirroring.oc_mirror.post_mirror.mapping.replace.list }}" - when: disconnected.mirroring.oc_mirror.post_mirror.mapping.replace.enabled + when: disconnected.mirroring.oc_mirror.post_mirror.mapping.replace.enabled - name: manually mirror patched mappings to registry ansible.builtin.shell: | set -o pipefail - oc image mirror -a {{ user_home.stdout }}/.docker/config.json -f {{ user_home.stdout }}/ocpinst/oc-mirror-workspace/{{ oc_mirror_results_dir_name.stdout }}/post_mapping.txt --insecure --filter-by-os='.*' --continue-on-error || true + oc image mirror -a {{ user_home.stdout }}/.docker/config.json -f {{ user_home.stdout }}/ocpinst_disconnected/oc-mirror-workspace/{{ oc_mirror_results_dir_name.stdout }}/post_mapping.txt --insecure --filter-by-os='.*' --continue-on-error || true args: - chdir: "{{ user_home.stdout }}/ocpinst" + chdir: "{{ user_home.stdout }}/ocpinst_disconnected" register: cmd_oc_image_mirror - name: print output of oc image mirror ansible.builtin.debug: var: cmd_oc_image_mirror - when: disconnected.mirroring.oc_mirror.post_mirror.mapping.replace.enabled - - name: Fetch the files that need to be applied on the cluster to the controller + - name: Fetch the files that need to be applied on the cluster to the controller for oc-mirror v1 plugin tags: mirror_artifacts ansible.posix.synchronize: mode: pull - src: "{{ user_home.stdout }}/ocpinst/oc-mirror-workspace/{{ oc_mirror_results_dir_name.stdout }}/*" + src: "{{ user_home.stdout }}/ocpinst_disconnected/oc-mirror-workspace/{{ oc_mirror_results_dir_name.stdout }}/*" dest: "../.oc-mirror-results" - when: disconnected.enabled + when: disconnected.mirroring.oc_mirror.image_set.apiVersion == "mirror.openshift.io/v1alpha2" + - name: Fetch the files that need to be applied on the cluster to the controller for oc-mirror v2 plugin + tags: mirror_artifacts + ansible.posix.synchronize: + mode: pull + src: "{{ user_home.stdout }}/ocpinst_disconnected/working-dir/cluster-resources/*" + dest: "../.oc-mirror-results" + when: disconnected.mirroring.oc_mirror.image_set.apiVersion == "mirror.openshift.io/v2alpha1" diff --git a/roles/disconnected_mirror_images/templates/imageset.yaml.j2 b/roles/disconnected_mirror_images/templates/imageset.yaml.j2 index 3ebfb32f..a7597636 100644 --- a/roles/disconnected_mirror_images/templates/imageset.yaml.j2 +++ b/roles/disconnected_mirror_images/templates/imageset.yaml.j2 @@ -1,11 +1,13 @@ kind: ImageSetConfiguration -apiVersion: mirror.openshift.io/v1alpha2 +apiVersion: {{ disconnected.mirroring.oc_mirror.image_set.apiVersion }} archiveSize: 4 +{% if (disconnected.enabled) and (disconnected.mirroring.oc_mirror.image_set.apiVersion == "mirror.openshift.io/v1alpha2") %} storageConfig: -{% if ( disconnected.enabled ) and ( disconnected.mirroring.oc_mirror.image_set.storageConfig.registry.enabled ) %} + {% if disconnected.mirroring.oc_mirror.image_set.storageConfig.registry.enabled %} registry: imageURL: {{ disconnected.registry.url | string + '/' + disconnected.mirroring.oc_mirror.image_set.storageConfig.registry.imageURL.org | string + '/' + disconnected.mirroring.oc_mirror.image_set.storageConfig.registry.imageURL.repo | string }} skipTLS: {{ disconnected.mirroring.oc_mirror.image_set.storageConfig.registry.skipTLS }} + {% endif %} {% endif %} mirror: {{ disconnected.mirroring.oc_mirror.image_set.mirror | to_nice_yaml | indent(2, true) }} diff --git a/roles/disconnected_mirror_images/templates/registries.conf.j2 b/roles/disconnected_mirror_images/templates/registries.conf.j2 new file mode 100644 index 00000000..4c9698f9 --- /dev/null +++ b/roles/disconnected_mirror_images/templates/registries.conf.j2 @@ -0,0 +1,11 @@ +{% for item in disconnected.mirroring.oc_mirror.post_mirror.mapping.replace.list if item.regexp is not none %} +[[registry]] +prefix = "{{ item.regexp }}" +location = "{{ item.replace }}" + +{% endfor %} +[[registry]] +location = "registry.redhat.io" + +[[registry.mirror]] +location = "{{ disconnected.mirroring.oc_mirror.release_image_registry }}" \ No newline at end of file diff --git a/roles/dns/templates/dns.db.j2 b/roles/dns/templates/dns.db.j2 index 40e64db9..d2240ddc 100644 --- a/roles/dns/templates/dns.db.j2 +++ b/roles/dns/templates/dns.db.j2 @@ -32,7 +32,7 @@ apps.{{ env.cluster.networking.metadata_name }} IN CNAME {{ env.bastion.netw {% if disconnected.enabled %} ;entry for mirror host. -{{ env.cluster.networking.metadata_name }}.{{ env.cluster.networking.metadata_name }}.{{ env.cluster.networking.base_domain }}. IN A {{ disconnected.mirroring.host.ip }} +{{ env.cluster.networking.metadata_name }}.{{ env.cluster.networking.metadata_name }}.{{ env.cluster.networking.base_domain }}. IN A {{ disconnected.registry.ip }} {% endif %} ;EOF