infra.aap_utilities.aap_setup_install failing after successful install at 'Wait for automation controller to be running"

[usmcguy]

Summary

Running the role infra.aap_utilities.aap_setup_install fails after successful install of update. Then fails when "Waiting for automation controller to be running" task executes on localhost.

Issue Type

• Bug Report

Ansible, Collection, Docker/Podman details

ansible --version

ansible [core 2.15.3]
  config file = /home/user/gitlab/aap-config/ansible.cfg
  configured module search path = ['/home/user/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/user/.local/lib/python3.11/site-packages/ansible
  ansible collection location = /home/user/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/user/.local/bin/ansible
  python version = 3.11.11 (main, Dec  9 2024, 15:32:27) [GCC 8.5.0 20210514 (Red Hat 8.5.0-22)] (/usr/bin/python3)
  jinja version = 3.1.4
  libyaml = True

ansible-galaxy collection list
Collection           Version
-------------------- -------
ansible.posix        2.0.0  
community.general    10.1.0 
community.postgresql 3.10.2 
infra.aap_utilities  2.6.0  
kubernetes.core      5.1.0

podman --version
podman version 4.9.4-rhel

• ansible installation method: one of source, pip, OS package, EE
  pip

OS / ENVIRONMENT

Red Hat Enterprise Linux 8.10

Desired Behavior

Expected the task to wait for the controllers to start

Actual Behavior

After the setup installation completes, successfully, when it gets to the task 'Wait for automation controller to be running' it has a problem with the variables. Unusual, because they were used to create the inventory file using aap_setup_download role. The error output

TASK [infra.aap_utilities.aap_setup_install : Wait for automation controller to be running] ***************************************
fatal: [localhost]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: {{ (aap_setup_prep_inv_nodes['automationcontroller'].keys() | list)[0] }}: 'list object' has no attribute 'keys'. 'list object' has no attribute 'keys'. {{ (aap_setup_prep_inv_nodes['automationcontroller'].keys() | list)[0] }}: 'list object' has no attribute 'keys'. 'list object' has no attribute 'keys'\n\nThe error appears to be in '/home/dking/.ansible/collections/ansible_collections/infra/aap_utilities/roles/aap_setup_install/tasks/setup.yml': line 96, column 7, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n    - name: Wait for automation controller to be running\n      ^ here\n"}

STEPS TO REPRODUCE

variables.yml
https://gist.github.com/usmcguy/28ce963d3d3b884d702f19e78f30c486

PROD.ini
https://gist.github.com/usmcguy/6fa7c186fd162d8d8596fa8facadc211

aap_install.yml
https://gist.github.com/usmcguy/c56edd3e2693bef5a4c1b6fd174efe09

ansible-playbook -i PROD.ini aap_install.yml -K

[djdanielsson]

thank you for reporting the bug, it might take some time for us to get around to debug what is going on and fix it

[sean-m-sullivan]

Can we also get the version of AAP you are reporting this on, There were some big changes in the API from 2.4 to 2.5, that might be causing this.

[usmcguy]

AAP is 2.4 with latest build package released earlier this month.

[amasolov]

I'm experiencing the same with install AAP 2.5 on RHEL9 using Ansible 2.14.17 (from RHEL9)

[snagoor]

@usmcguy hi, the problem he is that how you defined your inventory hosts groups. According to your varaible.yml setup, these would be set as a list not a dict.

aap_setup_prep_inv_nodes:
  automationcontroller: "{{ groups['aap_controllers'] }}"
  automationhub: "{{ groups['aap_hubs'] }}"
  execution_nodes: "{{ groups['aap_execs'] }}"

According to aap_setup_install role, this aap_setup_prep_inv_nodes variable should be a dictionary as follows

aap_setup_prep_inv_nodes:  # a dictionary of dictionaries!
  automationcontroller:
    ansible-ctrl.example.com:
  automationhub:
    ansible-hub.example.com:
  automationedacontroller:
    ansible-eda.example.com:
  database:
    database.example.com:  # If using an already existing DB, remove this group/node
                           # and adapt accordingly the following database related values
  execution_nodes:
    execution-1.example.com:
    execution-2.example.com:

If this aap_setup_prep_inv_nodes is not defined as a dictionary, then these default variable initializations will fail https://github.com/redhat-cop/aap_utilities/blob/devel/roles/aap_setup_install/defaults/main.yml#L56C1-L79C91

controller_hostname: "{{ (aap_setup_prep_inv_nodes['automationcontroller'].keys() | list)[0] }}"
ah_hostname: "{{ (aap_setup_prep_inv_nodes['automationhub'].keys() | list)[0] }}"
eda_hostname: "{{ (aap_setup_prep_inv_nodes[__aap_setup_inst_eda_group_name].keys() | list)[0] }}"
gateway_hostname: "{{ (aap_setup_prep_inv_nodes['automationgateway'].keys() | list)[0] }}"

To fix this, I did the following in my setup i.e. to over-ride these values as per my configuration

controller_hostname: "{{ groups['aap_controller'] | map('extract', hostvars, 'ansible_fqdn') | first }}"
gateway_hostname: "{{ groups['aap_gateway'] | map('extract', hostvars, 'ansible_fqdn') | first }}"

[usmcguy]

@snagoor You appear to be correct. Using a dictionary instead of a list fixed my issue.