AAP-43385: On-prem: Automate provision and related configuration changes

[manstis]

Jira Issue: https://issues.redhat.com/browse/AAP-43385

Description

This PR adds support for deploying ansible-chatbot-stack with our Operator.

It will automatically provision and register various MCP Servers along with llama-stack as necessary.

The behavior is defined by additional settings in chatbot_config_secret:

Testing

1. Deploy the Operator somewhere.
2. Configure it as normal for the Chatbot.
3. Add additional parameters for aap_gateway_url and aap_controller_url.
4. The MCP servers should be provisioned and be registered with llama-stack

• AAP Controller MCP Server
• AAP Lightspeed MCP Server

NOTE: We currently don't support AAP Gateway MCP Server due to issues relating to authentication tokens.

NOTE: It's difficult to test with MCP Server integration. It requires an AAP 2.6 instance for authentication.

CRO (tested with minikube)

apiVersion: aiconnect.ansible.com/v1alpha1
kind: AnsibleAIConnect
metadata:
  name: my-aiconnect
  namespace: ansible-ai-connect-operator-system
spec:
  no_log: false
  ingress_type: Ingress
  service_type: NodePort
  nodeport_port: 30109
  image_pull_secrets:
    - redhat-lightspeed-pull-secret
  image: quay.io/ansible/ansible-ai-connect-service
  image_version: latest
  image_pull_policy: Always
  chatbot_image: quay.io/ansible/ansible-chatbot-stack
  chatbot_image_version: latest
  chatbot_rag_db_image: quay.io/ansible/aap-rag-content
  chatbot_rag_db_image_version: latest
  auth_config_secret_name: 'my-secret-auth-configuration'
  chatbot_config_secret_name: 'my-secret-chatbot-configuration'
  database:
    postgres_storage_class: standard
  extra_settings:
    - setting: SOCIAL_AUTH_REDIRECT_IS_HTTPS
      value: false
    - setting: CSRF_TRUSTED_ORIGINS
      value: http://192.168.49.2:30109

Chatbot secret (without MCP)

---
apiVersion: v1
kind: Secret
metadata:
  name: my-secret-chatbot-configuration
  namespace: ansible-ai-connect-operator-system
data:
  chatbot_model: <whatever>
  chatbot_token: <whatever>
  chatbot_url: <whatever>
type: Opaque

Chatbot secret (with MCP)

---
apiVersion: v1
kind: Secret
metadata:
  name: my-secret-chatbot-configuration
  namespace: ansible-ai-connect-operator-system
data:
  chatbot_model: <whatever>
  chatbot_token: <whatever>
  chatbot_url: <whatever>
  aap_controller_url: <whatever>
  aap_gateway_url: <whatever>
type: Opaque

Scenarios tested

As above.

Production deployment

• This code change is ready for production on its own
• This code change requires the following considerations before going to production:

[manstis]

@rooftopcellist @rcarrillocruz You remain the most knowledgeable about operator development and therefore your review is welcome... you'll no doubt have some involvement with this and related works if and when it gets downstreamed (in some form or other) and so "early sight" won't do anybody any harm either.

[rcarrillocruz]

Operator wise I see where this change is going.
However, I think @matburt had concerns about exposing MCP servers in ports, so probably besides having the SDP to get LlamaStack as a thing in the product we would need to have another SDP for MCP in AAP overall.

Just thinking out loud: Would it be too crazy to ship MCP-component as a sidecar in component and expose it via an endpoint in the component nginx?

[manstis]

Hi @rcarrillocruz we are to conduct a SDP for llama-stack + MCP. It's in progress.

The MCP servers in this PR are added as sidecars in the main Chatbot Pod.. no ports are exposed outside of this.

All network traffic between llama-stack and the MCP servers is Pod-local.

Anyway.. this is a draft, SDP illustrating in essence this PR to be conducted. It's all likely to change 🤣

[rcarrillocruz]

Awesome, I thought the LlamaStack SDP was about downstreamification of it.

[rcarrillocruz]

What I meant by sidecars I meant as a service running in the upstream component project.
Then manage it with supervisord, expose it with the component nginx.
That way it's at the component layer.

But yeah, that's better discussed in the MCP architecture SDP, let's not continue this convo here.

[rooftopcellist]

The diff looks good. I did not get to deploy and poke at it, but I think we can merge and let it harden in downstream builds and test more there.