Skip to content

Conversation

@de1987
Copy link

@de1987 de1987 commented Oct 15, 2025

Jira Issue: AAP-55681

Description

To support the 3rd party models it's required to:

  • bump lightspeed-core-stack to 0.3.0
  • bump lightspeed-providers to 0.1.6

Testing

Steps to test

  1. Pull down the PR
  2. make clean
  3. make run
  4. open the chatbot and send prompts asking about Ansible/AAP

Scenarios tested

I run the chatbox locally with Granite 3.3 and asked at least 20 questions about AAP, playbooks, how to automate things... All the responses were as expected.

Production deployment

  • This code change is ready for production on its own
  • This code change requires the following considerations before going to production:

@de1987 de1987 self-assigned this Oct 15, 2025
bump lightspeed-providers to 0.1.6
@de1987 de1987 force-pushed the aap-55681-bump-lcs-and-lightspeed-providers branch from 3d164ad to 37d526e Compare October 16, 2025 17:16
@de1987 de1987 requested a review from jameswnl October 16, 2025 17:19
Copy link
Contributor

@romartin romartin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code looking good!! 👍

But can't test because of can't pull from quay....

$ podman login quay.io
Authenticating with existing credentials for quay.io
Existing credentials are invalid, please enter valid username and password
Username (romartin): 
Password: 
Error: authenticating creds for "quay.io": too many requests to registry

Copy link
Contributor

@romartin romartin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes are looking great, thanks @de1987

I have been able to test it quickly, in local, and I see a few things needs re-visiting before merging:

  • I still need tp riun a few AAP RAG & Inventory Gen. tests to ensure quality

  • Noticed slower inference times using LSC 0.3.0. I have to do more tests and compare with previous LSC versions anyway... may not block the PR, but we have to guarantee quality but also performance, otherwise let LSC team know

  • Please double check provider_id updates, after changes, as I see warning like:

WARNING  2025-10-27 21:47:34,406 llama_stack.core.routing_tables.common:257 core::routing_tables: WARNING: model identifier 'granite-3.3-8b-instruct' 
         not found in routing table. Falling back to searching in all providers. This is only for backwards compatibility and will stop working soon. 
         Migrate your calls to use fully scoped `provider_id/model_id` names. 

Thanks!

model_type: embedding
- metadata: {}
model_id: ${env.GEMINI_INFERENCE_MODEL:=gemini/gemini-2.5-flash}
model_id: ${env.GEMINI_INFERENCE_MODEL:=gemini-2.5-flash}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please redo this change as gemini model id works like this

Suggested change
model_id: ${env.GEMINI_INFERENCE_MODEL:=gemini-2.5-flash}
model_id: ${env.GEMINI_INFERENCE_MODEL:=gemini/gemini-2.5-flash}

model_id: ${env.GEMINI_INFERENCE_MODEL:=gemini-2.5-flash}
provider_id: gemini
provider_model_id: gemini/gemini-2.5-flash
provider_model_id: gemini-2.5-flash
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please redo this change also or set it to null

@ldjebran
Copy link
Contributor

@romartin that warning is not due to gemini having been changed, but to many other model ids.

I think redo back that gemeni changes and we will be ready to merge

llama-stack==0.2.16
llama-stack-client==0.2.16
llama-stack==0.2.22
llama-stack-client==0.2.23
Copy link
Contributor

@ldjebran ldjebran Oct 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why llama-stack-client version is different ?

it should be the same as llama-stack

https://github.com/lightspeed-core/lightspeed-stack/blob/main/pyproject.toml#L32C6-L32C24

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants