Skip to content

Conversation

@AlexCK-STFC
Copy link
Member

@AlexCK-STFC AlexCK-STFC commented Aug 13, 2025

Change default model from Deepseek to Microsoft Phi, set a default context limit, and update vLLM

Requires stfc/cloud-helm-charts#87 to be merged first

Change default model from Deepseek, and update vLLM
@AlexCK-STFC
Copy link
Member Author

AlexCK-STFC commented Aug 13, 2025

We might also want to add a note to azimuth_capi_operator_app_templates_huggingface_llm_description warning users not to use DeepSeek or other models which are not allowed as per UKRI security policy.

But I can't actually find a list of models which are not allowed.
I've found this: https://ukri.sharepoint.com/sites/thesource/SitePages/DeepSeek-AI-security-notice.aspx#

But what about other models with similar security concerns?
For example, Qwen is a popular OpenSource model that is developed by Alibaba. Hunyuan is developed by Tencent. Are these allowed?

It stops huggingface provisioning when the Monitoring cluster addon is disabled, similar to the issue we're having with FluentOperator
@AlexCK-STFC AlexCK-STFC force-pushed the switch-huggingface-repo branch from d77dd2a to c4d620d Compare August 14, 2025 13:29
@AlexCK-STFC
Copy link
Member Author

Also applied hotfix to disable monitoring, as it breaks deployment on services with monitoring disabled

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants