Assistant: Improve managing language models

[georgestagg]

This PR makes a few changes in an attempt to improve managing LLM provider models.

The manage models quickpick menu has been improved:

• Copilot's BYOK providers, particularly ones that Assistant does not currently support (e.g. Grok) are no longer shown.
• Only providers that the user has signed in with are shown
• The managementCommand property has been added to Assistant providers, so that the cog icon works.
• These changes address Assistant: "Manage Models" shows irrelevant providers #9283.

A new configuration setting positron.assistant.filterModels has been added. With this users (and in the future admins) can set glob filters. Models will only be shown if they match one of the given globs.

• This addresses Assistant: Include/Exclude filters for LLM models #9788.

A positron.assistant.preferredModel setting has been added. With this, when the Positron window first loads it should default to loading the model given in that setting. Here we use partial matching on the ID or model names.

• This partially addresses Assistant: configure default model per provider #9388, in that IIUC it matches the behaviour requested in the final comment of that thread.

A positron.assistant.defaultModels settings has been added. With this, users can specify the default model for a particular provider, which we will pre-select in the model selector if the user hasn't specified a preferred model instead.

• This addresses the remaining parts of Assistant: configure default model per provider #9388 based on Assistant: Improve managing language models #9799 (comment).
• We have set the default model for Anthropic to Claude Sonnet 4 and Bedrock to Claude 4 Sonnet Bedrock

---

Release Notes

New Features

• Assistant: added setting positron.assistant.filterModels to configure available models (Assistant: Include/Exclude filters for LLM models #9788)
• Assistant: added setting positron.assistant.preferredModel to set the preferred model to pre-select in the model selector (Assistant: configure default model per provider #9388)
• Assistant: added setting positron.assistant.defaultModels to set the default model per provider (Assistant: configure default model per provider #9388)

Bug Fixes

• Assistant: non-functional providers removed from "Manage models..." quickpick (Assistant: "Manage Models" shows irrelevant providers #9283)

QA Notes

Manage models quickpick

1. First, ensure you have multiple providers configured and signed in. I used Anthropic, AWS Bedrock, and GitHub Copilot Chat.
2. In the Assistant panel, invoke the model picker:
3. Ensure only useful entries are shown:
4. Select a signed in model, ensure a model list appears:

5. Do the same but select a signed out model, ensure the model configuration modal appears.
6. Do the same thing with a signed in model but click the cog on the far right, the model configuration modal should again appear.

Model filtering

1. In your user settings, set the following two filters:

"positron.assistant.filterModels": ["Claude*", "GPT-5"],

2. Restart Positron or Reload the window.
3. Ensure that only Claude family models or GPT-5 can be used:

4. This should also work with IDs, useful for e.g. OpenRouter:

"positron.assistant.filterModels": ["**/google/gemini-*"],

5. Finally, this should also restrict the manage models quickpick:

Setting preferred model

1. In your user settings, remove the positron.assistant.filterModels setting and set the following:

"positron.assistant.preferredModel": "GPT-5",

2. In the Assistant chat panel, switch to another provider and model, e.g. AWS Bedrock and Claude 4 Sonnet.
3. Create a new chat, ensure selected model does not change.
4. Reload the window, ensure that on reload the preferred model set in the configuration is first selected. In this case, it should switch to GPT-5 from the Copilot provider on reload.

Setting default model per provider

1. Sign into multiple providers, at least Anthropic and AWS Bedrock.
2. Try switching between them, confirm that the defaults we want by default are selected.
3. Set the following settings,

  "positron.assistant.defaultModels": { "anthropic-api": "Opus", "amazon-bedrock": "Opus" },

4. Reload the window, confirm that switching between the models now selects Opus models by default.

Note 1: Upstream has code to remember your last selection, so after the reload window you have to change providers to see the default kick in.

Note 2: Sometimes Assistant gets stuck in a state showing "Pick Model..." in the selector. Switching providers does nothing to clear that, but selecting a model first, and then switching providers, gets things working again. I don't know if this is related to this commit, or the changes in #9799.

[github-actions]

E2E Tests 🚀
This PR will run tests tagged with: @:critical

^{readme  valid tags}

[georgestagg]

Looks like I've broken something, will be busy over the new few days but will debug failing tests when I can.

[timtmok]

I can take over and see what's going on with the tests. It might be the test echo model that is broken here.

[georgestagg]

Sure, feel free to commit straight to this branch if you figure it out.

[sharon-wang]

Pushed a couple changes to

• hopefully fix the test error by adding the Echo language model back d521546
• filter the "manage models" quickpick to only show vendors/providers that the user has already logged into 4e65121

[georgestagg]

Thank you for fixing the tests!

Note that the change in 4e65121 does not work for me, because the display names for the "Copilot" vendor and the "GitHub Copilot Chat" provider do not match. So Copilot is missing from the model management quickpick.

We have access to the vendor IDs here, I think we should match on that instead.

[timtmok]

I'm not sure if the defaultModels setting works as intended. If I set the default to Claude Sonnet 4.5, it does set it as default but Claude Sonnet 4 is also set as default. I end up with 4 selected and not 4.5.

[timtmok]

I've changed it so that there will never be more than one default model.

[timtmok]

I somehow got into a bad state when using this with Bedrock and these settings:

"positron.assistant.filterModels": [
        "Claude*",
        "GPT-5"
]

When I went to manage models, I could see Bedrock as a provider in the quick pick. Selecting it resulted in nothing showing. I can see Bedrock as a provider in the chat view but it's filtered out due to how we track available providers. Available providers are calculated from available models but the model filter is essentially filtering out providers.

[timtmok]

I've added some eventing around the filterModels setting so that the provider selector updates.