Skip to content

feat(ai-core): add AiConfigurationService for AI preferences #319

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
AlexandraBuzila wants to merge 4 commits into from
Closed

feat(ai-core): add AiConfigurationService for AI preferences #319

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
7721f1a
feat(ai-core): add AiConfigurationService for AI preferences
AlexandraBuzila Jul 7, 2026
45045a6
refactor(ai-core): route AI preference reads through AiConfigurationS…
AlexandraBuzila Jul 7, 2026
213fff1
refactor: address review comments
AlexandraBuzila Jul 7, 2026
b3c23f6
refactor: remove unused isTrusted getter from TrustAwarePreferenceReader
AlexandraBuzila Jul 7, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion doc/Migration.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,7 @@ Custom agents are no longer stored in a single `customAgents.yml` per scope. Eac
**End-user-facing:**

- When an existing `customAgents.yml` is found, Theia asks before migrating it to the new layout: a notification offers **Migrate** or **Don't Show Again**. Nothing is written until you choose **Migrate**, which avoids unexpected file changes (for example in a workspace under version control). Declining is safe: legacy `customAgents.yml` files keep being loaded so agents continue to work, and you are asked again next session until you either migrate or dismiss the prompt for good with **Don't Show Again** (remembered in local storage, not a setting). After a successful migration the `customAgents.yml` is renamed to `customAgents.yml.bak` (never deleted); restoring it is a matter of renaming the `.bak` back by hand. The migration can also be triggered at any time from the command palette via `AI: Re-run custom-agent migration`.
- Prompts stored as a YAML *folded* block scalar (`prompt: >-`) keep their markdown heading structure: the folded scalar would otherwise merge each heading into the following paragraph (e.g. `## Task Your task is ...`), so migration and runtime loading preserve the original line breaks instead. If an earlier Theia version already migrated such an agent with merged headings, the generated `agent.md` is corrected automatically on the next migration, but only when you have not edited it since (the corrected content is rewritten from `customAgents.yml.bak`; user-modified files are left untouched).
- Prompts stored as a YAML _folded_ block scalar (`prompt: >-`) keep their markdown heading structure: the folded scalar would otherwise merge each heading into the following paragraph (e.g. `## Task Your task is ...`), so migration and runtime loading preserve the original line breaks instead. If an earlier Theia version already migrated such an agent with merged headings, the generated `agent.md` is corrected automatically on the next migration, but only when you have not edited it since (the corrected content is rewritten from `customAgents.yml.bak`; user-modified files are left untouched).
- The default prompt-override file created by "Edit prompt" changed from `<agent-name>_prompt.prompttemplate` to `prompt.prompttemplate` inside the agent folder. Existing sibling `<agent-name>_prompt*.prompttemplate` files are moved into the agent folder during migration.

**Adopter-facing:**
Expand All @@ -130,6 +130,15 @@ Custom agents are now scanned from both the `.agents/` and `.prompts/` folders o

- `PromptFragmentCustomizationProperties` gained an optional `agentDirectoryPaths` field carrying the absolute parent directories scanned for custom agents. The `.agents`/`.prompts` parents are exported as `CUSTOM_AGENT_WORKSPACE_DIRECTORIES`.

#### `AiConfigurationService` for reading/writing AI preferences

A new framework API, `AiConfigurationService` (`@theia/ai-core`), wraps `PreferenceService` for `ai-features.*` preferences and is the intended extension point for reading/writing AI configuration.

**Adopter-facing:**

- Prefer `AiConfigurationService` over `PreferenceService` for `ai-features.*` keys in frontend code. Its `get`/`inspect` are workspace-trust-aware (workspace/folder values are suppressed while the workspace is untrusted); writes (`set`/`update`) are never gated by trust.
- **Behavior change:** the AI terminal's shell-command allowlist/denylist (`ai-features.terminal.shellCommand{Allowlist,Denylist}`) are now read trust-aware via `AiConfigurationService`. Previously they were read with a raw `PreferenceService.get`, so an untrusted workspace could contribute allowlist entries. Now workspace/folder-scoped entries are suppressed until the workspace is trusted (an untrusted workspace can no longer widen the shell allowlist). User- and default-scoped entries are unaffected.

### v1.70.0

#### Removal of deprecated @theia/git extension from Theia codebase [#17148](https://github.com/eclipse-theia/theia/pull/17148)
Expand Down
91 changes: 43 additions & 48 deletions packages/ai-chat/src/browser/chat-tool-preference-bindings.spec.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,9 +23,7 @@ import {
TOOL_CONFIRMATION_PREFERENCE,
ToolConfirmationMode
} from '../common/chat-tool-preferences';
import { ToolRequest } from '@theia/ai-core';
import { PreferenceService } from '@theia/core/lib/common/preferences';
import { TrustAwarePreferenceReader } from '@theia/ai-core/lib/browser/trust-aware-preference-reader';
import { AiConfigurationService, ToolRequest } from '@theia/ai-core';

interface InspectResult<T> {
defaultValue?: T;
Expand All @@ -35,8 +33,7 @@ interface InspectResult<T> {

describe('ToolConfirmationManager', () => {
let manager: ToolConfirmationManager;
let preferenceServiceMock: sinon.SinonStubbedInstance<PreferenceService>;
let trustAwareReaderMock: sinon.SinonStubbedInstance<TrustAwarePreferenceReader>;
let aiConfigurationServiceMock: sinon.SinonStubbedInstance<AiConfigurationService>;
let storedPerToolPreferences: { [toolId: string]: ToolConfirmationMode };
let storedDefaultMode: ToolConfirmationMode | undefined;
let trusted: boolean;
Expand All @@ -58,27 +55,9 @@ describe('ToolConfirmationManager', () => {
perToolInspectResult = undefined;
defaultInspectResult = undefined;

preferenceServiceMock = {
updateValue: sinon.stub().callsFake((key: string, value: unknown) => {
if (key === TOOL_CONFIRMATION_PREFERENCE) {
storedPerToolPreferences = value as { [toolId: string]: ToolConfirmationMode };
} else if (key === DEFAULT_TOOL_CONFIRMATION_PREFERENCE) {
storedDefaultMode = value as ToolConfirmationMode;
}
return Promise.resolve();
}),
inspect: sinon.stub().callsFake((name: string) => {
if (name === TOOL_CONFIRMATION_PREFERENCE) {
return perToolInspectResult;
}
if (name === DEFAULT_TOOL_CONFIRMATION_PREFERENCE) {
return defaultInspectResult;
}
return undefined;
})
} as unknown as sinon.SinonStubbedInstance<PreferenceService>;

trustAwareReaderMock = {
// The manager talks only to AiConfigurationService. `get` mirrors the trust-aware read
// semantics, `update` the smart write, and `inspect` exposes schema defaults.
aiConfigurationServiceMock = {
get: sinon.stub().callsFake(<T>(name: string, fallback?: T): T | undefined => {
if (name === TOOL_CONFIRMATION_PREFERENCE) {
if (trusted) {
Expand All @@ -95,13 +74,29 @@ describe('ToolConfirmationManager', () => {
return ((value as unknown as T) ?? fallback);
}
return fallback;
}),
update: sinon.stub().callsFake((key: string, value: unknown) => {
if (key === TOOL_CONFIRMATION_PREFERENCE) {
storedPerToolPreferences = value as { [toolId: string]: ToolConfirmationMode };
} else if (key === DEFAULT_TOOL_CONFIRMATION_PREFERENCE) {
storedDefaultMode = value as ToolConfirmationMode;
}
return Promise.resolve();
}),
inspect: sinon.stub().callsFake((name: string) => {
if (name === TOOL_CONFIRMATION_PREFERENCE) {
return perToolInspectResult;
}
if (name === DEFAULT_TOOL_CONFIRMATION_PREFERENCE) {
return defaultInspectResult;
}
return undefined;
})
} as unknown as sinon.SinonStubbedInstance<TrustAwarePreferenceReader>;
} as unknown as sinon.SinonStubbedInstance<AiConfigurationService>;

const container = new Container();
container.bind(ToolConfirmationManager).toSelf().inSingletonScope();
container.bind(PreferenceService).toConstantValue(preferenceServiceMock as unknown as PreferenceService);
container.bind(TrustAwarePreferenceReader).toConstantValue(trustAwareReaderMock as unknown as TrustAwarePreferenceReader);
container.bind(AiConfigurationService).toConstantValue(aiConfigurationServiceMock as unknown as AiConfigurationService);
manager = container.get(ToolConfirmationManager);
});

Expand All @@ -124,9 +119,9 @@ describe('ToolConfirmationManager', () => {
describe('setDefaultConfirmationMode', () => {
it('persists the new default through the preference service', () => {
manager.setDefaultConfirmationMode(ToolConfirmationMode.ALWAYS_ALLOW);
expect(preferenceServiceMock.updateValue.calledOnce).to.be.true;
expect(preferenceServiceMock.updateValue.firstCall.args[0]).to.equal(DEFAULT_TOOL_CONFIRMATION_PREFERENCE);
expect(preferenceServiceMock.updateValue.firstCall.args[1]).to.equal(ToolConfirmationMode.ALWAYS_ALLOW);
expect(aiConfigurationServiceMock.update.calledOnce).to.be.true;
expect(aiConfigurationServiceMock.update.firstCall.args[0]).to.equal(DEFAULT_TOOL_CONFIRMATION_PREFERENCE);
expect(aiConfigurationServiceMock.update.firstCall.args[1]).to.equal(ToolConfirmationMode.ALWAYS_ALLOW);
expect(storedDefaultMode).to.equal(ToolConfirmationMode.ALWAYS_ALLOW);
});
});
Expand Down Expand Up @@ -223,31 +218,31 @@ describe('ToolConfirmationManager', () => {
describe('setConfirmationMode', () => {
it('persists ALWAYS_ALLOW for a regular tool when default is CONFIRM', () => {
manager.setConfirmationMode('regularTool', ToolConfirmationMode.ALWAYS_ALLOW);
expect(preferenceServiceMock.updateValue.calledOnce).to.be.true;
expect(aiConfigurationServiceMock.update.calledOnce).to.be.true;
expect(storedPerToolPreferences['regularTool']).to.equal(ToolConfirmationMode.ALWAYS_ALLOW);
});

it('persists ALWAYS_ALLOW for confirmAlwaysAllow tools', () => {
const toolRequest = createToolRequest('dangerousTool', true);
manager.setConfirmationMode('dangerousTool', ToolConfirmationMode.ALWAYS_ALLOW, toolRequest);
expect(preferenceServiceMock.updateValue.calledOnce).to.be.true;
expect(aiConfigurationServiceMock.update.calledOnce).to.be.true;
expect(storedPerToolPreferences['dangerousTool']).to.equal(ToolConfirmationMode.ALWAYS_ALLOW);
});

it('does not persist when mode matches the global default', () => {
storedDefaultMode = ToolConfirmationMode.ALWAYS_ALLOW;
manager.setConfirmationMode('regularTool', ToolConfirmationMode.ALWAYS_ALLOW);
expect(preferenceServiceMock.updateValue.called).to.be.false;
expect(aiConfigurationServiceMock.update.called).to.be.false;
});

it('does not persist CONFIRM for a regular tool when default is CONFIRM', () => {
manager.setConfirmationMode('regularTool', ToolConfirmationMode.CONFIRM);
expect(preferenceServiceMock.updateValue.called).to.be.false;
expect(aiConfigurationServiceMock.update.called).to.be.false;
});

it('persists DISABLED when default is CONFIRM', () => {
manager.setConfirmationMode('regularTool', ToolConfirmationMode.DISABLED);
expect(preferenceServiceMock.updateValue.calledOnce).to.be.true;
expect(aiConfigurationServiceMock.update.calledOnce).to.be.true;
expect(storedPerToolPreferences['regularTool']).to.equal(ToolConfirmationMode.DISABLED);
});

Expand All @@ -256,7 +251,7 @@ describe('ToolConfirmationManager', () => {
defaultValue: { 'myTool': ToolConfirmationMode.DISABLED }
};
manager.setConfirmationMode('myTool', ToolConfirmationMode.DISABLED);
expect(preferenceServiceMock.updateValue.called).to.be.false;
expect(aiConfigurationServiceMock.update.called).to.be.false;
});

it('removes an existing entry when mode matches the tool-specific schema default', () => {
Expand All @@ -265,14 +260,14 @@ describe('ToolConfirmationManager', () => {
};
storedPerToolPreferences['myTool'] = ToolConfirmationMode.ALWAYS_ALLOW;
manager.setConfirmationMode('myTool', ToolConfirmationMode.DISABLED);
expect(preferenceServiceMock.updateValue.calledOnce).to.be.true;
expect(aiConfigurationServiceMock.update.calledOnce).to.be.true;
expect(storedPerToolPreferences['myTool']).to.be.undefined;
});

it('does not persist CONFIRM for confirmAlwaysAllow tools (matches effective default)', () => {
const toolRequest = createToolRequest('dangerousTool', true);
manager.setConfirmationMode('dangerousTool', ToolConfirmationMode.CONFIRM, toolRequest);
expect(preferenceServiceMock.updateValue.called).to.be.false;
expect(aiConfigurationServiceMock.update.called).to.be.false;
});

it('persists ALWAYS_ALLOW for a confirmAlwaysAllow tool when global default is ALWAYS_ALLOW', () => {
Expand All @@ -281,21 +276,21 @@ describe('ToolConfirmationManager', () => {
storedDefaultMode = ToolConfirmationMode.ALWAYS_ALLOW;
const toolRequest = createToolRequest('dangerousTool', true);
manager.setConfirmationMode('dangerousTool', ToolConfirmationMode.ALWAYS_ALLOW, toolRequest);
expect(preferenceServiceMock.updateValue.calledOnce).to.be.true;
expect(aiConfigurationServiceMock.update.calledOnce).to.be.true;
expect(storedPerToolPreferences['dangerousTool']).to.equal(ToolConfirmationMode.ALWAYS_ALLOW);
});

it('removes an existing entry when setting CONFIRM for a confirmAlwaysAllow tool', () => {
const toolRequest = createToolRequest('dangerousTool', true);
storedPerToolPreferences['dangerousTool'] = ToolConfirmationMode.ALWAYS_ALLOW;
manager.setConfirmationMode('dangerousTool', ToolConfirmationMode.CONFIRM, toolRequest);
expect(preferenceServiceMock.updateValue.calledOnce).to.be.true;
expect(aiConfigurationServiceMock.update.calledOnce).to.be.true;
expect(storedPerToolPreferences['dangerousTool']).to.be.undefined;
});

it('persists DISABLED for any tool when default is CONFIRM', () => {
manager.setConfirmationMode('anyTool', ToolConfirmationMode.DISABLED);
expect(preferenceServiceMock.updateValue.calledOnce).to.be.true;
expect(aiConfigurationServiceMock.update.calledOnce).to.be.true;
expect(storedPerToolPreferences['anyTool']).to.equal(ToolConfirmationMode.DISABLED);
});
});
Expand Down Expand Up @@ -362,9 +357,9 @@ describe('ToolConfirmationManager', () => {

manager.resetAllConfirmationModeSettings();

expect(preferenceServiceMock.updateValue.calledOnce).to.be.true;
expect(preferenceServiceMock.updateValue.firstCall.args[0]).to.equal(TOOL_CONFIRMATION_PREFERENCE);
expect(preferenceServiceMock.updateValue.firstCall.args[1]).to.deep.equal({});
expect(aiConfigurationServiceMock.update.calledOnce).to.be.true;
expect(aiConfigurationServiceMock.update.firstCall.args[0]).to.equal(TOOL_CONFIRMATION_PREFERENCE);
expect(aiConfigurationServiceMock.update.firstCall.args[1]).to.deep.equal({});
});

it('does not modify the default-confirmation preference', () => {
Expand All @@ -386,7 +381,7 @@ describe('ToolConfirmationManager', () => {

manager.setConfirmationMode('shellExecute', ToolConfirmationMode.ALWAYS_ALLOW, toolRequest);

expect(preferenceServiceMock.updateValue.calledOnce).to.be.true;
expect(aiConfigurationServiceMock.update.calledOnce).to.be.true;
expect(storedPerToolPreferences['shellExecute']).to.equal(ToolConfirmationMode.ALWAYS_ALLOW);

mode = manager.getConfirmationMode('shellExecute', 'chat-1', toolRequest);
Expand All @@ -398,7 +393,7 @@ describe('ToolConfirmationManager', () => {

manager.setConfirmationMode('shellExecute', ToolConfirmationMode.DISABLED, toolRequest);

expect(preferenceServiceMock.updateValue.calledOnce).to.be.true;
expect(aiConfigurationServiceMock.update.calledOnce).to.be.true;
expect(storedPerToolPreferences['shellExecute']).to.equal(ToolConfirmationMode.DISABLED);

const mode = manager.getConfirmationMode('shellExecute', 'chat-1', toolRequest);
Expand Down
Loading
Loading