Skip to content

Adding configurable OPENAI_FAST_MODEL param#7935

Open
elijahsgh wants to merge 62 commits intoblock:mainfrom
elijahsgh:configurable-openai-fast-model
Open

Adding configurable OPENAI_FAST_MODEL param#7935
elijahsgh wants to merge 62 commits intoblock:mainfrom
elijahsgh:configurable-openai-fast-model

Conversation

@elijahsgh
Copy link

@elijahsgh elijahsgh commented Mar 16, 2026

Summary

The OpenAI provider currently has a default fast model (gpt-4o-mini) that it attempts to use. When using the OpenAI provider with another provider, such as Lemonade Server, goose will consistently attempt to call this model even though it doesn't exist.

We should make this configurable.

Type of Change

  • Feature
  • Bug fix
  • Refactor / Code quality
  • Performance improvement
  • Documentation
  • Tests
  • Security fix
  • Build / Release
  • Other (specify below)

AI Assistance

  • This PR was created or reviewed with AI assistance

The original

Testing

Manual build and retry.

Related Issues

Relates to #7934
Discussion: NA

Screenshots/Demos (for UX changes)

Before:
NA
After:
NA

Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e7ba96714b

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8a7ae21c8f

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f039f0418a

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@DOsinga
Copy link
Collaborator

DOsinga commented Mar 20, 2026

thanks. we should probably do this through the custom providers and add it as a field to DeclarativeProviderConfig

Copy link
Collaborator

@DOsinga DOsinga left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fast_model field on DeclarativeProviderConfig and the from_custom_config wiring are exactly right — that's the correct place for this.

I've pushed a small cleanup to your branch: removed the OPENAI_FAST_MODEL env-var path from from_env and the corresponding ConfigKey from the built-in OpenAI provider metadata. The built-in provider keeps its hardcoded gpt-4o-mini; custom/declarative providers (Lemonade, Groq, etc.) can now set fast_model in their provider JSON to override it.

@DOsinga
Copy link
Collaborator

DOsinga commented Mar 20, 2026

can you do the DCO check @elijahsgh

@elijahsgh elijahsgh force-pushed the configurable-openai-fast-model branch from 882f0ac to c23347f Compare March 21, 2026 00:27
@elijahsgh elijahsgh requested a review from angiejones as a code owner March 21, 2026 00:27
elijahsgh and others added 18 commits March 20, 2026 19:29
Signed-off-by: esnyder <elijah.snyder1@gmail.com>
Signed-off-by: esnyder <elijah.snyder1@gmail.com>
Signed-off-by: esnyder <elijah.snyder1@gmail.com>
Signed-off-by: esnyder <elijah.snyder1@gmail.com>
…_MODEL env var

Signed-off-by: Douwe Osinga <douwe@squareup.com>
Signed-off-by: esnyder <elijah.snyder1@gmail.com>
Co-authored-by: Douwe Osinga <douwe@squareup.com>
Signed-off-by: esnyder <elijah.snyder1@gmail.com>
Co-authored-by: Douwe Osinga <douwe@squareup.com>
Signed-off-by: esnyder <elijah.snyder1@gmail.com>
…ge (block#7931)

Signed-off-by: Adrian Cole <adrian@tetrate.io>
Co-authored-by: Adrian Cole <adrian@tetrate.io>
Signed-off-by: esnyder <elijah.snyder1@gmail.com>
Signed-off-by: Angie Jones <angie@Angies-MBP.lan>
Co-authored-by: Angie Jones <angie@Angies-MBP.lan>
Signed-off-by: esnyder <elijah.snyder1@gmail.com>
Signed-off-by: esnyder <elijah.snyder1@gmail.com>
Signed-off-by: Angie Jones <angie@Angies-MBP.lan>
Co-authored-by: Angie Jones <angie@Angies-MacBook-Pro.local>
Co-authored-by: Angie Jones <angie@Angies-MBP.lan>
Signed-off-by: esnyder <elijah.snyder1@gmail.com>
Signed-off-by: Adrian Cole <adrian@tetrate.io>
Signed-off-by: esnyder <elijah.snyder1@gmail.com>
Signed-off-by: esnyder <elijah.snyder1@gmail.com>
Signed-off-by: Rizel Scarlett <rizel@squareup.com>
Signed-off-by: esnyder <elijah.snyder1@gmail.com>
Signed-off-by: Rizel Scarlett <rizel@squareup.com>
Signed-off-by: esnyder <elijah.snyder1@gmail.com>
Signed-off-by: Rizel Scarlett <rizel@squareup.com>
Signed-off-by: esnyder <elijah.snyder1@gmail.com>
Signed-off-by: Angie Jones <angie@Angies-MBP.lan>
Co-authored-by: Douwe Osinga <douwe@squareup.com>
Co-authored-by: Angie Jones <jones.angie@gmail.com>
Signed-off-by: esnyder <elijah.snyder1@gmail.com>
jh-block and others added 22 commits March 20, 2026 19:29
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Signed-off-by: esnyder <elijah.snyder1@gmail.com>
Signed-off-by: Douwe Osinga <douwe@squareup.com>
Co-authored-by: Douwe Osinga <douwe@squareup.com>
Signed-off-by: esnyder <elijah.snyder1@gmail.com>
Signed-off-by: Michael Neale <michael.neale@gmail.com>
Signed-off-by: esnyder <elijah.snyder1@gmail.com>
…mprovements to codex (block#8000)

Signed-off-by: Michael Neale <michael.neale@gmail.com>
Signed-off-by: esnyder <elijah.snyder1@gmail.com>
Signed-off-by: Elias Posen <elias@posen.ch>
Signed-off-by: esnyder <elijah.snyder1@gmail.com>
Signed-off-by: Rabi Mishra <mishra.rabi@gmail.com>
Signed-off-by: esnyder <elijah.snyder1@gmail.com>
…ate (block#7818)

Signed-off-by: fre$h <fre5h3nough@gmail.com>
Signed-off-by: fre <anonwurcod@proton.me>
Signed-off-by: esnyder <elijah.snyder1@gmail.com>
Signed-off-by: Douwe Osinga <douwe@squareup.com>
Co-authored-by: Douwe Osinga <douwe@squareup.com>
Signed-off-by: esnyder <elijah.snyder1@gmail.com>
Signed-off-by: Douwe Osinga <douwe@squareup.com>
Co-authored-by: Douwe Osinga <douwe@squareup.com>
Co-authored-by: Tyler Longwell <tlongwell@squareup.com>
Signed-off-by: esnyder <elijah.snyder1@gmail.com>
Signed-off-by: esnyder <elijah.snyder1@gmail.com>
…k#8025)

Signed-off-by: Michael Neale <michael.neale@gmail.com>
Signed-off-by: esnyder <elijah.snyder1@gmail.com>
Signed-off-by: Andrew Harvard <aharvard@squareup.com>
Co-authored-by: Goose <opensource@block.xyz>
Signed-off-by: esnyder <elijah.snyder1@gmail.com>
Signed-off-by: Douwe Osinga <douwe@squareup.com>
Co-authored-by: Douwe Osinga <douwe@squareup.com>
Signed-off-by: esnyder <elijah.snyder1@gmail.com>
Signed-off-by: esnyder <elijah.snyder1@gmail.com>
Signed-off-by: alexyao2015 <alexyao2015@users.noreply.github.com>
Co-authored-by: alexyao2015 <alexyao2015@users.noreply.github.com>
Signed-off-by: esnyder <elijah.snyder1@gmail.com>
Signed-off-by: Abhijay007 <Abhijay007j@gmail.com>
Signed-off-by: Abhijay Jain <Abhijay007j@gmail.com>
Signed-off-by: esnyder <elijah.snyder1@gmail.com>
…lock#8030)

Signed-off-by: Douwe Osinga <douwe@squareup.com>
Co-authored-by: Douwe Osinga <douwe@squareup.com>
Signed-off-by: esnyder <elijah.snyder1@gmail.com>
Signed-off-by: Clyde <spitfire55@users.noreply.github.com>
Signed-off-by: Douwe Osinga <douwe@squareup.com>
Co-authored-by: Clyde <clyde@Clydes-Mac-Studio.local>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Douwe Osinga <douwe@squareup.com>
Signed-off-by: esnyder <elijah.snyder1@gmail.com>
Signed-off-by: Rabi Mishra <mishra.rabi@gmail.com>
Signed-off-by: esnyder <elijah.snyder1@gmail.com>
Signed-off-by: alexyao2015 <alexyao2015@users.noreply.github.com>
Signed-off-by: Douwe Osinga <douwe@squareup.com>
Co-authored-by: alexyao2015 <alexyao2015@users.noreply.github.com>
Co-authored-by: Douwe Osinga <douwe@squareup.com>
Signed-off-by: esnyder <elijah.snyder1@gmail.com>
…lock#7949)

Signed-off-by: Rizel Scarlett <rizel@squareup.com>
Signed-off-by: Angie Jones <angie@Angies-MBP.lan>
Signed-off-by: Douwe Osinga <douwe@squareup.com>
Co-authored-by: Douwe Osinga <douwe@squareup.com>
Co-authored-by: Angie Jones <jones.angie@gmail.com>
Co-authored-by: Angie Jones <angie@Angies-MBP.lan>
Signed-off-by: esnyder <elijah.snyder1@gmail.com>
…lock#8033)

Signed-off-by: Andrew Harvard <aharvard@squareup.com>
Signed-off-by: Douwe Osinga <douwe@squareup.com>
Co-authored-by: Goose <opensource@block.xyz>
Co-authored-by: Douwe Osinga <douwe@squareup.com>
Signed-off-by: esnyder <elijah.snyder1@gmail.com>
@elijahsgh elijahsgh force-pushed the configurable-openai-fast-model branch from c23347f to 0747875 Compare March 21, 2026 00:29
Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0747875b85

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +67 to +69
} else if (matchedProvider) {
await upsert('GOOSE_MODEL', matchedProvider.metadata.default_model, false);
setConfiguredModel(matchedProvider.metadata.default_model);

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Do not overwrite NanoGPT's configured model

When the new onboarding completes NanoGPT setup, the backend has already written GOOSE_MODEL=openai/gpt-4.1-nano in configure_nanogpt() (crates/goose/src/config/signup_nanogpt/mod.rs:10,123-126), but this fallback branch immediately replaces it with the provider metadata default. NanoGptProvider::metadata() currently advertises anthropic/claude-sonnet-4.6 (crates/goose/src/providers/nanogpt.rs:18,78-84), so every user who picks NanoGPT in the new onboarding exits with a different model than the signup flow selected. That changes the intended free/default experience and can send first requests to a model the signup flow never meant to choose.

Useful? React with 👍 / 👎.

Comment on lines +600 to +601
bus.cancel_request(&request.request_id).await;
axum::http::StatusCode::OK

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Honor stop requests that arrive before registration

session_cancel ignores whether cancel_request actually found an active request and always returns 200. That creates a race with the new stop flow in useChatStream.stopStreaming() (ui/desktop/src/hooks/useChatStream.ts:927-955): if the user clicks Stop before /sessions/{id}/reply reaches try_register_request(), /cancel is a no-op, the UI drops its listener anyway, and the original POST can still register and keep running on the server. In that window the user believes generation was stopped, but tool calls and other side effects can continue with no remaining way to cancel them.

Useful? React with 👍 / 👎.

@angiejones angiejones force-pushed the configurable-openai-fast-model branch from 0747875 to b77fa0e Compare March 21, 2026 00:47
Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b77fa0e6ec

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

base_path: params.base_path,
env_vars: None,
dynamic_models: None,
fast_model: None,

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Allow custom-provider API to persist fast_model

OpenAiProvider::from_custom_config now honors config.fast_model, but newly created custom providers still hardcode fast_model to None here, and the create/update request structs/routes do not pass any fast-model field through. In the normal /config/custom-providers workflow this makes the new fast-model override effectively unreachable, so users cannot configure the fast model without manually editing JSON on disk.

Useful? React with 👍 / 👎.

vincenzopalazzo and others added 2 commits March 20, 2026 22:08
Signed-off-by: Douwe Osinga <douwe@squareup.com>
Co-authored-by: Douwe Osinga <douwe@squareup.com>
Signed-off-by: esnyder <elijah.snyder1@gmail.com>
…r exists (block#8035)

Signed-off-by: esnyder <elijah.snyder1@gmail.com>
@elijahsgh elijahsgh force-pushed the configurable-openai-fast-model branch from b77fa0e to b5a4cc6 Compare March 21, 2026 03:08
Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b5a4cc6967

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

throw error;
}
}
await submitToSession(targetSessionId, updatedUserMessage, messagesForUI);

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Subscribe to the forked session before replaying edited turns

When editType === 'edit', this path posts the new turn to targetSessionId, but the hook's SSE subscription is still created by useSessionEvents(sessionId) for the original session. Unlike the previous /reply call, no stream is opened for the forked session, so the Message/Finish events for the edited replay never arrive and the UI stays stuck in a streaming state after the edit request starts.

Useful? React with 👍 / 👎.

Comment on lines +113 to +118
pub async fn get_or_create_event_bus(&self, session_id: &str) -> Arc<SessionEventBus> {
let mut buses = self.session_buses.lock().await;
buses
.entry(session_id.to_string())
.or_insert_with(|| Arc::new(SessionEventBus::new()))
.clone()

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Evict completed sessions from the event-bus cache

get_or_create_event_bus permanently inserts a SessionEventBus into session_buses, but this commit only removes buses from delete_session (crates/goose-server/src/routes/session.rs:301-304). Simply opening or replying in many sessions will therefore retain one replay buffer per session for the lifetime of the server, and each buffer can hold up to 512 cloned MessageEvents (including whole conversations/tool output), so resident memory now grows until restart.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.