Skip to content

Commit 65d3cd9

Browse files
brendan-kellambrendan-kellam
authored
feat(ask_sb): OpenAI compatible language models (#424)
* wip * docs + add option for api key * feedback * nit
1 parent 68107da commit 65d3cd9

File tree

13 files changed

+708
-25
lines changed

13 files changed

+708
-25
lines changed

‎CHANGELOG.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
77

88
## [Unreleased]
99

10+
### Added
11+
- [ask sb] Added OpenAI Compatible Language Provider. [#424](https://github.com/sourcebot-dev/sourcebot/pull/424)
12+
1013
## [4.6.2] - 2025-07-31
1114

1215
### Changed

‎docs/docs/configuration/language-model-providers.mdx

Lines changed: 55 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,12 @@ title: Language Model Providers
33
sidebarTitle: Language model providers
44
---
55

6+
import LanguageModelSchema from '/snippets/schemas/v3/languageModel.schema.mdx'
7+
8+
<Note>
9+
Looking to self-host your own model? Check out the [OpenAI Compatible](#openai-compatible) provider.
10+
</Note>
11+
612
To use [Ask Sourcebot](/docs/features/ask) you must define at least one Language Model Provider. These providers are defined within the [config file](/docs/configuration/config-file) you
713
provide Sourcebot.
814

@@ -45,7 +51,7 @@ For a detailed description of all the providers, please refer to the [schema](ht
4551

4652
### Amazon Bedrock
4753

48-
[Vercel AI SDK Amazon Bedrock Docs](https://v5.ai-sdk.dev/providers/ai-sdk-providers/amazon-bedrock)
54+
[Vercel AI SDK Amazon Bedrock Docs](https://ai-sdk.dev/providers/ai-sdk-providers/amazon-bedrock)
4955

5056
```json wrap icon="code" Example config with Amazon Bedrock provider
5157
{
@@ -70,7 +76,7 @@ For a detailed description of all the providers, please refer to the [schema](ht
7076

7177
### Anthropic
7278

73-
[Vercel AI SDK Anthropic Docs](https://v5.ai-sdk.dev/providers/ai-sdk-providers/anthropic)
79+
[Vercel AI SDK Anthropic Docs](https://ai-sdk.dev/providers/ai-sdk-providers/anthropic)
7480

7581
```json wrap icon="code" Example config with Anthropic provider
7682
{
@@ -91,7 +97,7 @@ For a detailed description of all the providers, please refer to the [schema](ht
9197

9298
### Azure OpenAI
9399

94-
[Vercel AI SDK Azure OpenAI Docs](https://v5.ai-sdk.dev/providers/ai-sdk-providers/azure)
100+
[Vercel AI SDK Azure OpenAI Docs](https://ai-sdk.dev/providers/ai-sdk-providers/azure)
95101

96102
```json wrap icon="code" Example config with Azure AI provider
97103
{
@@ -114,7 +120,7 @@ For a detailed description of all the providers, please refer to the [schema](ht
114120

115121
### Deepseek
116122

117-
[Vercel AI SDK Deepseek Docs](https://v5.ai-sdk.dev/providers/ai-sdk-providers/deepseek)
123+
[Vercel AI SDK Deepseek Docs](https://ai-sdk.dev/providers/ai-sdk-providers/deepseek)
118124

119125
```json wrap icon="code" Example config with Deepseek provider
120126
{
@@ -135,7 +141,7 @@ For a detailed description of all the providers, please refer to the [schema](ht
135141

136142
### Google Generative AI
137143

138-
[Vercel AI SDK Google Generative AI Docs](https://v5.ai-sdk.dev/providers/ai-sdk-providers/google-generative-ai)
144+
[Vercel AI SDK Google Generative AI Docs](https://ai-sdk.dev/providers/ai-sdk-providers/google-generative-ai)
139145

140146
```json wrap icon="code" Example config with Google Generative AI provider
141147
{
@@ -159,7 +165,7 @@ For a detailed description of all the providers, please refer to the [schema](ht
159165
<Note>If you're using an Anthropic model on Google Vertex, you must define a [Google Vertex Anthropic](#google-vertex-anthropic) provider instead</Note>
160166
<Note>The `credentials` paramater here expects a **path** to a [credentials](https://console.cloud.google.com/apis/credentials) file. This file **must be in a volume mounted by Sourcebot** for it to be readable.</Note>
161167

162-
[Vercel AI SDK Google Vertex AI Docs](https://v5.ai-sdk.dev/providers/ai-sdk-providers/google-vertex)
168+
[Vercel AI SDK Google Vertex AI Docs](https://ai-sdk.dev/providers/ai-sdk-providers/google-vertex)
163169

164170
```json wrap icon="code" Example config with Google Vertex provider
165171
{
@@ -185,7 +191,7 @@ For a detailed description of all the providers, please refer to the [schema](ht
185191
<Note>The `credentials` paramater here expects a **path** to a [credentials](https://console.cloud.google.com/apis/credentials) file. This file **must be in a volume mounted by Sourcebot** for it to be readable.</Note>
186192

187193

188-
[Vercel AI SDK Google Vertex Anthropic Docs](https://v5.ai-sdk.dev/providers/ai-sdk-providers/google-vertex#google-vertex-anthropic-provider-usage)
194+
[Vercel AI SDK Google Vertex Anthropic Docs](https://ai-sdk.dev/providers/ai-sdk-providers/google-vertex#google-vertex-anthropic-provider-usage)
189195

190196
```json wrap icon="code" Example config with Google Vertex Anthropic provider
191197
{
@@ -208,7 +214,7 @@ For a detailed description of all the providers, please refer to the [schema](ht
208214

209215
### Mistral
210216

211-
[Vercel AI SDK Mistral Docs](https://v5.ai-sdk.dev/providers/ai-sdk-providers/mistral)
217+
[Vercel AI SDK Mistral Docs](https://ai-sdk.dev/providers/ai-sdk-providers/mistral)
212218

213219
```json wrap icon="code" Example config with Mistral provider
214220
{
@@ -229,7 +235,7 @@ For a detailed description of all the providers, please refer to the [schema](ht
229235

230236
### OpenAI
231237

232-
[Vercel AI SDK OpenAI Docs](https://v5.ai-sdk.dev/providers/ai-sdk-providers/openai)
238+
[Vercel AI SDK OpenAI Docs](https://ai-sdk.dev/providers/ai-sdk-providers/openai)
233239

234240
```json wrap icon="code" Example config with OpenAI provider
235241
{
@@ -248,9 +254,36 @@ For a detailed description of all the providers, please refer to the [schema](ht
248254
}
249255
```
250256

257+
### OpenAI Compatible
258+
259+
[Vercel AI SDK OpenAI Compatible Docs](https://ai-sdk.dev/providers/openai-compatible-providers)
260+
261+
The OpenAI compatible provider allows you to use any model that is compatible with the OpenAI [Chat Completions API](https://github.com/ollama/ollama/blob/main/docs/openai.md). This includes self-hosted tools like [Ollama](https://ollama.ai/) and [llama.cpp](https://github.com/ggerganov/llama.cpp).
262+
263+
```json wrap icon="code" Example config with OpenAI Compatible provider
264+
{
265+
"$schema": "https://raw.githubusercontent.com/sourcebot-dev/sourcebot/main/schemas/v3/index.json",
266+
"models": [
267+
{
268+
"provider": "openai-compatible",
269+
"baseUrl": "BASE_URL_HERE",
270+
"model": "YOUR_MODEL_HERE",
271+
"displayName": "OPTIONAL_DISPLAY_NAME",
272+
"token": {
273+
"env": "OPTIONAL_API_KEY"
274+
}
275+
}
276+
]
277+
}
278+
```
279+
280+
<Accordion title="Troubleshooting">
281+
- When using [llama.cpp](https://github.com/ggml-org/llama.cpp), if you hit "Failed after 3 attempts. Last error: tools param requires --jinja flag", add the `--jinja` flag to your `llama-server` command.
282+
</Accordion>
283+
251284
### OpenRouter
252285

253-
[Vercel AI SDK OpenRouter Docs](https://v5.ai-sdk.dev/providers/community-providers/openrouter)
286+
[Vercel AI SDK OpenRouter Docs](https://ai-sdk.dev/providers/community-providers/openrouter)
254287

255288
```json wrap icon="code" Example config with OpenRouter provider
256289
{
@@ -271,7 +304,7 @@ For a detailed description of all the providers, please refer to the [schema](ht
271304

272305
### xAI
273306

274-
[Vercel AI SDK xAI Docs](https://v5.ai-sdk.dev/providers/ai-sdk-providers/xai)
307+
[Vercel AI SDK xAI Docs](https://ai-sdk.dev/providers/ai-sdk-providers/xai)
275308

276309
```json wrap icon="code" Example config with xAI provider
277310
{
@@ -288,4 +321,14 @@ For a detailed description of all the providers, please refer to the [schema](ht
288321
}
289322
]
290323
}
291-
```
324+
```
325+
326+
327+
## Schema reference
328+
329+
<Accordion title="Reference">
330+
[schemas/v3/languageModel.json](https://github.com/sourcebot-dev/sourcebot/blob/main/schemas/v3/languageModel.json)
331+
332+
<LanguageModelSchema />
333+
334+
</Accordion>

‎docs/snippets/schemas/v3/index.schema.mdx

Lines changed: 126 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1785,6 +1785,69 @@
17851785
],
17861786
"additionalProperties": false
17871787
},
1788+
"OpenAICompatibleLanguageModel": {
1789+
"type": "object",
1790+
"properties": {
1791+
"provider": {
1792+
"const": "openai-compatible",
1793+
"description": "OpenAI Compatible Configuration"
1794+
},
1795+
"model": {
1796+
"type": "string",
1797+
"description": "The name of the language model."
1798+
},
1799+
"displayName": {
1800+
"type": "string",
1801+
"description": "Optional display name."
1802+
},
1803+
"token": {
1804+
"description": "Optional API key. If specified, adds an `Authorization` header to request headers with the value Bearer <token>.",
1805+
"anyOf": [
1806+
{
1807+
"type": "object",
1808+
"properties": {
1809+
"secret": {
1810+
"type": "string",
1811+
"description": "The name of the secret that contains the token."
1812+
}
1813+
},
1814+
"required": [
1815+
"secret"
1816+
],
1817+
"additionalProperties": false
1818+
},
1819+
{
1820+
"type": "object",
1821+
"properties": {
1822+
"env": {
1823+
"type": "string",
1824+
"description": "The name of the environment variable that contains the token. Only supported in declarative connection configs."
1825+
}
1826+
},
1827+
"required": [
1828+
"env"
1829+
],
1830+
"additionalProperties": false
1831+
}
1832+
]
1833+
},
1834+
"baseUrl": {
1835+
"type": "string",
1836+
"format": "url",
1837+
"pattern": "^https?:\\/\\/[^\\s/$.?#].[^\\s]*$",
1838+
"description": "Base URL of the OpenAI-compatible chat completions API endpoint.",
1839+
"examples": [
1840+
"http://localhost:8080/v1"
1841+
]
1842+
}
1843+
},
1844+
"required": [
1845+
"provider",
1846+
"model",
1847+
"baseUrl"
1848+
],
1849+
"additionalProperties": false
1850+
},
17881851
"OpenRouterLanguageModel": {
17891852
"type": "object",
17901853
"properties": {
@@ -2528,6 +2591,69 @@
25282591
],
25292592
"additionalProperties": false
25302593
},
2594+
{
2595+
"type": "object",
2596+
"properties": {
2597+
"provider": {
2598+
"const": "openai-compatible",
2599+
"description": "OpenAI Compatible Configuration"
2600+
},
2601+
"model": {
2602+
"type": "string",
2603+
"description": "The name of the language model."
2604+
},
2605+
"displayName": {
2606+
"type": "string",
2607+
"description": "Optional display name."
2608+
},
2609+
"token": {
2610+
"description": "Optional API key. If specified, adds an `Authorization` header to request headers with the value Bearer <token>.",
2611+
"anyOf": [
2612+
{
2613+
"type": "object",
2614+
"properties": {
2615+
"secret": {
2616+
"type": "string",
2617+
"description": "The name of the secret that contains the token."
2618+
}
2619+
},
2620+
"required": [
2621+
"secret"
2622+
],
2623+
"additionalProperties": false
2624+
},
2625+
{
2626+
"type": "object",
2627+
"properties": {
2628+
"env": {
2629+
"type": "string",
2630+
"description": "The name of the environment variable that contains the token. Only supported in declarative connection configs."
2631+
}
2632+
},
2633+
"required": [
2634+
"env"
2635+
],
2636+
"additionalProperties": false
2637+
}
2638+
]
2639+
},
2640+
"baseUrl": {
2641+
"type": "string",
2642+
"format": "url",
2643+
"pattern": "^https?:\\/\\/[^\\s/$.?#].[^\\s]*$",
2644+
"description": "Base URL of the OpenAI-compatible chat completions API endpoint.",
2645+
"examples": [
2646+
"http://localhost:8080/v1"
2647+
]
2648+
}
2649+
},
2650+
"required": [
2651+
"provider",
2652+
"model",
2653+
"baseUrl"
2654+
],
2655+
"additionalProperties": false
2656+
},
25312657
{
25322658
"type": "object",
25332659
"properties": {

0 commit comments

Comments
 (0)