Groq support

[TymekV]

It would be nice to be able to use Groq as AI provider. It shouldn't be too hard to implement because Groq has an OpenAI compatible API.
API Reference

[sirkitree]

I was able to get pretty far using it today. Here's my settings:

"language_models": {
    "openai_compatible": {
      "groq": {
        "api_url": "https://api.groq.com/openai/v1",
        "available_models": [
          {
            "name": "moonshotai/kimi-k2-instruct-0905",
            "display_name": "kimi-k2",
            "max_tokens": 262144,
            "max_output_tokens": 16384,
            "max_completion_tokens": 16384,
            "capabilities": {
              "tools": true,
              "images": false,
              "parallel_tool_calls": false,
              "prompt_cache_key": false
            }
          }
        ]
      }
    }
  },
  "agent": {
    "default_model": {
      "provider": "groq",
      "model": "moonshotai/kimi-k2-instruct-0905"
    }
  },

However, it seems that Zed doesn't support the tool calling as I'm getting

tool call validation failed: parameters for tool edit_file did not match schema: errors: [additionalProperties 'source_content' not allowed]

But man - groq with kimi-k2 is SUPER fast

[imumesh18]

Hey @sirkitree are you trying this on stable? Will you be able to try the preview and see if this is fixed and tool calling work?

[sirkitree]

confirmed! it works in preview v0.203.2

thank you for such a quick response!

[imumesh18]

Hey @sirkitree thanks for confirmation this pr: #34921 was merged which has improved the tool calling schema for open_ai compatible provider. Currently it's only available right now in preview. Enjoy hopefully people will be able to use more providers.

[sirkitree]

hrm, just received another one:

tool call validation failed: parameters for tool edit_file did not match schema: errors: [missing properties: 'path']

[sirkitree]

a retry worked, but thought i'd mention it anyway

[MrBns]

they have added a lots of provider, why not groq ? please add groq support. vscode already support it.