Add Claude and local LLM functionality

[cebarboza]

Add other LLM service providers. In the case of local LLMs, try first ollama with Gemma 4.

[susanneBruun]

We are really interested in this, since we have Claude but also a local LLM model. How far are you with the development of this feature?

[cebarboza]

Hi @susanneBruun I will try to work on this as soon as possible, I already have some update for this. BTW, this repo is public so Faptimus420 must have access.

[Faptimus420]

Hi @cebarboza. That's great - in that case, I can help with implementing this if it's needed. Our local model is already implemented in Ollama, so if we can use it instead of procuring an OpenAI or Claude API key that would be great.

[cebarboza]

Which local model would you like to try? @Faptimus420

[Faptimus420]

@cebarboza We have Llama 4, Qwen 3.5, Qwen 3.6, and Gemma 4 available to run locally. We'll probably give all of them a try to see how each of them does, before we decide which to actually use or whether we need to procure API access for an online mode (and if we do, it will most likely be Claude rather than OpenAI as we can get it at a discount since we already have a non-Console subscription).

[cebarboza]

I will try to set and test Claude as soon as I can, probably I can do a release in gh tomorrow or Wednesday.

Then we can check the local models. I tried Gemma 4 but I have to see what is the issue requesting an structured response, since in my mac kinds of keeps thinking forever.

I have an M5 48GB of RAM, so Gemma 4 26B actually runs decent for some coding tasks, but probably it nees more tweeking when requesting a structured response from the local models.

[Faptimus420]

Have been distracted by another task last week, but would still like to help.

For using Claude - great, should be useful.
For local models - doing your own testing on that is appreciated, though I feel like the usage of local models should be considered advanced and not thoroughly tested usage (especially since different users could be using vastly different local models), so making sure things work properly and return sensible data should be up to the user; after all, if they know how to set up and use local Ollama models, they should be able to get that working themselves. I'd be happy to test myself on our models, I just need some documentation on how to tell PatientGenerator to use a local model, and what are the files I need to modify in terms of context prompts that are passed to the LLM.