fix: ollama detect context length automatically

[uinstinct]

Description

Use the Ollama configured default context length of 4096 (instead of Continue's 32,768).

resolves CON-3817

AI Code Review

• Team members only: AI review runs automatically when PR is opened or marked ready for review
• Team members can also trigger a review by commenting @continue-general-review or @continue-detailed-review

Checklist

• [] I've read the contributing guide
• [] The relevant docs, if any, have been updated or created
• [] The relevant tests, if any, have been updated or created

Screen recording or screenshot

before

after

Tests

[ What tests were added or updated to ensure the changes work as expected? ]

---

Summary by cubic

Use Ollama’s actual context length: we now default to 4096 (Ollama’s config) and use the model-provided value when available, instead of Continue’s 32,768. This prevents overestimating the token window and aligns with CON-3817.

• Bug Fixes
  • Added a contextLength getter that returns the detected model value, or 4096 as fallback.
  • Replaces the incorrect 32,768 default to avoid overflow and truncation issues.

[RomneyDa]

4096 might be a bit too restrictive, wondering if we should do 8192

[uinstinct]

4096 might be a bit too restrictive, wondering if we should do 8192

We can do it. It would take around 20% more memory in 8192 (attached image below).

Wondering if it would concern the users? Or should we put a user facing setting which can configure this num_ctx parameter (it also helps users know its changeable)?

[RomneyDa]

I think 20% is fine, 4096 makes sense for text in text out but for an agent application it's too restrictive