observability - telemetry #1
Description
pour débugger un LLM, j'analyse souvent les "thinking tokens" Il faudrait faire en sorte de les récupérer et les "logger" quelque part. L'idée c'est qu'en regardant ces thinking tokens, on peut checker si le modèle est en train de
- vraiment bosser sur la tâche ou
- tergiverse pour essayer de comprendre la tâche.
Dans le cas 2, il faut revoir le prompt et-ou les skills
voilà une possibilité:
Idea: combine Langfuse with OpenCode.ai, you first need a running instance of Langfuse. Since you want to self-host, the most reliable method is using Docker Compose.
Once Langfuse is up, you’ll configure OpenCode to route its LLM calls through a proxy or use Langfuse’s native OpenTelemetry (OTEL) support if your specific OpenCode version allows for custom telemetry endpoints.
Phase 1: Self-Host Langfuse (v2)
This setup includes the Langfuse web UI, the worker, a Postgres database, and Clickhouse for high-performance analytics.
- Clone the Repository:
git clone https://github.com/langfuse/langfuse.git
cd langfuse
- Configure Environment:
Open thedocker-compose.ymlfile. Look for entries marked with# CHANGEMEand replace them with secure, random strings (especiallyNEXTAUTH_SECRET,SALT, andENCRYPTION_KEY). - Launch the Stack:
docker compose up -d
- Access the UI:
Go tohttp://localhost:3000. Create your account (locally), then create a New Project. - Get Your Keys:
In your project settings, generate and copy your:
- Public Key (
pk-lf-...) - Secret Key (
sk-lf-...) - Host (This will be
http://localhost:3000or your server's IP)
Phase 2: Connecting OpenCode.ai to Langfuse
OpenCode is built to be model-agnostic. To get these traces into Langfuse, you have two primary methods:
Method A: The Gateway Approach (Recommended)
Since OpenCode.ai uses a baseURL for its model providers, you can use LiteLLM Proxy as a "man-in-the-middle." LiteLLM is natively supported by Langfuse and can relay everything OpenCode does.
- Run LiteLLM with Langfuse enabled:
# Set your keys
export LANGFUSE_PUBLIC_KEY="pk-lf-..."
export LANGFUSE_SECRET_KEY="sk-lf-..."
export LANGFUSE_HOST="http://localhost:3000"
# Start the proxy
pip install 'litellm[proxy]'
litellm --model gpt-4o --success_callback langfuse
- Configure OpenCode:
In youropencode.json(usually in~/.config/opencode/), point your model to the LiteLLM proxy:
{
"providers": {
"custom": {
"baseURL": "http://localhost:4000",
"apiKey": "anything"
}
}
}
Method B: Environment Variables (Direct OTEL)
OpenCode supports OpenTelemetry for its internal tracing. You can point this directly to your self-hosted Langfuse instance if you use the Langfuse OTEL endpoint.
- Set these variables in your terminal before running
opencode:
export OTEL_EXPORTER_OTLP_ENDPOINT="http://localhost:3000/api/public/otel"
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Basic <base64_encoded_pk:sk>"
export LANGFUSE_PUBLIC_KEY="pk-lf-..."
export LANGFUSE_SECRET_KEY="sk-lf-..."
(Note: You can encode your keys using echo -n "pk-lf-...:sk-lf-..." | base64)
Phase 3: Verification
- Run a task in OpenCode (e.g.,
opencode "refactor the auth module"). - Refresh your Langfuse dashboard at
http://localhost:3000. - You should see a new Trace appearing. You can click into it to see the exact system prompt OpenCode used, the files it read, and the code it generated.
Troubleshooting Tip
If you are running everything on the same machine, ensure your Docker containers can talk to each other. If OpenCode is running on the host and Langfuse is in Docker, use http://localhost:3000. If they are both in the same Docker network, use the service name (e.g., http://langfuse-server:3000).