docs(ai): add AI orchestration stack + LLM observability pages#87
docs(ai): add AI orchestration stack + LLM observability pages#87JacobPEvans-personal wants to merge 1 commit into
Conversation
Document the self-hosted AI orchestration layer (n8n, Dify, LangFlow, CrewAI, LangChain) and its LLM observability pipeline, generically and ahead of implementation. - ai-development/ai-orchestration-stack: the five tools, the services-vs- libraries distinction, and a blunt "which to reach for" guide. - observability/llm-observability: OpenLLMetry + OTEL GenAI emission, Cribl as the single ingest hub fanning out to Langfuse (trace/cost/eval) and Splunk (archival/SIEM); why Langfuse. - Wire both into docs.json nav. No homelab topology, VLANs, or addresses — concept and tool guidance only. Assisted-by: Claude:claude-opus-4-8 Claude-Session: https://claude.ai/code/session_013KC8izFrMx32DVFduQp2tU
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces two new documentation pages, 'AI orchestration stack' and 'LLM observability', and registers them in docs.json. The 'AI orchestration stack' page details the tools used for building and running LLM workflows, while the 'LLM observability' page outlines the telemetry pipeline using OpenTelemetry, Cribl, Langfuse, and Splunk. The review feedback points out two factual inaccuracies in the observability document regarding the self-hosted footprint of Langfuse (which does not use ClickHouse) and the licensing of Arize Phoenix (which is Apache-2.0 rather than the Elastic License).
| | License | MIT — self-host with no feature gates | | ||
| | Ingestion | Native OTLP, GenAI-convention aware | | ||
| | Built for | LLM apps — traces, cost, evals, prompt management | | ||
| | Footprint | Web + worker + Postgres + ClickHouse + Redis + object storage | |
There was a problem hiding this comment.
Langfuse does not use ClickHouse in its self-hosted architecture; it relies on PostgreSQL for both relational and analytical trace data (along with Redis for caching/queues and S3-compatible storage for blobs). We should remove ClickHouse from the footprint description to keep the documentation accurate.
| Footprint | Web + worker + Postgres + Redis + object storage |
| toward long-running agent debugging. Arize Phoenix is capable but ships under the | ||
| Elastic License, which gates self-host use. |
There was a problem hiding this comment.
Arize Phoenix is licensed under the Apache-2.0 license, not the Elastic License, making it fully open-source and self-hostable. We should correct this statement to maintain accuracy.
toward long-running agent debugging. Arize Phoenix is also Apache-2.0 but has a
different design focus.
1 participant
What
Two new public docs pages for the self-hosted AI orchestration buildout,
written ahead of implementation so the docs lead the work:
ai-development/ai-orchestration-stack— n8n, Dify, LangFlow, CrewAI,LangChain: what each is, the services-vs-libraries split, and a blunt
"which one to reach for" guide.
observability/llm-observability— OpenLLMetry + OTEL GenAI emission;Cribl as the single OTEL ingest hub fanning out to Langfuse (trace, cost,
eval) and Splunk (archival/SIEM); why Langfuse over Phoenix/Laminar.
Both wired into
docs.jsonnav (AI Development, Observability groups).Scope
Concept and tool guidance only — no homelab topology, VLANs, VMIDs, or
addresses. The infrastructure specifics live in the private docs; the
provisioning/config land in
terraform-proxmoxandansible-proxmox-apps.Notes
Mermaid diagrams follow the house hand-drawn theme; internal links resolve to
existing or same-PR pages; external links are upstream project/spec URLs.
🤖 Generated with Claude Code
https://claude.ai/code/session_013KC8izFrMx32DVFduQp2tU