Publish complete Logfire setup skills - #54
Merged
Conversation
![macroscopeapp[bot]](https://avatars.githubusercontent.com/in/900172?s=60&v=4){kind=small}
strawgate
force-pushed
the
codex/publish-logfire-setup-skills
branch
from
September 1, 2026 16:29
bb4ba9b to
e1211dc
Compare
strawgate
changed the title
|
|
||
| ## Step 3: Configure the Collector | ||
|
|
||
| Follow the [collector reference](./references/collector/host-and-infra-metrics.md) for the receiver(s) identified in Step 2 — it covers the shared exporter setup, then a dedicated section per source: host metrics, Docker, Kubernetes, database/queue/cache servers, and cloud-provider metrics, each with the exact receiver name, a working config, and the caveats that actually bite (Docker socket permissions, API version pinning, `host.docker.internal` vs `localhost`, IAM permissions, ADOT vs. Contrib collector images). |
There was a problem hiding this comment.
🟡 Medium logfire-infrastructure/SKILL.md:34
Step 3 directs users to a collector reference that does not provide runnable configurations for Kubernetes, database/queue/cache, or cloud sources, so following this skill leaves those advertised monitoring setups unconfigured. The reference only describes a Kubernetes deployment pattern and lists receiver names and requirements for the other sources; add complete receiver and pipeline examples to the reference (and its mirrored skill) or narrow this claim.
🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file @plugins/logfire/skills/logfire-infrastructure/SKILL.md around line 34:
Step 3 directs users to a collector reference that does not provide runnable configurations for Kubernetes, database/queue/cache, or cloud sources, so following this skill leaves those advertised monitoring setups unconfigured. The reference only describes a Kubernetes deployment pattern and lists receiver names and requirements for the other sources; add complete receiver and pipeline examples to the reference (and its mirrored skill) or narrow this claim.
|
|
||
| **On the Python CLI, always put `--non-interactive` immediately after `logfire`, on every invocation, for the rest of whichever skill sent you here too.** Without it, a question with nobody to answer it (which org? which project?) blocks on a read that never returns — there's no TTY for the CLI to notice is missing, so it can't detect this on its own. It's the only way to guarantee a clear error instead of a silent hang. The JS CLI doesn't have this flag yet; if a JS-CLI command needs to ask something (e.g. which account, when more than one token is cached) with no TTY attached, it fails with a clear "not running in a terminal" error instead of hanging — so the outcome is the same either way, just reached differently. | ||
|
|
||
| - Determine the region (US or EU) from the project's URL or the user's context *before* authenticating, and pass it up front — `--region {us,eu}` is global, right after `logfire --non-interactive`, before the subcommand. |
There was a problem hiding this comment.
🟠 High references/auth.md:33
This authentication flow cannot reach self-hosted Logfire projects: it requires --region {us,eu} before auth, which targets only the managed US/EU service. Add guidance to detect self-hosted projects and pass the mutually exclusive --base-url option through the authentication and subsequent project commands.
🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file @plugins/logfire/skills/logfire-instrumentation/references/auth.md around line 33:
This authentication flow cannot reach self-hosted Logfire projects: it requires `--region {us,eu}` before `auth`, which targets only the managed US/EU service. Add guidance to detect self-hosted projects and pass the mutually exclusive `--base-url` option through the authentication and subsequent project commands.
| | Framework | How | Coverage | | ||
| |-----------|-----|----------| | ||
| | PydanticAI | `instrument_pydantic_ai()` | Full — agent runs, tool calls, LLM requests | | ||
| | OpenAI Agents SDK | `instrument_openai_agents()` | Agent runs + tokens + tool calls + messages (no cost yet) | |
There was a problem hiding this comment.
🟡 Medium logfire-instrumentation/SKILL.md:250
The OpenAI Agents SDK row falsely tells users that instrument_openai_agents() captures tool calls and messages, so they will expect those fields in telemetry even though this integration currently provides only agent runs and tokens. Update the coverage description to match the canonical integration behavior.
-| OpenAI Agents SDK | `instrument_openai_agents()` | Agent runs + tokens + tool calls + messages (no cost yet) |
+| OpenAI Agents SDK | `instrument_openai_agents()` | Agent runs + tokens (no tool or message coverage yet) |🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file @plugins/logfire/skills/logfire-instrumentation/SKILL.md around line 250:
The OpenAI Agents SDK row falsely tells users that `instrument_openai_agents()` captures tool calls and messages, so they will expect those fields in telemetry even though this integration currently provides only agent runs and tokens. Update the coverage description to match the canonical integration behavior.
| | OpenAI Agents SDK | `instrument_openai_agents()` | Agent runs + tokens + tool calls + messages (no cost yet) | | ||
| | Claude Agent SDK | `instrument_claude_agent_sdk()` | LLM spans + cost (doesn't yet populate the Agents view) | | ||
| | AutoGen | `instrument_openai()` + native OpenTelemetry | Agent runs + model requests + cost; tool/message coverage varies | | ||
| | LangChain, LangGraph | Python: native OpenTelemetry — set `LANGSMITH_TRACING=true`, `LANGSMITH_OTEL_ENABLED=true`, and `LANGSMITH_OTEL_ONLY=true` (`langsmith>=0.4.25` — without `LANGSMITH_TRACING`, tracing itself never turns on and telemetry silently never appears), then just `logfire.configure()`; no instrument call. JS/TS: LangSmith's own OTel exporter — call `initializeOTEL()` (from `langsmith/experimental/otel/setup`) before importing the rest of the app, pointed at Logfire via `OTEL_EXPORTER_OTLP_ENDPOINT`/`OTEL_EXPORTER_OTLP_HEADERS`; see LangSmith's own JS OTel docs for the exact shutdown/flush call. LangGraph agents produce an agent root with nested node, model, and tool spans and appear in the Agents view; other LangChain workloads remain visible in Live view | Varies by framework | |
There was a problem hiding this comment.
🟡 Medium logfire-instrumentation/SKILL.md:253
The LangChain/LangGraph row falsely promises that LangGraph runs appear in the Agents view, so users following this guidance will expect agent-root coverage that neither the Python LangSmith OTel path nor its JS exporter produces. Update the coverage description to state that these spans remain in Live view unless the integration actually emits an agent root.
-| LangChain, LangGraph | Python: native OpenTelemetry — set `LANGSMITH_TRACING=true`, `LANGSMITH_OTEL_ENABLED=true`, and `LANGSMITH_OTEL_ONLY=true` (`langsmith>=0.4.25` — without `LANGSMITH_TRACING`, tracing itself never turns on and telemetry silently never appears), then just `logfire.configure()`; no instrument call. JS/TS: LangSmith's own OTel exporter — call `initializeOTEL()` (from `langsmith/experimental/otel/setup`) before importing the rest of the app, pointed at Logfire via `OTEL_EXPORTER_OTLP_ENDPOINT`/`OTEL_EXPORTER_OTLP_HEADERS`; see LangSmith's own JS OTel docs for the exact shutdown/flush call. LangGraph agents produce an agent root with nested node, model, and tool spans and appear in the Agents view; other LangChain workloads remain visible in Live view | Varies by framework |
+| LangChain, LangGraph | Python: native OpenTelemetry — set `LANGSMITH_TRACING=true`, `LANGSMITH_OTEL_ENABLED=true`, and `LANGSMITH_OTEL_ONLY=true` (`langsmith>=0.4.25` — without `LANGSMITH_TRACING`, tracing itself never turns on and telemetry silently never appears), then just `logfire.configure()`; no instrument call. JS/TS: LangSmith's own OTel exporter — call `initializeOTEL()` (from `langsmith/experimental/otel/setup`) before importing the rest of the app, pointed at Logfire via `OTEL_EXPORTER_OTLP_ENDPOINT`/`OTEL_EXPORTER_OTLP_HEADERS`; see LangSmith's own JS OTel docs for the exact shutdown/flush call. LangGraph and other LangChain workloads remain visible in Live view; these paths do not create an agent root or Agents view entry | Varies by framework |🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file @plugins/logfire/skills/logfire-instrumentation/SKILL.md around line 253:
The LangChain/LangGraph row falsely promises that LangGraph runs appear in the Agents view, so users following this guidance will expect agent-root coverage that neither the Python LangSmith OTel path nor its JS exporter produces. Update the coverage description to state that these spans remain in Live view unless the integration actually emits an agent root.
| ``` | ||
|
|
||
| Set `LOGFIRE_TOKEN` in your environment or use the Logfire CLI to select a project. | ||
| Set `LOGFIRE_TOKEN` in your environment, or don't — the `logfire` crate's `data-dir` feature (on by default) falls back to `.logfire/logfire_credentials.json` when it's unset, same as Python. Set it explicitly only to override that: a different token, or production, where it should be a separately-minted token per [Authenticate and Select the Exact Project](./references/auth.md)'s "If the calling skill needs a write token" section, not the local one. |
There was a problem hiding this comment.
🟠 High logfire-instrumentation/SKILL.md:122
An unset LOGFIRE_TOKEN disables telemetry for the Rust SDK, so this instruction causes locally authenticated Rust applications to export no data. The SDK's IfTokenPresent behavior does not fall back to .logfire/logfire_credentials.json; instruct users to set LOGFIRE_TOKEN instead.
Suggested change
| Set `LOGFIRE_TOKEN` in your environment, or don't — the `logfire` crate's `data-dir` feature (on by default) falls back to `.logfire/logfire_credentials.json` when it's unset, same as Python. Set it explicitly only to override that: a different token, or production, where it should be a separately-minted token per [Authenticate and Select the Exact Project](./references/auth.md)'s "If the calling skill needs a write token" section, not the local one. | |
| Set `LOGFIRE_TOKEN` in your environment or use the Logfire CLI to select a project. |
🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file @plugins/logfire/skills/logfire-instrumentation/SKILL.md around line 122:
An unset `LOGFIRE_TOKEN` disables telemetry for the Rust SDK, so this instruction causes locally authenticated Rust applications to export no data. The SDK's `IfTokenPresent` behavior does not fall back to `.logfire/logfire_credentials.json`; instruct users to set `LOGFIRE_TOKEN` instead.
| @@ -0,0 +1,60 @@ | |||
| --- | |||
| name: logfire-infrastructure | |||
| description: Monitor hosts, Docker containers, Kubernetes clusters, database/queue/cache servers, and cloud-provider metrics with Pydantic Logfire — no application code required. Use this skill whenever the user asks to "monitor my host/server/VM", "monitor my Docker containers", "monitor my Kubernetes cluster", "send infrastructure metrics to Logfire", "watch my database/Postgres/Redis/MongoDB/Kafka", "collect cloud metrics" (AWS/GCP), or mentions the OpenTelemetry Collector in the context of Logfire. This is infrastructure only — for instrumenting APPLICATION CODE (traces, logs, AI/agent spans) use the logfire-instrumentation skill instead. | |||
There was a problem hiding this comment.
🟡 Medium logfire-infrastructure/SKILL.md:3
This trigger routes application-telemetry requests that merely mention an OpenTelemetry Collector to the infrastructure-only skill, causing line 8 to reject the requested traces/logs workflow and send the user to the wrong skill. Narrow the frontmatter trigger to infrastructure Collector use (or add an explicit application-telemetry disambiguation), and apply the same correction to the standalone mirror.
🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file @plugins/logfire/skills/logfire-infrastructure/SKILL.md around line 3:
This trigger routes application-telemetry requests that merely mention an OpenTelemetry Collector to the infrastructure-only skill, causing line 8 to reject the requested traces/logs workflow and send the user to the wrong skill. Narrow the frontmatter trigger to infrastructure Collector use (or add an explicit application-telemetry disambiguation), and apply the same correction to the standalone mirror.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Validation
quick_validate.pybash scripts/check-skill-sync.shgit diff --checkRollout
This focused PR replaces the earlier autosync dependency and contains no unrelated Pydantic AI changes. After merge, unified-docs can add setup, infrastructure, and evals to the public publication allowlist.