Skip to content

feat: add deployment_type property to all PostHog telemetry events#421

Open
Freezaa9 wants to merge 2 commits intodevelopfrom
fix/issue-334-posthog-deployment-context
Open

feat: add deployment_type property to all PostHog telemetry events#421
Freezaa9 wants to merge 2 commits intodevelopfrom
fix/issue-334-posthog-deployment-context

Conversation

@Freezaa9
Copy link
Copy Markdown
Contributor

Summary

  • Adds deployment_type: "cloud" | "self-hosted" to every PostHog event across the platform — zero per-event instrumentation required
  • Frontend: detects by hostname (cloud.idunplatform.com"cloud", everything else → "self-hosted"), registered as a PostHog super property via posthog.register() in main.tsx
  • Backend (engine): reads IDUN_DEPLOYMENT_TYPE env var (default: "self-hosted"); cloud deployment sets IDUN_DEPLOYMENT_TYPE=cloud in its Docker environment
  • Adds unit test project to vite.config.ts so existing and new .test.ts files can run with vitest --project=unit

Closes #334

Changes

Area File What changed
Frontend utility src/utils/deployment.ts getDeploymentType(hostname) — pure function, easily testable
Frontend init src/main.tsx DeploymentSuperProperties component calls posthog.register() once at startup
Frontend tests src/utils/deployment.test.ts 8 unit tests covering cloud, self-hosted, edge cases
Frontend config vite.config.ts Added unit test project with node environment
Backend config telemetry/config.py get_deployment_type(environ) — reads IDUN_DEPLOYMENT_TYPE env var
Backend telemetry telemetry/telemetry.py _common_properties() now includes deployment_type
Backend tests tests/unit/telemetry/test_deployment_type.py 7 unit tests for detection + common properties
Docs services/idun_agent_web/CLAUDE.md Runtime config table + analytics section
Docs libs/idun_agent_engine/CLAUDE.md Telemetry module comment updated

Test plan

  • vitest --project=unit — 12 frontend unit tests pass (8 new deployment tests)
  • ruff check — no issues on modified Python files
  • Backend logic validated in isolation (Python REPL)
  • Verify in PostHog that new events include deployment_type after deploying

Future Improvements

These would make the analytics even more powerful for comparing cloud vs self-hosted usage:

Richer event context

  • Track platform version (library_version already on backend — surface it in frontend events too)
  • Add agent_count as a super property (requires a quick API call on startup)
  • Add workspace_count and user_count to understand deployment scale
  • Track auth_mode (basic vs oidc) to segment by enterprise vs hobbyist usage

Segmentation & funnels

  • Funnel analysis: compare conversion from "agent created" to "agent invoked" between cloud and self-hosted
  • Retention cohorts: do cloud users stay active longer than self-hosted users?
  • Feature adoption heatmaps: which integrations (Langfuse, MCP, guardrails) are more popular on each deployment type?

Alerting & anomaly detection

  • Alert on sudden drop in self-hosted telemetry events (could indicate a version with a telemetry bug shipped)
  • Alert when cloud usage spikes (scaling event, viral moment) vs self-hosted (could indicate a popular blog post or GitHub star surge)
  • Monitor error rates by deployment type to catch cloud-specific or self-hosted-specific regressions

Dashboard templates

  • Pre-built PostHog dashboard: "Cloud vs Self-Hosted" side-by-side KPIs (DAU, agents created, invocations per agent)
  • Separate funnel for each deployment type to identify where drop-off differs
  • Geographic distribution of self-hosted deployments (via IP geolocation in PostHog)

Licence & plan segmentation

  • Add license_type property once paid tiers are introduced (free / pro / enterprise)
  • Combine with deployment_type for a 2x2 analysis: cloud-free, cloud-paid, self-hosted-free, self-hosted-paid
  • Use PostHog feature flags to A/B test UI changes only on cloud users before rolling out to self-hosted

Self-hosted telemetry health

  • Track telemetry opt-out rate (IDUN_TELEMETRY_ENABLED=false) to understand what fraction of self-hosted installs are dark
  • Compare feature usage between opted-in and opted-out cohorts (if users consent to anonymous benchmarks)

🤖 Generated with Claude Code

Freezaa9 and others added 2 commits March 22, 2026 23:08
When a user exports a compiled graph (common in LangGraph tutorials), the
engine now recovers gracefully instead of raising a TypeError. The
CompiledStateGraph.builder attribute gives back the original StateGraph,
which is then recompiled with the engine-managed checkpointer/store.

- _validate_graph_builder: extract .builder from CompiledStateGraph and
  return the underlying StateGraph with a warning; raise TypeError only
  for genuinely unsupported types
- initialize: drop the now-dead CompiledStateGraph branch (always
  receives a StateGraph after _validate_graph_builder)
- Add unit test confirming compiled-graph input is accepted, the engine
  checkpointer is injected, and invoke() works end-to-end

Closes #401

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
All PostHog events now include a `deployment_type` super property that
automatically distinguishes cloud deployments from self-hosted ones.

Frontend: `src/utils/deployment.ts` detects cloud by matching the
hostname `cloud.idunplatform.com`; everything else is `self-hosted`.
The value is registered as a PostHog super property in `main.tsx` via
`posthog.register()`, so no per-event instrumentation is needed.

Backend: `IDUN_DEPLOYMENT_TYPE=cloud` env var tags engine telemetry
events. Defaults to `self-hosted` when unset, so existing self-hosted
deployments get the correct value without any configuration change.
The cloud deployment sets this env var in its Docker environment.

Also adds a unit test project to vite.config.ts so existing and new
`.test.ts` files can be run with `vitest --project=unit`.

Closes #334

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant