Skip to content

Comments

fix: Improve usage reporting#1108

Open
jsonbailey wants to merge 6 commits intomainfrom
jb/aic-1636/improve-reporting
Open

fix: Improve usage reporting#1108
jsonbailey wants to merge 6 commits intomainfrom
jb/aic-1636/improve-reporting

Conversation

@jsonbailey
Copy link
Contributor

@jsonbailey jsonbailey commented Feb 18, 2026

Note

Medium Risk
Changes analytics event keys/payloads and adds constructor-side tracking, which can affect downstream metrics pipelines and introduce unintended extra track calls, but does not alter core AI/config evaluation logic.

Overview
Improves usage reporting for the server AI SDK.

LDAIClientImpl now emits a one-time init event ($ld:ai:sdk:info) on construction with { aiSdkName, aiSdkVersion, aiSdkLanguage }, and renames the per-method tracking events to a consistent $ld:ai:usage:* scheme for completionConfig, agentConfig(s), judgeConfig, createChat, and createJudge.

SDK name/version were removed from LDAIConfigTracker’s per-metric getTrackData() payload (tests updated accordingly), and sdkInfo now includes a new aiSdkLanguage constant.

Written by Cursor Bugbot for commit a765834. This will update automatically on new commits. Configure here.

@github-actions
Copy link
Contributor

@launchdarkly/js-sdk-common size report
This is the brotli compressed size of the ESM build.
Compressed size: 25394 bytes
Compressed size limit: 26000
Uncompressed size: 124693 bytes

@github-actions
Copy link
Contributor

github-actions bot commented Feb 18, 2026

@launchdarkly/browser size report
This is the brotli compressed size of the ESM build.
Compressed size: 171304 bytes
Compressed size limit: 200000
Uncompressed size: 798320 bytes

@github-actions
Copy link
Contributor

github-actions bot commented Feb 18, 2026

@launchdarkly/js-client-sdk-common size report
This is the brotli compressed size of the ESM build.
Compressed size: 19370 bytes
Compressed size limit: 20000
Uncompressed size: 99722 bytes

@github-actions
Copy link
Contributor

github-actions bot commented Feb 18, 2026

@launchdarkly/js-client-sdk size report
This is the brotli compressed size of the ESM build.
Compressed size: 23373 bytes
Compressed size limit: 25000
Uncompressed size: 81215 bytes

@jsonbailey jsonbailey marked this pull request as ready for review February 19, 2026 21:59
@jsonbailey jsonbailey requested a review from a team as a code owner February 19, 2026 21:59
Copy link
Contributor

@joker23 joker23 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 4 potential issues.

variables?: Record<string, unknown>,
defaultAiProvider?: SupportedAIProvider,
): Promise<Judge | undefined> {
this._ldClient.track(TRACK_JUDGE_CREATE, context, key, 1);
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Duplicate usage events for chat creation

Medium Severity

createChat now emits a create-chat usage event and then calls completionConfig, which also emits a completion-config usage event for the same operation. This can double-count usage when consumers use createChat as their primary entrypoint, skewing usage reporting compared to direct completionConfig calls.

Additional Locations (1)

Fix in Cursor Fix in Web

defaultAiProvider?: SupportedAIProvider,
): Promise<Judge | undefined> {
this._ldClient.track(TRACK_JUDGE_CREATE, context, key, 1);
this._ldClient.track(TRACK_USAGE_CREATE_JUDGE, context, key, 1);
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Duplicate usage events for judge creation

Medium Severity

createJudge now emits a create-judge usage event and then calls judgeConfig, which also emits a judge-config usage event. This changes usage semantics and can double-count “judge config usage” for callers that primarily use createJudge.

Additional Locations (1)

Fix in Cursor Fix in Web

modelName: string;
providerName: string;
aiSdkName: string;
aiSdkVersion: string;
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SDK info removed from metrics payloads

Medium Severity

getTrackData no longer includes aiSdkName/aiSdkVersion, and the LDAIConfigTracker type was updated accordingly. Any analytics pipeline or downstream consumer that relied on these fields in token/duration/success events will silently lose SDK attribution, even though only a separate $ld:ai:sdk:info event is now emitted.

Additional Locations (1)

Fix in Cursor Fix in Web

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is intentional and we are making the change while still in pre-release. This is not something downstream consumers should be relying on which is part of the reason for this change.

aiSdkLanguage,
},
1,
);
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Constructor tracking can spam events

Medium Severity

The LDAIClientImpl constructor now unconditionally calls track for $ld:ai:sdk:info. If client instances are created per request/job (common in server code), this can generate high-volume duplicate events and unexpected side effects during object creation, increasing telemetry cost and potentially impacting latency.

Fix in Cursor Fix in Web

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants