Skip to content

feat(errors): Events API backend - Add error upsampling aggregation functions #95940

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Jul 21, 2025

Conversation

yuvmen
Copy link
Member

@yuvmen yuvmen commented Jul 18, 2025

Allow projects with error upsampling to use new sample_count(), sample_eps() and
sample_epm() function columns in Discover, returning the non extrapolated versions of these functions.

@yuvmen yuvmen requested a review from a team July 18, 2025 21:39
@yuvmen yuvmen requested review from a team as code owners July 18, 2025 21:39
@github-actions github-actions bot added Scope: Frontend Automatically applied to PRs that change frontend components Scope: Backend Automatically applied to PRs that change backend components labels Jul 18, 2025
Copy link
Contributor

🚨 Warning: This pull request contains Frontend and Backend changes!

It's discouraged to make changes to Sentry's Frontend and Backend in a single pull request. The Frontend and Backend are not atomically deployed. If the changes are interdependent of each other, they must be separated into two pull requests and be made forward or backwards compatible, such that the Backend or Frontend can be safely deployed independently.

Have questions? Please ask in the #discuss-dev-infra channel.

@roggenkemper
Copy link
Member

can we separate the FE changes into a different PR from the BE ones?

cursor[bot]

This comment was marked as outdated.

@yuvmen
Copy link
Member Author

yuvmen commented Jul 18, 2025

yea Ill break it apart into FE and BE

… flag

Allow projects with error upsampling to use new sample_count(), sample_eps() and
sample_epm() function columns in Discover, returning the non extrapolated versions of these functions.
@yuvmen
Copy link
Member Author

yuvmen commented Jul 18, 2025

@roggenkemper Separated it, the FE PR is now in: #95946

result[f"{function}()"] = result[function]
del result[function]

for function in upsampling_affected_functions:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could these loops be combined?

@yuvmen yuvmen changed the title feat(errors): Add error upsampling aggregation functions with feature flag feat(errors): Add error upsampling aggregation functions Jul 21, 2025
@yuvmen yuvmen changed the title feat(errors): Add error upsampling aggregation functions feat(errors): Backend - Add error upsampling aggregation functions Jul 21, 2025
@yuvmen yuvmen changed the title feat(errors): Backend - Add error upsampling aggregation functions feat(errors): Events API backend - Add error upsampling aggregation functions Jul 21, 2025
Copy link

codecov bot commented Jul 21, 2025

❌ 3 Tests Failed:

Tests completed Failed Passed Skipped
26889 3 26886 204
View the top 3 failed test(s) by shortest run time
tests.snuba.api.endpoints.test_organization_events.OrganizationEventsErrorsDatasetEndpointTest::test_sample_eps_with_allowlisted_project
Stack Traces | 16.8s run time
#x1B[1m#x1B[.../api/endpoints/test_organization_events.py#x1B[0m:6913: in test_sample_eps_with_allowlisted_project
    assert response.status_code == 200, response.content
#x1B[1m#x1B[31mE   AssertionError: b'{"detail":"sample_eps is not a valid function"}'#x1B[0m
#x1B[1m#x1B[31mE   assert 400 == 200#x1B[0m
#x1B[1m#x1B[31mE    +  where 400 = <Response status_code=400, "application/json">.status_code#x1B[0m
tests.snuba.api.endpoints.test_organization_events.OrganizationEventsErrorsDatasetEndpointTest::test_sample_epm_with_allowlisted_project
Stack Traces | 17.1s run time
#x1B[1m#x1B[.../api/endpoints/test_organization_events.py#x1B[0m:6966: in test_sample_epm_with_allowlisted_project
    assert response.status_code == 200, response.content
#x1B[1m#x1B[31mE   AssertionError: b'{"detail":"sample_epm is not a valid function"}'#x1B[0m
#x1B[1m#x1B[31mE   assert 400 == 200#x1B[0m
#x1B[1m#x1B[31mE    +  where 400 = <Response status_code=400, "application/json">.status_code#x1B[0m
tests.snuba.api.endpoints.test_organization_events.OrganizationEventsErrorsDatasetEndpointTest::test_sample_count_with_allowlisted_project
Stack Traces | 17.6s run time
#x1B[1m#x1B[.../api/endpoints/test_organization_events.py#x1B[0m:6864: in test_sample_count_with_allowlisted_project
    assert response.status_code == 200, response.content
#x1B[1m#x1B[31mE   AssertionError: b'{"detail":"sample_count is not a valid function"}'#x1B[0m
#x1B[1m#x1B[31mE   assert 400 == 200#x1B[0m
#x1B[1m#x1B[31mE    +  where 400 = <Response status_code=400, "application/json">.status_code#x1B[0m

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

fix issues with double conversion and moved logic into helper
Copy link
Contributor

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: Upsampling Function Data Corruption

The convert_fields_for_upsampling function has two issues: it can lead to incorrect data in query results and fails to convert metadata correctly for empty results. Specifically, if both a standard function (e.g., count()) and its upsampled equivalent (e.g., upsampled_count()) are present, the standard function's key will incorrectly hold the upsampled value after conversion. Additionally, fields_meta keys are not converted when a query returns no data, as the conversion logic relies on keys being present in the data, resulting in incorrect field names in the metadata.

src/sentry/api/helpers/error_upsampling.py#L41-L62

# Collect keys that need conversion and exist in data
all_present_keys: set[str] = set()
for result in data:
all_present_keys.update(result.keys())
# Filter the pre-ordered list to only include keys actually present
keys_to_convert = [key for key in _ORDERED_CONVERSION_KEYS if key in all_present_keys]
# Apply conversions to data
for result in data:
for original_key in keys_to_convert:
if original_key in result:
converted_key = _FUNCTION_KEY_CONVERSIONS[original_key]
result[converted_key] = result[original_key]
del result[original_key]
# Apply conversions to fields_meta
for original_key in keys_to_convert:
if original_key in fields_meta:
converted_key = _FUNCTION_KEY_CONVERSIONS[original_key]
fields_meta[converted_key] = fields_meta[original_key]
del fields_meta[original_key]

Fix in CursorFix in Web


Was this report helpful? Give feedback by reacting with 👍 or 👎

def test_sample_count_with_allowlisted_project(self):
"""Test that sample_count() returns raw sample count (not upsampled) for allowlisted projects."""
# Set up allowlisted project
with self.options({"issues.client_error_sampling.project_allowlist": [self.project.id]}):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this use the flagpole definition now

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm yea I can probably change the backend to use the flagpole instead of the existing option, but I am under some time pressure, that is a cleanup I can do and test afterwards, I don't need to do that yet really, I basically now have an option for the backend and a flappole for the frontend, and ill keep it like that for now and unite them later.

@yuvmen yuvmen merged commit af5153b into master Jul 21, 2025
64 checks passed
@yuvmen yuvmen deleted the yuvmen/discover-add-sample-count-functions branch July 21, 2025 22:51
andrewshie-sentry pushed a commit that referenced this pull request Aug 4, 2025
…unctions (#95940)

Events API now supports new sample_count(), sample_eps() and sample_epm() function columns in Discover for error upsampling projects, which return the original non extrapolated versions of these functions.
@github-actions github-actions bot locked and limited conversation to collaborators Aug 6, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Scope: Backend Automatically applied to PRs that change backend components Scope: Frontend Automatically applied to PRs that change frontend components
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants