Skip to content

Conversation

@SungJin1212
Copy link
Member

@SungJin1212 SungJin1212 commented Oct 10, 2025

This PR introduces a new parquet mode for the Store Gateway. In this initial version, the mode operates statelessly and does not perform operations such as block syncing. The primary improvement is to enhance cache utilization by leveraging the Store Gateway's block affinity.

Which issue(s) this PR fixes:
Fixes #6940

Checklist

  • Tests updated
  • Documentation added
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

@SungJin1212 SungJin1212 marked this pull request as draft October 10, 2025 09:46
@SungJin1212 SungJin1212 force-pushed the parquet-gateway-poc branch 7 times, most recently from 27d4ca3 to 7ab5bc7 Compare October 15, 2025 06:44
@SungJin1212 SungJin1212 marked this pull request as ready for review October 15, 2025 09:40
# Minimum time to wait for ring stability at startup. 0 to disable.
# CLI flag: -store-gateway.sharding-ring.wait-stability-min-duration
[wait_stability_min_duration: <duration> | default = 1m]
[wait_stability_min_duration: <duration> | default = 0s]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change should be removed

# start anyway.
# CLI flag: -store-gateway.sharding-ring.wait-stability-max-duration
[wait_stability_max_duration: <duration> | default = 5m]
[wait_stability_max_duration: <duration> | default = 5s]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change should be removed

assert.Empty(t, warnings)
}

//func TestParquetBucketStores_Series_ShouldNotCheckMaxInflightRequestsIfTheLimitIsDisabled(t *testing.T) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to keep this

assert.Equal(t, codes.InvalidArgument, s.Code())
}

func TestChunkToStoreEncoding(t *testing.T) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test seems irrelevant?

cortex_testutil "github.com/cortexproject/cortex/pkg/util/test"
)

func TestParquetGatewayWithBlocksStorageRunningInMicroservicesMode(t *testing.T) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer reusing the existing store gateway integration test instead of creating a new parquet gateway test. This adds a lot of duplicated code

@SungJin1212
Copy link
Member Author

@yeya24
I added some commits; the quay.io seems to be in the ' read-only` state temporarily.

@yeya24
Copy link
Contributor

yeya24 commented Oct 23, 2025

Can u please update the PR title and description?

@SungJin1212 SungJin1212 changed the title Parquet gateway poc Parquet Store gateway Oct 23, 2025
yeya24 and others added 11 commits October 24, 2025 10:54
Signed-off-by: SungJin1212 <[email protected]>
Signed-off-by: SungJin1212 <[email protected]>
Signed-off-by: SungJin1212 <[email protected]>
Signed-off-by: SungJin1212 <[email protected]>
Signed-off-by: SungJin1212 <[email protected]>
Signed-off-by: SungJin1212 <[email protected]>
Signed-off-by: SungJin1212 <[email protected]>
Signed-off-by: SungJin1212 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support Parquet Store Gateway

2 participants