**Describe the bug** ``` --- FAIL: TestQuerierWithBlocksStorageRunningInMicroservicesMode/blocks_default_sharding,_ingester_gRPC_streaming_disabled,_inmemory_index_cache,thanosEngine=true (35.49s) ``` **To Reproduce** It happens randomly, for example https://github.com/cortexproject/cortex/pull/5305