Support UTF-8 name

[SungJin1212]

Add a -name-validation-scheme flag for supporting UTF-8 on Cortex
We offer two encoding ways, legacy or utf8.
The utf8 mode enables UTF-8 encoding for validating label names and metric names.

• Distributor: Validate the series according to UTF-8 validation logic.
• Ruler: Accept rule groups that contain UTF-8 labels
• Alertmanager: The API allows UTF-8 labels.

Which issue(s) this PR fixes:
Fixes #

Checklist

• Tests updated
• Documentation added
• CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

[yeya24]

We should try to cover label names and label values API as well. But we can do it in next PR

[SungJin1212]

Should we remove https://github.com/cortexproject/cortex/blob/master/pkg/cortex/configinit/init.go as we start accepting UTF-8?

I've noticed the labelValues api with UTF-8 label name fails since we currently set the legacy scheme as default.

[yeya24]

We should. I think we might be missing some test cases so we didn't find this in previous PR. Any components that do label/metric validation needs to accept the validation scheme otherwise you modify the global validation scheme variable

[SungJin1212]

I removed it and added label names/values e2e test case.

[yeya24]

I think we are missing more test cases like alertmanager and even ruler configurations with UTF 8 labels.

[SungJin1212]

I added commits, testing the UTF-8 labels for Ruler and Alertmanager in the e2e test

[yeya24]

Same for this test case.

[yeya24]

How did this test even succeed. We don't even add config to enable UTF 8 labels. Does it mean that AM support UTF 8 labels by default? but I think this is wrong and a behavior change after we remove /pkg/cortex/configinit/init.go. The UTF 8 validation should be properly controlled by the new runtime config.

[yeya24]

I think Alertmanager is able to control which validation mode to use via InitFromFlags. https://github.com/prometheus/alertmanager/blob/main/matcher/compat/parse.go#L59C6-L59C19.

However, this is not per tenant and global. That's why I was asking if per tenant metric name validation scheme really works or not. Seems we need some code changes to make that configuration not global in alertmanager.

Also you may add more utf-8 configs to route's matchers? Adding it to group_by alone Idk if it is sufficient.

[SungJin1212]

The Alertmanager Cortex uses this code, so the test succeeds since the default value of the scheme is UTF-8.

// isValidClassicLabelName returns true if the string is a valid classic label name.
func isValidClassicLabelName(_ *slog.Logger) func(model.LabelName) bool {
	return func(name model.LabelName) bool {
		return name.IsValid()
	}
}

The upstream Alertmanager code is

func isValidClassicLabelName(_ *slog.Logger) func(model.LabelName) bool {
	return func(name model.LabelName) bool {
		return model.LegacyValidation.IsValidLabelName(string(name))
	}
}

So, we need to expose another UTF-8 config for the Alertmanager.

That's why I was asking if per per-tenant metric name validation scheme really works or not.

It seems we should apply a per-tenant scheme for external label validation in the Ruler and Distributor, but the configuration for the Alertmanager should be applied globally. WDYT?

[yeya24]

Thanks!