Proposal: Confidence Field

[CTM1]

When providing Sigma rules to user, some providers seem to use "Level" interchangeably with what would be "Confidence".

The doc says The level field describes the criticality of a triggered rule. and critical should never trigger a false positive and be of high relevance. The presence/absence of false positives and the relevance/criticality of an alert should be two different metrics to me.

Take for instance this Process Hollowing rule.

https://github.com/SigmaHQ/sigma/blob/master/rules/windows/process_tampering/proc_tampering_susp_process_hollowing.yml

This for me would be a high/critical level with medium confidence, but its level is now set to medium as many products (AV, Games etc.) modify their image in memory. But in itself, if this event is not a false-positive, it would be a high/critical alert.

Adding this field would allow for better modular configuration of rule sets.

I would volunteer to make a PR for this field myself, if it's something that seems relevant to the community.

EDIT: Confidence should be an optional field

[Neo23x0]

Could you please explain how a rule with "level: critical + confidence: medium" would be treated differently (before a match) than a rule with "level: medium" today?

[Neo23x0]

From my perspective, adding a "confidence" field to the Sigma rule standard raises one major and a few minor concerns.

Currently, when setting the level of a rule, we primarily evaluate its relevance by asking, "How significant is a match with this rule?" While the confidence we have about potential false positives does influence the level, it plays a relatively minor role. This is because, as detection engineers, we often lack the necessary visibility and telemetry to accurately assess confidence levels.

The example you mentioned is one of the rare cases where confidence can be reasonably estimated, as you’ve shown. However, only MSSPs or SIEM service providers have the advantage of working with a broad and diverse dataset from multiple sources and tenants. This gives them the ability to set confidence levels with the certainty I would expect for such a field. If we're forced to guess 90% of the time, and those guesses frequently miss the mark, I'd rather omit a "confidence" field altogether than risk adding misleading information.

[CTM1]

I failed to communicate this, but for me confidence would be an optional field, as level already is, so its usage wouldn't be forced onto people who don't have the capabilities to assess it.

For me adding this field would simply allow MSSPs/SIEM/whoever to stay compliant with this widely used standard if they so wish to add this granularity, not enforce a review of all current public SigmaHQ rules. I used SigmaHQ's rules as an example because those are public rules which could illustrate my point, but it's something contributors to SigmaHQ who don't have the resources to assess it won't add and it could be excluded from rules in which they wouldn't make sense. (But would allow contributors that do to add it with sufficient proof, I suppose).

Does this address your concern/raise others ?

[Neo23x0]

So far, in the SigmaHQ rule set, we've only added optional fields (such as level, falsepositives, and date) when at least 95% of the rules could be populated with them. We’ve avoided adding optional fields to only a small subset of rules to prevent user confusion and maintain consistency across the field selection for all rules.

We should bring in @nasbench to get his opinion on this.

[CTM1]

I understand the desire for consistency in the SigmaHQ ruleset. However, I think we need to consider Sigma more broadly as an open standard, not just in the context of the SigmaHQ ruleset.

While SigmaHQ maintains an excellent public ruleset, it's just one implementation of Sigma rules. Many organizations and vendors create their own private Sigma rulesets tailored to their specific needs and capabilities.

Adding an optional confidence field to the Sigma specification simply provides the capability for those who find it valuable. Organizations with the telemetry and expertise to accurately assess confidence levels could leverage this field in their own private rulesets or by enriching the public ones.

I see what you mean when you say "could be populated", but most optional fields, like "scope" aren't used in 95% of rules, so I fail to see the practical aspect of this in terms of consistency or user confusion.

Separate from its immediate application in any particular ruleset, I still believe it would be a positive addition to the specification. I do we believe we could benefit from second opinions from other maintainers 👍 .

[Neo23x0]

Many organizations and vendors create their own private Sigma rulesets tailored to their specific needs and capabilities.
Adding an optional confidence field to the Sigma specification simply provides the capability for those who find it valuable.

They can already do that.

Organizations with the telemetry and expertise to accurately assess confidence levels could leverage this field in their own private rulesets or by enriching the public ones.

I don't think many organisations can do that.
MSSPs, yes.
But even a single big corporation has standards, e.g. they use SAP instead of Peoplesoft as the ERP and cannot know it a certain rule triggers false positives caused by Peoplesoft applications.

but most optional fields, like "scope" aren't used in 95% of rules, so I fail to see the practical aspect of this in terms of consistency or user confusion.

I'm not a fan of the scope field. I'd like to remove it and any other field that isn't used in the majority of the the rules from the specification. It's a legacy issue and because others wanted to include it.