Add withStructuredMatching for linear-time array-consistent matching#245
Merged
Add withStructuredMatching for linear-time array-consistent matching#245
Conversation
Collaborator
|
Is the long-term plan to continue exposing this as an option or make it the default behavior? |
f22b78a to
88f5141
Compare
Collaborator
Author
As an option for now so users dont have unintended behavioural changes and also are able to test in shadow mode. Will change the default flag to true once we are confident enough. |
…ching Add a configuration flag withStructuredMatching(true) on Machine.Builder that enables StructuredFinder, a linear-time matching algorithm for rulesForJSONEvent(). The default behavior is UNCHANGED — rulesForJSONEvent() uses the original ACFinder with no modifications. Only when withStructuredMatching(true) is set does it use the new StructuredFinder. StructuredFinder indexes the event by field path and walks the compiled state machine trie with direct HashMap lookups instead of scanning all remaining fields per step. It also exits early once all rules are matched. No changes to ACFinder.java or Event.java. The original matching path is identical to origin/main, enabling safe shadow-mode validation. Validated against 748 in-repo tests + 2799 external correctness cases. Tested with 28 perf scenarios including 14 attack payloads up to 1.2MB.
88f5141 to
7677eaf
Compare
Collaborator
|
Haha, I remember when they built that, at the time we were introducing Ruler 2.0, and I was pestering Rishi or Shawn every day "Did you find anything?" After what seemed a long time they found a couple of things, all of them were number values matched numerically instead of literally. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add a configuration flag
withStructuredMatching(true)onMachine.Builderthat enables linear-time array-consistent matching via
rulesForJSONEvent().The default behavior is unchanged —
rulesForJSONEvent()uses theoriginal ACFinder with zero modifications. This enables safe shadow-mode
validation before switching over.
Problem
ACFinder exhibits O(N^2) step queue growth when events contain large arrays
at paths matching multi-field rules. Each array element match fans out to
all remaining fields via moveFrom(), producing N*(N+1)/2 steps.
Solution
StructuredFinder indexes the event by field path (HashMap), then walks the
compiled state machine trie with direct lookups instead of scanning all
fields. Early-exit stops the walk once all rules are matched.
Machine.builder().withStructuredMatching(true).build()Usage
Files Changed
Performance
Tested with payloads up to 1.2MB. All times in ms, "OOM" = OutOfMemoryError:
Correctness