Implement filters - #156
Draft
azmeuk wants to merge 11 commits into
Draft
Conversation
azmeuk
force-pushed
the
17-filters
branch
6 times, most recently
from
September 6, 2026 14:02
7b365f8 to
990c63c
Compare
ScimFilter parses the filter grammar of RFC7644 §3.4.2.2, on a new lark dependency, and evaluates it against Python objects. Parsing, resolution against a model and evaluation are separate stages, so FilterVisitor can transpile a tree into a backend query rather than evaluate it. Binding a filter checks it as it is created, against one resource type or against a union of them: §3.4.2.1 has an attribute only some of them declare evaluate to false on the others, so only an attribute none declares is refused. The grammar and the resolution sit above the package, attrPath being one ABNF rule that RFC7644 uses in both FILTER and PATH (§3.5.2).
Path validates against the PATH rule of RFC7644 §3.5.2, as corrected by errata 7122, instead of a permissive character pattern, so a.b.c is now rejected. get, set and delete honour the selection, letting PatchOp.patch address individual entries of a multi-valued attribute: a selection matching nothing raises noTarget for replace, as §3.5.2.3 mandates, and is a no-op for remove and add, which §3.5.2.2 and errata 8097 leave that way. The filter between the brackets is checked against the model before the resource is read, so an undeclared attribute is reported as invalidFilter whether or not the selected attribute holds values.
Path carried its own resolution next to the one the filter engine brought in, and the two disagreed: attr rendered the whole parsed node, so a value selection left every model-aware property None. Path.resolve is now the single resolution, and over the 153 paths iter_paths yields for User, User[EnterpriseUser] and Group, only x509Certificates.value changes, now reported unwrapped as bytes.
SearchRequest.filter is a ScimFilter instead of a str, so a malformed filter is rejected at validation time; it is a str subclass, so anything reading one still reads a string. Naming the resource types the endpoint serves, as in SearchRequest[User] or SearchRequest[User | Group], is what also refuses an attribute none of them declares, where an unparameterised request only has its syntax checked.
Parameterising a request names the resource types an endpoint serves, which makes an attribute none of them declares a client error rather than something to carry to the endpoint, so sort_by answers invalidPath at validation time the way filter answers invalidFilter. An unknown attributes entry stays ignored: dropping a projection degrades gracefully where dropping an order answers an arbitrary one the client cannot tell from what it asked for.
Adds a filters page covering the grammar, the operators, the deviations from the published ABNF and a worked transpiler built on FilterVisitor, and a tutorial section on filters and PATCH value selections. The framework guides now filter the collections they serve: a filter applies to the SCIM representation, so the store is mapped first, filtered, paginated last.
Replaces the in-memory storage layer of the integration guides with a database, where the filter becomes a WHERE clause next to the pagination, so a page costs a query over the matching rows rather than a walk over all of them. Twenty-three filters run both as a query and through match(), an oracle that caught SQLite's naive datetime, its case-folding LIKE, and a case test on str that skipped emails.value.
The examples already gather users and groups at the root, so the filter of a root query is checked against both models: an attribute only one of them declares evaluates to false on the other, as RFC7644 §3.4.2.1 requires, and one neither declares is refused outright.
The ORDER BY of the SQLAlchemy guide followed none of the three rules of RFC7644 §3.4.2.3: it kept the BINARY collation of SQLite where a non case-exact attribute ignores case, inverted the placement of the missing values in both directions, and refused a mapped sub-attribute such as meta.lastModified by looking it up under its own name. The primary key now closes the clause, so a page stays reproducible without a sortBy, and twelve orderings run both as a query and through sort_resources.
sort_resources read the primary flag off whatever get() returned, which holds entries for sortBy=emails but projected sub-attributes for sortBy=emails.value, so any path naming a sub-attribute raised an AttributeError. RFC7644 §3.4.2.3 picks the entry first, primary or else the first, and the sub-attribute is read from it, which turns sortBy=emails into the case where that sub-attribute is the value RFC7643 §2.4 reserves for it. A scalar entry is that value itself, and an extension left unset holds none.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #17