Policy Endpoints #57

lennertdr · 2025-07-08T07:07:03Z

Draft PR

To keep up with progress

Documentation is found within docs/policy-management.md

Added a script script:seed to seed and delete a hard coded set of policies for a variable webID, and a script script:demo-uma-policy as used in the demo:

Uma-Loama-Extended.mp4

Some APIs might need non-JSON/form input, this allows this possibility. In the future we probably want a more generic solution, but for now this at least allows the possibility.

Due to the RoutedHttpRequestHandler catching unsupported requests with a default handler, it was impossible to combine multiple with a WaterfallHandler. This is now solved by moving most of the behaviour to the canHandle, and having a static throw handler for the 404.

woutslabbinck

Nice work so far.
Some small changes would help.

After the changes, we can start with DELETE, and then with update (sparql update for patch and complete policy rewriting with PUT)

packages/uma/src/util/routeSpecific/policies/CreatePolicies.ts

packages/uma/src/util/routeSpecific/policies/GetPolicies.ts

scripts/test-uma-ODRL-policy.ts

…leted)

This prevents Components.js build errors

When looking into contracts in the future, this should be handled in a more robust way.

See the comment in AuxiliaryModesExtractor.ts for more details. This allows policies on resources to also apply to their auxiliary resources, such as .meta resources.

woutslabbinck

Some small remarks on the docs

docs/policy-management.md

…default.json

woutslabbinck

I have a small amount of nit picks, but in general it looks really good!

If these are resolved, I think this PR can be merged.

The checkboxes and comments from the docs that are still unresolved by then should become issues then

woutslabbinck · 2025-07-14T16:05:39Z

docs/policy-management.md

+The PUT works as a combination of DELETE and POST. It requires a body with the same content type as the [POST request](#creating-policies). This body will be interpreted as the requested policy with some rules.
+
+The PUT process:
+1. Find information about the policy. If it does not exist, return with a **status code 400** to indicate that you cannot rewrite a nonexistent policy.


When I read this now, I am not 100% if this is the behaviour we want.

Either we allow PUT to create a resource if it does not exist yet (following the POST method).
Or we use a 404, as we performed a GET request and did not find any resource.

By typing this out, I think creating the resource (given that the identifier in the policy and the PUT policy-id match).

I would also argue similar behaviour with PATCH if the resource does not exist.

Perhaps @joachimvh has another opinion?

It's pretty much up to us to determine what we want, so I guess @termontwouter might want to weigh in here as he is writing the new spec. Solid allows creating new resources with PUT, the UMA registration process does not, and only allows you to PUT to existing registered resources (where I agree a 404 would make more sense).

UMA indeed does not allow registering new resources with PUT (since you cannot know the asset id in advance); but this is not an UMA endpoint, so it could make sense to allow PUT to create a completely new policy. Ultimately, this does not matter that much, since it is a "proprietary" endpoint of the AS, which we don't spec; so feel free to implement whatever is easiest.

woutslabbinck · 2025-07-14T16:07:19Z

docs/policy-management.md

+2. The content type of the body gets validated. The content-type must be set to `application/sparql-query`. Any other type will result in a response with **status code 400**.
+3. We use the policy information to create a store which only contains groups **(1)**, **(3)** and **(4)** as explained [above](#get-one-policy). This will serve as an isolated store, on which we can execute the update query. This implementation has its advantages:
+    - We do not need to validate the sparql query, since we execute it on an isolated store.
+    - Performing DELETE queries on rules out of your scope will simply not work, since they are not part of the isolated store.


Does this mean that nothing then will happen?
I understand it as such and think that is valid behaviour.

woutslabbinck · 2025-07-14T16:08:24Z

docs/policy-management.md

+Some operations require the client to specify a policy ID in the URL. Since policy ID's might contain reserved characters (e.g. `/`, `:`, ...), we have chosen to encode them with the builtin [`encodeURIComponent()` function](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/encodeURIComponent). Using this method, reserved characters will be converted to their respective UTF-8 encodings.
+
+## Testing
+The current implementation is tested only by the script in `scripts\test-uma-ODRL-policy.ts`. This script tests every implemented endpoint with a designated flow. Since the script initiates with an empty storage, and there is no endpoint or other way to seed it, the first requests must test the POST endpoint. These tests are designed to ensure that the storage is filled. After the POST tests, the access endpoints can be tested. Every endpoint gets tested in this script, which makes sure that the added data is removed. The current testing will be replaced with proper unit tests in the near future.


and integrations tests of course :)

woutslabbinck · 2025-07-14T16:12:43Z

packages/ucp/src/storage/UCRulesStorage.ts

@@ -20,4 +20,7 @@ export interface UCRulesStorage {
     * @returns 
     */
    deleteRule: (identifier: string) => Promise<void>;
+
+    // Experimental endpoint
+    deleteRuleFromPolicy: (ruleID: string, PolicyID: string) => Promise<void>;


From the implementation, it seems like it deletes the usage control rule and its reference. Could you update the documentation as such?

Also remove the fact that it is experimental. At some point it will have unit tests and thus will not be experimental.

Could you also update the updateRule definition explaining that it does not delete the reference from policies?

woutslabbinck · 2025-07-14T16:14:11Z

packages/uma/config/rules/odrl/addedRules.ttl

is this an artefact of using the earliest POST method in combination with the FileStorage solution?
If so, I think this can be removed

woutslabbinck · 2025-07-14T16:18:36Z

packages/uma/src/util/routeSpecific/policies/EditPolicies.ts

+
+    // 2. Retrieve the Policy Body
+    const contentType = request.headers['content-type'];
+    if (!/(?:application\/sparql-update)$/i.test(contentType)) {


Perhaps it might be useful to use this utility for parsing the content type?
https://github.com/CommunitySolidServer/CommunitySolidServer/blob/main/src/util/HeaderUtil.ts#L381

The CSS is a dependency anyway

woutslabbinck · 2025-07-14T16:21:56Z

packages/uma/src/util/routeSpecific/policies/EditPolicies.ts

+    // Sanitization
+    sanitizeRule(policyStore, clientId);
+
+    // 3.1 Check that the other rules are unchanged
+    const newState = getPolicyInfo(policyId, policyStore, clientId);
+    if (newState.otherRules.length !== 0 || newState.otherPolicyRules.length !== 0)
+        throw new BadRequestHttpError("Update not allowed: attempted to modify rules not owned by client");
+
+    // 3.2 Check that only Policy/Rule changing quads are introduced and removed
+    // The only modifications we allow are policy definitions, policy rules that define owned rules and owned rules themselves
+    const newQuads = policyStore.getQuads(null, null, null, null);
+    if (newQuads.length - newState.ownedRules.length - newState.ownedPolicyRules.length - newState.policyDefinitions.length
+        !== initialQuads.length - ownedPolicyRules.length - ownedRules.length - policyDefinitions.length)
+        throw new BadRequestHttpError("Update not allowed: this query introduces quads that have nothing to do with the policy/rules you own");


Are these sanitisation checks the same as the ones of rewritepolicies?

If so, maybe it is better to create a util function out of it that is re-used both here and in rewrite

termontwouter

General note: the "config" in packages/uma/config/rules/odrl/ is not really config, but more scaffolding for demo's that was misplaced there out of laziness (probably my own). We should move this out of the config folder.

termontwouter · 2025-07-21T09:47:58Z

packages/uma/config/default.json

+              "comment": "Handles all remaining output. These handlers have to handle input/output parsing themselves. TODO: At some point we want more generic conneg.",
+              "@id": "urn:uma:default:RawRoutedHttpRequestHandler",
+              "@type": "RoutedHttpRequestHandler",
+              "routes": [
+                { "@id": "urn:uma:default:ContractRoute" },
+                { "@id": "urn:uma:default:LogRoute" },
+                { "@id": "urn:uma:default:VCRoute" },
+                { "@id": "urn:uma:default:PolicyRoute" },
+                { "@id": "urn:uma:default:OnePolicyRoute"}
+              ]


Not sure I fully understand these handlers...

What are the ContractRoute, LogRoute, and VCRoute?

Why are these bundled as "raw" handlers? Presumably, at least two of them talk RDF, so it might be better to put an RDF (de)serialiser around it, no?

I'm guessing something went wrong with the rebase resulting from #56

Which was a quick fix I did to make sure to remove a blocker for the work being done here, as the current API handler I had in there only supported JSON/form-encoded.

termontwouter · 2025-07-21T09:56:59Z

packages/uma/src/util/routeSpecific/policies/PolicyOptions.ts

+import { HttpHandlerRequest } from "../../http/models/HttpHandler";
+import { checkBaseURL, retrieveID } from "./PolicyUtil";
+
+export function policyOptions(request: HttpHandlerRequest, baseUrl: string) {


The CSS already contains a generic CorsHandler, so might as well use that one

termontwouter · 2025-07-21T10:54:18Z

docs/policy-management.md

+#### URI encoding decision
+Some operations require the client to specify a policy ID in the URL. Since policy ID's might contain reserved characters (e.g. `/`, `:`, ...), we have chosen to encode them with the builtin [`encodeURIComponent()` function](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/encodeURIComponent). Using this method, reserved characters will be converted to their respective UTF-8 encodings.


Might be a nit, but preferably policy IDs are URIs themselves, namely the URI to which these requests are sent.

namely the URI to which these requests are sent.

What do you mean with that if I may ask?

I mean that a policy at https://example.as/policies/abc0123 already has an identifier, which is exactly that URI (https://example.as/policies/abc0123). So while it makes sense to say that the suffix (abc0123) should be a safe URI component string (or encoded as one), the URI itself is the actual identifier i.m.o.

That could work as well. Though, maybe not ideal when you want to export your policies from one server to another.

Another option if we do not want to decide is to implement both, make it configurable and then experiment with what works best.

On a different note, having the policy be an URL would also make creating an API for rule editing very easy. (though also future work and open for discussion right now)

termontwouter · 2025-07-21T10:56:18Z

docs/policy-management.md

+#### Authorization/Authentication decisions
+The current implementation has insufficient authentication restrictions. Currently, the only requirement is that the 'Authorization' header is to be set to the webID of the "logged on" client. Proper procedures to authenticate this client are still to be implemented.


This should be resolved sooner rather than later. It should be fairly simple to add a classic OAuth authorization code flow to the UMA server, which we also need anyway to link up the AS to the RS. That might be something for @joachimvh though ...

termontwouter · 2025-07-21T11:02:29Z

docs/policy-management.md

+The algorithm to GET a single policy will use a procedure to separate the policy into different parts:
+1. Policy Quads
+    - Some quads in the policy define rules. Their general form looks like `<policyID> <relation> <ruleID> .`. These quads are split into rules owned by the client **(1)**, and those owned by another client **(2)**.
+    - Other quads define other parts about the policy. These quads are set into a group of `policy definitions` **(3)**.
+2. Rule Quads
+    - Owned rule quads are those where the client as an assigner **(4)**. 
+    - Other rule quads are owned by other clients **(5)**.


It is not entirely clear to me why this split is necessary, and which scenarios would have policies/rules by different assigners mixed up with eachother 🤔

termontwouter · 2025-07-21T11:12:13Z

All in all minor remarks. Seems like good work for such a short time!

lennertdr and others added 16 commits July 4, 2025 11:45

Basic POST /policies created

ecd6839

Practical addRule implementation to test the POST endpoint

846fcac

added Get One Policy endpoint, need a way to fix the ID (encoding?)

1c377a7

getOnePolicy works if a good encoding of ID's is implemented

e60eb75

follow implementation from Main

6f0916f

test content type

a183107

feat: Allow APIs with raw input

89eb900

Some APIs might need non-JSON/form input, this allows this possibility. In the future we probably want a more generic solution, but for now this at least allows the possibility.

Format checks removed, they are already in N3 Parser

f2de8ad

Merge remote-tracking branch 'upstream/main' into policyEndpoints

56727e8

Change to Memory structure for better testing on this branch

8295dbf

Merge remote-tracking branch 'upstream/main' into policyEndpoints

3d110ce

POST with proper content types

3ee80a6

Memory based tests

90ff950

Merge remote-tracking branch 'origin/policyEndpoints'

64ae909

import fix

b5dab70

woutslabbinck requested changes Jul 8, 2025

View reviewed changes

lennertdr and others added 13 commits July 8, 2025 11:13

GET /uma/policies/<id> first finished implementation

2dad0f7

Add extra checks to POST

2f43a30

More generic url handling and POST with sanitize function (to be comp…

73d32d8

…leted)

excessive documentation

9e1102b

test doc and very primitive way to detect fails

e11cc43

DELETE endpoint implemented, still needs tests

9f78328

fix: Export OperationLogger

944b52e

This prevents Components.js build errors

fix: Prevent contract creation errors from stopping the request

4c05231

When looking into contracts in the future, this should be handled in a more robust way.

fix: Request subject resource permissions for auxiliaries

1b08acb

See the comment in AuxiliaryModesExtractor.ts for more details. This allows policies on resources to also apply to their auxiliary resources, such as .meta resources.

feat: Allow relative URIs when using DirectoryUCRulesStorage

0f0e025

refactor: Remove unused seeding fields

0d91822

feat: Make containerURL of ContainerUCRulesStorage configurable

1e3c9e1

feat: Make App the root configured component

3e7d7b6

lennertdr added 2 commits July 10, 2025 16:28

extra PUT checks, extra documentation

d16071b

doc layout fix

a67b3d4

woutslabbinck requested changes Jul 10, 2025

View reviewed changes

lennertdr added 8 commits July 11, 2025 13:30

detailed documentation

615a9a9

Stronger POST checks

d66d1f4

DELETE idea, need to adjust tests

bb08a83

fixed small bug

29b8eaa

Merge remote-tracking branch 'upstream/main', kept memory storage in …

4c1cc6f

…default.json

doc update

9c189ec

doc update

7cbbedc

typos

0ba8846

woutslabbinck requested changes Jul 14, 2025

View reviewed changes

lennertdr added 3 commits July 16, 2025 09:49

temporary header against CORS, not the right solution

36934e8

script to seed for specific id

ed76ee6

options for other requests

155169e

termontwouter reviewed Jul 21, 2025

View reviewed changes

some requested changes

6eaf6f2

termontwouter mentioned this pull request Jul 28, 2025

Support asset collection policies #50

Merged

lennertdr added 8 commits July 28, 2025 14:34

undo wrong import

b8de385

quick workaround

10c7247

demo script

dc950da

script shortcut

80bf03b

added test again

e6044f7

Removed logs and finetuned docs

eb605dc

docfix

46c177f

TODO's

9a5f4e8

woutslabbinck approved these changes Aug 4, 2025

View reviewed changes

woutslabbinck marked this pull request as ready for review August 4, 2025 10:08

woutslabbinck merged commit abac429 into SolidLabResearch:feat/policy-endpoint Aug 4, 2025

		#### URI encoding decision
		Some operations require the client to specify a policy ID in the URL. Since policy ID's might contain reserved characters (e.g. `/`, `:`, ...), we have chosen to encode them with the builtin [`encodeURIComponent()` function](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/encodeURIComponent). Using this method, reserved characters will be converted to their respective UTF-8 encodings.

		#### Authorization/Authentication decisions
		The current implementation has insufficient authentication restrictions. Currently, the only requirement is that the 'Authorization' header is to be set to the webID of the "logged on" client. Proper procedures to authenticate this client are still to be implemented.

Policy Endpoints #57

Policy Endpoints #57

Uh oh!

Conversation

lennertdr commented Jul 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Draft PR

Uh oh!

woutslabbinck left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

woutslabbinck left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

woutslabbinck left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

termontwouter left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

woutslabbinck Jul 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

termontwouter commented Jul 21, 2025

Uh oh!

Uh oh!

lennertdr commented Jul 8, 2025 •

edited

Loading

woutslabbinck Jul 28, 2025 •

edited

Loading