Adding Documentation for NEURLFilter (#83)

* Adding Documentation for NEURLFilter

---------

Co-authored-by: Akshay Wadia <awadia@apple.com>

Author: fboemer

README.md

@@ -11,7 +11,10 @@ callers and display identity information on the incoming call screen.
   <img alt="Overview of the Live Caller ID Lookup" src="https://github.com/apple/pir-service-example/raw/main/Sources/PIRService/PIRService.docc/Resources/overview@2x.png">
 </picture>
 
-This repository provides a functional server backend to test the Live Caller ID Lookup feature.
+[NEURLFilter](https://developer.apple.com/documentation/networkextension/neurlfiltermanager) is a new feature for iOS and MacOS that allows the system
+to communicate with a third party service to privately check if a requested URL should be allowed or not. This allows your app to implement URL filtering in a privacy preserving manner.
+
+This repository provides a functional server backend to test the Live Caller ID Lookup and NEURLFilter features.
 
 > [!WARNING]
 > While functional, this is just an example service and should not be run in production.

Sources/PIRService/PIRService.docc/Authentication.md

@@ -24,7 +24,7 @@ that different level of details can be provided based on the user tier.
 7. The system does not know which user tier is associated with which public key, so it sends the User Token to the
    authentication server.
 8. Authentication server verifies the User Token and returns the public key that is associated with the User Tier. The
-   system verifies that the returned public key is present in the Token Issuer Directory and it is valid based in the
+   system verifies that the returned public key is present in the Token Issuer Directory and it is valid based on the
    current time.
 9. The system constructs a Privacy Pass token request using the specific public key. The token request is sent along
    with the User Token to the authentication server.
@@ -33,7 +33,7 @@ that different level of details can be provided based on the user tier.
     * the token request uses the public key that is associated with the right user tier,
 
     and issues the token response that the system uses to get a Privacy Pass token.
-11. When a PIR request is made, the system attached an unused Privacy Pass token to the request. The PIR node can use
+11. When a PIR request is made, the system attaches an unused Privacy Pass token to the request. The PIR node can use
     the public key to verify that the token is valid and that assures that the request is authorized.
 12. Response to the PIR request is returned to the system.
 

Sources/PIRService/PIRService.docc/CommonErrors.md

@@ -4,8 +4,8 @@ Learn about the common errors that the device logs and why they happen.
 
 ## Overview
 
-While using the example service or when developing your own service, you will most probably encounter one of the errors
-that will be listed below. This page gives more details about the errors, like why they happen, what they mean, and how
+While using the example service or when developing your own service, you may encounter one of the common errors
+listed below. This page gives more details about these errors, like why they happen, what they mean, and how
 to work around them.
 
 ### How to look for errors
@@ -20,7 +20,7 @@ You can use the Console app to see on-device logs. Search for error messages fro
 
 ```
 "The request timed out." UserInfo={NSLocalizedDescription=The request timed out.,
-NSErrorFailingURLKey=http://MacBook-Pro.local:8080/.well-known/private-token-issuer-director
+NSErrorFailingURLKey=http://MacBook-Pro.local:8080/.well-known/private-token-issuer-directory
 ```
 
 **Reason:**
@@ -69,11 +69,16 @@ Device issued a request to the configuration endpoint (`/config`), but did not f
 was looking for.
 
 **Workaround:**
-Double-check that the service has usecases configured the way the device expects them. Usecase names should be:
-* `<bundleIdentifier>.block`
-* `<bundleIdentifier>.identity`
+Double-check that the service has usecases configured the way the device expects them. Usecase names for Live Caller ID Lookup should be:
+* `<app_extension_bundle_identifier>.block`
+* `<app_extension_bundle_identifier>.identity`
 
-where `<bundleIdentifier>` is replaced with bundle identifier of your Live Caller ID Lookup extension.
+In the above, replace `<app_extension_bundle_identifier>` with your app extension's bundle identifier.
+
+And usecase name for NEURLFilter should be:
+* `<app_bundle_identifier>.url.filtering`
+
+In the above, replace `<app_bundle_identifier>` with your application's bundle identifier.
 
 #### No token key found with key id
 
@@ -102,8 +107,9 @@ you restarted the service, which means that new keys were generated, and the exa
 therefore making the cached tokens invalid.
 
 **Workaround:**
-Once the device runs out of cached tokens, it will fetch new ones. One way to speed up the process is to call
-[refreshPIRParameters(forExtensionWithIdentifier:)](https://developer.apple.com/documentation/sms_and_call_reporting/livecalleridlookupmanager/4418043-refreshpirparameters).
+Once the device runs out of cached tokens, it will fetch new ones. One way to speed up the process is to explicitly refresh the PIR parameters.
+* For example, for Live Caller ID Lookup, call [refreshPIRParameters(forExtensionWithIdentifier:)](https://developer.apple.com/documentation/sms_and_call_reporting/livecalleridlookupmanager/4418043-refreshpirparameters).
+* For NEURLFilter, see [NEURLFilter API documentation](https://developer.apple.com/documentation/networkextension/neurlfiltermanager).
 
 #### Evaluation key not found
 
@@ -123,7 +129,6 @@ A real service should persist the evaluation key, but the example service only s
 copy of the evaluation key is gone after you restart the example service.
 
 **Workaround:**
-The device will periodically refetch the configuration from the server and will notice the missing evaluation key. You
-can call
-[refreshPIRParameters(forExtensionWithIdentifier:)](https://developer.apple.com/documentation/sms_and_call_reporting/livecalleridlookupmanager/4418043-refreshpirparameters)
-to let system know that it should refetch the configuration immediately.
+The device will periodically refetch the configuration from the server and will notice the missing evaluation key. You can also force the system to refresh configuration by calling the appropriate refresh API.
+* For Live Caller ID Lookup, you can call [refreshPIRParameters(forExtensionWithIdentifier:)](https://developer.apple.com/documentation/sms_and_call_reporting/livecalleridlookupmanager/4418043-refreshpirparameters).
+* For NEURLFilter, refer to [NEURLFilter API documentation](https://developer.apple.com/documentation/networkextension/neurlfiltermanager).

Sources/PIRService/PIRService.docc/HTTPEndpoints.md

@@ -1,12 +1,10 @@
-# Setting up the HTTP endpoints for Live Caller ID Lookup
+# Setting up the HTTP endpoints
 
 Learn about the required endpoints that the system expects from your service.
 
 ## Overview
 
-Your client side app needs your
-[serviceURL](https://developer.apple.com/documentation/sms_and_call_reporting/livecalleridlookupextensioncontext/4365180-serviceurl)
-so the system can fetch information from your server. Communication between the system and server uses Protocol
+Communication between the system and server uses Protocol
 Buffer (Protobuf) messages over HTTP. For the Protobuf schema please see [Homomorphic Encryption
 Protobuf](https://github.com/apple/swift-homomorphic-encryption-protobuf).
 

Sources/PIRService/PIRService.docc/NEURLFilterDataFormat.md

@@ -0,0 +1,19 @@
+# Data format for NEURLFilter
+
+Understand the data format that Network Extension URL Filter expects.
+
+## Overview
+
+Network Extension on-device examines all URL requests, checking each URL against a Bloom filter built from your URL data set, containing URLs you want to block.
+If the Bloom filter match is negative, the URL is not in your data set and it will be allowed immediately. If the Bloom filter match is positive, the match result is not
+definitive due to potential false positives. In this case, an off-device URL query will be sent to your PIR server to match against your URL data set to get a final verdict.
+If your PIR server responds with a match, the system on-device will block the URL, and the URL request will fail.
+
+## URL Data Record
+
+The URL data record has the URL (excluding the scheme portion) as the key. For example, for URL `https://www.example.com`, the URL key should be
+`www.example.com`. If an entry is found matching the URL key, it already indicates a match. So the value for the record is simply set to the integer 1 serving as place holder.
+
+Value | Description
+----- | -----------
+1     | <Place holder>

Sources/PIRService/PIRService.docc/Onboarding.md

@@ -1,10 +1,10 @@
-# How to onboard Live Caller ID Lookup extension with Apple's Oblivious HTTP relay
+# How to onboard with an Oblivious HTTP relay hosted by Apple
 
-Understand the requirements for running a Live Caller ID Lookup service.
+Understand the requirements for running PIRService.
 
 ## Overview
 
-To hide client’s IP address all network request made by the system to the Live Caller ID Lookup server will go over
+To hide client’s IP address all network request made by the system to the server will go over
 [Oblivious HTTP](https://www.rfc-editor.org/rfc/rfc9458). Apple will provide the oblivious relay and therefore there is
 a required onboarding step to make sure that Apple’s oblivious HTTP relay has been configured to forward requests to
 your chosen oblivious HTTP gateway.
@@ -13,14 +13,14 @@ your chosen oblivious HTTP gateway.
 
 ### How to test without private relay
 
-The system does not use private relay, when the application is installed directly from Xcode. This allows the
+The system does not use private relay when the application is installed directly from Xcode. This allows the
 application & the service deployment to be tested before filling out the onboarding form and setting up Oblivious HTTP
 relay.
 
 
 ### Requirements
 
-Before filling out the form, there are a few requirements you have to satisfy to ensure smooth operations.
+Before filling out the onboarding form, there are a few requirements you have to satisfy to ensure smooth operations.
 
 #### OHTTP gateway
 
@@ -31,10 +31,10 @@ openssl s_client -alpn h2 -connect $(OHTTP_GATEWAY_FQDN):443 </dev/null
 In the output check if the SSL session was established or not.
 
 #### URLs
-Please provide the URLs as you would put them into the
-[LiveCallerIDLookupExtensionContext](https://developer.apple.com/documentation/identitylookup/livecalleridlookupextensioncontext).
+You will need to provide the URLs you would put into your feature configuration. For example, for Live Caller ID Lookup, you would provide the URLs configured in
+[LiveCallerIDLookupExtensionContext](https://developer.apple.com/documentation/identitylookup/livecalleridlookupextensioncontext). For NEURLFilter, you will provide the URLs specified in [NEURLFilter API documentation](https://developer.apple.com/documentation/networkextension/neurlfiltermanager).
 
-> Important: In addition, we strongly advice you to use subdomains instead of paths. Support for custom paths for the
+> Important: In addition, we strongly advise you to use subdomains instead of paths. Support for custom paths for the
 > service URL and token issuer URL will be deprecated in a future iOS version.
 
 Good example:
@@ -50,20 +50,20 @@ http://example.net:8080/lookup - No HTTPS, non standard port, path instead of su
 ```
 
 #### HTTP Bearer Token / UserToken
-The `userToken` field is a of type `Data` and the system sets the "Authorization" header like this:
+The `userToken` field is of type `String` and the system sets the "Authorization" header like this:
 ```swift
-request.setValue("Bearer \(userToken.base64EncodedString())", forHTTPHeaderField: "Authorization")
+request.setValue("Bearer \(userToken)", forHTTPHeaderField: "Authorization")
 ```
-This implies that when filling out the form, you must enter a valid base64 encoding.
-> Warning: For example JSON Web Tokens are not valid base64 encodings and are therefore unsupported.
 
 #### Checklist
 
-1. You must know the bundle identifier of your Live Caller ID Lookup extension.
+1. You must know the bundle identifier of your extension for Live Caller ID Lookup, or bundle identifier of your application for NEURLFilter.
 2. You need to provide expected request and response size and per continent traffic estimates that include:
     * peak requests per second
     * total requests per day
-3. You have added a test identity (+14085551212 with name “Johnny Appleseed”) to your dataset.
+3. You have added a test identity to your dataset.
+    * For Live Caller ID Lookup, it should be +14085551212 with name “Johnny Appleseed”.
+    * For NEURLFilter, it should be `www.apple.com/url-filter-test` with value as the integer `1`.
 4. You have set up an Oblivious HTTP gateway.
 5. You must provide your Oblivious HTTP Gateway configuration resource.
 6. You must provide your Oblivious HTTP Gateway resource -- a URL used to make oblivious HTTP requests to your service.
@@ -72,16 +72,22 @@ This implies that when filling out the form, you must enter a valid base64 encod
 9. You must provide an HTTP Bearer token, that allows us to validate that your deployment is set up correctly and we can
    successfully fetch the test identity.
 10. You must add a DNS TXT record to your service URL to prove your ownership, control, and intent to serve Live Caller
-    ID Lookups. More specifically you need to add the following record:
-    `apple-live-caller-id-lookup=<bundle_identifier>` where `<bundle_identifier>` is replaced with your extension's
-    bundle identifier. For example, if your Live Caller ID Lookup extension's bundle identifier is `net.example.lookup`
-    the DNS TXT record should be: `apple-live-caller-id-lookup=net.example.lookup`.
+    ID Lookups or NEURLFilter. More specifically you need to add one of the following records, depending on whether you are hosting
+    Live Caller ID Lookups, or NEURLFilter:
+    * For Live Caller ID Lookup, add `apple-live-caller-id-lookup=<app_extension_bundle_identifier>`
+    * For NEURLFilter, add `apple-url-filter=<app_bundle_identifier>`
+
+    Here, `<app_extension_bundle_identifier>` is replaced with your extension's bundle identifier for Live Caller ID Lookup; and `<app_bundle_identifier>` is replaced with your application bundle identifier for NEURLFilter. For example, if your Live Caller ID Lookup
+    extension's bundle identifier is `net.example.lookup` the DNS TXT record should be: `apple-live-caller-id-lookup=net.example.lookup`.
 11. You must ensure that your deployment (including Oblivious HTTP gateway & PIR service) is running so that we can
     perform the validation test.
 
-### Onboarding form
+### Onboarding forms
 
 The onboarding form should be filled out when you have a working service, but before you start distributing your
-application with the Live Caller ID Lookup extension.
+application.
 
-> Important: [Link to the onboarding form.](https://developer.apple.com/contact/request/live-caller-id-lookup/)
+> Important:
+> [Link to the onboarding form for Live Caller ID Lookup.](https://developer.apple.com/contact/request/live-caller-id-lookup/)
+>
+> [Link to onboarding form for NEURLFilter.](https://developer.apple.com/contact/request/network-extension-url-filter)

Sources/PIRService/PIRService.docc/PIRService.md

@@ -1,18 +1,25 @@
 # ``PIRService``
 
-Example service for Live Caller ID Lookup
+Example service for Live Caller ID Lookup and NEURLFilter
 
 ## Overview
 
+This service is a non-scalable example that can be used to test both Live Caller ID Lookup and NEURLFilter features. Please see
+<doc:TestingInstructionsLiveCallerIdLookup> and <doc:TestingInstructionsNEURLFilter> to see instructions on how to run the example service and connect your phone to the service.
+
+### Live Caller ID Lookup
+
 [Live Caller ID
 Lookup](https://developer.apple.com/documentation/sms_and_call_reporting/getting_up-to-date_calling_and_blocking_information_for_your_app)
-is a new feature that allows the system to communicate with a third party service to privately retrieve information
+is a new feature in iOS 18.0 that allows the system to communicate with a third party service to privately retrieve information
 about a phone number for an incoming call. This allows the system to automatically block known spam callers and display
 identity information on the incoming call screen.
 
 ![Live Caller ID Lookup data flow diagram](overview.png)
 
-This service is an non-scalable example that can be used to test the Live Caller ID Lookup feature. Please see
-<doc:TestingInstructions> to see intructions on how to run the example service and connect your phone to the service.
+### NEURLFilter
+
+[NEURLFilter](https://developer.apple.com/documentation/networkextension/neurlfiltermanager) is a new feature for iOS and MacOS that allows the system
+to communicate with a third party service to privately check if a requested URL should be allowed or not. This allows your app to implement URL filtering in a privacy preserving manner.
 
 ## Topics