-
Notifications
You must be signed in to change notification settings - Fork 12
API
Manage has two API's:
- Client API for the Manage JavaScript GUI application which is secured using federative authentication (using shibboleth)
- Internal Metadata READ / WRITE API for other applications secured by basic authentication and scopes (using ansible)
- Internal Stats READ API for the statistics application secured by basic authentication (using ansible)
The first API is self-explaining and only interesting for Manage developers.
The read API is very simple. It has two endpoints: one for searching and one for retrieval of entire documents by primary key.
You can specify which fields you want to fetch with the REQUESTED_ATTRIBUTES key and optionally which fields to search. Example:
curl -H 'Content-Type: application/json' -u pdp:secret -X POST -d \
'{"metaDataFields.AssertionConsumerService:0:Location":".*","metaDataFields.coin:policy_enforcement_decision_required":"1","REQUESTED_ATTRIBUTES":["metaDataFields.AssertionConsumerService:0:Location","metaDataFields.coin:policy_enforcement_decision_required"]}' \
'http://localhost:8080/manage/api/internal/search/saml20_sp'
Or against a deployed application on a test environment:
curl -H 'Content-Type: application/json' -u pdp:secret -X POST -d '{"metaDataFields.AssertionConsumerService:0:Location":".*","REQUESTED_ATTRIBUTES":["entityid"]}' 'https://manage.test2.surfconext.nl/manage/api/internal/search/saml20_sp'
The query will default AND the different inputs. To query with the logical OR operator you need to specify the optional LOGICAL_OPERATOR_IS_AND post parameter to false.
curl -H 'Content-Type: application/json' -u pdp:secret -X POST -d '{"metaDataFields.contacts:0:contactType":"technical","metaDataFields.contacts:1:contactType":"technical","metaDataFields.contacts:2:contactType":"technical","REQUESTED_ATTRIBUTES":["metaDataFields.contacts:0:emailAddress", "metaDataFields.contacts:0:contactType","metaDataFields.contacts:1:emailAddress", "metaDataFields.contacts:1:contactType","metaDataFields.contacts:2:emailAddress", "metaDataFields.contacts:2:contactType"],"LOGICAL_OPERATOR_IS_AND": false }' 'http://localhost:8080/manage/api/internal/search/saml20_sp' | python -m json.tool
Wildcards like .*surf.* are translated to a regular expression search. Specify booleans with 0 or 1 and leave the value empty for a does not exists query. The result will always - depending on the type of Metadata specified as a path variable - return some defaults fields. For SP / IdP Metadata the default fields are:
- entityid
- state
- metaDataFields.name:en
- metaDataFields.name:nl
The following example will return the default fields and the ARP for the SP with the specified entityid:
curl -H 'Content-Type: application/json' -u pdp:secret -X POST -d \
'{"entityid":"https://dmsonline.uvt.nl/nl/home","REQUESTED_ATTRIBUTES":["arp"]}' \
'http://localhost:8080/manage/api/internal/search/saml20_sp'
And will return the following JSON:
[{
"_id": "362c928f-e80c-4318-a04e-c7dc072c131c",
"data": {
"entityid": "https://dmsonline.uvt.nl/nl/home",
"state": "testaccepted",
"arp": {
"attributes": {
"urn:mace:dir:attribute-def:displayName": [{
"source": "idp",
"value": "*"
}],
"urn:mace:dir:attribute-def:eduPersonAffiliation": [{
"source": "idp",
"value": "*"
}],
"urn:mace:dir:attribute-def:cn": [{
"source": "idp",
"value": "*"
}],
"urn:mace:dir:attribute-def:eduPersonPrincipalName": [{
"source": "idp",
"value": "*"
}],
"urn:mace:dir:attribute-def:mail": [{
"source": "idp",
"value": "*"
}],
"urn:mace:terena.org:attribute-def:schacHomeOrganization": [{
"source": "idp",
"value": "*"
}]
},
"enabled": true
},
"metaDataFields": {
"name:en": "DMS | Delcomkje",
"name:nl": "DMS | Delcomkje"
}
}
}]
If you omit the REQUESTED_ATTRIBUTES and add the parameter ALL_ATTRIBUTES with the value true all the fields will be returned. Example:
curl -H 'Content-Type: application/json' -u pdp:secret -X POST -d '{"ALL_ATTRIBUTES":true}' 'http://localhost:8080/manage/api/internal/search/saml20_sp' | python -m json.tool
Will return (result shortened for readability):
[
{
"_id": "0eda9504-0be6-4966-ad01-37b2daae2439",
"data": {
"active": true,
"allowedEntities": [],
"allowedall": true,
"arp": {
"attributes": {
"urn:mace:dir:attribute-def:displayName": [
{
"source": "idp",
"value": "*"
}
],
"urn:mace:dir:attribute-def:eduPersonPrincipalName": [
{
"source": "idp",
"value": "*"
}
]
},
"enabled": true
},
"created": "2016-01-28T15:22:02+01:00",
"eid": 54,
"entityid": "http://orcid.test.surfconext.nl/authentication/metadata",
"id": 348,
"ip": "145.100.191.122",
"manipulation": null,
"metaDataFields": {
"AssertionConsumerService:0:Binding": "urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST",
"AssertionConsumerService:0:Location": "http://orcid.test.surfconext.nl/authentication/consume-assertion",
"AssertionConsumerService:0:index": "0",
"NameIDFormat": "urn:oasis:names:tc:SAML:2.0:nameid-format:transient",
"contacts:0:contactType": "technical",
"contacts:1:contactType": "technical",
"contacts:2:contactType": "technical",
"description:en": "Connection server - only used for OrcID now",
"description:nl": "Connection server - only used for OrcID now",
"logo:0:height": "60",
"logo:0:url": "https://.png",
"logo:0:width": "120",
"name:en": "OpenConext connection server",
"name:nl": "OpenConext connection server"
},
"metadataurl": "http://orcid.test.surfconext.nl/authentication/metadata",
"notes": null,
"revisionid": 4,
"revisionnote": "No revision note",
"state": "prodaccepted",
"type": "saml20-sp",
"user": "urn:collab:person:example.com:admin"
},
"revision": {
"created": 1453990922000,
"number": 4,
"updatedBy": "urn:collab:person:example.com:admin"
},
"type": "saml20_sp",
"version": 0
}
]
Another example agains the test2 environment to retrieve all single tenant templates:
curl -H 'Content-Type: application/json' -u pdp:secret -X POST -d '{"ALL_ATTRIBUTES":true}' 'https://manage.test2.surfconext.nl/manage/api/internal/search/single_tenant_template' | python -m json.tool
Or to retrieve a single / complete IDP by entityId:
curl -H 'Content-Type: application/json' -u pdp:secret -X POST -d '{"ALL_ATTRIBUTES":true, "entityid":"http://mock-idp"}' 'https://manage.test2.surfconext.nl//manage/api/internal/search/saml20_idp' | python -m json.tool
In order to use this API call you need to know the primary key of the MetaData which is returned in the Search API. You can not store the primary key in your own database as it might change (as we re-migrate the data).
curl -u sp-portal:secret -H 'Content-Type: application/json' 'http://localhost:8080/manage/api/internal/metadata/saml20_sp/176584e4-6f0c-4ec2-a0b9-e1bac1ad1aab'
There are basically two Write API's. One is based on a merge principle and one is based on a you-provide-everything principle.
You specify the exact path of the values and the actually new values of the fields you want to update.
The following example adds an IdP to the array of allowedEntities of the specified SP, sets the top-level attribute allowedall to false and finally updates or adds the metadata field description:en. Note that the id of the Metadata you want to update must already be retrieved by a search query:
curl -H 'Content-Type: application/json' -u sp-portal:secret \
-X PUT -d '{"id": "bac56ecd-29aa-449f-81ff-109cff1f90c4", "type": "saml20_sp","pathUpdates": { "allowedall": false, "allowedEntities": [{ "name": "https://allow-me" }], "metaDataFields.description:en": "New description" }}' \
'http://localhost:8080/manage/api/internal/merge'
The entire updated document is send back in response to the update:
{
"id": "fa4fd71d-b9d1-4da7-95c2-3be6bbeaf1f0",
"version": 9,
"type": "saml20_sp",
"revision": {
"number": 22,
"created": 1504195745.015000000,
"parentId": null,
"updatedBy": "sp-portal"
},
"data": {
"id": 281,
"eid": 1,
"entityid": "https://engine.test.surfconext.nl/authentication/sp/metadata",
"revisionid": 13,
"state": "prodaccepted",
"type": "saml20-sp",
"metadataurl": "https://metatdataurl",
"allowedall": false,
"manipulation": null,
"user": "urn:collab:person:example.com:admin",
"created": "2015-12-16T17:11:17+01:00",
"ip": "145.100.191.122",
"revisionnote": "No revision note",
"active": true,
"arp": {
"attributes": {},
"enabled": false
},
"notes": null,
"metaDataFields": {
"displayName:nl": "OpenConext Engine",
"NameIDFormats:0": "urn:oasis:names:tc:SAML:1.1:nameid-format:unspecified",
"url:en": "https://engine.test.surfconext.nl",
"description:en": "OpenConext SSO Proxy",
"contacts:2:givenName": "Support",
"contacts:1:surName": "OpenConext",
"contacts:2:surName": "OpenConext",
"contacts:2:emailAddress": "[email protected]",
"description:nl": "OpenConext SSO Proxy",
"logo:0:width": "96",
"logo:0:url": "https://static.test.surfconext.nl/media/conext_logo.png",
"name:en": "OpenConext Engine",
"contacts:0:givenName": "Support",
"contacts:0:surName": "OpenConext Engine",
"contacts:1:emailAddress": "[email protected]",
"contacts:0:emailAddress": "[email protected]",
"AssertionConsumerService:0:Location": "https://engine.test.surfconext.nl/authentication/sp/consume-assertion",
"contacts:0:contactType": "technical",
"contacts:2:contactType": "administrative",
"logo:0:height": "96",
"contacts:1:contactType": "technical",
"contacts:1:givenName": "Support",
"name:nl": "OpenConext Engine",
"NameIDFormat": "urn:oasis:names:tc:SAML:1.1:nameid-format:unspecified",
"AssertionConsumerService:0:Binding": "urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST",
"url:nl": "https://engine.test.surfconext.nl",
"displayName:en": "OpenConext Engine"
},
"allowedEntities": []
}
}
If an invalid request results in errors because of schema validations then a BAD_REQUEST is returned with the validation errors. The invalid request - invalid enum for NameIDFormat -:
curl -H 'Content-Type: application/json' -u sp-portal:secret -X PUT \
-d '{"id": "fa4fd71d-b9d1-4da7-95c2-3be6bbeaf1f0","type": "saml20_sp","pathUpdates": {"metaDataFields.NameIDFormats:0": "bogus"}}' \
'http://localhost:8080/manage/api/internal/metadata'
And the result:
{
"timestamp": 1506060789246,
"status": 400,
"error": "org.everit.json.schema.ValidationException",
"exception": "org.everit.json.schema.ValidationException",
"message": "#/metaDataFields/NameIDFormats:0: bogus is not a valid enum value",
"path": "/manage/api/internal/metadata",
"validations": "#/metaDataFields/NameIDFormats:0: bogus is not a valid enum value"
}
To create a new MetaData you can POST the entire document to the API:
curl -H 'Content-Type: application/json' -u sp-portal:secret \
-X POST -d '${THE_ENTIRE_DOCUMENT}' \
'http://localhost:8080/manage/api/internal/metadata'
With the provide-everything WRITE API you can also change the document by a PUT of the entire valid MetaData to the server. It is required that you have all the data and thus it is required you fetched the MetaData by its primary key and not using the Search API:
After changing MetaData you can use the following PUT to make updates:
curl -H 'Content-Type: application/json' -u sp-portal:secret \
-X PUT -d '${THE_ENTIRE_DOCUMENT}' \
'http://localhost:8080/manage/api/internal/metadata'
There is also a Validate endpoint which can be used to validate metadata before actually saving the metadata:
curl -H 'Content-Type: application/json' -u sp-portal:secret -X POST \
-d '{"type":"saml20_sp","data":{"entityid":"Duis ad do 2","state":"testaccepted","allowedall":true,"metaDataFields":{"name:en":"Test","AssertionConsumerService:0:Binding":"bogus","AssertionConsumerService:0:Location":"https://acs"}}}' \ 'http://localhost:8080/manage/api/internal/validate/metadata'
Note that you need to handle bad requests, because that is the result if there are validation exceptions
HTTP/1.1 400
{
"timestamp": 1506676418561,
"status": 400,
"error": "org.everit.json.schema.ValidationException",
"exception": "org.everit.json.schema.ValidationException",
"message": "#/metaDataFields/AssertionConsumerService:0:Binding: bogus is not a valid enum value",
"path": "/manage/api/internal/validate/metadata",
"validations": "#/metaDataFields/AssertionConsumerService:0:Binding: bogus is not a valid enum value"
}
After updating the MetaData the changes need to be pushed by Manage to EB. There is a endpoint to initiate the push. The configured API client will need the PUSH scope in order to perform the PUSH.
curl -u sp-portal:secret -H 'Content-Type: application/json' 'http://localhost:8080/manage/api/internal/push'
The Stats API has dedicated endpoints to retrieve consolidated metadata.
All SP's / IdP's and Revisions sorted on eid / revision.number and without any where clause:
curl -H 'Content-Type: application/json' -u pdp:secret 'http://localhost:8080/manage/api/internal/stats/revisions' | python -m json.tool
Respectively all unique SP's / IdP's and Revisions sorted on eid / revision.number. A provider is considered unique based on the entityId, coin:instutiontion_id and state (prod or acc).
curl -H 'Content-Type: application/json' -u pdp:secret 'http://localhost:8080/manage/api/internal/stats/uniques/saml20_sp' | python -m json.tool
curl -H 'Content-Type: application/json' -u pdp:secret 'http://localhost:8080/manage/api/internal/stats/uniques/saml20_idp' | python -m json.tool
All unique and allowed SP / IdP connections based on the unique list retrieved on the previous query and filtered further on the allowedall and allowedEntities attributes.
curl -H 'Content-Type: application/json' -u pdp:secret 'http://localhost:8080/manage/api/internal/stats/connections' | python -m json.tool
All SP's / IdPs and Revisions that can not be connected to:
curl -H 'Content-Type: application/json' -u pdp:secret 'http://localhost:8080/manage/api/internal/stats/none_allowed' | python -m json.tool
All SP's / IdPs and Revisions that can not be connected to - this list MUST be empty:
curl -H 'Content-Type: application/json' -u pdp:secret 'http://localhost:8080/manage/api/internal/stats/new_providers' | python -m json.tool
In the Manage application configuration you specify - or override using ansible - the location of the Manage api user configuration. This YML file contains all the internal API users, passwords and scopes. Example:
apiUsers:
- {
name: "attribute-aggregator",
password: "secret",
scopes: [READ]
}
- {
name: "pdp",
password: "secret",
scopes: [READ]
}
- {
name: "sp-portal",
password: "secret",
scopes: [READ, WRITE, PUSH]
}