Add support for format validation

[jdesrosiers]

JSON Schema has never required that the format keyword be validated. Since 2019-09, it's been required that it's not validated by default. For these reasons, I never chose to add support for validating format. I do want to support it eventually, but it's never been a priority.

I think this would be a good task for someone looking to contribute to open source. It's relatively straightforward, but requires proficiency with regular expressions, attention to detail, and the ability to read and understand RFCs. For each format, you'll need to understand the syntax and determine the best way to validate it. Sometimes that will be a regular expression, sometimes you'll need to find an appropriate library. There might be a case or two where you need to write the code to validate the format yourself, but mostly this should be about finding and using the right resource. The rest is just configuring the implementation for the keyword/vocabulary.

Acceptance Criteria

• Must include a test runner for and pass all format tests in the official test suite
  • draft-04 - tests
  • draft-06 - tests
  • draft-07 - tests
  • 2019-09 - tests
  • 2020-12 - tests
• All defined formats are supported (Spec)
  • date-time
  • date
  • time
  • duration
  • email
  • idn-email
  • hostname
  • idn-hostname
  • ipv4
  • ipv6
  • uri (Use @hyperjump/uri)
  • uri-reference (Use @hyperjump/uri)
  • iri (Use @hyperjump/uri)
  • iri-reference (Use @hyperjump/uri)
  • uuid
  • uri-template
  • json-pointer
  • relative-json-pointer
  • regex
• Validation is off by default in all dialects and can be enabled by setting a config
• In 2019-09, the format vocabulary set to true enables validation by default but can still be disabled by config
• By default, only formats defined by the dialect the schema uses are validated. Other formats are not enabled even if they are defined for another dialect. (Example: uuid should not be validated in draft-07)
  • draft-04 - date-time, email, hostname, ipv4, ipv6, uri
  • draft-06 - date-time, email, hostname, ipv4, ipv6, uri, uri-reference, uri-template, json-pointer
  • draft-07 - date-time, date, time, email, idn-email, hostname, idn-hostname, ipv4, ipv6, uri, uri-reference, iri, iri-reference, uri-template, json-pointer, relative-json-pointer, regex
  • 2019-09 - date-time, date, time, duration, email, idn-email, hostname, idn-hostname, ipv4, ipv6, uri, uri-reference, iri, iri-reference, uuid, uri-template, json-pointer, relative-json-pointer, regex
  • 2020-12 - date-time, date, time, duration, email, idn-email, hostname, idn-hostname, ipv4, ipv6, uri, uri-reference, iri, iri-reference, uuid, uri-template, json-pointer, relative-json-pointer, regex
• In draft-04/6/7, unknown formats return true
• In 2019-09 with the format vocabulary set to false, unknown formats return true
• In 2019-09 with the format vocabulary set to true, unknown formats throw an error
• In 2020-12 with the format-annotation vocabulary, unknown formats return true
• In 2020-12 with the format-assertion vocabulary, unknown formats throw an error
• Users can enable a defined format in a draft that didn't define it yet. (Example: enabling uuid in draft-07 schema)
• Users can define and enable custom formats for specific dialects
• Annotations are collected when validation is enabled
• Works in node.js and the browser
• When validation is off, its code must not be bundled
• Bundle size for browsers should be kept as small as possible (Be careful about including large dependencies)
• Update README

[techmannih]

Hey @jdesrosiers I’m available and interested in contributing to this issue.

[jdesrosiers]

Thanks for helping out! The format keyword implementation is a good place to start.

[Vishv04]

Hi @jdesrosiers ,

I would like to work on this issue and believe I understand the problem described. I have forked and cloned the repository and tried exploring the codebase, but I couldn’t identify where to make the necessary modifications for the "format" validation.

Could you please guide me on which files/modules are relevant? Also, it would be great if you could provide instructions on how to run this project locally to test my changes.

[jdesrosiers]

@Vishv0407 Thanks for offering, but @techmannih is already working on it. I have no problem with multiple people working together on it, but I'll leave it up to @techmannih to decide if and how they want to bring someone else in.

[techmannih]

@Vishv0407 Thank you for showing your interest. I'm glad! Yes, we can discuss this issue on Slack, and we can work together in collaboration.

[Vishv04]

@techmannih sure!, let's start discussion on slack.

[mgrist]

I get an error when trying to validate this date format field. I am guessing it has something to do with this open issue, or am I doing something wrong here?

test-schema.json

{
  "$schema": "https://json-schema.org/draft/2019-09/schema",
  "$id": "https://schemas.test.com/some-schema",
  "type": "object",
  "required": ["start_at"],
  "properties": {
    "start_at": {
      "title": "Offering Release Date",
      "description": "Your price start date",
      "editable": true,
      "hidden": false,
      "examples": ["2017-06-30"],
      "type": "string",
      "oneOf": [
        {
          "type": "string",
          "format": "date"
        },
        {
          "type": "string",
          "format": "date-time"
        }
      ]
    }
  },
  "additionalProperties": false
}

validator.ts

import { readFileSync } from "fs";
import {
  registerSchema,
  validate,
  OutputFormat,
} from "@hyperjump/json-schema/draft-2019-09";

const schemaId = "https://schemas.test.com/some-schema";

const schemaPath = "test-schema.json";

const payload = {
  start_at: "2017-06-30",
};

const outputFormat: OutputFormat = "BASIC";

const parsedSchema = JSON.parse(readFileSync(schemaPath, "utf8"));
registerSchema(parsedSchema, schemaId);

validate(schemaId, payload, outputFormat).then((result) => {
  console.log(result);
});

output

{
  valid: false,
  errors: [
    {
      keyword: 'https://json-schema.org/keyword/oneOf',
      absoluteKeywordLocation: 'https://schemas.test.com/some-schema#/properties/start_at/oneOf',
      instanceLocation: '#/start_at'
    }
  ]
}

[jdesrosiers]

@mgrist By default, format keyword is just an annotation. It doesn't influence validation. So, your schema is really saying that the value must be one of a string or a string. Since both of those are true, the oneOf fails because oneOf allows only on schema to pass. This is the expected and correct result.

Optionally, implementations can provide a configuration option to enable evaluating format as a validation keyword. I currently don't provide that option. That's what this issue was for. I had someone working on it for me, but they appear to have given up.

[mgrist]

Okay, thank you for the information. I may look into the codebase and see if I can figure out how to implement it.

[jdesrosiers]

@mgrist If you want, you can just use addKeyword with the keyword URI https://json-schema.org/keyword/format to replace the default implementation of format with a custom one that validates the formats you want to validate. You already have experience with custom keywords, so that should be trivial for you.

If you want to contribute a PR that would be great too! That would be more work with configuration and testing requirements for multiple dialects and might require you to implement more formats than you personally need. So, I understand if you just want to just do enough for your needs.

[mgrist]

@mgrist If you want, you can just use addKeyword with the keyword URI https://json-schema.org/keyword/format to replace the default implementation of format with a custom one that validates the formats you want to validate. You already have experience with custom keywords, so that should be trivial for you.

If you want to contribute a PR that would be great too! That would be more work with configuration and testing requirements for multiple dialects and might require you to implement more formats than you personally need. So, I understand if you just want to just do enough for your needs.

@jdesrosiers I was able to add a keyword and implement the date and date-time formats. Thanks for the tip! After I finish up this project I am working on, I will look into adding this implementation. Would love to contribute.