Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .claude/settings.local.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
{
"permissions": {
"allow": ["Bash(npx vitest:*)", "Bash(bash:*)", "Bash(node test-debug.js:*)"]
},
"enableAllProjectMcpServers": false
}
662 changes: 331 additions & 331 deletions .yarn/releases/yarn-4.9.4.cjs → .yarn/releases/yarn-4.11.0.cjs

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion .yarnrc.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
nodeLinker: node-modules

yarnPath: .yarn/releases/yarn-4.9.4.cjs
yarnPath: .yarn/releases/yarn-4.11.0.cjs
152 changes: 152 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,152 @@
# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## Development Commands

### Building
```bash
yarn build # Compile TypeScript to dist/
yarn typecheck # Run TypeScript type checking without emitting files
```

### Testing
```bash
yarn test # Run all tests with coverage (Vitest)
yarn test:node # Run tests in Node.js environment
yarn test:browser # Run tests in browser environment (jsdom)
yarn test:watch # Run tests in watch mode
yarn test:update # Update test snapshots
```

### Code Quality
```bash
yarn lint # Run ESLint on lib/ directory
yarn prettier # Format all code files
```

### Running Individual Tests
To run a single test file:
```bash
npx vitest test/specs/circular/circular.spec.ts
```

To run tests matching a pattern:
```bash
npx vitest --grep "circular"
```

## Architecture Overview

### Core Purpose
This library parses, resolves, and dereferences JSON Schema `$ref` pointers. It handles references to:
- External files (local filesystem)
- HTTP/HTTPS URLs
- Internal JSON pointers within schemas
- Mixed JSON and YAML formats
- Circular/recursive references

### Key Architecture Components

#### 1. $RefParser (lib/index.ts)
The main entry point and orchestrator class. Provides four primary operations:
- **parse()**: Reads a single schema file (JSON/YAML) without resolving references
- **resolve()**: Parses and resolves all `$ref` pointers, returns a `$Refs` object mapping references to values
- **bundle()**: Converts external `$ref` pointers to internal ones (single file output)
- **dereference()**: Replaces all `$ref` pointers with their actual values (fully expanded schema)

All methods support both callback and Promise-based APIs, with multiple overload signatures.

#### 2. $Refs (lib/refs.ts)
A map/registry of all resolved JSON references and their values. Tracks:
- All file paths/URLs encountered
- Circular reference detection
- Helper methods to query references by type (file, http, etc.)

#### 3. Pointer (lib/pointer.ts)
Represents a single JSON pointer (`#/definitions/person`) and implements JSON Pointer RFC 6901 spec:
- Parses JSON pointer syntax (`/`, `~0`, `~1` escaping)
- Resolves pointers to actual values within objects
- Handles edge cases (null values, missing properties)

#### 4. $Ref (lib/ref.ts)
Wraps a single reference with metadata:
- The reference path/URL
- The resolved value
- Path type (file, http, etc.)
- Error information (when continueOnError is enabled)

#### 5. Plugin System
Two types of plugins, both configurable via options:

**Parsers** (lib/parsers/):
- JSON parser (json.ts)
- YAML parser (yaml.ts) - uses js-yaml
- Text parser (text.ts)
- Binary parser (binary.ts)
- Execute in order based on `order` property and `canParse()` matching

**Resolvers** (lib/resolvers/):
- File resolver (file.ts) - reads from filesystem (Node.js only)
- HTTP resolver (http.ts) - fetches from URLs using native fetch
- Custom resolvers can be added via options
- Execute in order based on `order` property and `canRead()` matching

#### 6. Core Operations

**lib/parse.ts**: Entry point for parsing a single schema file
**lib/resolve-external.ts**: Crawls schema to find and resolve external `$ref` pointers
**lib/bundle.ts**: Replaces external refs with internal refs
**lib/dereference.ts**: Replaces all `$ref` pointers with actual values, handles circular references

#### 7. Options System (lib/options.ts)
Hierarchical configuration with defaults for:
- Which parsers/resolvers to enable
- Circular reference handling (boolean or "ignore")
- External reference resolution (relative vs root)
- Continue on error mode (collect all errors vs fail fast)
- Bundle/dereference callbacks and matchers
- Input mutation control (mutateInputSchema)

### Key Design Patterns

1. **Flexible Arguments**: normalizeArgs() (lib/normalize-args.ts) unifies various call signatures into consistent internal format

2. **Path Handling**: Automatic conversion between filesystem paths and file:// URLs. Cross-platform support via util/url.ts and util/convert-path-to-posix.ts

3. **Error Handling**:
- Fail-fast by default
- Optional continueOnError mode collects errors in JSONParserErrorGroup
- Specific error types: JSONParserError, InvalidPointerError, MissingPointerError, ResolverError, ParserError

4. **Circular Reference Management**:
- Detected during dereference
- Can throw error, ignore, or handle via dereference.circular option
- Reference equality maintained (same `$ref` → same object instance)

5. **Browser/Node Compatibility**:
- Uses native fetch (requires Node 18+)
- File resolver disabled in browser builds (package.json browser field)
- Tests run in both environments

## Testing Strategy

Tests are organized in test/specs/ by scenario:
- Each scenario has test files (*.spec.ts) and fixture data
- Tests validate parse, resolve, bundle, and dereference operations
- Extensive coverage of edge cases: circular refs, deep nesting, special characters in paths
- Browser-specific tests use test/fixtures/server.ts for HTTP mocking

Test utilities:
- test/utils/helper.js: Common test patterns
- test/utils/path.js: Path handling for cross-platform tests
- test/utils/serializeJson.ts: Custom snapshot serializer

## Important Constraints

1. **TypeScript Strict Mode**: Project uses strict TypeScript including exactOptionalPropertyTypes
2. **JSON Schema Support**: Compatible with JSON Schema v4, v6, and v7
3. **Minimum Node Version**: Requires Node >= 20 (for native fetch support)
4. **Circular JSON**: Dereferenced schemas may contain circular references (not JSON.stringify safe)
5. **Path Normalization**: Always converts filesystem paths to POSIX format internally
6. **URL Safety**: HTTP resolver has safeUrlResolver option to block internal URLs (default: unsafe allowed)
4 changes: 0 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,19 +29,15 @@ format. Maybe some of the files contain cross-references to each other.
{
"definitions": {
"person": {
// references an external file
"$ref": "schemas/people/Bruce-Wayne.json"
},
"place": {
// references a sub-schema in an external file
"$ref": "schemas/places.yaml#/definitions/Gotham-City"
},
"thing": {
// references a URL
"$ref": "http://wayne-enterprises.com/things/batmobile"
},
"color": {
// references a value in an external file via an internal reference
"$ref": "#/definitions/thing/properties/colors/black-as-the-night"
}
}
Expand Down
21 changes: 12 additions & 9 deletions lib/bundle.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,10 @@ import $Ref from "./ref.js";
import Pointer from "./pointer.js";
import * as url from "./util/url.js";
import type $Refs from "./refs.js";
import type $RefParser from "./index";
import type { ParserOptions } from "./index";
import type { JSONSchema } from "./index";
import type { BundleOptions } from "./options";
import type $RefParser from "./index.js";
import type { ParserOptions } from "./index.js";
import type { JSONSchema } from "./index.js";
import type { BundleOptions } from "./options.js";

export interface InventoryEntry {
$ref: any;
Expand Down Expand Up @@ -40,7 +40,7 @@ function bundle<S extends object = JSONSchema, O extends ParserOptions<S> = Pars
crawl<S, O>(parser, "schema", parser.$refs._root$Ref.path + "#", "#", 0, inventory, parser.$refs, options);

// Remap all $ref pointers
remap(inventory);
remap<S, O>(inventory, options);
}

/**
Expand Down Expand Up @@ -203,7 +203,10 @@ function inventory$Ref<S extends object = JSONSchema, O extends ParserOptions<S>
*
* @param inventory
*/
function remap(inventory: InventoryEntry[]) {
function remap<S extends object = JSONSchema, O extends ParserOptions<S> = ParserOptions<S>>(
inventory: InventoryEntry[],
options: O,
) {
// Group & sort all the $ref pointers, so they're in the order that we need to dereference/remap them
inventory.sort((a: InventoryEntry, b: InventoryEntry) => {
if (a.file !== b.file) {
Expand Down Expand Up @@ -254,10 +257,10 @@ function remap(inventory: InventoryEntry[]) {
// This $ref already resolves to the main JSON Schema file
entry.$ref.$ref = entry.hash;
} else if (entry.file === file && entry.hash === hash) {
// This $ref points to the same value as the prevous $ref, so remap it to the same path
// This $ref points to the same value as the previous $ref, so remap it to the same path
entry.$ref.$ref = pathFromRoot;
} else if (entry.file === file && entry.hash.indexOf(hash + "/") === 0) {
// This $ref points to a sub-value of the prevous $ref, so remap it beneath that path
// This $ref points to a sub-value of the previous $ref, so remap it beneath that path
entry.$ref.$ref = Pointer.join(pathFromRoot, Pointer.parse(entry.hash.replace(hash, "#")));
} else {
// We've moved to a new file or new hash
Expand All @@ -267,7 +270,7 @@ function remap(inventory: InventoryEntry[]) {

// This is the first $ref to point to this value, so dereference the value.
// Any other $refs that point to the same value will point to this $ref instead
entry.$ref = entry.parent[entry.key] = $Ref.dereference(entry.$ref, entry.value);
entry.$ref = entry.parent[entry.key] = $Ref.dereference(entry.$ref, entry.value, options);

if (entry.circular) {
// This $ref points to itself
Expand Down
7 changes: 3 additions & 4 deletions lib/dereference.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,8 @@ import Pointer from "./pointer.js";
import * as url from "./util/url.js";
import type $Refs from "./refs.js";
import type { DereferenceOptions, ParserOptions } from "./options.js";
import type { JSONSchema } from "./types";
import type $RefParser from "./index";
import { TimeoutError } from "./util/errors";
import { type $RefParser, type JSONSchema } from "./index.js";
import { TimeoutError } from "./util/errors.js";

export default dereference;

Expand Down Expand Up @@ -278,7 +277,7 @@ function dereference$Ref<S extends object = JSONSchema, O extends ParserOptions<
}

// Dereference the JSON reference
let dereferencedValue = $Ref.dereference($ref, pointer.value);
let dereferencedValue = $Ref.dereference($ref, pointer.value, options);

// Crawl the dereferenced value (unless it's circular)
if (!circular) {
Expand Down
2 changes: 1 addition & 1 deletion lib/normalize-args.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import type { Options, ParserOptions } from "./options.js";
import { getNewOptions } from "./options.js";
import type { JSONSchema, SchemaCallback } from "./types";
import type { JSONSchema, SchemaCallback } from "./index.js";

// I really dislike this function and the way it's written. It's not clear what it's doing, and it's way too flexible
// In the future, I'd like to deprecate the api and accept only named parameters in index.ts
Expand Down
8 changes: 8 additions & 0 deletions lib/options.ts
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,13 @@ export interface DereferenceOptions {
* Default: `relative`
*/
externalReferenceResolution?: "relative" | "root";

/**
* Whether duplicate keys should be merged when dereferencing objects.
*
* Default: `true`
*/
mergeKeys?: boolean;
}

/**
Expand Down Expand Up @@ -229,6 +236,7 @@ export const getJsonSchemaRefParserDefaultOptions = () => {
*/
excludedPathMatcher: () => false,
referenceResolution: "relative",
mergeKeys: true,
},

mutateInputSchema: true,
Expand Down
Loading
Loading