nodedb-json

A small JSON-file database for Node.js. It is useful for local tools, examples, desktop apps, small services, tests, and configuration-like data where a full database server would be too much.

It stores one JSON document on disk and gives you CRUD helpers, array operations, indexes, batch writes, and query helpers.

Install

npm install nodedb-json

Quick Start

const NodedbJson = require('nodedb-json');

const db = new NodedbJson('data/db.json', {
  defaultValue: { users: [] }
});

db.createIndex('users', { field: 'id', type: 'unique' });

db.push('users', { id: 1, name: 'Bob', age: 31 });
db.update('users', user => user.id === 1, { role: 'admin' });

const user = db.findByField('users', 'id', 1);
console.log(user);

TypeScript can type the root document with a schema:

import NodedbJson, { DuplicateIndexError } from 'nodedb-json';

type User = {
  id: number;
  name: string;
  age?: number;
};

type AppConfig = {
  theme: string;
  language?: string;
};

interface Schema {
  users: User[];
  config: AppConfig;
}

const db = new NodedbJson<Schema>('data/db.json', {
  defaultValue: {
    users: [],
    config: { theme: 'dark' }
  }
});

const users = db.collection('users');

try {
  users.createIndex({ field: 'id', type: 'unique' });
  users.insert({ id: 1, name: 'Bob' });

  const user = users.findOne({ id: 1 }); // User | undefined
  const config = db.get('config'); // AppConfig
} catch (error) {
  if (error instanceof DuplicateIndexError) {
    console.error(error.message);
  }
}

What It Does

• Reads and writes a JSON file.
• Creates missing parent directories automatically.
• Writes through a same-directory temp file and rename.
• Keeps a .bak backup by default and can restore from it if the JSON file is corrupt.
• Creates a .lock file by default to prevent two live writers from opening the same database.
• Supports dot-path keys such as config.theme.
• Returns deep copies from read APIs so callers cannot mutate internal state by accident.
• Supports unique and multi indexes for array fields.
• Persists index definitions to a .meta.json file by default.
• Supports NodedbJson<Schema> and typed collection() helpers in TypeScript.
• Includes Vitest coverage and GitHub Actions for Node 18, 20, and 22.

Important Limits

This package is still a single-file, synchronous JSON database.

• It is best used by one process / one writer at a time.
• File locking is cooperative and based on a sibling .lock file.
• Call close() when a long-lived instance is done so the lock is released.
• If fileLock: false is used, two live instances can overwrite each other's in-memory snapshots.
• Every save serializes the whole JSON document.
• For many small writes, use batch() or autoSave: false plus flush().
• The package publishes separate ESM and CommonJS entries through exports.

Constructor Options

type DbOptions = {
  autoSave?: boolean;          // default: true
  createIfNotExists?: boolean; // default: true
  defaultValue?: Partial<TSchema>;

  enableIndexing?: boolean;    // default: true
  autoIndex?: boolean;         // default: true
  indexValueMode?: 'strict' | 'coerce'; // default: 'strict'
  indexes?: IndexOptions;
  persistIndexes?: boolean;    // default: true
  indexMetaPath?: string;      // default: `${dbFile}.meta.json`

  atomicWrites?: boolean;      // default: true
  backupOnWrite?: boolean;     // default: true
  backupPath?: string;         // default: `${dbFile}.bak`
  tempPath?: string;           // default: `${dbFile}.tmp`

  fileLock?: boolean;          // default: true
  lockPath?: string;           // default: `${dbFile}.lock`
  staleLockMs?: number;        // remove old locks only when the recorded PID is gone
};

Example:

const db = new NodedbJson('data/db.json', {
  autoSave: false,
  defaultValue: {
    users: [],
    config: { theme: 'dark' }
  },
  indexes: {
    users: [
      { field: 'id', type: 'unique' },
      { field: 'department', type: 'multi' }
    ]
  }
});

Basic Usage

Values And Objects

db.set('config.theme', 'dark');
db.set('config.language', 'zh-CN');

console.log(db.get('config.theme')); // dark
console.log(db.has('config.language')); // true

db.update('config', { timezone: 'Asia/Shanghai' });
db.delete('config', ['language']);

get() returns a deep copy:

const config = db.get('config');
config.theme = 'light';

console.log(db.get('config.theme')); // still "dark"

For controlled in-place style changes, use mutate():

db.mutate('config', config => {
  config.theme = 'light';
});

getUnsafeReference() exists for advanced cases, but mutating its return value bypasses change tracking, auto-save, and index refreshes.

Arrays

db.push('users', { id: 1, name: 'Bob', age: 31 });
db.push('users', [
  { id: 2, name: 'Charlie', age: 35 },
  { id: 3, name: 'Dave', age: 40 }
]);

const user = db.find('users', user => user.id === 2);
const adults = db.filter('users', user => user.age >= 18);

db.update('users', user => user.id === 1, { role: 'admin' });
db.delete('users', [2, 3], 'id');

Collections

collection() is a small typed wrapper around an array path. It keeps common array operations readable without changing the stored JSON shape:

const users = db.collection<User>('users');

users.insert({ id: 1, name: 'Bob' });
users.updateOne({ id: 1 }, { name: 'Robert' });
users.deleteOne({ id: 1 });

const activeUsers = users.findMany(user => user.active === true);

When the database has a schema, the item type is inferred:

const db = new NodedbJson<Schema>('data/db.json');
const users = db.collection('users'); // Collection<User>

Manual Save

autoSave is enabled by default. Disable it when you want several changes to be written once:

const db = new NodedbJson('data/db.json', { autoSave: false });

db.set('config.theme', 'dark');
db.push('logs', { time: new Date().toISOString(), action: 'updated config' });

db.flush();

Lifecycle

const db = new NodedbJson('data/db.json', {
  autoSave: false,
  staleLockMs: 60_000
});

db.push('users', { id: 1 });
db.flush();  // force pending changes to disk
db.reload(); // read the JSON file again and rebuild indexes
db.close();  // flush and release the lock

Opening a database creates db.json.lock by default. A second instance opening the same file throws FileLockError until the first one calls close(). If close() cannot save pending changes, the instance stays open and keeps the lock so callers can retry flush() or close(). Use close({ force: true }) only when you intentionally want to discard pending in-memory changes and release the lock. When staleLockMs is set, an old lock is removed only if its recorded process is no longer alive.

If the main JSON file is corrupt and a valid .bak exists, the backup is restored automatically. If both are invalid, construction throws CorruptedFileError.

Batch

batch() runs allowed write operations with autoSave temporarily disabled. If one operation throws, in-memory data and indexes are rolled back.

db.batch([
  { method: 'set', args: ['config.theme', 'dark'] },
  { method: 'push', args: ['logs', { action: 'config updated' }] }
]);

Allowed batch methods are set, update, delete, push, and mutate.

Indexes

Indexes are for arrays of objects.

db.createIndex('users', { field: 'id', type: 'unique' });
db.createIndex('users', { field: 'department', type: 'multi' });

const bob = db.findByField('users', 'id', 1);
const devUsers = db.filterByField('users', 'department', ['dev']);

db.dropIndex('users', 'department');
console.log(db.getIndexes());

Index types:

• unique: one item per field value. Duplicate non-null values throw DuplicateIndexError.
• multi: many items can share the same field value.

Index definitions created with createIndex() are persisted by default. You can also define them in options:

const db = new NodedbJson('data/db.json', {
  indexes: {
    users: [
      { field: 'id', type: 'unique' },
      { field: 'department', type: 'multi' }
    ]
  }
});

Index values are type-strict by default. A numeric 1 and a string "1" are different index values, which keeps indexed lookups consistent with non-indexed === lookups:

db.findByField('users', 'id', 1);   // matches { id: 1 }
db.findByField('users', 'id', '1'); // matches { id: '1' }

For compatibility with older releases, set indexValueMode: 'coerce' to use the old string-coercion behavior.

Query

Use query() for filtering, sorting, pagination, projection, and aggregation. Queries run as an in-memory eager pipeline over the current array.

const result = db.query('users', {
  where: {
    department: 'dev',
    age: { $gte: 18, $lt: 60 },
    status: { $in: ['active', 'pending'] }
  },
  sort: { field: 'age', direction: 'desc' },
  pagination: { page: 1, pageSize: 10 },
  select: ['id', 'name', 'age'],
  aggregation: [
    { type: 'count' },
    { type: 'avg', field: 'age' }
  ]
});

console.log(result.data);
console.log(result.pagination);
console.log(result.aggregations);
console.log(result.stats.usedIndex);

Object where conditions can use indexes when indexed equality or $in fields are present. Multiple indexed fields are intersected before the remaining conditions are checked. Function predicates are supported too, but they scan the collection:

const result = db.query('users', {
  where: user => user.age >= 18 && user.active === true
});

Supported object operators:

db.query('users', {
  where: {
    age: { $gte: 18, $lt: 60 },
    status: { $in: ['active', 'pending'] },
    name: { $startsWith: '张' }
  }
});

Operators: $eq, $ne, $gt, $gte, $lt, $lte, $in, $nin, $startsWith, $endsWith, $contains, and $regex.

skip must be a non-negative integer. limit, pagination.page, and pagination.pageSize must be positive integers. When skip, limit, and pagination are used together, the order is:

where -> sort -> skip -> limit -> pagination -> select

Aggregations run against the filtered data before sort, skip, limit, pagination, and select are applied.

In TypeScript, literal select fields narrow the returned item type:

const result = db.query<User>('users', {
  select: ['id'] as const
});

result.data[0].id;   // ok
result.data[0].name; // TypeScript error

Dynamic string[] selects and nested path selects stay conservatively typed.

Convenience helpers:

db.orderBy('users', { field: 'age', direction: 'desc' }, 5);
db.paginate('users', 1, 20, { department: 'dev' });
db.count('users', { active: true });
db.aggregate('users', [{ type: 'max', field: 'age' }]);
db.distinct('users', 'department');

1.4.1 Notes

This is a correctness patch for the 1.4.x line.

Fixed:

• Collection object conditions now use the same matcher as root queries.
• Persisted index definitions no longer cause false indexed lookups when autoIndex: false.
• Nested object field deletion works with dot-path keys.
• close() no longer closes the instance or releases the lock if saving fails.
• staleLockMs no longer removes a lock owned by a live process.
• Group aggregations and distinct() no longer expose internal references.
• $regex queries are isolated from stateful g / y RegExp lastIndex.
• createIndex() and dropIndex() roll back in-memory index state if metadata persistence fails.
• TypeScript select results now reflect partial selected records.

Usage impact:

• Normal runtime usage is unchanged.
• close({ force: true }) is available when you explicitly want to discard pending in-memory changes after a close/save failure.
• TypeScript users may now see compile errors when accessing fields that were not selected by a literal select query. The runtime already returned partial objects; the type now matches that behavior.

1.4.0 Reliability Notes

The reliability work adds:

• Atomic writes, backup recovery, and parent directory creation.
• Persistent index metadata.
• File locking through a sibling .lock file.
• flush(), reload(), and close().
• DatabaseError, CorruptedFileError, and FileLockError.

Remaining limits:

• The database is still synchronous and single-file.
• File locking is cooperative and local-filesystem oriented.
• There is no cross-process transaction isolation.

1.3.1 Notes

This release focuses on correctness and test coverage.

Fixed:

• Multi-value indexes now correctly return items stored at array position 0.
• Indexed batch deletion no longer deletes the wrong items after array indexes shift.
• Unique indexes now reject duplicate values instead of silently overwriting entries.
• Object updates now merge plain objects as documented.
• Indexed object updates now verify all query conditions before modifying an item.
• Read APIs return deep copies by default.
• Tests now fail properly through Vitest and CI.

Also included:

• Atomic JSON writes with temp files and rename.
• Backup creation and corrupt JSON recovery from .bak.
• Parent directory creation.
• Persistent index metadata.

Development

npm run lint
npm run typecheck
npm test
npm run build
npm pack --dry-run

License

MIT

Support

Issues: https://github.com/zxbb1190/nodedb-json/issues

Email: mailto:douyaj33@gmail.com