nosqlite

trying to build a small embedded nosql database from scratch in rust.

main idea is simple: sqlite like single file database but for document/nosql style storage.

this project is mostly for learning database internals and low level systems stuff.

things i want to learn while building this; pager, page layouts, b+ tree, binary storage, write ahead logging, indexes, recovery not trying to make next mongodb or some distributed cloud thing.

features

currently working on:

• single file database storage
• bson document storage
• collections
• document insertion
• document lookup
• deletion
• ffi layer
• shell
• language bindings

planned:

• b+ tree indexes
• query engine
• filters
• update operations
• transactions
• write ahead logging
• crash recovery
• secondary indexes

documentation

Full documentation for now is at here.

architecture

project currently contains:

• nosqlite-core: storage engine and pager internals
• nosqlite-ffi: c abi bindings layer
• nosqlite-shell: cli shell for interacting with database
• bindings
  • go
  • c
  • dart/flutter (planned)
  • python (planned)

installation

prebuilt shared/static libraries are released for:

• linux: x86, x86_64, arm64, armv7

• android: x86, x86_64, arm64, armv7

• windows: x86, x86_64

• macos: x86_64, arm64

linux / macos

install latest runtime:

curl -fsSL https://raw.githubusercontent.com/PriyanshuPz/nosqlite/master/scripts/install.sh | sh

install runtime + shell:

curl -fsSL https://raw.githubusercontent.com/PriyanshuPz/nosqlite/master/scripts/install.sh | sh -s -- --shell

windows powershell

install latest runtime:

irm https://raw.githubusercontent.com/PriyanshuPz/nosqlite/master/scripts/install.ps1 | iex

install runtime + shell:

& ([scriptblock]::Create(
    (irm https://raw.githubusercontent.com/P8labs/PriyanshuPz/nosqlite/master/scripts/install.ps1)
)) -shell

installed layout

runtime installs into:

linux

~/.nosqlite

macos

/Users/<user>/.nosqlite

windows

C:\Users\<user>\.nosqlite

layout:

.nosqlite/
├── include/
├── lib/
│   ├── linux/
│   ├── windows/
│   ├── macos/
│   └── android/
└── bin/

releases

github actions automatically builds:

• shared libraries
• static libraries
• shell binaries
• headers

for all supported platforms and architectures.

release artifacts include:

• dynamic libraries (.so, .dll, .dylib)
• static libraries (.a)
• generated nosqlite.h
• shell binaries

building locally

requirements:

• rust
• cargo
• zig
• cargo-zigbuild
• cbindgen

build ffi:

cargo zigbuild \
  --release \
  -p nosqlite-ffi

build shell:

cargo build \
  --release \
  -p nosqlite-shell

AI disclosure

for this project i am not using any in editor AI tools or autocomplete AI. no copilot. no AI code generation inside editor. using zed editor normally and writing code manually.

AI tools like ChatGPT or Google Gemini can still be used sometimes for theory, explanations, debugging help, searching things

main reason is i want to actually learn how databases work internally instead of vibe coding random storage engine.

resources

will keep adding useful stuff here while building.

• https://youtu.be/aZjYr87r1b8?si=CLWGrI-eb3JdWYB8
• https://rust.code-maven.com/rust-programming/rust/stdin/rust-hello-name.html
• https://www.mongodb.com/docs/mongodb-shell/crud/
• https://en.wikipedia.org/wiki/8-bit_computing
• https://donsnotes.com/tech/charsets/ascii.html
• https://bsonspec.org/spec.html
• SQLite Database System Design and Implementation Book by Sibsankar Haldar
• https://ratatui.rs/recipes/apps
• https://bsonspec.org/spec.html
• https://doc.rust-lang.org/book/ch16-03-shared-state.html

status

very experimental. internal formats and APIs will change frequently. not production ready.