Skip to content

Commit fcda284

Browse files
semihalevsemihalev
committed
Initial commit: stoolap-mariadb storage engine plugin
MariaDB 11.4 storage engine plugin (ha_stoolap.so) backed by the Stoolap Rust engine via the C ABI. Apache 2.0. Plugin source (~10K LOC C++) under src/: - Whole-SELECT, derived, unit pushdown factories. - Direct UPDATE/DELETE through pre_direct_*_rows hooks, with constraint errors mapped to ER_DUP_ENTRY / ER_NO_REFERENCED_ROW_2. - Per-session isolation: register_trx reads thd_tx_isolation, opens STOOLAP_ISOLATION_SNAPSHOT for REPEATABLE READ / SERIALIZABLE. start_consistent_snapshot handlerton callback anchors snapshot at BEGIN time. - Composite PRIMARY KEY / UNIQUE / FK and self-FK refused at CREATE. - Multi-row INSERT IGNORE / REPLACE / ODKU bypass bulk batching so MariaDB drives per-row dup recovery. - Three-layer records cache (handler-local, tx-local, engine-global) with success-only delta application and end-of-tx invalidation; Stoolap_records_live_counts telemetry pins the in-tx cache reuse. - Data-driven error mapping with Stoolap_unmapped_errors drift detector; case_18 asserts every known stoolap error class round-trips to the right MariaDB errno. - Per-THD stoolap clone matches the PHP daemon threading contract. Build glue (CMakeLists.txt, .clang-format) and vendored MariaDB wsrep headers under third_party/. Python integration harness under tests/: runner.py spawns a private mariadbd, 17 case files (520+ assertions, all green on memory:// and file:// DSNs). Benchmark dashboard (vs InnoDB under the same MariaDB 11.4) under benchmarks/. Recent runs: STOOLAP 44 wins / InnoDB 9 wins. CI (macOS + Ubuntu, MariaDB 11.4) and release workflows under .github/.
0 parents  commit fcda284

90 files changed

Lines changed: 24147 additions & 0 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.clang-format‎

Lines changed: 76 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,76 @@
1+
---
2+
# Matches the existing hand-rolled style of src/*.{cc,h}:
3+
# - 4-space indent, no tabs
4+
# - opening brace on the same line as functions / control flow
5+
# - `Type* name` (left-attached pointer)
6+
# - ~80 column wrap with continuation aligned under the open paren
7+
# - short single-statement bodies kept on one line
8+
# - C-style casts get no space (`(int) x` -> `(int)x`); we use C++ casts.
9+
Language: Cpp
10+
BasedOnStyle: Google
11+
12+
IndentWidth: 4
13+
ContinuationIndentWidth: 4
14+
AccessModifierOffset: -4
15+
ColumnLimit: 80
16+
UseTab: Never
17+
TabWidth: 4
18+
19+
PointerAlignment: Left
20+
ReferenceAlignment: Pointer
21+
DerivePointerAlignment: false
22+
23+
AlignAfterOpenBracket: Align
24+
AlignConsecutiveAssignments:
25+
Enabled: false
26+
AlignConsecutiveDeclarations:
27+
Enabled: false
28+
AlignTrailingComments: true
29+
AlignEscapedNewlines: Left
30+
AlignOperands: Align
31+
32+
AllowShortBlocksOnASingleLine: Empty
33+
AllowShortCaseLabelsOnASingleLine: false
34+
AllowShortFunctionsOnASingleLine: Inline
35+
AllowShortIfStatementsOnASingleLine: WithoutElse
36+
AllowShortLambdasOnASingleLine: All
37+
AllowShortLoopsOnASingleLine: false
38+
39+
BreakBeforeBraces: Attach
40+
BreakConstructorInitializers: BeforeColon
41+
BreakInheritanceList: BeforeColon
42+
BreakBeforeTernaryOperators: true
43+
BreakStringLiterals: false
44+
45+
# Don't reflow long URL / mid-comment text every time someone touches a
46+
# block; keep the carefully formatted block comments stable.
47+
ReflowComments: false
48+
49+
# Local includes first (we have a flat src/ + a vendored third_party/);
50+
# system headers next; trust hand-ordered grouping inside a block.
51+
IncludeBlocks: Preserve
52+
SortIncludes: Never
53+
54+
# Preserve the visual wrap on very long string literals (error messages
55+
# split across lines for log readability).
56+
PenaltyBreakString: 1000
57+
58+
NamespaceIndentation: None
59+
# We sometimes annotate the closing brace with a parenthetical
60+
# explanation (`} // namespace (re-opened so helpers below have
61+
# external linkage)`); FixNamespaceComments would rewrite those to
62+
# the canonical bare form and orphan the rest of the comment.
63+
FixNamespaceComments: false
64+
65+
SpaceAfterCStyleCast: false
66+
SpaceBeforeAssignmentOperators: true
67+
SpaceBeforeParens: ControlStatements
68+
SpaceBeforeRangeBasedForLoopColon: true
69+
SpacesBeforeTrailingComments: 2
70+
SpacesInAngles: Never
71+
SpacesInContainerLiterals: false
72+
SpacesInCStyleCastParentheses: false
73+
SpacesInParentheses: false
74+
SpacesInSquareBrackets: false
75+
76+
Standard: c++17

‎.github/workflows/ci.yml‎

Lines changed: 210 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,210 @@
1+
name: CI
2+
3+
on:
4+
push:
5+
branches: [main]
6+
tags-ignore: ['**']
7+
pull_request:
8+
workflow_dispatch:
9+
10+
concurrency:
11+
group: ${{ github.workflow }}-${{ github.ref }}
12+
cancel-in-progress: true
13+
14+
env:
15+
STOOLAP_VERSION: main
16+
MARIADB_VERSION: '11.4'
17+
18+
jobs:
19+
# ----------------------------------------------------------------------
20+
# Build libstoolap.{so,dylib} once per OS; share via artifact so the
21+
# plugin build / test jobs don't need a Rust toolchain.
22+
# ----------------------------------------------------------------------
23+
build-lib:
24+
name: Build libstoolap (${{ matrix.target }})
25+
strategy:
26+
fail-fast: false
27+
matrix:
28+
include:
29+
- os: ubuntu-latest
30+
target: x86_64-unknown-linux-gnu
31+
lib: libstoolap.so
32+
- os: macos-latest
33+
target: aarch64-apple-darwin
34+
lib: libstoolap.dylib
35+
runs-on: ${{ matrix.os }}
36+
steps:
37+
- name: Clone stoolap engine
38+
run: git clone --depth 1 --branch $STOOLAP_VERSION https://github.com/stoolap/stoolap.git stoolap-engine
39+
40+
- name: Install Rust
41+
uses: dtolnay/rust-toolchain@stable
42+
43+
- name: Cache Rust build
44+
uses: actions/cache@v4
45+
with:
46+
path: |
47+
stoolap-engine/target
48+
~/.cargo/registry
49+
key: rust-${{ matrix.target }}-${{ env.STOOLAP_VERSION }}-${{ github.sha }}
50+
restore-keys: |
51+
rust-${{ matrix.target }}-${{ env.STOOLAP_VERSION }}-
52+
53+
- name: Build libstoolap
54+
working-directory: stoolap-engine
55+
run: cargo build --release --features ffi
56+
57+
- name: Upload library + header
58+
uses: actions/upload-artifact@v4
59+
with:
60+
name: stoolap-${{ matrix.target }}
61+
path: |
62+
stoolap-engine/target/release/${{ matrix.lib }}
63+
stoolap-engine/include/stoolap.h
64+
if-no-files-found: error
65+
66+
# ----------------------------------------------------------------------
67+
# macOS build + test. Brew's mariadb@11.4 ships the server-internal
68+
# headers in the same layout MARIADB_PREFIX expects, so the only setup
69+
# is `brew install mariadb@11.4`.
70+
# ----------------------------------------------------------------------
71+
test-macos:
72+
name: Test on macOS (MariaDB ${{ env.MARIADB_VERSION }})
73+
needs: build-lib
74+
runs-on: macos-latest
75+
steps:
76+
- uses: actions/checkout@v4
77+
78+
- name: Install MariaDB ${{ env.MARIADB_VERSION }}
79+
run: |
80+
brew install mariadb@${{ env.MARIADB_VERSION }} cmake
81+
MARIADB_PREFIX="$(brew --prefix)/opt/mariadb@${{ env.MARIADB_VERSION }}"
82+
echo "MARIADB_PREFIX=$MARIADB_PREFIX" >> $GITHUB_ENV
83+
echo "$MARIADB_PREFIX/bin" >> $GITHUB_PATH
84+
85+
- name: Download libstoolap
86+
uses: actions/download-artifact@v4
87+
with:
88+
name: stoolap-aarch64-apple-darwin
89+
path: stoolap-staging
90+
91+
- name: Stage stoolap (CMakeLists layout)
92+
run: |
93+
mkdir -p stoolap-fake/include stoolap-fake/target/release
94+
cp stoolap-staging/include/stoolap.h stoolap-fake/include/
95+
cp stoolap-staging/target/release/libstoolap.dylib stoolap-fake/target/release/
96+
97+
- name: Build plugin
98+
run: |
99+
cmake -S . -B build \
100+
-DMARIADB_PREFIX=$MARIADB_PREFIX \
101+
-DSTOOLAP_DIR=$PWD/stoolap-fake
102+
cmake --build build -j
103+
104+
- name: Install plugin
105+
run: install -m 0755 build/ha_stoolap.so $MARIADB_PREFIX/lib/plugin/
106+
107+
- name: Install Python deps
108+
run: python3 -m pip install --break-system-packages --user mysql-connector-python
109+
110+
- name: Run tests
111+
run: python3 tests/runner.py
112+
113+
- name: Upload server log on failure
114+
if: failure()
115+
uses: actions/upload-artifact@v4
116+
with:
117+
name: macos-mariadbd-err
118+
path: /tmp/stoolap-test.err
119+
if-no-files-found: ignore
120+
121+
# ----------------------------------------------------------------------
122+
# Linux build + test. Best-effort while the Ubuntu header story
123+
# stabilises -- MariaDB's apt packages don't always ship the
124+
# server-internal headers (`handler.h` etc.) we need; libmariadbd-dev
125+
# is the closest match. Marked continue-on-error so the macOS job
126+
# gates merges until this lands cleanly.
127+
# ----------------------------------------------------------------------
128+
test-linux:
129+
name: Test on Ubuntu (MariaDB ${{ env.MARIADB_VERSION }})
130+
needs: build-lib
131+
runs-on: ubuntu-latest
132+
continue-on-error: true
133+
steps:
134+
- uses: actions/checkout@v4
135+
136+
- name: Install MariaDB ${{ env.MARIADB_VERSION }} + dev headers
137+
run: |
138+
curl -LsS https://r.mariadb.com/downloads/mariadb_repo_setup \
139+
| sudo bash -s -- --mariadb-server-version=$MARIADB_VERSION --skip-maxscale --skip-tools
140+
sudo apt-get update
141+
sudo apt-get install -y mariadb-server libmariadbd-dev cmake g++ python3-pip
142+
sudo systemctl stop mariadb || true
143+
sudo systemctl disable mariadb || true
144+
145+
- name: Find server-internal headers
146+
id: mariadb
147+
run: |
148+
PREFIX="$RUNNER_TEMP/mariadb-prefix"
149+
mkdir -p "$PREFIX/include/mysql" "$PREFIX/lib/plugin" "$PREFIX/bin"
150+
# libmariadbd-dev installs server-internal headers under
151+
# /usr/include/mysql/server/. Mirror brew's MARIADB_PREFIX
152+
# layout: $PREFIX/include/mysql/server/private/handler.h
153+
if [ -f /usr/include/mysql/server/private/handler.h ]; then
154+
ln -s /usr/include/mysql "$PREFIX/include/mysql"
155+
elif [ -f /usr/include/mariadb/server/private/handler.h ]; then
156+
ln -s /usr/include/mariadb "$PREFIX/include/mysql"
157+
else
158+
echo "ERROR: server-internal handler.h not found in any expected location" >&2
159+
echo " Falling back to building MariaDB headers from source" >&2
160+
ls -la /usr/include/mysql /usr/include/mariadb 2>&1 || true
161+
exit 1
162+
fi
163+
ln -sf /usr/sbin/mariadbd "$PREFIX/bin/mariadbd"
164+
ln -sf /usr/bin/mariadb-install-db "$PREFIX/bin/mariadb-install-db"
165+
ln -sf /usr/bin/mariadb "$PREFIX/bin/mariadb"
166+
for so in /usr/lib/mysql/plugin/*.so; do
167+
[ -e "$so" ] && ln -sf "$so" "$PREFIX/lib/plugin/" || true
168+
done
169+
echo "prefix=$PREFIX" >> $GITHUB_OUTPUT
170+
171+
- name: Download libstoolap
172+
uses: actions/download-artifact@v4
173+
with:
174+
name: stoolap-x86_64-unknown-linux-gnu
175+
path: stoolap-staging
176+
177+
- name: Install libstoolap + stage CMakeLists layout
178+
run: |
179+
sudo install -m 0755 stoolap-staging/target/release/libstoolap.so /usr/local/lib/
180+
sudo ldconfig
181+
mkdir -p stoolap-fake/include stoolap-fake/target/release
182+
cp stoolap-staging/include/stoolap.h stoolap-fake/include/
183+
cp stoolap-staging/target/release/libstoolap.so stoolap-fake/target/release/
184+
185+
- name: Build plugin
186+
run: |
187+
cmake -S . -B build \
188+
-DMARIADB_PREFIX=${{ steps.mariadb.outputs.prefix }} \
189+
-DSTOOLAP_DIR=$PWD/stoolap-fake
190+
cmake --build build -j$(nproc)
191+
192+
- name: Install plugin
193+
run: |
194+
sudo install -m 0755 build/ha_stoolap.so /usr/lib/mysql/plugin/
195+
196+
- name: Install Python deps
197+
run: pip install --user mysql-connector-python
198+
199+
- name: Run tests
200+
env:
201+
MARIADB_PREFIX: ${{ steps.mariadb.outputs.prefix }}
202+
run: python3 tests/runner.py
203+
204+
- name: Upload server log on failure
205+
if: failure()
206+
uses: actions/upload-artifact@v4
207+
with:
208+
name: linux-mariadbd-err
209+
path: /tmp/stoolap-test.err
210+
if-no-files-found: ignore

0 commit comments

Comments
 (0)