[SKETCH] Add support for CrateDB

[amotl]

Introduction

Thanks a stack for conceiving this excellent package. After learning more details about the ingredients of dlt just recently, this library certainly sparks my interest.

About

This module is designed for SQLite but the abstraction pattern can be extended to other database systems by implementing database-specific executors.

This patch attempts to add support for CrateDB. However, it is really just a sketch.

Preview

uv pip install --upgrade \
  'git+https://github.com/crate-workbench/sqlalchemy-upsert-kit.git@cratedb' \
  'sqlalchemy-cratedb>=0.42.0.dev2'

Details

Module

• Because CrateDB does not support CREATE TEMPORARY TABLE ..., usage was disabled, resorting to a regular CREATE TABLE .... Stale temporary tables would need to be pruned by other means with CrateDB.
• Because CrateDB does not support microseconds-granularity, the get_utc_now utility function needed to be adjusted.

Software tests

• Because CrateDB is eventually consistent, a synthetic table refresh after DML operations is provided by the refresh_after_dml SQLAlchemy utility function. Within the scope of this patch, that applies to adjusting test cases only, but it would need to be documented for all users using the library in a regular runtime setting together with CrateDB. It is just a single function call to be invoked after creating an SQLAlchemy engine, but it needs to take place somewhere.
• Because CrateDB does not support transactions, relevant software tests needed to be disabled.
• Because CrateDB returns non-standard result counts, relevant spots in software tests needed to be adjusted.

Problems

Other than disabling a few other software tests, this one still fails. We did not investigate closer about the root cause yet.

FAILED tests/sqlite/test_sqlite_insert_or_merge.py::test_success - AssertionError: assert 'v2' == 'v1'

• MERGE strategy fails with CrateDB: AssertionError: assert 'v2' == 'v1' crate-workbench/sqlalchemy-upsert-kit#1