Store NUMERIC columns as NUMERIC

[florinutz]

sa.Numeric(10, 2) created a BIGINT column. Decimal("1.25") went in and Decimal("1.00") came
back, with no error at any layer — SQLAlchemy re-applies the declared scale on read, so the truncated
integer returned looking like a plausible decimal.

amount = sa.Column(sa.Numeric(10, 2))     # -> amount LONG
session.add(Ledger(amount=Decimal("1.25")))
...
session.query(Ledger).one().amount        # Decimal('1.00')

Closes #292. Also delivers the write half of #163 — see the note on reads below.

Root cause

Two constants in CrateTypeCompiler, added in 2018 when CrateDB had no NUMERIC at all and mapping
elsewhere was the only way to make the column creatable:

def visit_DECIMAL(self, type_, **kw):  return "DOUBLE"
def visit_NUMERIC(self, type_, **kw):  return "LONG"

Both discard type_.precision and type_.scale. CrateDB gained NUMERIC as a cast target in 4.4.0
and storage for it in 5.9.0, and the mapping was never revisited. docs/data-types.rst had since
published it as the intended design.

There is a second, quieter half. The dialect leaves supports_native_decimal at its default of
False, which makes SQLAlchemy's Numeric.bind_processor return processors.to_float — so a
Decimal was already a float before the driver saw it. On a NUMERIC(38, 24) column:

bound as Decimal -> stored 1.234567890123456789012345
bound as float   -> stored 1.234567890123456700000000

Fix

Render NUMERIC(precision, scale), in DDL and in casts alike. NUMERIC and DECIMAL are the same
type in both SQLAlchemy and CrateDB, so visit_DECIMAL defers to visit_NUMERIC; sa.Numeric and
sa.NUMERIC already share that entry point through the generic compiler.

Bind Decimal values unconverted, via a Numeric entry in colspecs, so the driver serializes them
as strings — which it already does — and CrateDB stores every digit. Note this is not
supports_native_decimal = True: that flag also tells SQLAlchemy the DBAPI returns Decimal objects
and to skip result conversion, and this driver returns floats, so setting it would make
Numeric(asdecimal=True) hand back floats.

Two points worth a maintainer's opinion:

• Declaring the type without a precision now raises CompileError. CrateDB rejects a bare
  NUMERIC column ("NUMERIC storage is only supported if precision and scale are specified"), so
  there is nothing correct to emit. A cast target may omit the precision and still does, since the
  server accepts CAST(x AS NUMERIC). The alternatives were to emit bare NUMERIC and let the server
  refuse it at execution, or to keep emitting LONG when no precision is given — the latter would
  leave exactly this bug in place for anyone writing sa.Numeric().
• This makes CrateDB 5.9 the effective minimum for Numeric columns. Users on older servers get a
  working (if lossy) BIGINT today and would start seeing server errors. The repository documents no
  supported-version floor, so I have not invented one — the requirement is stated in CHANGES.md and
  in the type map instead. Say the word if you would rather it degraded on older servers.

Reads are unchanged and remain bounded by the transport: responses are parsed as JSON, whose numbers
become floats, so a value returned from the database carries at most a double's digits. The server
sends the full value, so this is fixable in crate-python rather than here; I will open an issue
there with the measurement, which also answers the question left open on crate/crate-python#652.

Tests

• tests/numeric_test.py — DDL and cast rendering for all three spellings, precision without scale,
  a live round-trip of Decimal("1.25"), and a live write of a value with more digits than a double
  holds. DECIMAL is deliberately tested with such a value: with 1.25 it would pass even while
  rendering DOUBLE, since a double holds 1.25 exactly.
• tests/create_table_test.py — the basic-types table now declares its exact-numeric columns with a
  precision and expects NUMERIC(10, 2).

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 9946c390-061e-4bc9-8109-aab732ec95d4

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}