You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
ddl: Fix default value filling with finer granularity (#10682) (#10688)
This is an automated cherry-pick of #10682
### What problem does this PR solve?
Issue Number: close#10680
Problem Summary:
```SQL
DROP DATABASE IF EXISTS db_1483421321;
CREATE DATABASE db_1483421321;
USE db_1483421321;
CREATE TABLE t0(c0 TINYINT, handle INT NOT NULL AUTO_INCREMENT, PRIMARY KEY(handle));
ALTER TABLE t0 SET TIFLASH REPLICA 1;
-- this SLEEP will make sure the case always reproducable, it not, make it longer
select sleep(5);
SELECT /*+ read_from_storage(tiflash[t0]) */ * FROM t0 order by 1;
SELECT /*+ read_from_storage(tikv[t0]) */ * FROM t0 order by 1;
ALTER TABLE t0 ADD COLUMN c1 DECIMAL NULL;
ALTER TABLE t0 ADD COLUMN c2 FLOAT NULL DEFAULT 0.0795439693286002;
ALTER TABLE t0 ADD COLUMN c5 DECIMAL NOT NULL;
ALTER TABLE t0 ADD COLUMN c4 DECIMAL DEFAULT 247262911;
ALTER TABLE t0 MODIFY COLUMN c2 FLOAT NOT NULL;
ALTER TABLE t0 MODIFY COLUMN c5 DECIMAL NULL;
ALTER TABLE t0 MODIFY COLUMN c1 BIGINT DEFAULT -56083770 NOT NULL;
-- first insert
INSERT IGNORE INTO t0 (c1, c2, c5, c4) VALUES (-2051270087, 0.44141045099028775, 0.0, 15523);
UPDATE t0 SET c5 = 870337888;
ALTER TABLE t0 MODIFY COLUMN c1 BIGINT NULL;
-- second insert
INSERT INTO t0 (c2, c5, c4) VALUES (0.004406799693866592, 0.8752311290235516, 14652);
---- TiFlash data inconsistent with TiKV after modifying a column from NOT NULL to NULL
-- in tiflash, handle=2, c1=0
SELECT /*+ read_from_storage(tiflash[t0]) */ * FROM t0 order by 1;
+--------+--------+-------------+--------------+-----------+-------+
| c0 | handle | c1 | c2 | c5 | c4 |
+--------+--------+-------------+--------------+-----------+-------+
| <null> | 1 | -2051270087 | 0.44141045 | 870337888 | 15523 |
| <null> | 2 | 0 | 0.0044067996 | 1 | 14652 |
+--------+--------+-------------+--------------+-----------+-------+
-- in tikv, handle=2, c1=null
SELECT /*+ read_from_storage(tikv[t0]) */ * FROM t0 order by 1;
+--------+--------+-------------+--------------+-----------+-------+
| c0 | handle | c1 | c2 | c5 | c4 |
+--------+--------+-------------+--------------+-----------+-------+
| <null> | 1 | -2051270087 | 0.44141045 | 870337888 | 15523 |
| <null> | 2 | <null> | 0.0044067996 | 1 | 14652 |
+--------+--------+-------------+--------------+-----------+-------+
```
#### Root cause
- TiDB can omit a column in the row value when the column is NULL and
both DefaultValue and OriginDefaultValue are nil (see TiDB CanSkip).
- If TiFlash is still on the old schema (column is NOT NULL),
addDefaultValueToColumnIfPossible currently accepts the missing column
and fills defaultValueToField().
- When origin_default_value is empty, defaultValueToField() for NOT NULL
falls back to GenDefaultField (zero), so TiFlash returns 0 while TiKV
(new schema) returns NULL.
### What is changed and how it works?
```commit-message
ddl: Fix default value filling with finer granularity
- Tightens addDefaultValueToColumnIfPossible
- For NOT NULL missing columns, force_decode=false now returns false unless there is an origin default.
- This forces a schema sync instead of silently filling 0.
- force_decode=true still fills a default value (best-effort).
```
With old schema (NOT NULL, no origin default), missing column now
triggers schema sync. After schema sync, column becomes nullable, so
missing column can be decode and stored in tiflash with `NULL`, matching
TiKV.
### Check List
Tests <!-- At least one of them must be included. -->
- [x] Unit test
- [x] Integration test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No code
Side effects
- [ ] Performance regression: Consumes more CPU
- [ ] Performance regression: Consumes more Memory
- [ ] Breaking backward compatibility
Documentation
- [ ] Affects user behaviors
- [ ] Contains syntax changes
- [ ] Contains variable changes
- [ ] Contains experimental features
- [ ] Changes MySQL compatibility
### Release note
<!-- bugfix or new feature needs a release note -->
```release-note
Fixed a TiFlash/TiKV mismatch after altering a column from NOT NULL to NULL
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Smarter decoding: missing columns are auto-filled from default or
origin-default values when safe, reducing schema-sync interruptions.
* Better NOT NULL handling across schema/state transitions, improving
data visibility and consistency.
* **Tests**
* Expanded unit and end-to-end tests for default/origin-default
behavior, NOT NULL/nullable transitions, and cross-engine consistency.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>
Signed-off-by: JaySon-Huang <tshent@qq.com>
Co-authored-by: JaySon <tshent@qq.com>
0 commit comments