Running PR to keep track of custom changes to support YB - #127
Open
fourpointfour wants to merge 85 commits into
Open
Running PR to keep track of custom changes to support YB#127fourpointfour wants to merge 85 commits into
fourpointfour wants to merge 85 commits into
Conversation
Initial changes required for the Debezium Connector for Postgres to work with YugabyteDB source.
…st YugabyteDB (#105) This PR includes the changes required for the tests so that they can work against YugabyteDB. YugabyteDB issue: yugabyte/yugabyte-db#21394
Modified Dockerfile to package custom log4j.properties so that the log files can be rolled over when their size exceeds 100MB. Also changed the Kafka connect JDBC jar being used - this new jar has a custom change to log every sink record going to the target database.
Changes in this PR: 1. Modification of Dockerfile to include transformers for aiven at the time of docker image compilation a. Aiven source: https://github.com/Aiven-Open/transforms-for-apache-kafka-connect
…BC driver (#107) ## Problem The Debezium connector for Postgres uses a single host model where the JDBC driver connects to a PG instance and continues execution. However, when we move to YugabyteDB where we have a multi node deployment, the current model can fail in case the node it has connected to goes down. ## Solution To address that, we have made changes in this PR and replaced the Postgres JDBC driver with [YugabyteDB smart driver](https://github.com/yugabyte/pgjdbc) which allows us to specify multiple hosts in the JDBC url so that the connector does not fail or run into any fatal error while maintaining the High Availability aspect of YugabyteDB. Changes in this PR include: 1. Changing of version in `pom.xml` from `2.5.2.Final` to `2.5.2.ybpg.20241-SNAPSHOT` a. This is done to ensure that upon image compilation, the changed code from Debezium Code is picked up. 2. Replacing of all packages from `org.postgresql.*` to `com.yugabyte.*` to comply with the new JDBC driver. 3. Masking the validator method in debezium-core which disallowed characters like `: (colon)` in the configuration property `database.hostname`
**Summary** This PR is to support consistent snapshot in the case of an existing slot. In this case, the consistent_point hybrid time is determined from the pg_replication_slots view, specifically from the yb_restart_commit_ht column. There is an assumption here that this slot has not been used for streaming till this point. If this holds, then the history retention barrier will be in place as of the consistent snapshot time (consistent_point). The snapshot query will be run as of the consistent_point and subsequent streaming will start from the consistent_point of the slot. **Test Plan** Added new test mvn -Dtest=PostgresConnectorIT#initialSnapshotWithExistingSlot test
**Changes:** 1. Providing JMX Exporter jar to KAFKA_OPTS to be further provided to java options. 2. Modifying `metrics.yaml` to include correct regex to be scraped as per Postgres connector.
…peruser (#115) **Summary** This PR adds the support for a non superuser to be configured as the connector user (database.user). Such a user is required to have the privileges listed in https://debezium.io/documentation/reference/2.5/connectors/postgresql.html#postgresql-permissions Specifically, the changes in this revision relate to how the consistent_point is specified to the YugabyteDB server in order to execute a consistent snapshot. **Test Plan** Added new test mvn -Dtest=PostgresConnectorIT#nonSuperUserSnapshotAndStreaming test
…nectorTask (#114) This PR is to add a higher level retry whenever there's a where while starting a PostgresConnectorTask, the failures can include but not limited to the following: 1. Failure of creating JDBC connection 2. Failure to execute query 3. Tserver/master restart 4. Node restart 5. Connection failure
…streaming (#116) ## Problem PG connector does not wait for acknowledgement of snapshot completion offset before transitioning to streaming. This can lead to an issue if there is a connector restart in the streaming phase and it goes for a snapshot on restart. In streaming phase, as soon as the 1st GetChanges call is made on the server, the retention barriers are lifted and so the server can no longer serve the snapshot records on a restart. Therefore it is important that the connector waits for acknowledgement of snapshot completion offset before it actually transitions to streaming. ## Solution This PR introduces a waiting mechanism for acknowledgement of snapshot completion offset before transitioning to streaming. We have introduced a custom heartbeat implementation that will dispatch heartbeat when forced heartbeat method is called but we'll dispatch nothing when a normal heartbeat method is called. With this PR, connector will dispatch heartbeats while waiting for the snapshot completion offset i.e during the transition phase. For these heartbeat calls, there is no need to set the `heartbeat.interval.ms` since we are making forced heartbeat calls which do not rely on this config. Note, this heartbeat call is only required to support applications using debezium engine/embedded engine. It is not required when the connector is run with kakfa-connect. ### Test Plan Manually deployed connector in a docker container and tested two scenarios: 0 snapshot records & non-zero snapshot records. Unit tests corresponding to these scenarios will be added in a separate PR.
#119) **Summary** This PR adds support for the INITIAL_ONLY snapshot mode for Yugabyte. In the case of Yugabyte also, the snapshot is consumed by executing a snapshot query (SELECT statement) . To ensure that the streaming phase continues exactly from where the snapshot left, this snapshot query is executed as of a specific database state. In YB, this database state is represented by a value of HybridTime. Changes due to transactions with commit_time strictly greater than this snapshot HybridTime will be consumed during the streaming phase. This value for HybridTime is the value of the "yb_restart_commit_ht" column of the pg_replication_slots view of the associated slot. Thus, in the case of Yugabyte, even for the INITIAL_ONLY snapshot mode, a slot needs to be created if one does not exist. With this approach, a connector can be deployed in INITIAL_ONLY mode to consume the initial snapshot. This can be followed by the deployment of another connector in NEVER mode. This connector will continue the streaming from exactly where the snapshot left. **Test Plan** 1. Added new test -` mvn -Dtest=PostgresConnectorIT#snapshotInitialOnlyFollowedByNever test ` 2. Enabled existing test - `mvn -Dtest=PostgresConnectorIT#shouldNotProduceEventsWithInitialOnlySnapshot test` 3. Enabled existing test - `mvn -Dtest=PostgresConnectorIT#shouldPerformSnapshotOnceForInitialOnlySnapshotMode test `
… image (#118) This PR adds the dependencies for the `AvroConverter` to function in the Kafka Connect environment. The dependencies will only be added at the time of building the docker image.
This PR adds a log which will be print the IP of the node every time a connection is created.
Retry in case of failures while task is restarting. Right now any kind of failure will lead to task throwing RetriableException exception causing Task restart.
…te source (#120) **Summary** This PR enables 30/33 tests in IncrementalSnapshotIT for Yugabyte source The tests that are excluded are 1. updates 2. updatesLargeChunk 3. updatesWithRestart **Test Plan** `mvn -Dtest=IncrementalSnapshotIT test`
This PR comments out the part in the init_database i.e. the startup script during tests where some extensions are being installed - it is taking more than 2 minutes at this stage and since we do not need it in the tests we use, it can be skipped.
Throw retry for all exceptions. In future, we will need to throw runtime exception for wrong configurations.
|
Hi @vaibhav-yb, thanks for your contribution. Please prefix the commit message(s) with the DBZ-xxx JIRA issue key. |
…r image (#129) This PR only changes the link in the `Dockerfile` to fetch the latest custom sink connector jar from GitHub. According to PR yugabyte/kafka-connect-jdbc#3, changes include the following: 1. Addition of 3 new configuration properties * `log.table.balance`: i. Default is `false` but when set to `true`, the sink connector will execute a query to get the table balance from the target table. ii. Note that this is only applicable for consistency related tests where the given query is applicable - it will fail if set in any other tests. * `expected.total.balance` i. Default is `1000000` (1M) which can be changed to whatever value we are expecting the total balance to be in the target table. * `tables.for.balance` i. This takes a comma separated string value for all the table names from which the sink connector is supposed to extract balances from. ii. This property will only be valid when `log.table.balance` is specified as `true` iii. There is no default for this property so if `log.table.balance` is set to `true` and `tables.for.balance` is not specified then we will throw a `RuntimeException` 2. Log additions to aid debugging.
|
Hi @vaibhav-yb, thanks for your contribution. Please prefix the commit message(s) with the DBZ-xxx JIRA issue key. |
…is set (#131) ## Problem PG connector filters out record based on its starting point (WAL position), which in turn depends on the offset received from Kafka. So, in case, the starting point corresponds to a record in the middle of a transaction, PG connector will filter out the records of that transaction with LSN < starting point. This creates a problem in the downstream pipeline expects consistency of data. Filtering of records leads to PG connector shipping transaction with missing records. When such a transaction is applied on the sink, consistency breaks. ## Solution When 'provide.transaction.metadata' connector configuration is set, PG connector ships transaction boundary records BEGIN/COMMIT. Based on these boundary records, sink connector writes data maintaining consistency. Therefore, when this connector config is set, we will disable filtering records based on WAL Resume position. ## Testing Manually testing - Ran the connector with this fix in our QA runs where the issue was first discovered. All 10 runs triggered passed. Unit testing - Cannot reproduce the above mentioned scenario in a unit test.
|
Hi @vaibhav-yb, thanks for your contribution. Please prefix the commit message(s) with the DBZ-xxx JIRA issue key. |
|
Hi @vaibhav-yb, thanks for your contribution. Please prefix the commit message(s) with the DBZ-xxx JIRA issue key. |
…rds with CHANGE (#106)
|
Hi @vaibhav-yb, thanks for your contribution. Please prefix the commit message(s) with the DBZ-xxx JIRA issue key. |
This PR adds a configuration to let the user disable consistent snapshot i.e. `yb.consistent.snapshot` to the connector which is enabled by default. Setting consistent snapshot means setting/establishing the boundary between snapshot records (records that existed at the time of stream creation) and streaming records. If `yb.consistent.snapshot` is disabled i.e. set to `false`: - We will not be setting a boundary for snapshot - If the connector restarts after taking a snapshot but before acknowledging the streaming LSN, the snapshot will be taken again. This can result in some records being received both during the snapshot phase and the streaming phase.
|
Hi @fourpointfour, thanks for your contribution. Please prefix the commit message(s) with the DBZ-xxx JIRA issue key. |
…7 in YB connector (#190) ## Problem The current YB connector is based on Java 11. This PR upgrades the Java version and necessary dependencies to Java 17-compatible versions. The upstream Debezium project upgraded to Java 17 in version 3.0. While we are currently on version 2.5.2, upgrading to Java 17 in this version is safe and beneficial because it simplifies cherry-picking from upstream patches from upstream (which uses Java 17) will not require Java version modifications. ## Solution To upgrade the Java version we will do the following: - Update `pom.xml` and change the min. jdk version and maven release version - Update `debezium-schema-generator/pom.xml` and change maven-plugin-plugin from 3.6.0 to 3.9.0. Required because 3.6.0 uses ASM 7.x which cannot read Java 17 bytecode. - Update `Dockerfile` - Install Java 17 runtime ## Test Plan Manual testing + Few existing
|
Hi @fourpointfour, thanks for your contribution. Please prefix the commit message(s) with the DBZ-xxx JIRA issue key. |
## Problem The current YugabyteDB connector is based on Debezium version 2.5.2, which does not include support for pgVector. ## Solution We will cherry pick the upstream commit debezium@7cf7af5 which adds support for pgvector in Postgres Connector in version 3.0: Test Compatibility - Updated VectorDatabaseIT.java to use YugabyteDB-specific classes: - PostgresConnector.class → YugabyteDBConnector.class - SnapshotMode.NO_DATA → SnapshotMode.NEVER Additional changes: For Java 17: - Upgrade impsort version to 1.12.0 in `pom.xml` - Upgrade format.imports.source.compliance to 17 in `debezium-parent/pom.xml` ## Test Plan ``` mvn -pl debezium-connector-postgres -Dtest=VectorDatabaseTest,VectorDatabaseIT test ``` --------- Co-authored-by: Jiri Pechanec <jpechane@redhat.com>
|
Hi @fourpointfour, thanks for your contribution. Please prefix the commit message(s) with the DBZ-xxx JIRA issue key. |
…OT TO EXPORT_SNAPSHOT (#188) ## Problem With 2025.2.2 and later branches, we plan to enable EXPORT_SNAPSHOT by default instead of USE_SNAPSHOT. However, we need to maintain backward compatibility with old branches where we didn't have EXPORT_SNAPSHOT on by default and needed a g-flag to be set to true for it to even work. Otherwise, it ends up throwing the below error. ``` ERROR: cannot export or import snapshot when ysql_enable_pg_export_snapshot is disabled. ``` Recommended to read [https://github.com/yugabyte/debezium/pull/113](https://github.com/yugabyte/debezium/pull/113) for getting a better context of the change. ## Solution 1. Add a variable (`exportSnapshotUsed`) to `SlotCreationResult` which will tell the connector if the slot was created with EXPORT_SNAPSHOT or USE_SNAPSHOT. 2. First, to maintain backward compatibility with versions where the flag `ysql_enable_pg_export_snapshot` was not enabled, we will add a fallback mechanism. i. First, the connector will try to create a replication slot by explicitly mentioning `EXPORT_SNAPSHOT` in the command (or not if parallel streaming is off). If this fails with an error `cannot export or import snapshot when ysql_enable_pg_export_snapshot is disabled.`, then we fallback to the old code of the connector which will create a replication slot by explicitly mentioning `USE_SNAPSHOT` (or not if parallel streaming is off). ii. **Note**: Even if we don't pass anything in the create replication slot command, it uses the default, which in 2025.2.2 and above versions is `EXPORT_SNAPSHOT` and in 2025.2.1 and below versions is `USE_SNAPSHOT`. 3. When a new replication slot is created, we store all the data in `slotCreatedInfo` including whether EXPORT_SNAPSHOT was used. i. For versions where `EXPORT_SNAPSHOT` is enabled, `slotCreatedInfo.isExportSnapshotUsed()` will be `true` and snapshot_name will be the snapshot ID. ``` yugabyte=# CREATE_REPLICATION_SLOT test_replication_slot_99d2fc58624f4aa29f1317d434e LOGICAL pgoutput; slot_name | consistent_point | snapshot_name | output_plugin ---------------------------------------------------+------------------+-------------------------------------------------------------------+--------------- test_replication_slot_99d2fc58624f4aa29f1317d434e | 0/2 | e928c3c6c4d54deca3c449dae1e00233-ae0666b99cc6d3be93494b0d1022eb8f | pgoutput ``` and then use this snapshot ID to set it in the snapshot session: ``` String setSnapshotQuery = "SET TRANSACTION SNAPSHOT '" + slotCreatedInfo.snapshotName() + "';"; jdbcConnection.executeWithoutCommitting(setSnapshotQuery); ``` ii. For versions where `EXPORT_SNAPSHOT` is **not** enabled, snapshot_name will be the hybrid time. ``` yugabyte=# CREATE_REPLICATION_SLOT test_replication_slot_99d2fc58624f4aa29f1317d434e4831c LOGICAL pgoutput; slot_name | consistent_point | snapshot_name | output_plugin --------------------------------------------------------+------------------+---------------------+--------------- test_replication_slot_99d2fc58624f4aa29f1317d434e4831c | 0/2 | 7227389791448002560 | pgoutput ``` and then we use this hybrid time to set the local read time of the snapshot session (Note: This is pre-existing code): ``` private String ybSnapshotStatement(String ybReadTime) { return String.format("DO " + "LANGUAGE plpgsql $$ " + "BEGIN " + "SET LOCAL yb_read_time TO '%s ht'; " + "EXCEPTION " + "WHEN OTHERS THEN " + "CALL set_yb_read_time('%s ht'); " + "END $$;", ybReadTime, ybReadTime); } ``` 4. For connector restart case: i. If the slot is re-created (parallel streaming mode drops and recreates the slot on mid-snapshot restart): Steps 1, 2 & 3 will be repeated. ii. If the slot already exists (slotCreatedInfo is null): a. For EXPORT_SNAPSHOT, we cannot use the old snapshot ID as the moment the session closes, the snapshot is deleted, and when the connector is restarted, there is no way to access it. So for this case, we will fallback to the old code where we set the hybrid time from the slot (`slotState.slotRestartCommitHT()`) as the local read time. ## Test Plan ``` ./mvnw test -pl debezium-connector-postgres -Dtest="PostgresConnectorIT#shouldStreamWithExportSnapshotDisabledAndPreCreatedSlotWithUseSnapshot,PostgresConnectorIT#shouldStreamWithDefaultFlagsAndConnectorCreatedSlot,PostgresConnectorIT#shouldStreamWithDefaultFlagsAndPreCreatedSlotViaReplicationConnection" ```
|
Hi @fourpointfour, thanks for your contribution. Please prefix the commit message(s) with the DBZ-xxx JIRA issue key. |
… in Yugabyte Connector (#191) ## Problem PostgreSQL's logical replication protocol includes ORIGIN messages that identify the source of transactions in cascaded replication setups. This is particularly important for hub-and-spoke replication topologies where changes may originate from different clusters and need to be tracked to prevent replication loops. **Current Limitation: When a PostgreSQL connector receives an ORIGIN message, it simply acknowledges and skips it without processing.** Read more about the Origin message format at [Logical Replication Message Formats](https://www.postgresql.org/docs/12/protocol-logicalrep-message-formats.html) ## Solution Extend the PostgreSQL connector to capture and propagate replication origin metadata from PostgreSQL's logical replication stream. ### Implementation Details 1. **Add ORIGIN as a ReplicationMessage Operation** - Added `ORIGIN` to the `ReplicationMessage.Operation` enum - Created `OriginMessage` class (similar to `TransactionMessage`) to represent ORIGIN messages 2. **Parse and Emit ORIGIN Messages in PgOutputMessageDecoder** - Parse `origin_name` and `origin_lsn` when an ORIGIN message is received - ORIGIN message format: Int64 (origin LSN) followed by null-terminated string (origin name) - Emit `OriginMessage` to the processor instead of caching values in the decoder 3. **Handle ORIGIN in PostgresStreamingChangeEventSource** - Process `Operation.ORIGIN` messages to update origin state in `PostgresOffsetContext` - ORIGIN messages are "swallowed" (not dispatched to EventDispatcher) - they only update state - Subsequent DML events automatically include origin info from `SourceInfo` 4. **Propagate Origin to Change Events** - `SourceInfo` stores origin metadata (`originName`, `originLsn`) - `PostgresSourceInfoStructMaker` adds optional `origin` and `origin_lsn` fields to the source schema - All DML events within a transaction include the origin info in their source block 5. **Restart Recovery Handling** - Always process ORIGIN messages even during WAL position recovery (skip phase) - `shouldMessageBeSkipped()` explicitly returns `false` for ORIGIN messages - This ensures that when replaying from transaction BEGIN after a crash, the origin state is populated before processing resumed events Example: ``` { "before": null, "after": { "id": { "value": 48417, "set": true } }, "source": { "version": "dz.2.5.2.yb.2025.1.SNAPSHOT.3", "connector": "postgresql", "name": "dbserver2", "ts_ms": 1767343556846, "snapshot": "false", "db": "yugabyte", "sequence": "[\"5\",\"38423\"]", "schema": "public", "table": "t1", "txId": 3, "lsn": 38423, "xmin": null, "origin": "origin1", // ← New field added "origin_lsn": 100006 // ← New field added }, "op": "c", "ts_ms": 1767343722393, "transaction": null } ``` ## Test plan ``` # Unit tests SourceInfoTest#originInfoIsNullByDefault+originInfoCanBeSetAndCleared+originInfoIncludedInToString" # Integration tests YBRecordsStreamProducerIT#shouldIncludeOriginInfoInSourceMetadataWhenOriginIsSet+shouldHaveNullOriginInfoWhenNoOriginIsSet+shouldNotLeakOriginInfoBetweenTransactions+shouldCorrectlyTrackDifferentOriginsAcrossTransactions+shouldPreserveOriginInfoAfterConnectorRestartMidTransaction ``` --------- Signed-off-by: Shishir Sharma <ssharma@yugabyte.com>
|
Hi @fourpointfour, thanks for your contribution. Please prefix the commit message(s) with the DBZ-xxx JIRA issue key. |
…hot is disabled (#192) ## Problem When ysql_enable_pg_export_snapshot & streamingMode.isParallel() are disabled, the current implementation doesn't properly fall back to the alternative snapshot mechanism, resulting in the error below. ``` Caused by: com.yugabyte.util.PSQLException: ERROR: cannot export or import snapshot when ysql_enable_pg_export_snapshot is disabled. ``` ## Solution Determine snapshot value based on canExportSnapshot; if false, fall back to streamingMode.isParallel()
|
Hi @fourpointfour, thanks for your contribution. Please prefix the commit message(s) with the DBZ-xxx JIRA issue key. |
…tXLogLocation() for YugabyteDB (#193) ## Problem With the commit yugabyte/yugabyte-db@f1d0a1fb6158 we have disabled `pg_current_wal_lsn()` since each tablet maintains its own WAL, making a single global WAL LSN is unsupported. Before this change, the function returned a garbage value (e.g. 0/1000118) that was never meaningful for YugabyteDB. Now calling it throws an error, causing the connector to fail during the streaming init phase: ``` Caused by: com.yugabyte.util.PSQLException: ERROR: pg_current_wal_lsn() is not yet supported Hint: See yugabyte/yugabyte-db#30243. React with thumbs up to raise its priority. ``` ## Solution Skip calling `currentXLogLocation()` in `initialContext()` when YugabyteDB is enabled, passing `null` as the LSN instead. This is safe because the value is never meaningfully used for YugabyteDB: - **Snapshot path** (`determineSnapshotOffset`): The LSN from `initialContext()` is immediately overwritten by `updateOffsetForSnapshot()` → `getTransactionStartLsn()`, which already has a YB-specific path returning `slotLastFlushedLsn()`. - **Streaming init path** (when `offsetContext` is null): The streaming start position is determined by the replication slot, not this value (`hasStartLsnStoredInContext` is false → `replicationConnection.startStreaming(walPosition)` without an LSN). - **Pre-snapshot catch-up streaming** (`updateOffsetForPreSnapshotCatchUpStreaming`): Never entered because `shouldStreamEventsStartingFromSnapshot()` defaults to `true` and no snapshotter overrides it. - **`getTransactionStartLsn()` fallback**: Never reached for YugabyteDB due to the early return at the `YugabyteDBServer.isEnabled()` branch. ## Test Plan ``` mvn test -pl debezium-connector-postgres -Dtest=PostgresConnectionIT#shouldReportValidXLogPos ``` <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Medium Risk** > Changes startup offset initialization behavior by allowing a `null` LSN for YugabyteDB, which could affect any code paths that implicitly assume a non-null starting LSN. > > **Overview** > Prevents the connector from failing during streaming initialization on YugabyteDB by skipping `currentXLogLocation()` in `PostgresOffsetContext.initialContext()` and initializing the starting LSN as `null` when `YugabyteDBServer.isEnabled()`. > > Updates `PostgresConnectionIT#shouldReportValidXLogPos` to assert that `currentXLogLocation()` throws a `SQLException` with the expected “pg_current_wal_lsn() is not yet supported” message (reflecting YugabyteDB behavior). > > <sup>Written by [Cursor Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit e7f550a. This will update automatically on new commits. Configure [here](https://cursor.com/dashboard?tab=bugbot).</sup> <!-- /CURSOR_SUMMARY -->
|
Hi @fourpointfour, thanks for your contribution. Please prefix the commit message(s) with the DBZ-xxx JIRA issue key. |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 4 potential issues.
Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.
This is the final PR Bugbot will review for you during this billing cycle
Your free Bugbot reviews will reset on February 25
Details
Your team is on the Bugbot Free tier. On this plan, Bugbot will review limited PRs each billing cycle for each member of your team.
To receive Bugbot reviews on all of your PRs, visit the Cursor dashboard to activate Pro and start your 14-day free trial.
|
|
||
| protected boolean isRetryRemaining() { | ||
| return (connectorConfig.getMaxRetriesOnError() == -1) || getRetries() <= connectorConfig.getMaxRetriesOnError(); | ||
| } |
There was a problem hiding this comment.
Off-by-one in retry count comparison logic
Medium Severity
isRetryRemaining() uses getRetries() <= getMaxRetriesOnError() while the base class hasMoreRetries() uses retries < maxRetries. When retries equals maxRetries, setProducerThrowable calls hasMoreRetries() which returns false and queues a ConnectException, but then isRetryRemaining() returns true, causing the start() catch block to throw a RetriableException. This allows one extra retry beyond the configured maximum.
Additional Locations (1)
|
|
||
| public List<String> getSlotRanges() { | ||
| return List.of(getConfig().getString(SLOT_RANGES).trim().split(";")); | ||
| } |
There was a problem hiding this comment.
NPE in parallel config getters with no defaults
Medium Severity
getSlotNames(), getPublicationNames(), and getSlotRanges() call .trim() on the result of getConfig().getString() which returns null when the corresponding fields (SLOT_NAMES, PUBLICATION_NAMES, SLOT_RANGES) are not configured, since they have no default values. The validation only checks that these fields aren't used outside parallel mode — it doesn't require them when streaming.mode is parallel. This causes a NullPointerException if a user enables parallel streaming without setting these fields.
| public SchemaBuilder datatypeSparseVectorSchema() { | ||
| return SchemaBuilder.struct() | ||
| .name(SparseVector.LOGICAL_NAME) | ||
| .name(SparseVector.LOGICAL_NAME) |
There was a problem hiding this comment.
Duplicate .name() call in sparse vector schema
Low Severity
datatypeSparseVectorSchema() calls .name(SparseVector.LOGICAL_NAME) twice in succession on the SchemaBuilder. This is a redundant duplicate call that appears to be a copy-paste error.
| @Override | ||
| public Map<String, String> getSourcePartition() { | ||
| return Collect.hashMapOf(SERVER_PARTITION_KEY, serverName); | ||
| return Collect.hashMapOf(SERVER_PARTITION_KEY, getPartitionIdentificationKey()); |
There was a problem hiding this comment.
Partition equality inconsistent with source partition key
Medium Severity
getSourcePartition() now returns a key derived from serverName, taskId, and slotName via getPartitionIdentificationKey(), but equals() and hashCode() still compare only serverName. In parallel mode, partitions with different taskId/slotName are considered equal by equals() and produce the same hashCode(), despite having distinct source partition keys used for offset storage. This semantic inconsistency could cause subtle bugs when partitions are used in hash-based collections.
Additional Locations (1)
…tor resumes after table drop (#195) ## Problem `PgOutputMessageDecoder.handleRelationMessage()` resolves primary key columns by querying the live database via `connection.readPrimaryKeyNames()`. When the connector is stopped, rows are inserted (accumulating in WAL), the table is dropped, and the walsender is killed, upon restart the connector establishes a fresh replication connection, receives a new relation message for the dropped table, and the PK query returns empty. This results in all pending Kafka records being produced with a **null key**. The WAL relation message already carries a per-column `flags` byte that identifies PK columns, but the existing code reads it without using it for PK resolution. ## Solution Refactor PK resolution in `handleRelationMessage()` to use the relation message `flags` byte instead of querying the database. ### Implementation Details 1. **Use Relation message flags for DEFAULT/INDEX/CHANGE replica identities** - Derive `primaryKeyColumns` from the relation message `flags` byte (`flags & 1 == 1` means PK column) - No out-of-band DB query needed — works even if the table has been dropped 2. **Fall back to DB query for FULL/NOTHING replica identities** - `FULL` sets all flags to 1, `NOTHING` sets all to 0, flags are ambiguous for PK resolution - Query `connection.readPrimaryKeyNames()` only for these cases 3. **Backward-compatible CHANGE fallback** - YugabyteDB's `yboutput` may not set flags for CHANGE identity in older versions (YB#22555) - If CHANGE yields an empty PK list from flags, fall back to a DB query 4. **Remove unnecessary out-of-band DB queries** - Removed queries for `columnDefaults` and `columnOptionality`, use safe defaults (`optional=true`, `hasDefault=false`) instead - Extracted `queryPrimaryKeysFromDatabase()` and `parseReplicaIdentity()` helper methods 5. **Integration tests (`YBHandleRelationMessageIT`)** - PK resolution tests for DEFAULT, FULL, and CHANGE replica identities - Table-drop + connector restart scenario: stop connector, insert rows, drop table, kill walsender, restart, verify all records have non-null keys - Red-green verified: test fails with null key on original code, passes with the fix ### Trade-off: Column metadata defaults The previous code queried the database for per-column optionality (`IS NOT NULL` constraints) and default value expressions. With this change, all columns default to `optional=true` and `hasDefault=false`. This means the connector's schema metadata for columns will not reflect `NOT NULL` constraints or default values from the database. This is an acceptable trade-off because: - These metadata fields do not affect the Kafka record key (the core fix) - The relation message protocol does not carry optionality or default info - Querying the DB for this metadata is what causes the failure when the table is dropped ## Test plan ``` YBHandleRelationMessageIT#shouldResolvePkWithReplicaIdentityDefault YBHandleRelationMessageIT#shouldResolvePkWithReplicaIdentityFull YBHandleRelationMessageIT#shouldResolvePkWithReplicaIdentityChange YBHandleRelationMessageIT#shouldResolvePkAfterTableDropWithReplicaIdentityDefault YBHandleRelationMessageIT#shouldResolvePkAfterTableDropWithReplicaIdentityFull YBHandleRelationMessageIT#shouldResolvePkAfterTableDropWithReplicaIdentityChange ``` <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Medium Risk** > Changes primary-key resolution during pgoutput relation decoding and removes DB-derived column metadata, which can affect emitted record keys and schema details during streaming/restarts. Adds new Yugabyte-focused integration tests covering multiple replica identities and restart-after-drop scenarios. > > **Overview** > Fixes a restart edge case where pending WAL events could be emitted with **null Kafka keys** after a table is dropped by changing `PgOutputMessageDecoder.handleRelationMessage()` to derive primary-key columns from the replication relation message `flags` (with DB-metadata fallback for `FULL/NOTHING`, and a `CHANGE` fallback for older `yboutput`). > > Reduces reliance on out-of-band metadata by dropping per-column default/optionality lookups and defaulting columns to `optional=true` / `hasDefault=false`, and adds `YBHandleRelationMessageIT` to verify non-null keys across replica identities and a stop/insert/drop/restart scenario. > > <sup>Written by [Cursor Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit 4052df0. This will update automatically on new commits. Configure [here](https://cursor.com/dashboard?tab=bugbot).</sup> <!-- /CURSOR_SUMMARY -->
|
You have used all of your free Bugbot PR reviews. To receive reviews on all of your PRs, visit the Cursor dashboard to activate Pro and start your 14-day free trial. |
|
Hi @fourpointfour, thanks for your contribution. Please prefix the commit message(s) with the DBZ-xxx JIRA issue key. |
…TION for filtered publication (#196) ## Problem When `publication.autocreate.mode=filtered` is configured, the connector unconditionally executes `ALTER PUBLICATION ... SET TABLE ...` on **every restart**, even when the publication's table list has not changed. This causes unnecessary DDL operations against the database on each connector startup, which can: - Cause unnecessary lock contention on the database (ALTER PUBLICATION acquires locks) - Potentially interfere with ongoing database operations. ## Solution Cherry-pick of upstream Debezium [DBZ-9395](https://issues.redhat.com/browse/DBZ-9395) ([PR debezium#6709](debezium#6709), merged as [a8e1557](debezium@a8e1557)). Before issuing `ALTER PUBLICATION`, query `pg_publication_tables` to compare the publication's current table set against the desired captured tables. Only issue the DDL when the sets actually differ. ### Modifications from upstream The upstream PR targets Debezium 3.x (`main` branch) which has diverged significantly from our 2.5.2 fork. A direct `git cherry-pick` was not possible due to: - **Missing upstream-only code**: The 3.x codebase has `isReadOnlyDb()`, `SQL_LOCK_NOT_AVAILABLE`, `validatePublications()`, `executeWithTimeout()`, and `createSlotCommandTimeout()` none of which exist in 2.5.2 - **Method name difference**: Our fork has `createOrUpdatePublicationModeFilterted` (typo preserved from 2.5.2 base), upstream fixed it to `createOrUpdatePublicationModeFiltered` - **`setQueryTimeout` → `statement_timeout` replacement**: The upstream PR also replaced client-side `Statement.setQueryTimeout()` with server-side `SET statement_timeout`. This part is **not applicable** to our fork since we don't use `setQueryTimeout` in this code path The following parts were manually adapted for 2.5.2: 1. **`getCurrentPublicationTables(Statement)`**: queries `pg_publication_tables` for the publication's current table list. Uses `jdbcConnection.createTableId()` (returns `TableId(null, schema, table)`) to match the format from `determineCapturedTables()` for correct `Set.equals()` comparison 2. **`isPublicationUpdateRequired(Statement)`**: compares current vs desired tables, logs additions/removals, returns `false` if already in sync 3. **Guard in `initPublication()`**: wraps the existing `createOrUpdatePublicationModeFilterted(stmt, true)` call with the `isPublicationUpdateRequired()` check 4. **`mockito-core` test dependency**: added to `debezium-connector-postgres/pom.xml` (version managed by BOM, was not previously a direct dependency of this module) ### Implementation Details - `getCurrentPublicationTables()` returns `Optional.empty()` on SQL exceptions (e.g., insufficient privileges), causing the guard to fall through to `ALTER PUBLICATION` as a safe default - `isPublicationUpdateRequired()` returns `false` when desired tables are empty to avoid clearing the publication - Detailed logging of which tables need to be added/removed when an update is required ## Test plan ``` PostgresPublicationTableComparisonTest#testPublicationUpdateRequiredWhenTablesAdded PostgresPublicationTableComparisonTest#testPublicationUpdateRequiredOnSQLException PostgresPublicationTableComparisonTest#testPublicationUpdateRequiredWhenPublicationEmpty PostgresPublicationTableComparisonTest#testNoUpdateRequiredWhenTablesMatch PostgresPublicationTableComparisonTest#testPublicationUpdateRequiredWhenTablesDiffer PostgresPublicationTableComparisonTest#testPublicationUpdateRequiredWhenQueryFails PostgresPublicationTableComparisonTest#testNoUpdateRequiredWhenNoDesiredTables ``` <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Medium Risk** > Changes publication initialization logic to conditionally run `ALTER PUBLICATION` based on live catalog comparison, which affects startup behavior and DDL execution in a replication-critical code path. > > **Overview** > Avoids running `ALTER PUBLICATION ... SET TABLE ...` on every restart when `publication.autocreate.mode=filtered` by querying `pg_publication_tables` and only updating the publication when the current and desired table sets differ (with a safe fallback to update if the query fails). > > Adds unit coverage for the table-set comparison logic and includes `mockito-core` as a test dependency to support the new tests. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 63d4d59. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY -->
|
You have used all of your free Bugbot PR reviews. To receive reviews on all of your PRs, visit the Cursor dashboard to activate Pro and start your 14-day free trial. |
|
Hi @fourpointfour, thanks for your contribution. Please prefix the commit message(s) with the DBZ-xxx JIRA issue key. |
… tables with non-FULL replica identity (#197) ## Problem When a table without a primary key is under a CDC publication, UPDATE and DELETE records may reach the connector even though they lack sufficient key data to identify the affected row. This happens when: - The `yb_cdcsdk_allow_dml_without_pk` GUC is set to `true`, allowing DMLs through on non-PK tables. - The table's replica identity is changed to FULL after the CDC stream was created — the server allows DMLs, but the stream still has the original non-FULL replica identity. In both cases, the streamed UPDATE/DELETE records don't have enough information for downstream consumers. ## Solution Add connector-side filtering to skip UPDATE/DELETE records for tables that have no primary key and a non-FULL stream replica identity. INSERTs are never filtered. Added a `shouldFilterNoPkRecord`heck in `PostgresStreamingChangeEventSource.processReplicationMessages()`, after `offsetContext.updateWalPosition()` and before `dispatcher.dispatchDataChangeEvent()`. When the check triggers, a log message is emitted at INFO level with the operation type, table name, and stream replica identity value, and the record is skipped. The filtering only applies when running against YugabyteDB (`YugabyteDBServer.isEnabled()`). ## Test Plan YugabyteReplicaIdentityIT#shouldFilterUpdateAndDeleteForNoPkTableWithNonFullRI YugabyteReplicaIdentityIT#shouldNotFilterUpdateAndDeleteForNoPkTableWithFullRI YugabyteReplicaIdentityIT#shouldNotFilterInsertForNoPkTableRegardlessOfRI YugabyteReplicaIdentityIT#shouldBlockUpdateAndDeleteForNoPkTableWithFlagFalse YugabyteReplicaIdentityIT#shouldFilterUpdateAndDeleteForNoPkTableWithDefaultRIAndFlagTrue YugabyteReplicaIdentityIT#shouldFilterUpdateDeleteAfterAlterToFullBecauseStreamRIIsStale YugabyteReplicaIdentityIT#shouldFilterUpdateDeleteForDefaultRIAfterAlterToFull YugabyteReplicaIdentityIT#shouldVerifyStreamReplicaIdentityAfterAlterToFull <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Medium Risk** > Changes streaming dispatch behavior by dropping certain UPDATE/DELETE events for YugabyteDB, which could affect downstream data completeness if the filtering conditions are mis-detected. Scoped to no-primary-key tables with non-`FULL` stream replica identity and covered by new integration tests. > > **Overview** > Prevents YugabyteDB CDC from emitting ambiguous `UPDATE`/`DELETE` events by **skipping dispatch** when the target table has *no primary key* and the stream’s replica identity is **non-`FULL`** (in `PostgresStreamingChangeEventSource`). Inserts and `FULL` replica identity tables continue to flow normally. > > Adds *rate-limited debug logging* to track how many events were filtered, and extends `YugabyteReplicaIdentityIT` with new scenarios for filtering vs non-filtering behavior (including stale stream replica identity after `ALTER ... REPLICA IDENTITY FULL`) plus more resilient replication slot cleanup via backend termination retries. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 36a808a. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY -->
|
Hi @fourpointfour, thanks for your contribution. Please prefix the commit message(s) with the DBZ-xxx JIRA issue key. |
## Problem The PgOutput relation-message handler contained a fallback path that issued an out-of-band DB query whenever CHANGE replica identity yielded no primary-key columns from the relation message flags. This was originally added while testing table rewrite, since older yboutput builds did not set the PK flag for CHANGE identity (YB#22555), leaving Debezium without a key. The fallback is no longer needed and is in fact undesirable: The table-rewrite feature is only supported from YugabyteDB 2026.1, and that version already includes the fix for yboutput correctly setting the PK flag for CHANGE replica identity in relation messages. In any prior version, dropping/altering a primary key is not possible, so the "missing PK on CHANGE" condition that motivated the fallback cannot occur. The fallback issues a synchronous metadata query inline with relation-message processing, which is undesirable on a hot path that runs frequently. ## Solution Remove the CHANGE-identity DB fallback from PgOutputMessageDecoder#decodeRelation. The flags byte in the relation message is now the sole source of PK information for DEFAULT, INDEX, and CHANGE replica identities (matching upstream behavior); FULL / NOTHING continue to use the DB-backed lookup as before, and queryPrimaryKeysFromDatabase is retained for that path. ## Test plan Existing PgOutput relation-message tests continue to pass.
|
Hi @fourpointfour, thanks for your contribution. Please prefix the commit message(s) with the DBZ-xxx JIRA issue key. |
…o.known.offset.on.start property (#200) ## Problem The internal configuration property `slot.seek.to.known.offset.on.start` is not supported by the YugabyteDB connector. When enabled, the connector calls `validateSlotIsInExpectedState()`, which in turn invokes `pg_replication_slot_advance()` to seek the replication slot to the last known offset. This mechanism is not applicable to YugabyteDB's virtual WAL / CDC implementation, and leaving the option available lets users configure a code path that cannot work correctly against YugabyteDB. Today the connector silently accepts the property and attempts to use it, instead of telling the user it is unsupported. ## Solution Reject the property explicitly and make the unsupported code path unreachable for YugabyteDB, with defense in depth across the three layers where the option is consumed: - Start path: `YugabyteDBConnector.start()` now calls a new `rejectUnsupportedProperties()` helper that throws a `DebeziumException` if `slot.seek.to.known.offset.on.start` is present in the configuration (regardless of its value). This transitions the connector to the FAILED state with a clear message and prevents any task from starting. Following existing precedent in the connector, the thrown `DebeziumException` is not subject to the task-level retry logic in `PostgresErrorHandler`, so a configuration that can never succeed is not retried. - Validate path: `YugabyteDBConnector.validateConnection()` now surfaces the same condition as a per-property validation error (added to the `ConfigValue` map) before any database connection is attempted, so the failure is reported by the Kafka Connect `/validate` REST endpoint before the connector is even created. Both layers share a single `unsupportedPropertyMessage()` helper so the wording stays consistent. - Runtime path: `PostgresReplicationConnection.startStreaming()` now guards the `validateSlotIsInExpectedState()` call with `!YugabyteDBServer.isEnabled()`, so even if the flag were somehow set, the `pg_replication_slot_advance()` path is skipped entirely when running against YugabyteDB. The error message reads: `Configuration property 'slot.seek.to.known.offset.on.start' is not supported. Please remove it from the connector configuration.` ## Test Plan YBValidateTest#shouldRejectUnsupportedSlotSeekToKnownOffsetProperty YBValidateTest#shouldRejectUnsupportedSlotSeekToKnownOffsetPropertyEvenWhenFalse YBValidateTest#shouldSurfaceUnsupportedPropertyAsValidationError
|
Hi @fourpointfour, thanks for your contribution. Please prefix the commit message(s) with the DBZ-xxx JIRA issue key. |
…avior during streaming phase (#198) ## Problem The YugabyteDB heartbeat implementation only dispatches heartbeats during the snapshot-to-streaming transition phase via `forcedBeat()`. The regular timer-driven `heartbeat()` method is a no-op, so during the streaming phase neither `heartbeat.interval.ms` nor `heartbeat.action.query` has any effect on YugabyteDB. ## Solution Extend the YB-specific heartbeat to support upstream behavior during the streaming phase, while keeping snapshot-phase behavior and the snapshot-to-streaming transition behavior unchanged. - `YBHeartbeatImpl`: `heartbeat(...)` now delegates to the upstream timer-driven path when `heartbeat.interval.ms > 0` and the offset shows snapshot is no longer in effect. `forcedBeat(...)` is unchanged. - `YBDatabaseHeartbeatImpl` (new): extends `DatabaseHeartbeatImpl` with the same streaming-only gate on `heartbeat(...)`. Inherited `forcedBeat(...)` runs `heartbeat.action.query` on each emitted heartbeat. - `PostgresConnectorConfig.createHeartbeat`: returns the no-op heartbeat only when neither the transition wait nor streaming heartbeats are needed; picks `YBDatabaseHeartbeatImpl` when an interval and an action query are both configured; falls back to `YBHeartbeatImpl` otherwise. ### Behavior after this change - Snapshot phase: no change. - Snapshot-to-streaming transition wait: no change. - Streaming phase: when `heartbeat.interval.ms > 0`, periodic heartbeat records are produced; when `heartbeat.action.query` is also set, the query runs on each tick.
|
Hi @fourpointfour, thanks for your contribution. Please prefix the commit message(s) with the DBZ-xxx JIRA issue key. |
…ution for CHANGE replica identity (#201) ## Problem The connector needs a table's primary-key columns to build the Kafka record key, and how it discovers them depends on the replica identity: - For `DEFAULT` and `INDEX` replica identity, the pgoutput/yboutput RELATION message has **always** marked the PK columns in its per-column flags, so the PK is read directly from the message. - For YugabyteDB's `CHANGE` replica identity, **only newer releases(from 2025.2.3.0) mark the PK in the RELATION message, older releases do not carry PK information for `CHANGE` at all**, so on those versions the PK must be looked up with a database metadata query. PR #195 added that DB-query fallback for `CHANGE`. PR #199 removed it, assuming the RELATION message always carries the PK for `CHANGE` which is only true from the release that shipped that change (stable `2025.2.3.0`, preview `2.31.0.0`). As a result, on older servers the connector resolves an **empty** primary key for `CHANGE` tables: emitted records have a `null` Kafka key, and downstream consumers break e.g: the `YBExtractNewRecordState` SMT throws a `NullPointerException` on the null key and kills the sink task. Reported in yugabyte/yugabyte-db#22555. ## Solution Select the PK-resolution strategy from the detected YugabyteDB server version, with the boundary at the release where the RELATION message begins carrying the PK **for `CHANGE` replica identity** (stable `2025.2.3.0`, preview `2.31.0.0`): - **Version model** — a new `YugabyteDBVersion` value type parses the YB token from `version()` (e.g. `2025.2.0.0-b131`), ignores any build suffix, and compares format-aware: stable/year-based (`YYYY.minor.patch.rev`) and preview (`major.minor.patch.rev`) lines are only ever compared against the threshold of the same format. `pkInRelationMessage()` reports whether this version marks the PK in the RELATION message for `CHANGE`; an unknown/unparseable version is treated as "does not", so the safe DB-query path is used. - **Version lookup**: `PostgresConnection` gains a cached `getYugabyteDBVersion()` and a `fetchLatestYugabyteDbVersion(int maxRetries)` that runs `SELECT substring(version() from 'YB-([^\s]+)')`, retrying transient failures up to the configured `slot.max.retries` with a 30s gap (mirroring the existing replication-slot retrieval retry), and throwing `DebeziumException` if the version cannot be read. The version is resolved once, up front, when the replication connection is created, so it is stable for the lifetime of the stream. - **Decoder gating**: `PgOutputMessageDecoder.handleRelationMessage()` trusts the RELATION-message PK flags for `CHANGE` only when `pkInRelationMessage()` is true, otherwise it resolves the PK with a DB metadata query (`readPrimaryKeyNames`, falling back to unique indices). `DEFAULT`/`INDEX` continue to read the flags and `FULL`/`NOTHING` continue to query the DB — behaviour for those is unchanged. **Operational note:** the server version is read when each replication connection is created (every connector start/restart) and cached for that connection's lifetime. A continuously-running connector keeps its strategy across an upgrade and stays correct, the one thing to avoid is starting/restarting the connector *during* a rolling upgrade at least for REPLICA IDENTITY CHANGE, when `version()` may report an already-upgraded node while the stream still carries old-format RELATION messages. ## Test Plan `YugabyteDBVersionTest`: - previewVersionsAtOrAboveThresholdHavePkInRelationMessage - previewVersionsBelowThresholdDoNotHavePkInRelationMessage - yearBasedVersionsAtOrAboveThresholdHavePkInRelationMessage - yearBasedVersionsBelowThresholdDoNotHavePkInRelationMessage - unknownVersionDoesNotHavePkInRelationMessage
|
Hi @fourpointfour, thanks for your contribution. Please prefix the commit message(s) with the DBZ-xxx JIRA issue key. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
9 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Note
High Risk
Touches critical CDC connector paths (schema generation, snapshot/streaming coordination, offset/LSN flushing, and retry semantics) and changes defaults/drivers toward YugabyteDB, so regressions could impact correctness or recoverability of change capture.
Overview
This PR adds release/packaging automation for the YugabyteDB connector: new GitHub Actions workflows to build and publish artifacts (GitHub releases, Confluent zip packaging, and Quay image pushes) plus a new
Dockerfilethat builds a slimmer Debezium Connect image with Java 17 and bundled Yugabyte/Confluent dependencies.It shifts the connector toward YugabyteDB defaults and distribution: many Maven modules now inherit
${revision}, the PG connector artifact is renamed toyugabytedb-source-connector, adds Yugabyte JDBC dependencies and S3distributionManagement+ ans3-deployprofile, and updates test connection defaults for local YB (port/user/db) while swappingorg.postgresql.*driver types forcom.yugabyte.*.Core runtime behavior is updated for YugabyteDB: introduces
LogicalDecoder.YBOUTPUTand a YB-specificPGTableSchemaBuilder/value wrapping model, adds parallel snapshot/streaming configuration fields, changes partition identity to includetaskId/slotName, adds snapshot-to-streaming coordination that waits for snapshot completion offsets, and extends streaming/offset logic for YB (hybrid-time vs sequence LSN handling, origin metadata propagation, replica-identity handling, and revised retry/heartbeat behavior).Written by Cursor Bugbot for commit 5dd9e17. This will update automatically on new commits. Configure here.