Skip to content

Conversation

tsg
Copy link
Member

@tsg tsg commented Oct 11, 2025

Description

This adds two config options for the snapshot configuration:

  • ignore_row_processing_errors (PGSTREAM_POSTGRES_SNAPSHOT_IGNORE_ROW_PROCESSING_ERRORS) - if set to true, it will continue the snapshot in case there are errors on a particular row / batch.
  • log_row_on_error" (PGSTREAM_POSTGRES_SNAPSHOT_LOG_ROW_ON_ERROR`) - if set to true, it will log the row (including user data) before continuing / exiting.

Both of these options are meant for troubleshooting and should be used with care in production environments:

  • Ignoring errors can result in consistency issues, so while the immediate error is ignored, foreign-key or other constraints might fail later in the process.
  • Logging the user data might leak PII into the logs.

Type of Change

Please select the relevant option(s):

  • 🐛 Bug fix (non-breaking change that fixes an issue)
  • ✨ New feature (non-breaking change that adds functionality)
  • 💥 Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • 📚 Documentation update
  • 🔧 Refactoring (no functional changes)
  • ⚡ Performance improvement
  • 🧪 Test coverage improvement
  • 🔨 Build/CI changes
  • 🧹 Code cleanup

Testing

  • Unit tests added/updated
  • Integration tests added/updated
  • Manual testing performed
  • All existing tests pass

Checklist

  • Code follows project style guidelines
  • Self-review completed
  • Code is well-commented
  • Documentation updated where necessary

@eminano eminano force-pushed the ignore_errors_option branch from 3e4c3d0 to 135cb35 Compare October 14, 2025 12:53
Copy link

Merging this branch will increase overall coverage

Impacted Packages Coverage Δ 🤖
github.com/xataio/pgstream/cmd/config 83.45% (+0.15%) 👍
github.com/xataio/pgstream/pkg/snapshot/generator/postgres/data 80.95% (+1.03%) 👍

Coverage by file

Changed files (no unit tests)

Changed File Coverage Δ Total Covered Missed 🤖
github.com/xataio/pgstream/cmd/config/config_env.go 89.00% (+0.11%) 200 (+2) 178 (+2) 22 👍
github.com/xataio/pgstream/cmd/config/config_yaml.go 89.25% (+0.12%) 186 (+2) 166 (+2) 20 👍
github.com/xataio/pgstream/pkg/snapshot/generator/postgres/data/config.go 0.00% (ø) 12 0 12
github.com/xataio/pgstream/pkg/snapshot/generator/postgres/data/pg_snapshot_generator.go 86.44% (+0.91%) 236 (+8) 204 (+9) 32 (-1) 👍

Please note that the "Total", "Covered", and "Missed" counts above refer to code statements instead of lines of code. The value in brackets refers to the test coverage of that file in the old version of the code.

Changed unit test files

  • github.com/xataio/pgstream/pkg/snapshot/generator/postgres/data/pg_snapshot_generator_test.go

@eminano eminano merged commit edfb54a into main Oct 14, 2025
7 checks passed
@eminano eminano deleted the ignore_errors_option branch October 14, 2025 13:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants