PT-2237 - fixed PostgreSQL DB logs collection (Reopen)#1099
Merged
svetasmirnova merged 25 commits into3.xfrom Apr 27, 2026
Merged
PT-2237 - fixed PostgreSQL DB logs collection (Reopen)#1099svetasmirnova merged 25 commits into3.xfrom
svetasmirnova merged 25 commits into3.xfrom
Conversation
This fix ensures `pt-k8s-debug-collector` collects postgreSql databse logs which is stored in `/pgdata/<cluster_name>/pg_log`. A test `TestIndividualFiles` was refactored, and modified for a new case with new feature.
This refactor includes replacing all of the `kubectl` cli calls with golang sdk for k8s. Additionaly dumper now has new structure, new logger, tar file path controll, and multithreaded approach for downloading and exporting files form multiple pods.
…7_fix_pg_log_collector
…7_fix_pg_log_collector
…cona-toolkit into PT-2237_fix_pg_log_collector
svetasmirnova
requested changes
Mar 31, 2026
Collaborator
svetasmirnova
left a comment
There was a problem hiding this comment.
pgo test fails:
=== RUN TestCollectorRunner/Operator_pgo/TestIndividualFiles/Resource_pgo
main_test.go:401:
Error Trace: /home/sveta/src/percona/percona-toolkit/src/go/pt-k8s-debug-collector/main_test.go:401
/home/sveta/go/pkg/mod/github.com/stretchr/testify@v1.11.1/suite/suite.go:115
Error: Received unexpected error:
exit status 2
Test: TestCollectorRunner/Operator_pgo/TestIndividualFiles/Resource_pgo
main_test.go:403:
Error Trace: /home/sveta/src/percona/percona-toolkit/src/go/pt-k8s-debug-collector/main_test.go:403
/home/sveta/go/pkg/mod/github.com/stretchr/testify@v1.11.1/suite/suite.go:115
Error: Preprocessor Check
Test: TestCollectorRunner/Operator_pgo/TestIndividualFiles/Resource_pgo
Messages: test pgo_pg_logs_exist
resource:pgo
namespace: pgo
output is not as expected
Output:
Wanted: [.log]
=== RUN TestCollectorRunner/Operator_pgo/TestIndividualFiles/Resource_auto
main_test.go:401:
Error Trace: /home/sveta/src/percona/percona-toolkit/src/go/pt-k8s-debug-collector/main_test.go:401
/home/sveta/go/pkg/mod/github.com/stretchr/testify@v1.11.1/suite/suite.go:115
Error: Received unexpected error:
exit status 2
Test: TestCollectorRunner/Operator_pgo/TestIndividualFiles/Resource_auto
main_test.go:403:
Error Trace: /home/sveta/src/percona/percona-toolkit/src/go/pt-k8s-debug-collector/main_test.go:403
/home/sveta/go/pkg/mod/github.com/stretchr/testify@v1.11.1/suite/suite.go:115
Error: Preprocessor Check
Test: TestCollectorRunner/Operator_pgo/TestIndividualFiles/Resource_auto
Messages: test pgo_pg_logs_exist
resource:auto
namespace: pgo
output is not as expected
Output:
Wanted: [.log]
Since we will deprecate pgo soon as per EOL note here: https://docs.percona.com/percona-operator-for-postgresql/1.6.0/, you can simply remove this test.
svetasmirnova
approved these changes
Apr 2, 2026
There was a problem hiding this comment.
Pull request overview
This PR fixes PostgreSQL log collection in the Kubernetes debug collector by enabling extraction of all files from a log directory (where filenames vary at runtime), and updates the integration test to validate that at least one PostgreSQL .log file is included in the produced dump archive.
Changes:
- Added support for dumping an entire directory of “individual files” (used for PostgreSQL
pg_log) instead of only known, fixed filenames. - Added pg log directory mappings for PG v1 (“pgo”) and PG v2 (“pgv2”) resources.
- Refactored
TestIndividualFilesto run namespace-specific assertions and added checks for PostgreSQL log presence.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| src/go/pt-k8s-debug-collector/main_test.go | Refactors individual-file tests and adds checks that PG .log files exist in the archive. |
| src/go/pt-k8s-debug-collector/dumper/resources.go | Registers PostgreSQL log directories (pg_log) for PG v1 and PG v2 dump collection. |
| src/go/pt-k8s-debug-collector/dumper/kube_utils.go | Introduces streaming tar execution helpers used for directory/file extraction. |
| src/go/pt-k8s-debug-collector/dumper/individual_files.go | Reworks individual file dumping to support env substitution and directory extraction via tar streaming. |
| src/go/pt-k8s-debug-collector/dumper/dumper.go | Normalizes CR type handling when matching pods and dumping summaries/files. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
artemgavrilov
approved these changes
Apr 27, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What was added:
To keep changes minimal, the existing test
TestIndividualFileswas modified to cover the new case.The test was refactored because the previous implementation couldn't handle files with variable names.
In
/pgdata/<cluster_name>/pg_log, logs are named according to the day/week, so filenames are not known at runtime.A new function
getAllFilesFromDirectorywas added todumper.go.Its purpose is the same as the existing
getIndividualFiles, but it supports extracting multiple files from a directory. It is useful because pg log filenames can vary, and there can be multiple log files.(
/libhas not been changed)