feat: separate pruner - #195
Conversation
fix: batch insert db by max postgres limit
chore: bump movevm.
bump modules
impv: improve docs + moved local setup out of dockerfiles/
feat(informative): bump dependencies
fix(informative): finalize block event mode extraction
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.37.0 to 0.38.0. - [Commits](golang/net@v0.37.0...v0.38.0) --- updated-dependencies: - dependency-name: golang.org/x/net dependency-version: 0.38.0 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore: remove unnecessary extension from docker * use alpine instead of ubuntu
This reverts commit dc2ec60.
* feat: refactor common code to package * fix: bring back go mod * chore: import name * feat: index transaction data * chore: import order * feat: fix as comment * fix: sentry span description * fix: GetRowsToPruneByBlockHeight * fix: refactor error * feat: index validator data (#27) * feat: index validator data * fix: typo * fix: as comment * feat: initialize api (#28) * feat: initialize api * fix: remove chain id hardcode * feat: add /tx/v1/txs/hash (#31) * feat: add /tx/v1/txs/hash * chore: error order * chore: public key struct - consistency * Feat/account transactions (#29) * feat: index account transactions * fix: delete empty file * fix: panic * refactor: parse coin amt func * fix: conflict * Feat/db migration (#35) * Merge branch 'v2' * feat(informative-sweeper): early return if migration fails * feat(informative-prunner): optimize chunk stream gcs upload * feat(informative-flusher): nullable transaction error message * feat(db): add custom types & better temp dir management * chore(db): endline * feat(db): edit migration * fix(informative-prunner): interface to any * feat(db): filter clauses and batch size * feat(db): separate sql types * feat(db): interface to any * feat(informative-flusher): fix inserting transactions related tables * feat: validator commit signatures (#36) * feat: validator commit signatures * fix: comments * feat: merge conflict --------- Co-authored-by: benzbeeb <benz@initia.xyz> * feat: use gorm in api, apply to /collections (#39) * feat: use gorm in api, apply to collections * fix: as comments * fix: refactor * feat: capture new module (#32) * feat: capture new module * fix: as comment * fix: query * feat: merge v2 * feat: and query height * feat: use generated struct for nft (#40) * Feat: add L1 indexer API endpoint for scan api overview (#30) * feat: add l1 indexer endpoint for scan api overview * feat: add gorm * fix: rename, refactor --------- Co-authored-by: evilpeach <kanisorn.peach@gmail.com> * Feat/index block (#41) * feat: index block * fix: comments * fix: remove unused * feat: capture new collection (#42) * feat: capture new collection * fix: remove comment * feat: add indexer txs (#33) * feat: add l1 indexer endpoint for scan api overview * feat: add indexer txs * feat: add gorm * feat: txs * feat: update txs * fix: comments * chore: add comments, fix order * feat: add nft api (#37) * feat: add l1 indexer endpoint for scan api overview * feat: add indexer txs * feat: add nft api * feat: add gorm * feat: txs * feat: update txs * fix: count * fix: comments * feat: module txs (#47) * feat: capture new nft (#44) * feat: capture new nft * feat: handle transfer + owner * feat: new owners * fix: type * chore: add comment * feat(api): validator (#48) * feat(api): initial validator dto, repo, routes and services * feat(api): more validator routes * feat(api): all validator routes implementation * fix(api): validator bugs and improvements * feat(api): resolve merge conflict * fix(api): validator routes merge bug * fix(api): imports order * fix(api): validator routes naming * feat: capture related nft collection transaction and nft histories (#50) * chore: refactor type * feat: insert nft history * fix: as comment * Support module routes (#46) * feat: modules, module by id * feat: modules route * feat: get module by id * feat: module publish info * docs: add swagger * feat: enhance proposal in module publish info * fix publish info bug, add module proposals * feat: module txs * feat: module stats for table counts * fix: update module structure * feat: add nft txs api (#38) * feat: add l1 indexer endpoint for scan api overview * feat: add indexer txs * feat: add nft api * feat: add nft txs api * feat: add gorm * feat: txs * feat: update txs * fix: count * fix: comments * Feat/nft collection api (#43) * feat: update nft collection * feat: add collections api * feat: add blocks api (#45) * feat: add blocks api * feat: update naming convention * feat: update route name * feat: add accounts route (#49) * feat: add accounts route * feat: update account proposals response * feat: update api format * feat: update nft naming convention * feat: validator naming format (#53) * feat: add l1 indexer endpoint for scan api overview * feat: add indexer txs * feat: add nft api * feat: add nft txs api * feat: add gorm * feat: txs * feat: update txs * fix: count * fix: comments * Feat/nft collection api (#43) * feat: update nft collection * feat: add collections api * feat: add blocks api (#45) * feat: add blocks api * feat: update naming convention * feat: update route name * feat: add accounts route (#49) * feat: add accounts route * feat: update account proposals response * feat: update api format * feat: update nft naming convention * feat: update naming convention * feat: udpate * Feat/create proposal (#52) * feat: create proposals * feat: add insert to db * feat:capture mutation event and burn (#51) * feat:capture mutation event and burn * fix: nft bug * fix: remove use for test code * feat(api): proposal (#54) * feat(api): proposal routes * feat(api): proposal imports order * feat(api): improve proposals query param * fix(api): merge conflict * feat(api): update swagger docs * fix(api): proposal route types * feat: capture opinit and ibc events (#55) * feat: capture opinit and ibc events * fix: remove comment * feat: capture module histories and opint tx (#56) * feat: capture module histories and opint tx * fix: add new batch insert * feat(docker): update Dockerfiles for v2 (#57) * feat: api workflow (#59) * feat: api workflow * fix: remove tag prefix * feat: proposal status changes (#58) * feat: proposal status changes * fix: update: proposal * fix: sql * fix: tally * fix: proposal expedited change * chore refactor --------- Co-authored-by: benzbeeb <benz@initia.xyz> * fix: go mod tidy * feat: emergency tally event (#62) * Fix: API minor change (#63) * fix: api minor change * fix: pr comments * feat: capture vote deposit proposal and gov publish (#64) * feat: capture vote deposit proposal and gov publish * feat: fix as comment * fix: update proposal votes (#65) * Fix/start sweep from block height (#68) * fix: start sweep on latest block height * fix: get latest block height * fix: tx hash in transaction table (#69) * fix: hex byte * fix: decode tx hash * fix: tx hash * fix: tx hash * feat(chart): add api charts (#67) * fix: remove height * fix: bring back val uptime (#71) * feat: update tx count (#70) * fix: resolve voting power (#72) * fix: resolve voting power * fix: nil pointer * fix: remove debug * fix: as comment * fix: wrong owner nft (#73) * fix: wrong owner nft * fix: styling * fix * fix: remove endline * fix: change ProposalID to pointer type in CollectionMutationEvent and NftMutationEvent * Update nft and collection (#75) * fix: and update mutate event * refactor: update Flush and UpdateTxCount methods to include block height parameter * fix: tracking table * fix: mutate event * fix: latest block height (#74) * fix: latest block height * fix: proposal and module * fix: pr comments --------- Co-authored-by: Poafs1 <prach.yot@gmail.com> * fix: add collection transaction insertion in FlushTransferredNft method * fix: handle proposal fetch errors gracefully in updateProposals method * feat: generic indexer (#76) * fix: refactor proposal deposit handling in handleProposalDepositEvent method * fix: remove publish tx_id * fix: publish tx * feat: add validator slash event handling in batch insert - Introduced a new field for validator slash events in DBBatchInsert. - Implemented logic to insert validator slash events in the Flush method. - Enhanced validator event processing to capture and store slash events from SDK messages. * fix: proposal (#78) * fix: proposal * fix: remove unused * fix: enhance handleRedelegateEvent to validate attributes - Updated handleRedelegateEvent to return errors if source or destination validator addresses and amount are not found in the event attributes. - Improved error handling for better debugging and reliability. * fix: ensure handleRedelegateEvent returns nil for successful processing - Updated handleRedelegateEvent to return nil after processing, ensuring consistent behavior when no errors occur. * patch generic indexer to use msg from informative-sweeper (#77) * refactor: update module paths and dependencies in go.mod, remove sweeper command, and adjust flusher and validatorcron imports - Changed module path from `github.com/alleslabs/initia-mono/generic-indexer` to `github.com/initia-labs/core-indexer/generic-indexer`. - Updated various dependencies to their latest versions in `go.mod`. - Removed the sweeper command from the CLI. - Adjusted imports in flusher and validatorcron files to reflect the new module path. - Updated function signatures and logic in flusher to accommodate changes in message handling and database interactions. * refactor: update dependencies and remove unused files - Updated `go.mod` and `go.sum` to include new dependencies and remove outdated ones. - Removed unused files related to database handling and validator cron functionality. - Adjusted import paths in various files to align with the updated module structure. - Modified function signatures in the flusher to accommodate changes in data handling. - Commented out the validator cron command in the root command file. * fix: refactor flusher * feat: reintroduce validator cron functionality - Added the `validatorcron` command back to the root command, enabling its execution. - Restored the `validatorcron` package with necessary imports and command structure. - Updated the `FlushCmd` function to handle validator data flushing from Kafka to the database. - Enhanced database interactions for historical voting powers and validator uptimes. - Refactored the `ValidatorCron` struct to utilize GORM for database operations. * refactor: remove disable LCD transaction response flag - Removed the `--disable-lcd-tx-response` flag from the `run.sh` script and the associated handling in the `FlushCmd` function. - Simplified the transaction insertion logic in the `decodeAndInsertTxs` method by removing the conditional check for disabling account transaction indexing. * refactor: streamline validator address querying in flusher * refactor: remove error handling for validator updates in Run method * refactor: rename getAccountTransaction to getAccountTransactions for clarity * feat: processor (#80) * feat: processor * fix: statetracker * fix: remove dup init * fix: remove old validator * fix: package name * fix: use findattr * feat: validator error * feat: slash event * fix: split fns * fix: use flusher dbbatchinsert * fix: comment * fix: combine loops * Adjusted the sweeper configuration to handle multiple Kafka topics in the sweeper logic. (#81) * fix: redelegate event (#82) * enhance database upsert functionality (#84) * feat: jail event (#83) * feat: handle module proposal (#87) * fix: inactive and voting times (#90) * fix: inactive and voting times * fix: proposal * fix: db voting times * feat: dockerize generic-indexer (#89) * feat: dockerize generic-indexer * fix: newline * fix: block hash decode (#85) * QA-403: Fix/modules repository query (#86) * fix: modules repository query * refactor: remove unnecessay int parser --------- Co-authored-by: evilpeach <kanisorn.peach@gmail.com> * fix: get @type from msg (#88) * fix: add height filter height 0 (#92) * feat: new processor logic (#91) * feat: new processor logic * fix: reset tx processor * fix: comment * fix: todo * fix: split processTx to four fns * fix: error * [merge to #91] feat: total deposit changes (#93) * feat: total deposit changes * fix: replace txID with txData * fix: remove move event and finalize block events * fix: function name and error * fix: improve error handling in processTransactions function * feat: bump initia sdk * update initia version * fix: update proposal ID query in UpdateProposalTotalDeposit function * chore: update dependencies in go.mod and go.sum - Bump initia version to v1.1.3 - Add movevm version v1.0.0 - Update iavl version to v1.2.6 - Update OPinit and its API versions to v1.1.1 and v1.1.0 respectively * fix: change remark field to json.RawMessage in MutateEventModel (#94) * chore: update dependencies in go.mod and go.sum (#95) - Bump iavl version to v1.2.6 - Update initia version to v1.1.3 - Set movevm version to v1.0.0 - Update OPinit and its API versions to v1.1.1 and v1.1.0 respectively * feat: add sweeper functionality (#96) - Introduced a new Dockerfile for the sweeper service, enabling building and running the application in a containerized environment. - Created main entry point for the sweeper application. - Implemented command structure using Cobra for sweeping blockchain data. - Added run script to facilitate execution with necessary parameters. - Established configuration and logging for the sweeper service, integrating with Kafka and Sentry for message handling and error tracking. - Included Go module files for dependency management. * fix(informative): proposal deposit double spend * fix: change ResolvedHeight to a pointer in AccountProposal struct (#97) * feat(informative): validator cache via db (#98) * feat(informative): load validator cache from db * feat(informative): update validators every block * feat: initialize event indexer service (#99) * feat: initialize event indexer service * feat(event-indexer): update event-indexer/.env.example * feat(event-indexer): update event-indexer/cmd/root.go --------- Co-authored-by: David Tumcharoen <42636319+traviolus@users.noreply.github.com> * feat(docker): update Dockerfiles for event-indexer * fix(informative): validator unjail type * fix(informative): remove missing imports * feat(informative): indexer package (#100) * feat(informative): replace flusher with indexer * feat(informative): replace flusher with indexer * refactor: generic-indexer (#101) * refactor: generic-indexer - Updated `go.mod` and `go.sum` to include new dependencies and remove outdated ones. - Removed unused files related to database handling and validator cron functionality. - Adjusted import paths in various files to align with the updated module structure. - Modified function signatures in the flusher to accommodate changes in data handling. - Commented out the validator cron command in the root command file. * chore: import order * feat: migrate blockheight to int64 (#102) * feat: migrate blockheight to int64 * fix: migrate * feat(informative): handle uppercase addresses * fix: rename migrate file (#103) * fix(processor): append module proposals to dbBatchInsert (#104) * fix: account proposals (#105) * fix: update types field to json.RawMessage in ModuleProposalModel and change Timestamp to a pointer in ValidatorVotedProposal (#106) * fix: update types field to json.RawMessage in ModuleProposalModel and change Timestamp to a pointer in ValidatorVotedProposal * remove typo * fix: update schema types from array to object in API documentation and DTOs * QA-467: Fix - latest republished txs (#107) * fix: latest republished txs nullable * fix: pr comments * fix(state-tracker): update proposal fields to use getter methods for SubmitTime, DepositEndTime, VotingTime, and VotingEndTime (#110) * feat(workflow): cicd (#109) * feat: account procesor (#108) * feat: account procesor * fix: revert parser * feat: add base processor * fix: remove unused fn * fix(generic): block height type (#112) * replace proposal expedited changes with status updates in end block event handling (#111) * refactor(processor): replace proposal expedited changes with status updates in end block event handling * fix: proposal status --------- Co-authored-by: songwongtp <16089160+songwongtp@users.noreply.github.com> * feat(informative): bank processor (#114) * feat(informative): bank processor * feat(informative): use bank processor * fix: filter account txs (#113) * fix: refactor ibc and op (#118) * fix: refactor ibc and op * fix: remove unused fns * feat(charts): create charts for indexers (#115) * feat(charts): create indexer charts * remove probe * feat: move processor (#121) * feat: move processor * fix: add comment * fix: removed unnecesary logic * fix(storage): use gcs only * feat: new move event (#122) * feat(generic): bump go mod * fix(generic): go mod * feat(uploader): implementation (#123) * feat(uploader): implementation * chore(uploader): refactor comment * feat(uploader): update env example * feat(charts): remove unnecessary value (#127) * fix: add metadata (#128) * fix: go mod tidy * Feat/validator map (#125) * feat(charts): modify charts (#130) * FE-1517: Fix - validator type message empty (#120) * fix: bonded toke change message type * feat: add note * feat(charts): create charts for uploader (#126) * feat(workflow): create ci for uploader (#124) * Fix/refactor–accounts (#129) * fix: query db outside of cacher * fix: add space * fix: refactor common tx fns * fix: rename file * fix: remove unused fn * fix: comments * fix: remove unnecessary nonretryable * fix: remove unused tx parser * feat: retry fn (#131) * fix: query db outside of cacher * fix: add space * fix: refactor common tx fns * fix: rename file * fix: remove unused fn * fix: comments * fix: remove unnecessary nonretryable * fix: remove unused tx parser * feat: retry fn * fix(generic): validators query * feat(charts): modify charts (#132) * feat(charts): modify charts - add ID env to deployment - add command value - modify generic-indexer value * remove command * fix: api proposal resolved height * fix: proposal cancel resolved (#133) * fix: proposal cancel resolved * fix: cancel * feat(charts): update sweeper chart (#134) * fix(api): handle invalid transaction hash in GetAccountTxs (#136) * fix(api): handle invalid transaction hash in GetAccountTxs * fix: return empty slice instead of nil * Fix/proposal validator votes (#137) * fix: ProposalVote and related models to use pointer types for Timestamp and TxHash fields * fix: remove unnecessary blank line in GetProposalVotes function * fix: change VotingEndTime field to use pointer type in ProposalSummary struct * fix: update ProposalVoteModel to use pointer for Timestamp field in proposal.go * feat(docker): update Dockerfiles for sweeper (#135) * feat(docker): update Dockerfiles for sweeper * feat(docker): update Dockerfiles for sweeper * feat(docker): remove db from sweeper stage build * feat: update built binary name * feat(chart): add sa to charts (#138) * feat(chart): add sa to charts * fix --------- Co-authored-by: LizJeong <liz@23labs.io> * fix(docker): sweeper binary duplicate name (#139) * fix(docker): sweeper binary * feat(docker): update binaries * fix(docker): uploader binary * fix charts (#140) * feat: handle ibc msg type url (#141) * Feat/genesis (#142) * feat(indexer): implement StartFromGenesis method and add Genesis RPC support - Introduced StartFromGenesis method in the indexer to initialize tracking from the genesis block. - Added Genesis method to the Hub and Client for fetching genesis data from RPC. - Implemented database functions for inserting the genesis block and checking tracking initialization status. - Enhanced the StartIndexing method to call StartFromGenesis if tracking is not initialized. * refactor(indexer): streamline validator creation in StartFromGenesis - Replaced the previous validator fetching logic with a new function to create a GenesisValidator directly from the MsgCreateValidator. - Simplified the StartFromGenesis method by removing unnecessary unpacking and error handling related to the validator. - Improved code readability and maintainability by consolidating validator initialization logic. * fix: remove debug * fix: panic to error * fix: return error * fix: return err * feat(charts): create chart for generic-indexer cron (#144) * feat(charts): create chart for generic-indexer cron * modify * feat(charts): modify charts (#145) * fix(charts): fix parse error (#146) * fix(informative): proper nft processor maps key (#148) * fix(db): update is_emergency when handling event (#150) * feat(mq): update dlq topics (#149) * Fix informative dup block (#151) * feat(db): add function to retrieve latest informative block height and skip already processed messages in Kafka handler * refactor(db): remove GetLatestBlockHeight function and update sweeper to use GetLatestInformativeBlockHeight * Update pruning logic to use GetLatestInformativeBlockHeight function (#152) * fix: api regression test (#147) * fix: api regression test * feat: update swagger * feat: parsed proposal id int32 * fix: test * feat: add pagination limit * feat: update pagination logic * fix Dockerfile (#153) * fix: update pagination limit validation to allow a maximum of 1000 (#154) * fix: coderabbit comments * fix: comment * fix: rpc check --------- Co-authored-by: benzbeeb <benz@initia.xyz> Co-authored-by: Nattharat Kondanna Wiriyakulnan <mercedesmwit20@gmail.com> Co-authored-by: Wasin <wasinwatt@gmail.com> Co-authored-by: David Tumcharoen <42636319+traviolus@users.noreply.github.com> Co-authored-by: Kanisorn Peach <kanisorn.peach@gmail.com> Co-authored-by: Kanisorn Peach <peach@initia.xyz> Co-authored-by: Prach Yothaprasert <prach.yot@gmail.com> Co-authored-by: Prach Yothaprasert <prach@initia.xyz> Co-authored-by: LizJeong <liz@23labs.io> Co-authored-by: Nattharat Kondanna Wiriyakulnan <nattharat.wi@gmail.com>
* refactor: ci workflow * remove * Update docker/build-push-action to v6
…ent (#157) - Introduced a check for the block message version in the parseBlockAndRebalanceRPCClient function to ensure it is set to 0, returning an error if it is not. - Added a new Version field to the BlockResultMsg struct to track message versions, allowing consumers to handle messages accordingly.
Bumps [github.com/gofiber/fiber/v2](https://github.com/gofiber/fiber) from 2.52.2 to 2.52.9. - [Release notes](https://github.com/gofiber/fiber/releases) - [Commits](gofiber/fiber@v2.52.2...v2.52.9) --- updated-dependencies: - dependency-name: github.com/gofiber/fiber/v2 dependency-version: 2.52.9 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* refactor: remove apperror package and update error handling - Deleted the apperror package, consolidating error handling directly within the handlers and repositories. - Updated error responses in various handlers to utilize a new standardized error response method. - Adjusted error creation in repositories to return native errors instead of custom error types. - Enhanced pagination error handling in DTOs to improve clarity and maintainability. * fix: update error handling for proposal ID validation - Refactored error response in GetProposalAnswerCounts to use a standardized error message for invalid proposal IDs, improving clarity and consistency in error handling. * chore: update dependencies and improve error messages * feat: enhance error handling and add new dependency - Introduced a new error message for height validation in the apperror package. - Updated the BlockHandler to utilize the new height error response. - Improved error formatting in the ModuleRepository and ProposalRepository for better clarity. - Added a new indirect dependency on github.com/stretchr/objx in go.mod.
* feat(readme): update v2 * fix(readme): update section
#163) * feat(mocks): add mock implementations for various repositories and services * fix: correct typo in AccountTxsResponse and update mock imports - Fixed a typo in the AccountTxsResponse struct, changing `AccounTxs` to `AccountTxs`. - Updated import statements in mock files to ensure consistency and clarity. - Removed the unused validator.go file from mocks directory. * chore(tests): standardize import statements in test files - Added the `github.com/stretchr/testify/assert` import to the beginning of the account_test.go, block_test.go, and nft_test.go files for consistency. - Removed duplicate import statements to streamline the code.
* feat: add tx response in informative * fix: import order
|
Important Review skippedReview was skipped as selected files did not have any reviewable changes. 💤 Files selected but had no reviewable changes (1)
⛔ Files ignored due to path filters (1)
⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: ⛔ Files ignored due to path filters (1)
📒 Files selected for processing (1)
You can disable this status message by setting the Use the checkbox below for a quick retry:
WalkthroughPruner is extracted into a standalone service: new Go module, CLI entrypoint, main, runner script, Dockerfile, Helm chart, and CI matrix entry added. Pruner-related registration and run tasks removed from event-indexer. API/command names changed (Prune → Run, PruneCmd → RunCmd). ChangesEvent-indexer cleanup
Pruner CLI & entrypoint
Pruner core rename & lifecycle
Module, env, and examples
Build & deployment
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Tip Issue Planner is now in beta. Read the docs and try it out! Share your feedback on Discord. Comment |
There was a problem hiding this comment.
Actionable comments posted: 7
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
pruner/pruner/pruner.go (1)
230-243:⚠️ Potential issue | 🟠 MajorMake
Run()idempotent across calls (currentsync.Oncescope is ineffective).
onceis a local variable, so eachRun()call gets a freshsync.Once, defeating the “avoid multiple prunner instances” intent. Move the guard toPrunner(or package-level) so multiple invocations don’t spawn parallel pruning loops.🔧 Proposed fix
type Prunner struct { dbClient *gorm.DB storageClient storage.Client config *PrunnerConfig + once sync.Once } func (p *Prunner) Run() { // Mutex to avoid multiple prunner instances - var once sync.Once - once.Do(func() { + p.once.Do(func() { ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM) defer stop() go p.StartPruning(ctx) @@ logger.Info().Msgf("Stopping prunner ...") p.close() }) }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@pruner/pruner/pruner.go` around lines 230 - 243, The sync.Once used in Prunner.Run is local and thus ineffective; make the guard a field on the Prunner struct (e.g., add once sync.Once to Prunner) and change the method to call p.once.Do(...) instead of a local once so multiple Run() calls are idempotent and only a single pruning goroutine is started; inside the p.once.Do closure keep the existing signal.NotifyContext, go p.StartPruning(ctx), the <-ctx.Done() wait and p.close() call so shutdown behavior is preserved.
🧹 Nitpick comments (2)
pruner/run.sh (1)
24-28: Prefer env-configurable defaults for pruning/backup settings.
The script loads.envbut ignoresPRUNING_*andBACKUP_*values, which can surprise local users.♻️ Suggested tweak
- ./pruner run --db $DB_CONNECTION_STRING \ - --backup-bucket-name ${chain}-local-core-event-data-backup \ - --backup-file-prefix events \ - --pruning-keep-block 10 \ - --pruning-interval 1 \ + ./pruner run --db "$DB_CONNECTION_STRING" \ + --backup-bucket-name "${BACKUP_BUCKET_NAME:-${chain}-local-core-event-data-backup}" \ + --backup-file-prefix "${BACKUP_FILE_PREFIX:-events}" \ + --pruning-keep-block "${PRUNING_KEEP_BLOCK:-10}" \ + --pruning-interval "${PRUNING_INTERVAL:-1}" \🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@pruner/run.sh` around lines 24 - 28, The pruner invocation is hardcoding pruning and backup flags instead of using environment-configurable defaults; update the script so the pruner run command uses environment variables (e.g., PRUNING_KEEP_BLOCK, PRUNING_INTERVAL, BACKUP_BUCKET_NAME, BACKUP_FILE_PREFIX) with sensible fallbacks if unset, for the flags --pruning-keep-block, --pruning-interval, --backup-bucket-name and --backup-file-prefix (still use DB_CONNECTION_STRING and the existing chain variable), so local users can override these values via .env or environment without changing the script.charts/pruner/templates/deployment.yaml (1)
28-65: Add a pod-levelsecurityContextto satisfy KSV-0118 and harden defaults.Even with container-level settings, scanners typically expect pod defaults to prevent any future sidecar/initContainer from running as root.
🛡️ Suggested hardening
template: metadata: labels: app: {{ quote .Release.Name }} spec: + securityContext: + runAsNonRoot: true + runAsUser: 1000 + runAsGroup: 1000 + fsGroup: 1000 {{- if .Values.serviceAccountName }} serviceAccountName: {{ quote .Values.serviceAccountName }} {{- end }} @@ - name: pruner image: {{ quote .Values.image }} @@ securityContext: readOnlyRootFilesystem: true runAsNonRoot: true runAsUser: 1000 runAsGroup: 1000 + allowPrivilegeEscalation: false + capabilities: + drop: ["ALL"]🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@charts/pruner/templates/deployment.yaml` around lines 28 - 65, Add a pod-level securityContext under spec (alongside the existing containers block) to enforce non-root defaults for any sidecars/initContainers: create a spec.securityContext with runAsNonRoot: true, runAsUser: 1000, runAsGroup: 1000 and fsGroup: 1000 so pod-wide defaults align with the container-level securityContext already set for the pruner container; reference the existing spec, containers, and the per-container securityContext for placement and consistent values.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@dockerfiles/Dockerfile.pruner`:
- Around line 3-5: The Dockerfile uses COPY with parent paths (COPY ../pruner,
COPY ../pkg) which are outside the build context and will fail; update the COPY
instructions in the Dockerfile.pruner to reference files/dirs inside the build
context (or adjust the build step to set the correct context) so that the
sources being copied (pruner and pkg) are available from the context, keeping
WORKDIR /build unchanged and ensuring the COPY targets remain /build/pruner and
/build/pkg.
- Around line 9-18: The image runs as root; create a non-root user and switch to
it after installing packages and copying the binary by adding user creation and
chown steps for the /app directory and the /pruner binary (referencing the COPY
--from=builder /pruner-binary /pruner and WORKDIR /app entries), then add USER
<username> before the container is started so runtime processes do not run as
root; ensure the new user has a home or minimal shell and ownership of /pruner
and /app is set to that user.
- Around line 19-30: The Dockerfile currently downloads MoveVM artifacts
(referenced by MOVEVM_VERSION and MOVEVM_ARCH) with wget without integrity
checks; update the RUN block to also fetch the corresponding checksum files
(e.g., .sha256 or .sha256sum) for /lib/libmovevm.${MOVEVM_ARCH}.so and
/lib/libcompiler.${MOVEVM_ARCH}.so, verify them using sha256sum -c (or echo
<checksum> | sha256sum -c) and fail the build on mismatch, and only move the
verified artifacts into /lib; ensure the verification step uses the same
MOVEVM_VERSION and MOVEVM_ARCH variables and exits non‑zero if verification
fails.
In `@pruner/go.mod`:
- Line 58: Update the indirect dependency versions in go.mod to remediate the
reported vulnerabilities: change github.com/dvsekhvalnov/jose2go to v1.7.0 or
later and golang.org/x/oauth2 to v0.27.0 or later, then run `go get`/`go mod
tidy` (or your repo's dependency update command) to update go.sum and ensure the
new versions are resolved; verify there are no breaking changes by running the
test suite and CI.
- Line 3: The Dockerfile.pruner base image and the Go toolchain version in
pruner/go.mod are mismatched; update Dockerfile.pruner to use the same Go
version declared in pruner/go.mod by changing the FROM line to use golang:1.25
so the build container matches Go 1.25.0 (alternatively, if you prefer to keep
the image older, modify the pruner/go.mod go directive to 1.24 or 1.23 to match
the rest of the repo), ensuring the unique artifacts affected are
Dockerfile.pruner and pruner/go.mod.
In `@pruner/run.sh`:
- Line 8: The current ARGS assignment uses ARGS=${@:2} which flattens/loses
quoting; change it to capture positional arguments as an array with ARGS=(
"${@:2}" ) and wherever ARGS is expanded (the later invocation around the
existing uses at lines referenced in the comment) replace plain $ARGS with
"${ARGS[@]}" so each original argument and quoting is preserved.
- Around line 5-22: The script computes SCRIPT_DIR but never uses it, so
task__run() may build the wrong Go module; modify task__run (in run.sh) to
change directory into SCRIPT_DIR before running go build and sourcing .env
(e.g., cd "$SCRIPT_DIR" or pushd "$SCRIPT_DIR" then popd after), ensuring go
build -o pruner . and source .env execute from the script directory rather than
the caller's CWD.
---
Outside diff comments:
In `@pruner/pruner/pruner.go`:
- Around line 230-243: The sync.Once used in Prunner.Run is local and thus
ineffective; make the guard a field on the Prunner struct (e.g., add once
sync.Once to Prunner) and change the method to call p.once.Do(...) instead of a
local once so multiple Run() calls are idempotent and only a single pruning
goroutine is started; inside the p.once.Do closure keep the existing
signal.NotifyContext, go p.StartPruning(ctx), the <-ctx.Done() wait and
p.close() call so shutdown behavior is preserved.
---
Nitpick comments:
In `@charts/pruner/templates/deployment.yaml`:
- Around line 28-65: Add a pod-level securityContext under spec (alongside the
existing containers block) to enforce non-root defaults for any
sidecars/initContainers: create a spec.securityContext with runAsNonRoot: true,
runAsUser: 1000, runAsGroup: 1000 and fsGroup: 1000 so pod-wide defaults align
with the container-level securityContext already set for the pruner container;
reference the existing spec, containers, and the per-container securityContext
for placement and consistent values.
In `@pruner/run.sh`:
- Around line 24-28: The pruner invocation is hardcoding pruning and backup
flags instead of using environment-configurable defaults; update the script so
the pruner run command uses environment variables (e.g., PRUNING_KEEP_BLOCK,
PRUNING_INTERVAL, BACKUP_BUCKET_NAME, BACKUP_FILE_PREFIX) with sensible
fallbacks if unset, for the flags --pruning-keep-block, --pruning-interval,
--backup-bucket-name and --backup-file-prefix (still use DB_CONNECTION_STRING
and the existing chain variable), so local users can override these values via
.env or environment without changing the script.
* feat: implement Keybase service with caching for validator images - Added a new `KeybaseService` to handle interactions with the Keybase API, including caching of validator image URLs. - Introduced a `TTLCache` for efficient image URL storage with a configurable time-to-live. - Updated `ValidatorService` and related handlers to utilize the new Keybase service for fetching and caching validator images. - Modified pagination and validator retrieval logic to support new parameters and improve functionality. * feat: introduce ValidatorStatusFilter for improved validator querying - Added a new `ValidatorStatusFilter` type to represent the status of validators (active, inactive, all). - Updated the `GetValidators` method across services, repositories, and handlers to utilize the new status filter instead of the deprecated `is_active` parameter. - Enhanced query logic to support filtering by validator status, improving the flexibility and clarity of validator retrieval. * feat: enhance validator block statistics calculation - Introduced a constant for the number of blocks to look back when calculating validator block statistics. - Updated the logic in `GetValidatorBlockStats` to use the new constant for determining the block range. - Improved handling of total blocks and signed blocks for each validator, ensuring all operators are included in the results even if they have no signed blocks. * fix: improve validator block statistics handling - Added checks to ensure maxHeight and totalBlocksInRange are at least 1 in the GetValidatorBlockStats method, preventing potential errors in block range calculations. - Updated the GetValidators method to ensure the Keybase service is initialized before attempting to retrieve validator images, enhancing robustness. * refactor: streamline validator block statistics handling - Removed the GetValidatorBlockStats method from the ValidatorRepository interface and its implementation, as it is no longer needed. - Updated the GetValidators method to utilize pre-calculated vote counts for the last 10,000 blocks, improving performance by avoiding expensive on-demand calculations. - Introduced a new column in the validator_vote_counts table to store the last 10,000 block vote counts, enhancing data retrieval efficiency. * refactor: streamline validator block statistics handling - Removed the GetValidatorBlockStats method from the ValidatorRepository interface and its implementation in the ValidatorRepository struct to simplify the codebase. - Updated the GetValidators method to utilize pre-calculated vote counts for the last 10,000 blocks, enhancing performance by avoiding expensive on-demand calculations. - Introduced a new column in the validator_vote_counts table to store the last 10,000 block vote counts, improving data retrieval efficiency. - Adjusted the calculateValidatorUptimes function to handle both last 100 and last 10,000 blocks, ensuring comprehensive uptime calculations for validators. * refactor: remove unused Keybase service and TTL cache - Deleted the KeybaseService implementation and its associated TTLCache, as they are no longer needed in the codebase. - This cleanup helps streamline the code and reduce complexity by removing redundant components. * refactor: update validator image handling to use base64-encoded data - Changed the image handling in the validator service to store base64-encoded image data instead of URLs, improving performance by reducing external API calls. - Updated database migrations to replace the `image_url` column with `identity_image` for better clarity and functionality. - Refactored the image fetching logic to retrieve and encode images directly from Keybase, ensuring a more efficient data retrieval process. * feat: add endpoint to retrieve all validators with simplified response - Introduced a new endpoint `/indexer/validator/v1/all` to fetch a simplified list of all validators, including both active and inactive ones. - Created `AllValidatorsResponse` type to structure the response data, enhancing the API's usability. - Updated the `GetAllValidators` handler to implement pagination and return the necessary validator information in JSON format. * refactor: enhance validator image handling and update cron job intervals - Updated the database migration to set the `identity_image` column as NOT NULL, ensuring data integrity for validator images. - Introduced a new flag for `ValidatorIdentityImageUpdateInterval` in the indexer cron command, allowing configurable intervals for updating validator identity images. - Refactored the indexer cron job to include a scheduled task for updating validator identity images from Keybase, improving the efficiency of image retrieval. - Added functions to upsert validator identity images and vote counts, streamlining the database operations related to validator data. * refactor: update validator uptime calculations and database queries - Changed the `Uptime` field in `ValidatorInfo` to be non-optional, ensuring consistent data representation. - Updated SQL queries in the `ValidatorRepository` to use `last_10000` instead of `last_100` for improved accuracy in uptime calculations. - Adjusted the `GetValidators` method in the `validatorService` to calculate uptime as a percentage based on the last 10,000 blocks. - Refactored the `UpsertValidatorIdentityImages` function to enhance batch processing and update logic for validator identity images. * refactor: simplify validator status check in GetValidators method - Updated the condition in the GetValidators method to always calculate uptime for active validators, removing the dependency on the status filter. - This change streamlines the logic for uptime calculation, ensuring it is consistently applied to all active validators.
There was a problem hiding this comment.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Duplicate comments:
In `@dockerfiles/Dockerfile.pruner`:
- Around line 19-30: The RUN block that computes MOVEVM_ARCH and uses wget to
fetch libmovevm.${MOVEVM_ARCH}.so and libcompiler.${MOVEVM_ARCH}.so must verify
integrity: update the Dockerfile's RUN sequence to also download the
corresponding checksum file (or a .sha256 file) for the given MOVEVM_VERSION and
MOVEVM_ARCH, compute the local checksum (e.g., sha256sum) and compare it to the
expected value before moving files into /lib, failing the build on mismatch;
alternatively fetch and verify a signed checksum using gpg if available (ensure
to reference MOVEVM_VERSION, MOVEVM_ARCH, and the two target filenames
libmovevm.${MOVEVM_ARCH}.so and libcompiler.${MOVEVM_ARCH}.so in the same RUN
block so verification happens immediately after wget and before using the
files).
- Around line 3-5: The Dockerfile uses invalid COPY sources ("COPY ../pruner"
and "COPY ../pkg") which reference files outside the build context; update the
COPY commands to use paths available inside the build context (e.g., "COPY
pruner /build/pruner" and "COPY pkg /build/pkg") or alternatively move
Dockerfile.pruner into the repository root or change your docker build context
to the parent directory so WORKDIR /build and the COPY targets resolve
correctly; adjust the Dockerfile entries referencing WORKDIR /build, COPY
../pruner, and COPY ../pkg accordingly.
- Around line 9-17: The Dockerfile (Dockerfile.pruner) currently installs
packages with `apt-get install -y` and leaves the container running as root;
update the RUN instruction that contains `apt-get update && apt-get install -y
ca-certificates wget` to include `--no-install-recommends` and cleanup (apt-get
clean && rm -rf /var/lib/apt/lists/*) and add steps to create a non-root user
(e.g., adduser or groupadd/useradd), chown the copied binary (`/pruner`) to that
user, and switch to it with a `USER` instruction after the `COPY --from=builder
/pruner-binary /pruner` line so the final image does not run as root.
* refactor: update validator retrieval to include all statuses - Modified the `Validators` method calls in `updateValidatorHistoricalPower` and `updateValidators` to fetch all validators (bonded, unbonding, unbonded) by using an empty string as the status parameter. This change ensures that the database remains in sync with the complete set of validator data. * feat: enhance validator image handling with compression and encoding - Updated the `ValidatorInfo` struct to include `ImageEncoding` for specifying the encoding type of the image. - Implemented `compressImageBase64` function to gzip base64-encoded images, reducing payload size for improved performance. - Modified the `GetValidators` and `GetValidatorInfo` methods to utilize the new image compression logic, ensuring efficient image data retrieval. - Added `normalizeImageToJPEG` function in the indexer cron job to convert images to JPEG format, further optimizing storage and transmission.
* feat: add bimg dependency for image processing - Introduced `github.com/h2non/bimg` v1.1.9 to the project for enhanced image processing capabilities. - Refactored the `normalizeImageToJPEG` function in `jobs.go` to utilize bimg for converting images to JPEG format and resizing them to a specified avatar size, improving image handling efficiency. * refactor: update image normalization logic in normalizeImageToJPEG function - Changed the image cropping option in the normalizeImageToJPEG function from true to false, allowing the entire image to fit within the specified dimensions instead of center cropping. This adjustment improves the handling of wide and tall images, enhancing overall image quality. * refactor: simplify image cropping comment in normalizeImageToJPEG function - Updated the comment for the cropping option in the normalizeImageToJPEG function to clarify its purpose, removing unnecessary details while maintaining the focus on fitting the entire image within the specified dimensions. * build: update Dockerfile to include dependencies for bimg and enable CGO - Added necessary packages (build-essential, libvips-dev, pkg-config) for building the bimg dependency in the Dockerfile. - Enabled CGO during the build process for the generic-indexer binary to support image processing capabilities. - Updated the Ubuntu base image installation to include libvips42 for runtime image handling. * build: remove CGO_ENABLED flag from Dockerfile build command - Updated the Dockerfile to remove the CGO_ENABLED=1 flag during the build of the generic-indexer binary, simplifying the build process while maintaining functionality.
#200) - Modified the query in the `updateValidatorImages` function to remove the condition checking for an empty `identity_image`, allowing for a broader selection of validators for image updates. This change simplifies the logic and ensures that all validators with a non-empty identity are considered for image processing.
- Changed the Swagger documentation URL from `/swagger/doc.json` to `/indexer/swagger/doc.json` in the main application file. - Updated the README to reflect the new endpoint for accessing Swagger documentation.
…203) * chore: update Go modules and dependencies across multiple components - Upgraded Go version to 1.24.0 in various modules. - Updated `cometbft` dependency from v0.38.19 to v0.38.20. - Updated `go-logr/logr` from v1.4.2 to v1.4.3. - Updated `google/go-cmp` from v0.6.0 to v0.7.0. - Updated `testify` from v1.10.0 to v1.11.1. - Added new indirect dependencies including `github.com/alitto/pond` v1.8.3 and `github.com/ledgerwatch/erigon-lib`. - Updated `initia-labs` dependencies to their latest versions, including `OPinit`, `initia`, and `movevm`. * build: upgrade Go version to 1.24 in multiple Dockerfiles
- Updated `cometbft` dependency from v0.38.19-initia.2 to v0.38.21-initia.2 across multiple modules. - Updated `cosmos-sdk` dependency from v0.50.14-initia.2 to v0.50.15-initia.0 in various components. - Ensured consistency in dependency versions across `api`, `generic-indexer`, and `informative-indexer` modules.
- Updated the MOVEVM_VERSION environment variable to v1.2.0 in Dockerfiles for api, event, generic, informative, and sweeper components, ensuring consistency across the project.
* chore: update initia dependency to v1.4.2 across multiple modules - Upgraded the `initia` dependency from v1.4.0 to v1.4.2 in the `api`, `event-indexer`, `generic-indexer`, `informative-indexer`, `pkg`, and `sweeper` modules. - Updated related `go.sum` files to reflect the new version of `initia` and ensure consistency across the project. * feat: enhance go.mod update process with missing replace directive checks - Introduced a mechanism to track and report missing replace directives in go.mod files during the update process. - Added error handling to notify users of missing replacements, improving the robustness of the dependency management script.
- Upgraded the `initia` dependency from v1.4.2 to v1.4.3 in the `api`, `event-indexer`, `generic-indexer`, `informative-indexer`, `pkg`, and `sweeper` modules. - Updated related `go.sum` files to reflect the new version of `initia` and ensure consistency across the project. - Enhanced the dependency management script to automatically add missing replace directives in go.mod files.
#209) * feat: implement custom memory-based LRU cache, add runtime debug metrics, and externalize performance configurations * fix: remove duplicate runtime metric * feat: replace dynamic size estimation with precalculated CacheSizeBytes in TxByHashResponse * fix: add context cancellation support to GCS task result collection loop * refactor: use range over integer in GCS worker initialization for cleaner syntax * refactor: replace custom worker pool with errgroup for concurrent transaction fetching in GCS service
…202) * fix: enhance error handling in GetBlock and GetBlockResults methods - Updated the retry logic in the GetBlock and GetBlockResults methods to trigger a rebalance of RPC clients every 10 retries, improving error handling and client management. - Added logging for errors encountered during the rebalance process to aid in debugging and monitoring. * fix: improve retry logic in GetBlock and GetBlockResults methods - Added a retryCount increment in the error handling of GetBlock and GetBlockResults methods to ensure accurate tracking of retries. - This change enhances the existing error handling by maintaining a consistent retry count for better management of RPC client rebalances. * fix: streamline error handling in GetBlock and GetBlockResults methods - Refactored error handling in the GetBlock and GetBlockResults methods to improve clarity and maintainability. - Simplified the rebalance error logging by using a single variable for the rebalance error, enhancing readability and reducing redundancy. * fix: enhance RPC client rebalance handling in GetBlock and GetBlockResults methods - Introduced a new RebalanceRPCs method to encapsulate rebalance logic and error handling, improving code organization and readability. - Updated GetBlock and GetBlockResults methods to utilize the new RebalanceRPCs method, streamlining error handling and reducing redundancy in logging. - Added logging for active RPC clients to aid in monitoring and debugging. * feat: add error handling for zero active clients in RPC methods - Introduced a new errors.go file with a custom error for zero active clients. - Updated GetBlock, GetBlockResults, and RebalanceRPCs methods to return errors, improving error handling and clarity in RPC interactions. - Enhanced the StartSweeping method to handle errors from GetBlockFromRPCAndProduce, ensuring robust error management during block processing. * fix: update logging level for error handling in GetBlockFromRPCAndProduce method - Changed logging from Fatal to Error for block retrieval and message production errors in the GetBlockFromRPCAndProduce method, improving error handling and preventing application termination on recoverable errors.
…oss repositories (#196) - Updated multiple repository files to replace calls to utils.CountWithTimeout with the new db.CountWithTimeout function, ensuring consistent timeout handling for count queries. - Removed the .air.toml file from the api directory and added Go binaries to .gitignore to streamline the build process. - Introduced a new count.go file in the db package to implement the CountWithTimeout function, enhancing query execution with timeout management.
#210) * chore: remove hardcoded environment variables and clean up Helm chart YAML formatting * chore: reformat Helm chart values YAML for improved readability * chore: remove trailing whitespace in annotations configuration
* fix: dependabot issues * build: upgrade base Go image version to 1.25 across all Dockerfiles
2ae5828 to
1a902e8
Compare
1a902e8 to
2472c5a
Compare
There was a problem hiding this comment.
Actionable comments posted: 3
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
pruner/pruner/pruner.go (2)
231-243:⚠️ Potential issue | 🟠 Major | ⚡ Quick win
p.close()races with theStartPruninggoroutine — no graceful drain before DB shutdown.
Run()callsp.close()(closes the DB connection) immediately after<-ctx.Done()returns, without waiting for theStartPruninggoroutine to finish its current pruning cycle. If the goroutine is mid-way throughpruningTable(insidestreamBackupToGCSorpruneRows) when the DB is closed, it will hit "sql: database is closed" errors on in-flight operations.Additionally,
StartPruninguses a plaintime.Sleep(pruningInterval)(up to 24 hours) rather than a context-aware select, so the goroutine cannot respond promptly to cancellation during the sleep phase.Suggested fix:
🛠️ Proposed fix for graceful shutdown
+func (p *Prunner) Run() { + p.once.Do(func() { + ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM) + defer stop() + + var wg sync.WaitGroup + wg.Add(1) + go func() { + defer wg.Done() + p.StartPruning(ctx) + }() + + <-ctx.Done() + + wg.Wait() // drain the current pruning cycle before closing + logger.Info().Msgf("Stopping prunner ...") + p.close() + }) +}And replace
time.SleepinStartPruningwith a context-aware sleep:- time.Sleep(pruningInterval) + select { + case <-signalCtx.Done(): + return + case <-time.After(pruningInterval): + }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@pruner/pruner/pruner.go` around lines 231 - 243, The Run method currently calls p.close() right after signaling cancel which can race with the StartPruning goroutine (which may be inside pruningTable/streamBackupToGCS/pruneRows); update Run to wait for StartPruning to finish before closing the DB (e.g., add a sync.WaitGroup or a done channel on the Prunner struct, have StartPruning signal done when it exits, and only call p.close() after waiting). Also make StartPruning responsive to cancellation by replacing time.Sleep(pruningInterval) with a context-aware wait (select between time.After(pruningInterval) and ctx.Done()) so the goroutine wakes promptly on shutdown and drains in-flight work before Run closes the DB.
44-44:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winLogger component name
"event-indexer-prunner"is misleading for a standalone service.Since the pruner is now its own binary/service, the component label in structured logs should reflect that rather than inheriting the old event-indexer name.
✏️ Proposed fix
-logger = zerolog.Ctx(log.With().Str("component", "event-indexer-prunner").Str("chain", config.Chain)... +logger = zerolog.Ctx(log.With().Str("component", "pruner").Str("chain", config.Chain)...🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@pruner/pruner/pruner.go` at line 44, The structured logger assignment sets the component label to "event-indexer-prunner", which is misleading for this standalone service; update the logger initialization (the logger variable assignment that uses zerolog.Ctx and config.Chain/config.Environment/config.CommitSHA) to use a correct component name such as "pruner" (or another agreed service name) instead of "event-indexer-prunner" so logs correctly identify this binary.
♻️ Duplicate comments (5)
pruner/run.sh (2)
5-5:⚠️ Potential issue | 🟠 Major | ⚡ Quick win
SCRIPT_DIRis computed but never used;go buildandsource .envrun from the caller's working directory.If
run.shis invoked from the repo root,go buildwill build the wrong module andsource .envwill look for.envin the wrong place.Also applies to: 20-22
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@pruner/run.sh` at line 5, The SCRIPT_DIR variable is computed but never used, so go build and source .env run from the caller's CWD; update run.sh to use SCRIPT_DIR by either changing to that directory before executing commands or running commands with SCRIPT_DIR as their working dir (e.g., cd "$SCRIPT_DIR" && go build ... and source "$SCRIPT_DIR/.env"), and apply the same change to the other occurrences on lines 20-22 so all go build and source invocations operate relative to SCRIPT_DIR.
8-8:⚠️ Potential issue | 🟡 Minor | ⚡ Quick win
ARGS=${@:2}flattens the argument array; arguments with spaces will be split incorrectly.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@pruner/run.sh` at line 8, The ARGS assignment uses ARGS=${@:2}, which collapses positional parameters into a single word and will split arguments containing spaces; change ARGS to be an array capturing the leftover positional parameters (use shell array assignment from "${@:2}") and whenever ARGS is passed to commands expand it with the array form (use "${ARGS[@]}") so quoted arguments are preserved; update the assignment for ARGS and any subsequent usage (e.g., command invocations) that currently expand ARGS plain to the array-safe forms.pruner/go.mod (1)
58-58:⚠️ Potential issue | 🟠 Major | ⚡ Quick winOSV vulnerabilities previously flagged — still unresolved.
The past review already identified these indirect dependencies as needing upgrades:
github.com/dvsekhvalnov/jose2gov1.6.0 (line 77): upgrade to v1.7.0+ (CVE-2025-63811 / GHSA-9mj6-hxhv-w67j — DoS via JWT bomb).golang.org/x/oauth2v0.24.0 (line 200): upgrade to v0.27.0+ (GO-2025-3488 / GHSA-6v2p-p543-phr9 — memory exhaustion).github.com/cometbft/cometbftv0.38.19 (line 58): previously considered patched, but two new high-severity advisories have since been published (GO-2025-3442 — node stuck in blocksync; GO-2026-4361 — block time derivation inconsistency). A newer patch release is required.Also applies to: 77-77, 200-200
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@pruner/go.mod` at line 58, Update the indirect vulnerable dependencies in go.mod by bumping the module versions: change github.com/dvsekhvalnov/jose2go to v1.7.0+ (use the latest patch >= v1.7.0), golang.org/x/oauth2 to v0.27.0+, and upgrade github.com/cometbft/cometbft to the latest patched release addressing GO-2025-3442/GO-2026-4361; perform the changes by running go get github.com/dvsekhvalnov/jose2go@v1.7.0 golang.org/x/oauth2@v0.27.0 github.com/cometbft/cometbft@<latest-patch>, then run go mod tidy and run the test suite to ensure nothing breaks.dockerfiles/Dockerfile.pruner (2)
4-5:⚠️ Potential issue | 🔴 Critical | ⚡ Quick win
COPY ../prunerandCOPY ../pkgreference paths outside the build context — build will fail.Docker forbids COPY sources that traverse above the build context. These paths are still invalid.
🛠️ Proposed fix
-COPY ../pruner /build/pruner -COPY ../pkg /build/pkg +COPY pruner /build/pruner +COPY pkg /build/pkg🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@dockerfiles/Dockerfile.pruner` around lines 4 - 5, The Dockerfile.pruner uses invalid sources COPY ../pruner and COPY ../pkg which reference files outside the build context and will fail; change the Dockerfile to COPY pruner /build/pruner and COPY pkg /build/pkg (or otherwise remove the leading ../) and ensure the docker build is executed with a build context that includes those directories (e.g., run docker build from the repository root or update CI build context) so the paths in Dockerfile.pruner resolve correctly.
9-30:⚠️ Potential issue | 🟠 Major | ⚡ Quick winThree previously flagged hardening issues remain unresolved.
- Running as root (no
USERinstruction): Add a non-root user and switch to it beforeCMD.apt-getwithout--no-install-recommends: Inflates image size unnecessarily.- No checksum verification on MoveVM libraries: Downloads are trusted without integrity checks — a supply-chain risk.
🔐 Combined fix sketch
-RUN apt-get update && apt-get install -y ca-certificates wget +RUN apt-get update && apt-get install -y --no-install-recommends ca-certificates wget \ + && rm -rf /var/lib/apt/lists/* RUN ARCH=$(dpkg --print-architecture) && \ ... \ - wget -O /lib/libmovevm.${MOVEVM_ARCH}.so \ - https://github.com/initia-labs/movevm/releases/download/${MOVEVM_VERSION}/libmovevm.${MOVEVM_ARCH}.so && \ - wget -O /lib/libcompiler.${MOVEVM_ARCH}.so \ - https://github.com/initia-labs/movevm/releases/download/${MOVEVM_VERSION}/libcompiler.${MOVEVM_ARCH}.so + wget -O /tmp/libmovevm.${MOVEVM_ARCH}.so \ + https://github.com/initia-labs/movevm/releases/download/${MOVEVM_VERSION}/libmovevm.${MOVEVM_ARCH}.so && \ + wget -O /tmp/libcompiler.${MOVEVM_ARCH}.so \ + https://github.com/initia-labs/movevm/releases/download/${MOVEVM_VERSION}/libcompiler.${MOVEVM_ARCH}.so && \ + wget -O /tmp/checksums.txt \ + https://github.com/initia-labs/movevm/releases/download/${MOVEVM_VERSION}/checksums.txt && \ + sha256sum -c /tmp/checksums.txt --ignore-missing && \ + mv /tmp/libmovevm.${MOVEVM_ARCH}.so /lib/ && \ + mv /tmp/libcompiler.${MOVEVM_ARCH}.so /lib/ +RUN useradd -r -u 10001 pruner && chown 10001:10001 /pruner +USER 10001🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@dockerfiles/Dockerfile.pruner` around lines 9 - 30, The Dockerfile currently installs packages as root, uses apt-get without --no-install-recommends, and downloads MoveVM libraries without integrity checks; update the Dockerfile to (1) install packages with apt-get update && apt-get install -y --no-install-recommends ca-certificates wget (and clean apt caches in the same RUN), (2) add a non-root user (e.g., create user/group and chown /app and binaries) and include a USER instruction so the image does not run as root before CMD, and (3) verify MoveVM library integrity by fetching and validating a checksum/signature before trusting the wget downloads for libmovevm.${MOVEVM_ARCH}.so and libcompiler.${MOVEVM_ARCH}.so (use the MOVEVM_VERSION variable to locate the checksum file and validate via sha256sum or signature check, failing the build on mismatch). Ensure permissions for the non-root user allow execution of /pruner and the libraries.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@charts/pruner/templates/deployment.yaml`:
- Around line 13-16: Deployment-level annotations are being rendered from
.Values.annotations under the Deployment metadata but pod-level annotations must
live under spec.template.metadata; add support for a separate
.Values.podAnnotations used in the pod template (rendered under
spec.template.metadata via toYaml | nindent) while keeping .Values.annotations
for Deployment metadata, and update values.yaml to document/define
podAnnotations; locate the annotations block in the Deployment template (the
current annotations rendering) and add a similar conditional block inside
spec.template.metadata for podAnnotations (e.g., check .Values.podAnnotations
and render it).
In `@pruner/go.mod`:
- Line 133: Update the indirect dependencies in go.mod to the patched versions
to address the listed OSV advisories: bump github.com/jackc/pgx/v5 to v5.9.2 (or
at least v5.9.2+), google.golang.org/grpc to v1.79.3+, and
go.opentelemetry.io/otel/sdk to v1.43.0+; run the suggested command (go get
github.com/jackc/pgx/v5@v5.9.2 google.golang.org/grpc@v1.79.3
go.opentelemetry.io/otel/sdk@v1.43.0 && go mod tidy) to update go.mod/go.sum and
ensure the fixes for GHSA-j88v-2chj-qfwx, GHSA-9jj7-4m8r-rfcm,
GHSA-p77j-4mvh-x3m3, GHSA-9h8m-3fm2-qjrq and GHSA-hfvc-g4fc-pqhx are applied.
In `@pruner/run.sh`:
- Line 11: The help string for the run command is a copy-paste artifact — update
the variable help__run in run.sh so it contains a meaningful description instead
of "run run"; replace the value of help__run with a concise, accurate message
(e.g., describing that it executes the pruner or runs the pruning job and
accepts additional args) so users see a helpful usage line.
---
Outside diff comments:
In `@pruner/pruner/pruner.go`:
- Around line 231-243: The Run method currently calls p.close() right after
signaling cancel which can race with the StartPruning goroutine (which may be
inside pruningTable/streamBackupToGCS/pruneRows); update Run to wait for
StartPruning to finish before closing the DB (e.g., add a sync.WaitGroup or a
done channel on the Prunner struct, have StartPruning signal done when it exits,
and only call p.close() after waiting). Also make StartPruning responsive to
cancellation by replacing time.Sleep(pruningInterval) with a context-aware wait
(select between time.After(pruningInterval) and ctx.Done()) so the goroutine
wakes promptly on shutdown and drains in-flight work before Run closes the DB.
- Line 44: The structured logger assignment sets the component label to
"event-indexer-prunner", which is misleading for this standalone service; update
the logger initialization (the logger variable assignment that uses zerolog.Ctx
and config.Chain/config.Environment/config.CommitSHA) to use a correct component
name such as "pruner" (or another agreed service name) instead of
"event-indexer-prunner" so logs correctly identify this binary.
---
Duplicate comments:
In `@dockerfiles/Dockerfile.pruner`:
- Around line 4-5: The Dockerfile.pruner uses invalid sources COPY ../pruner and
COPY ../pkg which reference files outside the build context and will fail;
change the Dockerfile to COPY pruner /build/pruner and COPY pkg /build/pkg (or
otherwise remove the leading ../) and ensure the docker build is executed with a
build context that includes those directories (e.g., run docker build from the
repository root or update CI build context) so the paths in Dockerfile.pruner
resolve correctly.
- Around line 9-30: The Dockerfile currently installs packages as root, uses
apt-get without --no-install-recommends, and downloads MoveVM libraries without
integrity checks; update the Dockerfile to (1) install packages with apt-get
update && apt-get install -y --no-install-recommends ca-certificates wget (and
clean apt caches in the same RUN), (2) add a non-root user (e.g., create
user/group and chown /app and binaries) and include a USER instruction so the
image does not run as root before CMD, and (3) verify MoveVM library integrity
by fetching and validating a checksum/signature before trusting the wget
downloads for libmovevm.${MOVEVM_ARCH}.so and libcompiler.${MOVEVM_ARCH}.so (use
the MOVEVM_VERSION variable to locate the checksum file and validate via
sha256sum or signature check, failing the build on mismatch). Ensure permissions
for the non-root user allow execution of /pruner and the libraries.
In `@pruner/go.mod`:
- Line 58: Update the indirect vulnerable dependencies in go.mod by bumping the
module versions: change github.com/dvsekhvalnov/jose2go to v1.7.0+ (use the
latest patch >= v1.7.0), golang.org/x/oauth2 to v0.27.0+, and upgrade
github.com/cometbft/cometbft to the latest patched release addressing
GO-2025-3442/GO-2026-4361; perform the changes by running go get
github.com/dvsekhvalnov/jose2go@v1.7.0 golang.org/x/oauth2@v0.27.0
github.com/cometbft/cometbft@<latest-patch>, then run go mod tidy and run the
test suite to ensure nothing breaks.
In `@pruner/run.sh`:
- Line 5: The SCRIPT_DIR variable is computed but never used, so go build and
source .env run from the caller's CWD; update run.sh to use SCRIPT_DIR by either
changing to that directory before executing commands or running commands with
SCRIPT_DIR as their working dir (e.g., cd "$SCRIPT_DIR" && go build ... and
source "$SCRIPT_DIR/.env"), and apply the same change to the other occurrences
on lines 20-22 so all go build and source invocations operate relative to
SCRIPT_DIR.
- Line 8: The ARGS assignment uses ARGS=${@:2}, which collapses positional
parameters into a single word and will split arguments containing spaces; change
ARGS to be an array capturing the leftover positional parameters (use shell
array assignment from "${@:2}") and whenever ARGS is passed to commands expand
it with the array form (use "${ARGS[@]}") so quoted arguments are preserved;
update the assignment for ARGS and any subsequent usage (e.g., command
invocations) that currently expand ARGS plain to the array-safe forms.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 426c1bf3-4112-4cf0-8b7e-19b258e75b56
⛔ Files ignored due to path filters (1)
pruner/go.sumis excluded by!**/*.sum
📒 Files selected for processing (14)
.github/workflows/unified-deployment.ymlcharts/pruner/Chart.yamlcharts/pruner/templates/deployment.yamlcharts/pruner/values.yamldockerfiles/Dockerfile.prunerevent-indexer/cmd/root.goevent-indexer/run.shpruner/.env.examplepruner/cmd/pruner/cmd.gopruner/cmd/root.gopruner/go.modpruner/main.gopruner/pruner/pruner.gopruner/run.sh
💤 Files with no reviewable changes (2)
- event-indexer/run.sh
- event-indexer/cmd/root.go
✅ Files skipped from review due to trivial changes (4)
- .github/workflows/unified-deployment.yml
- charts/pruner/values.yaml
- charts/pruner/Chart.yaml
- pruner/main.go
🚧 Files skipped from review as they are similar to previous changes (1)
- pruner/cmd/root.go
| {{- with .Values.annotations }} | ||
| annotations: | ||
| {{- toYaml . | nindent 6 }} | ||
| {{- end }} |
There was a problem hiding this comment.
Deployment-level annotations will not apply to pods — consider adding a separate pod template annotations block.
.Values.annotations is rendered under the Deployment's own metadata, not under spec.template.metadata. Annotations commonly expected on pods (Prometheus scraping, Datadog agent configuration, etc.) must be on the pod template to take effect. If this values key is intended only for Deployment-level annotations, the naming is fine; if pod annotations are also needed, a separate podAnnotations value is required.
✏️ Proposed fix (add pod-level annotations support)
template:
metadata:
labels:
app: {{ quote .Release.Name }}
+ {{- with .Values.podAnnotations }}
+ annotations:
+ {{- toYaml . | nindent 8 }}
+ {{- end }}
spec:And add to values.yaml:
+podAnnotations: {}Also applies to: 24-27
🧰 Tools
🪛 YAMLlint (1.38.0)
[error] 14-14: syntax error: could not find expected ':'
(syntax)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@charts/pruner/templates/deployment.yaml` around lines 13 - 16,
Deployment-level annotations are being rendered from .Values.annotations under
the Deployment metadata but pod-level annotations must live under
spec.template.metadata; add support for a separate .Values.podAnnotations used
in the pod template (rendered under spec.template.metadata via toYaml | nindent)
while keeping .Values.annotations for Deployment metadata, and update
values.yaml to document/define podAnnotations; locate the annotations block in
the Deployment template (the current annotations rendering) and add a similar
conditional block inside spec.template.metadata for podAnnotations (e.g., check
.Values.podAnnotations and render it).
| ARGS=${@:2} | ||
|
|
||
|
|
||
| help__run="run <..args> : run run" |
There was a problem hiding this comment.
Help string "run run" appears to be a copy-paste artifact.
The description just reads "run run" instead of describing the task.
✏️ Proposed fix
-help__run="run <..args> : run run"
+help__run="run <chain> : build and run the pruner for the given chain"📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| help__run="run <..args> : run run" | |
| help__run="run <chain> : build and run the pruner for the given chain" |
🧰 Tools
🪛 Shellcheck (0.11.0)
[warning] 11-11: help__run appears unused. Verify use (or export if used externally).
(SC2034)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@pruner/run.sh` at line 11, The help string for the run command is a
copy-paste artifact — update the variable help__run in run.sh so it contains a
meaningful description instead of "run run"; replace the value of help__run with
a concise, accurate message (e.g., describing that it executes the pruner or
runs the pruning job and accepts additional args) so users see a helpful usage
line.
Summary by CodeRabbit
New Features
Refactor