Introduce MDBX stream storage #1908
Draft
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull request introduces significant updates to the relay server, gRPC integration, and data stream handling, as well as minor improvements to the codebase and configuration files. The most important changes include the addition of gRPC support, refactoring of the relay server for modularity, and updates to the data stream storage configuration.
Relay Server and gRPC Integration:
cmd/relay/main.goto modularize the relay server setup, add gRPC server support, and improve logging. Introduced aRelayConfigstruct for configuration and created helper functions likecreateRelayServer,setupGRPCServer, andhandleSignalsfor better code organization. [1] [2]cmd/relay/mock_factory.goto provide a mock implementation ofDatastreamFactoryfor testing gRPC-related functionality.docker-compose.ymlto expose both relay and gRPC ports and adjusted thecommandto include the gRPC port.Data Stream Enhancements:
DataStreamStorageTypeconfiguration option to support different storage backends (e.g.,file,mdbx). This change affects multiple files, includingcmd/rpcdaemon/cli/httpcfg/http_cfg.go,cmd/utils/flags.go, andturbo/cli/default_flags.go. [1] [2] [3]eth/backend.goto use the newDataStreamStorageTypeconfiguration when creating the data stream server. [1] [2]Testing and Build Improvements:
cmd/relay/main_test.goto test flag parsing, gRPC server setup, and signal handling for the relay server.Makefileto include gRPC code generation fordatastream_service.proto.Dependency and Import Updates:
go.modto include the latest version ofzkevm-data-streamerand other dependencies. [1] [2] [3]zk/datastream/client/stream_client.goandzk/datastream/client/stream_client_test.goto use the correct proto package. [1] [2]Configuration and Dockerfile Changes:
7070toDockerfile,Dockerfile.debian, andDockerfile.releasefor consistent exposure across environments. [1] [2] [3]