Skip to content

Commit a3dfde5

Browse files
JoshuahokyJoshuahoky
committed
Merge remote-tracking branch 'upstream/main' into feat/shmipc_fallback
2 parents 6b3a4a0 + 01a614b commit a3dfde5

File tree

8 files changed

+660
-0
lines changed

8 files changed

+660
-0
lines changed

‎CLAUDE.md‎

Lines changed: 102 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,102 @@
1+
# CLAUDE.md - Volo Workspace
2+
3+
For detailed per-crate documentation, see the CLAUDE.md in each sub-crate directory.
4+
5+
## Project Overview
6+
7+
[Volo](https://github.com/cloudwego/volo) is a high-performance Rust RPC framework by CloudWeGo (ByteDance). It supports Thrift, gRPC, and HTTP protocols with fully async design (Tokio), zero-copy optimizations, and middleware via Service/Layer abstractions (motore).
8+
9+
- Rust Edition: 2024
10+
- MSRV: 1.85.0
11+
- Current Version: 0.12.x
12+
13+
## Workspace Structure
14+
15+
```
16+
volo/
17+
├── volo/ # Core library
18+
├── volo-build/ # Code generation from IDL (Thrift/Protobuf)
19+
├── volo-cli/ # CLI tool (project scaffolding)
20+
├── volo-grpc/ # gRPC implementation
21+
├── volo-http/ # HTTP implementation
22+
├── volo-macros/ # Procedural macros (reserved)
23+
├── volo-thrift/ # Thrift implementation
24+
├── examples/ # Example code
25+
├── benchmark/ # Performance benchmarks
26+
└── tests/code-generation/ # Code generation tests
27+
```
28+
29+
## Crate Dependency Graph
30+
31+
```
32+
volo-macros (reserved)
33+
|
34+
v
35+
+------------------ volo ------------------+
36+
| | |
37+
v v v
38+
volo-thrift volo-grpc volo-http
39+
| | |
40+
+----------+----------+ |
41+
v |
42+
volo-build <----------------------------+
43+
|
44+
v
45+
volo-cli
46+
```
47+
48+
## Crate Overview
49+
50+
- **volo**: Core abstractions -- service discovery (`Discover`), load balancing (`LoadBalance`), network transport (`Address`, `Conn`, TCP/Unix/TLS/ShmIPC), context (`RpcCx`, `RpcInfo`, `Endpoint`), hot restart, panic capture
51+
- **volo-thrift**: TTHeader/Framed transport, Binary/Compact protocols, Ping-Pong/Multiplex modes, connection pooling, ISN-based multi-service routing, BizError
52+
- **volo-grpc**: HTTP/2 (hyper), unary/streaming calls, compression (gzip/zlib/zstd), gRPC-Web, metadata
53+
- **volo-http**: Server (Router/Handler/Extractor), Client (connection pooling/DNS/proxy), JSON/Form/Multipart/WebSocket/SSE, TLS (Rustls/Native-TLS)
54+
- **volo-build**: Generates Rust code from Thrift/Protobuf IDL. Config: `volo.yml` / `volo.workspace.yml`
55+
- **volo-cli**: `volo init`, `volo http init`, `volo idl add`, `volo repo add/update`, `volo migrate`
56+
- **volo-macros**: Reserved. Active macros: `#[service]` (from motore), `volo_unreachable!`, `new_type!` (from volo)
57+
58+
## Feature Flags Summary
59+
60+
| Feature | volo | volo-thrift | volo-grpc | volo-http |
61+
| ------------- | ---- | ----------- | ---------- | ---------- |
62+
| `rustls` | Y | - | Y | Y |
63+
| `native-tls` | Y | - | Y | Y |
64+
| `shmipc` | Y | Y | - | - |
65+
| `multiplex` | - | Y | - | - |
66+
| `gzip`/`zlib` | - | - | Y(default) | - |
67+
| `zstd` | - | - | Y | - |
68+
| `grpc-web` | - | - | Y | - |
69+
| `json` | - | - | - | Y(default) |
70+
| `ws` | - | - | - | Y |
71+
| `cookie` | - | - | - | Y |
72+
73+
## Core Abstractions
74+
75+
All built on the `motore` crate's `Service<Cx, Request>` and `Layer<S>` traits.
76+
77+
- **Service Discovery**: `Discover` trait (volo) -- `StaticDiscover`, `WeightedStaticDiscover`
78+
- **Load Balancing**: `LoadBalance` trait (volo) -- weighted random, consistent hashing
79+
80+
## Design Patterns
81+
82+
- **Builder**: `XxxBuilder::new().xxx().build()`
83+
- **Make**: `MakeXxx` trait creates `Xxx` instances
84+
- **Layer**: `XxxLayer` implements `motore::layer::Layer`
85+
- **Service**: Implements `motore::service::Service`
86+
- **Private features**: Prefixed with `__` (e.g., `__tls`)
87+
88+
## Commit Conventions
89+
90+
Follow [Conventional Commits](https://www.conventionalcommits.org/): `feat(volo-thrift): add multi-service support`, `fix(volo-http): resolve connection pool leak`, `chore: update dependencies`
91+
92+
## Release Order
93+
94+
Publish in this order:
95+
96+
1. `volo-macros`
97+
2. `volo`
98+
3. `volo-build`
99+
4. `volo-cli`
100+
5. `volo-thrift`
101+
6. `volo-grpc`
102+
7. `volo-http` (released independently)

‎volo-build/CLAUDE.md‎

Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,78 @@
1+
# CLAUDE.md - volo-build
2+
3+
## Overview
4+
5+
`volo-build` is the code generation tool for the Volo framework, compiling Thrift and Protobuf IDL files into Rust code at build time. It is the underlying dependency of `volo-cli` and is typically used in `build.rs`.
6+
7+
## Directory Structure
8+
9+
```
10+
volo-build/src/
11+
├── lib.rs # Library entry, defines Builder and public exports
12+
├── model.rs # Configuration model (SingleConfig, Entry, Service, Idl, etc.)
13+
├── config_builder.rs # ConfigBuilder and InitBuilder
14+
├── thrift_backend.rs # Thrift code generation backend
15+
├── grpc_backend.rs # gRPC/Protobuf code generation backend
16+
├── util.rs # Git operations, file operations, config read/write
17+
├── workspace.rs # Workspace mode support
18+
└── legacy/ # Legacy configuration format compatibility
19+
```
20+
21+
## Builder (`lib.rs`)
22+
23+
Main code generation builder supporting both Thrift and Protobuf protocols. Created via `Builder::thrift()` or `Builder::protobuf()`.
24+
25+
Key methods: `add_service(path)`, `out_dir(path)`, `filename(name)`, `plugin(p)`, `ignore_unused(bool)`, `touch(items)`, `keep_unknown_fields(paths)`, `split_generated_files(bool)`, `special_namings(namings)`, `dedup(list)`, `common_crate_name(name)`, `with_descriptor(bool)`, `with_field_mask(bool)`, `with_comments(bool)`, `include_dirs(dirs)`, `write()`, `init_service()`.
26+
27+
## ConfigBuilder (`config_builder.rs`)
28+
29+
Configuration file-based (`volo.yml`) code generation builder. Use `ConfigBuilder::default().write()` for the default config file, or `ConfigBuilder::new(path)` for a custom one. Supports adding plugins via `.plugin(p)`.
30+
31+
Also provides `InitBuilder` for initializing new services.
32+
33+
## Configuration Model (`model.rs`)
34+
35+
`SingleConfig` is the root structure. Key types: `Entry` (code generation entry), `Service` (service definition with IDL and codegen options), `Idl` (IDL file source and path), `Source` (Local or Git), `Repo` (Git repository config), `CodegenOption`, `CommonOption`.
36+
37+
Example `volo.yml`:
38+
39+
```yaml
40+
entries:
41+
default:
42+
filename: volo_gen.rs
43+
protocol: thrift
44+
repos:
45+
my_repo:
46+
url: https://github.com/example/idl.git
47+
ref: main
48+
lock: abc123
49+
services:
50+
- idl:
51+
source: local
52+
path: ./idl/service.thrift
53+
codegen_option:
54+
touch: ["MyService"]
55+
touch_all: false
56+
dedups: []
57+
special_namings: []
58+
split_generated_files: false
59+
```
60+
61+
## Thrift Backend (`thrift_backend.rs`)
62+
63+
Implements `pilota_build::CodegenBackend` for Thrift services. Generates: `{ServiceName}Server`, `{ServiceName}Client`, `{ServiceName}GenericClient`, `{ServiceName}OneShotClient`, `{ServiceName}ClientBuilder`, `{ServiceName}RequestSend/Recv`, `{ServiceName}ResponseSend/Recv`. Supports exception handling, oneway methods, multi-service routing, and split file generation.
64+
65+
## gRPC Backend (`grpc_backend.rs`)
66+
67+
Implements `pilota_build::CodegenBackend` for gRPC services. Generates the same type pattern as Thrift (`Server`, `Client`, `GenericClient`, `OneShotClient`, `ClientBuilder`, `RequestSend/Recv`, `ResponseSend/Recv`). Supports client streaming, server streaming, and bidirectional streaming.
68+
69+
## Workspace Support (`workspace.rs`)
70+
71+
Supports code generation for multi-crate workspaces via `volo.workspace.yml`. Use `workspace::Builder::thrift().gen()` or `workspace::Builder::protobuf().gen()`.
72+
73+
## Notes
74+
75+
1. **OUT_DIR**: Must be run in `build.rs`; depends on the `OUT_DIR` environment variable.
76+
2. **Git Operations**: Requires system `git` CLI installed.
77+
3. **Config Migration**: Legacy configuration formats need migration to the new format. See https://www.cloudwego.io/docs/volo/guide/config/
78+
4. **Split Files**: Enabling `split_generated_files` generates multiple files, suitable for large projects.

‎volo-cli/CLAUDE.md‎

Lines changed: 76 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,76 @@
1+
# CLAUDE.md - volo-cli
2+
3+
## Project Overview
4+
5+
`volo-cli` is the command line tool for the Volo framework, used for creating and managing Volo-based RPC and HTTP service projects from IDL files or templates.
6+
7+
## Directory Structure
8+
9+
```
10+
volo-cli/
11+
└── src/
12+
├── bin/
13+
│ └── volo.rs # CLI entry point (main function)
14+
├── lib.rs # Library root, defines macros and exports modules
15+
├── command.rs # CliCommand trait and define_commands! macro
16+
├── context.rs # Context struct
17+
├── model.rs # RootCommand and subcommand definitions
18+
├── init.rs # `volo init` command implementation
19+
├── http.rs # `volo http` command implementation
20+
├── migrate.rs # `volo migrate` command implementation
21+
├── idl/
22+
│ ├── mod.rs # `volo idl` command entry
23+
│ └── add.rs # `volo idl add` subcommand
24+
├── repo/
25+
│ ├── mod.rs # `volo repo` command entry
26+
│ ├── add.rs # `volo repo add` subcommand
27+
│ └── update.rs # `volo repo update` subcommand
28+
└── templates/ # Project template files
29+
├── thrift/ # Thrift project templates
30+
├── grpc/ # gRPC project templates
31+
└── http/ # HTTP project templates
32+
```
33+
34+
## Commands
35+
36+
| Command | Module | Description |
37+
| -------------------------- | ---------------- | ------------------------------------------- |
38+
| `volo init <name> <idl>` | `init.rs` | Initialize Thrift/gRPC project |
39+
| `volo http init <name>` | `http.rs` | Initialize HTTP project |
40+
| `volo idl add <idl>` | `idl/add.rs` | Add IDL file to existing project |
41+
| `volo repo add -g <git>` | `repo/add.rs` | Add Git repository as IDL source |
42+
| `volo repo update [repos]` | `repo/update.rs` | Update specified or all Git repository IDLs |
43+
| `volo migrate` | `migrate.rs` | Migrate legacy configuration to new format |
44+
45+
## Key Macros
46+
47+
### `define_commands!` (`src/command.rs`)
48+
49+
Batch-defines subcommand enums and auto-implements the `CliCommand` trait, simplifying command dispatch logic. All commands implement the `CliCommand` trait (`fn run(&self, cx: Context) -> anyhow::Result<()>`).
50+
51+
### `templates_to_target_file!` (`src/lib.rs`)
52+
53+
Outputs template files to a target path with parameter substitution:
54+
55+
```rust
56+
templates_to_target_file!(folder, "templates/thrift/cargo_toml", "Cargo.toml", name = &name);
57+
```
58+
59+
## Project Templates
60+
61+
- **Thrift** (`templates/thrift/`): Standard Thrift RPC project with `volo-gen` sub-crate for code generation
62+
- **gRPC** (`templates/grpc/`): Protobuf-based gRPC project, similar structure to Thrift
63+
- **HTTP** (`templates/http/`): Pure HTTP project using `volo-http` Router pattern, no `volo-gen` sub-crate
64+
65+
## Logging
66+
67+
- Default log level is `WARN`
68+
- Use `-v` to raise to `INFO`, `-vv` for `DEBUG`, `-vvv` for `TRACE`
69+
- Version update check runs at startup; disable with env var `VOLO_DISABLE_UPDATE_CHECK`
70+
71+
## Notes
72+
73+
1. The init command checks if `volo.yml` already exists to avoid overwriting existing configuration
74+
2. IDL protocol must be consistent with existing entry configuration (cannot mix Thrift and Protobuf)
75+
3. After initialization, `cargo fmt --all` is automatically run to format generated code
76+
4. After initialization, Git repository is automatically initialized (if not already present)

‎volo-grpc/CLAUDE.md‎

Lines changed: 92 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,92 @@
1+
# CLAUDE.md - volo-grpc
2+
3+
## Overview
4+
5+
`volo-grpc` is the gRPC implementation of the Volo framework, providing async gRPC client and server based on HTTP/2 (hyper). Supports unary, client streaming, server streaming, and bidirectional streaming calls with gzip/zlib/zstd compression, gRPC-Web, and TLS.
6+
7+
**Documentation:** https://docs.rs/volo-grpc
8+
9+
## Directory Structure
10+
11+
```
12+
volo-grpc/src/
13+
├── lib.rs # Public API exports
14+
├── body.rs # BoxBody type
15+
├── codegen.rs # Code generation helpers
16+
├── context.rs # ClientContext, ServerContext (RpcInfo, stats, extensions)
17+
├── message.rs # RecvEntryMessage, SendEntryMessage traits (prost::Message)
18+
├── request.rs # Request<T> wrapper (metadata + message/Streaming)
19+
├── response.rs # Response<T> wrapper (metadata + message/Streaming)
20+
├── status.rs # gRPC Status (code, message, details, metadata) and Code enum
21+
├── tracing.rs # Span provider
22+
├── client/ # ClientBuilder, Client ("clone and use" pattern)
23+
│ ├── callopt.rs # Per-call options (CallOpt)
24+
│ ├── dns.rs # DNS resolution
25+
│ ├── meta.rs # MetaService (metadata handling)
26+
│ └── layer/timeout.rs
27+
├── server/ # Server, Router, ServiceBuilder, NamedService
28+
│ ├── router.rs # Multi-service routing
29+
│ ├── service.rs # ServiceBuilder::new(svc).build()
30+
│ ├── incoming.rs # Connection acceptance
31+
│ ├── meta.rs # MetaService
32+
│ └── layer/timeout.rs
33+
├── codec/ # Codec trait, encode/decode, compression (gzip/zlib/zstd)
34+
├── metadata/ # MetadataMap, MetadataKey, MetadataValue (binary keys use `-bin` suffix)
35+
├── layer/ # Shared layers: loadbalance, grpc_timeout, grpc_web, user_agent, CORS
36+
└── transport/ # Client transport, connection, TLS config
37+
```
38+
39+
## Key Components
40+
41+
**Client** -- `ClientBuilder` configures: `rpc_timeout`, `connect_timeout`, `discover`, `load_balance`, `layer`/`layer_front`, `compression`.
42+
43+
**Server** -- Built on hyper HTTP/2. Methods: `add_service`, `layer`/`layer_front`/`layer_tower`, `run`/`run_with_shutdown`, `tls_config`, plus HTTP/2 tuning options.
44+
45+
**Router** -- Supports multiple gRPC services via `add_service`:
46+
47+
```rust
48+
Server::new()
49+
.add_service(ServiceBuilder::new(service_a).build())
50+
.add_service(ServiceBuilder::new(service_b).build())
51+
.run(addr).await?;
52+
```
53+
54+
**NamedService** -- Services implement this trait (provides `const NAME`) for routing.
55+
56+
**Codec** -- Encoder/Decoder abstraction. Compression: gzip and zlib enabled by default, zstd optional.
57+
58+
**Metadata** -- `MetadataMap` stores key-value pairs. Binary keys use `-bin` suffix.
59+
60+
## Feature Flags
61+
62+
| Feature | Description |
63+
| --------------------- | ------------------------ |
64+
| `default` | Enables gzip and zlib |
65+
| `gzip` | gzip compression |
66+
| `zlib` | zlib compression |
67+
| `zstd` | zstd compression |
68+
| `compress` | Compression base support |
69+
| `rustls` | Rustls TLS |
70+
| `native-tls` | Native TLS |
71+
| `native-tls-vendored` | Vendored Native TLS |
72+
| `grpc-web` | gRPC-Web support |
73+
74+
## HTTP/2 Configuration Options
75+
76+
Server HTTP/2 settings:
77+
78+
- `http2_init_stream_window_size` / `http2_init_connection_window_size` (default 1MB)
79+
- `http2_adaptive_window`
80+
- `http2_max_concurrent_streams`
81+
- `http2_keepalive_interval` / `http2_keepalive_timeout` (default 20s)
82+
- `http2_max_frame_size`
83+
- `http2_max_send_buf_size`
84+
- `http2_max_header_list_size` (default 16MB)
85+
- `accept_http1`: Accept HTTP/1 (required for gRPC-Web)
86+
87+
## Notes
88+
89+
1. **Users should not use volo-grpc directly**: Use code generated by `volo-build`
90+
2. **gRPC-Web requires additional configuration**: Enable `grpc-web` feature and set `accept_http1(true)`
91+
3. **Compression is optional**: gzip and zlib enabled by default, zstd requires manual enabling
92+
4. **TLS requires backend selection**: rustls or native-tls

0 commit comments

Comments
 (0)