Skip to content

Commit b34c64f

Browse files
committed
added the ttrpc stress utility
This change adds a stress utility which can be used for stress testing the ttrpc connection. This tool represents a simple client-server interaction where the client sends continuous requests to the server, and the server responds with the same data, allowing for testing of concurrent request handling and response verification. This tool is adapted from https://github.com/kevpar/ttrpcstress.git which was written by Kevin Parsons. Signed-off-by: Harsh Rawat <[email protected]>
1 parent ababa3f commit b34c64f

File tree

10 files changed

+549
-1
lines changed

10 files changed

+549
-1
lines changed

Makefile

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@ WHALE = "🇩"
2424
ONI = "👹"
2525

2626
# Project binaries.
27-
COMMANDS=protoc-gen-go-ttrpc protoc-gen-gogottrpc
27+
COMMANDS=protoc-gen-go-ttrpc protoc-gen-gogottrpc ttrpc-stress
2828

2929
ifdef BUILDTAGS
3030
GO_BUILDTAGS = ${BUILDTAGS}

cmd/ttrpc-stress/README.md

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
# ttrpc-stress
2+
3+
This is a simple client/server utility designed to stress-test a TTRPC connection. It aims to identify potential deadlock cases during repeated, rapid TTRPC requests.
4+
5+
## Overview of the tool
6+
7+
- The **server** listens for connections and exposes a single, straightforward method. It responds immediately to any requests.
8+
- The **client** creates multiple worker goroutines that send a large number of requests to the server as quickly as possible.
9+
10+
This tool represents a simple client-server interaction where the client sends requests to the server, and the server responds with the same data, allowing for testing of concurrent request handling and response verification. By utilizing core TTRPC facilities like `ttrpc.(*Client).Call` and `ttrpc.(*Server).Register` instead of generated client/server code, the test remains straightforward and effective.
11+
12+
## Usage
13+
14+
The `ttrpc-stress` command provides two modes of operation: server and client.
15+
16+
### Run the Server
17+
18+
To start the server, specify a Unix socket or named pipe as the address:
19+
```bash
20+
ttrpc-stress server <ADDR>
21+
```
22+
- `<ADDR>`: The Unix socket or named pipe to listen on.
23+
24+
### Run the Client
25+
26+
To start the client, specify the address, number of iterations, and number of workers:
27+
```bash
28+
ttrpc-stress client <ADDR> <ITERATIONS> <WORKERS>
29+
```
30+
- `<ADDR>`: The Unix socket or named pipe to connect to.
31+
- `<ITERATIONS>`: The total number of iterations to execute.
32+
- `<WORKERS>`: The number of workers handling the iterations.
33+
34+
## Version Compatibility Testing
35+
36+
One of the primary motivations for developing this stress utility was to identify potential deadlock scenarios when using different versions of the server and client. The goal is to test the current version of TTRPC, which is used to build `ttrpc-stress`, against the following older versions in both server and client scenarios:
37+
38+
- `v1.0.2`
39+
- `v1.1.0`
40+
- `v1.2.0`
41+
- `v1.2.4`
42+
- `latest`
43+
44+
### Known Issues in TTRPC Versions
45+
46+
| Version Range | Description | Comments |
47+
|---------------------|-------------------------------------------|------------------------------------------------------------------|
48+
| **v1.0.2 and before** | Original deadlock bug | [#94](https://github.com/containerd/ttrpc/pull/94) for fixing deadlock in `v1.1.0` |
49+
| **v1.1.0 - v1.2.0** | No known deadlock bugs | |
50+
| **v1.2.0 - v1.2.4** | Streaming with a new deadlock bug | [#107](https://github.com/containerd/ttrpc/pull/107) introduced deadlock in `v1.2.0` |
51+
| **After v1.2.4** | No known deadlock bugs | [#168](https://github.com/containerd/ttrpc/pull/168) for fixing deadlock in `v1.2.4` |
52+
---
53+
54+
Clients before `v1.1.0` and between `v1.2.0`-`v1.2.3` can encounted the deadlock.
55+
56+
However, if the **server** version is `v1.2.0` or later, deadlock issues in the client may be avoided.
57+
58+
Please refer to https://github.com/containerd/ttrpc/issues/72 for more information about the deadlock bug.
59+
60+
---
61+
62+
Happy testing!
Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
//go:build !windows
2+
// +build !windows
3+
4+
/*
5+
Copyright The containerd Authors.
6+
7+
Licensed under the Apache License, Version 2.0 (the "License");
8+
you may not use this file except in compliance with the License.
9+
You may obtain a copy of the License at
10+
11+
http://www.apache.org/licenses/LICENSE-2.0
12+
13+
Unless required by applicable law or agreed to in writing, software
14+
distributed under the License is distributed on an "AS IS" BASIS,
15+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
16+
See the License for the specific language governing permissions and
17+
limitations under the License.
18+
*/
19+
20+
package main
21+
22+
import "net"
23+
24+
// listenConnection listens for incoming Unix domain socket connections at the specified address.
25+
func listenConnection(addr string) (net.Listener, error) {
26+
return net.Listen("unix", addr)
27+
}
28+
29+
// dialConnection dials a Unix domain socket connection to the specified address.
30+
func dialConnection(addr string) (net.Conn, error) {
31+
return net.Dial("unix", addr)
32+
}
Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
//go:build windows
2+
// +build windows
3+
4+
/*
5+
Copyright The containerd Authors.
6+
7+
Licensed under the Apache License, Version 2.0 (the "License");
8+
you may not use this file except in compliance with the License.
9+
You may obtain a copy of the License at
10+
11+
http://www.apache.org/licenses/LICENSE-2.0
12+
13+
Unless required by applicable law or agreed to in writing, software
14+
distributed under the License is distributed on an "AS IS" BASIS,
15+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
16+
See the License for the specific language governing permissions and
17+
limitations under the License.
18+
*/
19+
20+
package main
21+
22+
import (
23+
"net"
24+
25+
"github.com/Microsoft/go-winio"
26+
)
27+
28+
// listenConnection listens for incoming named pipe connections at the specified address.
29+
func listenConnection(addr string) (net.Listener, error) {
30+
return winio.ListenPipe(addr, &winio.PipeConfig{
31+
// 0 buffer sizes for pipe is important to help deadlock to occur.
32+
// It can still occur if there is buffering, but it takes more IO volume to hit it.
33+
InputBufferSize: 0,
34+
OutputBufferSize: 0,
35+
})
36+
}
37+
38+
// dialConnection dials a named pipe connection to the specified address.
39+
func dialConnection(addr string) (net.Conn, error) {
40+
return winio.DialPipe(addr, nil)
41+
}

cmd/ttrpc-stress/main.go

Lines changed: 203 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,203 @@
1+
/*
2+
Copyright The containerd Authors.
3+
4+
Licensed under the Apache License, Version 2.0 (the "License");
5+
you may not use this file except in compliance with the License.
6+
You may obtain a copy of the License at
7+
8+
http://www.apache.org/licenses/LICENSE-2.0
9+
10+
Unless required by applicable law or agreed to in writing, software
11+
distributed under the License is distributed on an "AS IS" BASIS,
12+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
See the License for the specific language governing permissions and
14+
limitations under the License.
15+
*/
16+
17+
package main
18+
19+
import (
20+
"context"
21+
"flag"
22+
"fmt"
23+
"log"
24+
"os"
25+
"strconv"
26+
27+
ttrpc "github.com/containerd/ttrpc"
28+
payload "github.com/containerd/ttrpc/cmd/ttrpc-stress/payload"
29+
30+
"golang.org/x/sync/errgroup"
31+
)
32+
33+
// main is the entry point of the stress utility.
34+
func main() {
35+
// Define a flag for displaying usage information.
36+
flagHelp := flag.Bool("help", false, "Display usage")
37+
flag.Parse()
38+
39+
// Check if help flag is set or if there are insufficient arguments.
40+
if *flagHelp || flag.NArg() < 2 {
41+
usage()
42+
}
43+
44+
// Switch based on the first argument to determine mode (server or client).
45+
switch flag.Arg(0) {
46+
case "server":
47+
// Ensure correct number of arguments for server mode.
48+
if flag.NArg() != 2 {
49+
usage()
50+
}
51+
52+
addr := flag.Arg(1)
53+
54+
// Run the server and handle any errors.
55+
err := runServer(context.Background(), addr)
56+
if err != nil {
57+
log.Fatalf("error: %s", err)
58+
}
59+
60+
case "client":
61+
// Ensure correct number of arguments for client mode.
62+
if flag.NArg() != 4 {
63+
usage()
64+
}
65+
66+
addr := flag.Arg(1)
67+
68+
// Parse iterations and workers arguments.
69+
iters, err := strconv.Atoi(flag.Arg(2))
70+
if err != nil {
71+
log.Fatalf("failed parsing iters: %s", err)
72+
}
73+
74+
workers, err := strconv.Atoi(flag.Arg(3))
75+
if err != nil {
76+
log.Fatalf("failed parsing workers: %s", err)
77+
}
78+
79+
// Run the client and handle any errors.
80+
err = runClient(context.Background(), addr, iters, workers)
81+
if err != nil {
82+
log.Fatalf("runtime error: %s", err)
83+
}
84+
85+
default:
86+
// Display usage information if the mode is unrecognized.
87+
usage()
88+
}
89+
}
90+
91+
// usage prints the usage information and exits the program.
92+
// usage prints the usage information for the program and exits.
93+
func usage() {
94+
fmt.Fprintf(os.Stderr, `Usage:
95+
stress server <ADDR>
96+
Run the server with the specified unix socket or named pipe.
97+
stress client <ADDR> <ITERATIONS> <WORKERS>
98+
Run the client with the specified unix socket or named pipe, number of ITERATIONS, and number of WORKERS.
99+
`)
100+
os.Exit(1)
101+
}
102+
103+
// runServer sets up and runs the server.
104+
func runServer(ctx context.Context, addr string) error {
105+
log.Printf("Starting server on %s", addr)
106+
107+
// Listen for connections on the specified address.
108+
l, err := listenConnection(addr)
109+
if err != nil {
110+
return fmt.Errorf("failed listening on %s: %w", addr, err)
111+
}
112+
113+
// Create a new ttrpc server.
114+
server, err := ttrpc.NewServer()
115+
if err != nil {
116+
return fmt.Errorf("failed creating ttrpc server: %w", err)
117+
}
118+
119+
// Register a service and method with the server.
120+
server.Register("ttrpc.stress.test.v1", map[string]ttrpc.Method{
121+
"TEST": func(_ context.Context, unmarshal func(interface{}) error) (interface{}, error) {
122+
req := &payload.Payload{}
123+
// Unmarshal the request payload.
124+
if err := unmarshal(req); err != nil {
125+
log.Fatalf("failed unmarshalling request: %s", err)
126+
}
127+
id := req.Value
128+
log.Printf("got request: %d", id)
129+
// Return the same payload as the response.
130+
return &payload.Payload{Value: id}, nil
131+
},
132+
})
133+
134+
// Serve the server and handle any errors.
135+
if err := server.Serve(ctx, l); err != nil {
136+
return fmt.Errorf("failed serving server: %w", err)
137+
}
138+
return nil
139+
}
140+
141+
// runClient sets up and runs the client.
142+
func runClient(ctx context.Context, addr string, iters int, workers int) error {
143+
log.Printf("Starting client on %s", addr)
144+
145+
// Dial a connection to the specified pipe.
146+
c, err := dialConnection(addr)
147+
if err != nil {
148+
return fmt.Errorf("failed dialing connection to %s: %w", addr, err)
149+
}
150+
151+
// Create a new ttrpc client.
152+
client := ttrpc.NewClient(c)
153+
ch := make(chan int)
154+
var eg errgroup.Group
155+
156+
// Start worker goroutines to send requests.
157+
for i := 0; i < workers; i++ {
158+
eg.Go(func() error {
159+
for {
160+
i, ok := <-ch
161+
if !ok {
162+
return nil
163+
}
164+
// Send the request and handle any errors.
165+
if err := send(ctx, client, uint32(i)); err != nil {
166+
return fmt.Errorf("failed sending request: %w", err)
167+
}
168+
}
169+
})
170+
}
171+
172+
// Send iterations to the channel.
173+
for i := 0; i < iters; i++ {
174+
ch <- i
175+
}
176+
close(ch)
177+
178+
// Wait for all goroutines to finish.
179+
if err := eg.Wait(); err != nil {
180+
return fmt.Errorf("failed waiting for goroutines: %w", err)
181+
}
182+
return nil
183+
}
184+
185+
// send sends a request to the server and verifies the response.
186+
func send(ctx context.Context, client *ttrpc.Client, id uint32) error {
187+
req := &payload.Payload{Value: id}
188+
resp := &payload.Payload{}
189+
190+
log.Printf("sending request: %d", id)
191+
// Call the server method and handle any errors.
192+
if err := client.Call(ctx, "ttrpc.stress.test.v1", "TEST", req, resp); err != nil {
193+
return err
194+
}
195+
196+
ret := resp.Value
197+
log.Printf("got response: %d", ret)
198+
// Verify the response matches the request.
199+
if ret != id {
200+
return fmt.Errorf("expected return value %d but got %d", id, ret)
201+
}
202+
return nil
203+
}

cmd/ttrpc-stress/payload/doc.go

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
/*
2+
Copyright The containerd Authors.
3+
4+
Licensed under the Apache License, Version 2.0 (the "License");
5+
you may not use this file except in compliance with the License.
6+
You may obtain a copy of the License at
7+
8+
http://www.apache.org/licenses/LICENSE-2.0
9+
10+
Unless required by applicable law or agreed to in writing, software
11+
distributed under the License is distributed on an "AS IS" BASIS,
12+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
See the License for the specific language governing permissions and
14+
limitations under the License.
15+
*/
16+
17+
package payload

0 commit comments

Comments
 (0)