go

Generate Go Client stubs

To generate the stubs:

# Clone the repo containing the proto definitions, from within client/src/grpc_generated/go/
git clone https://github.com/triton-inference-server/common.git

# Compiles *.proto to *.pb.go
# install protoc-gen-go-grpc with
# go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@latest
./gen_go_stubs.sh

Example Go Client

To run the grpc client example:

# Clone the repo containing the example model, from within client/src/grpc_generated/go/.
git clone https://github.com/triton-inference-server/server.git

# Setup "simple" model from example model_repository
cd server/docs/examples
./fetch_models.sh

# Launch (detached) Triton
docker run -d -p8000:8000 -p8001:8001 -p8002:8002 -it -v $(pwd)/model_repository:/models nvcr.io/nvidia/tritonserver:22.11-py3 tritonserver --model-store=/models

# Use client
cd ../../../
go run grpc_simple_client.go

Sample Output:

$ go run grpc_simple_client.go
  FLAGS: {simple  1 localhost:8001}
  Triton Health - Live: true
  Triton Health - Ready: true
  name:"simple"  versions:"1"  platform:"tensorflow_graphdef"  inputs:{name:"INPUT0"  datatype:"INT32"  shape:-1  shape:16}  inputs:{name:"INPUT1"  datatype:"INT32"  shape:-1  shape:16}  outputs:{name:"OUTPUT0"  datatype:"INT32"  shape:-1  shape:16}  outputs:{name:"OUTPUT1"  datatype:"INT32"  shape:-1  shape:16}

  Checking Inference Outputs
  --------------------------
  0 + 1 = 1
  0 - 1 = -1
  1 + 1 = 2
  1 - 1 = 0
  2 + 1 = 3
  2 - 1 = 1
  3 + 1 = 4
  3 - 1 = 2
  4 + 1 = 5
  4 - 1 = 3
  5 + 1 = 6
  5 - 1 = 4
  6 + 1 = 7
  6 - 1 = 5
  7 + 1 = 8
  7 - 1 = 6
  8 + 1 = 9
  8 - 1 = 7
  9 + 1 = 10
  9 - 1 = 8
  10 + 1 = 11
  10 - 1 = 9
  11 + 1 = 12
  11 - 1 = 10
  12 + 1 = 13
  12 - 1 = 11
  13 + 1 = 14
  13 - 1 = 12
  14 + 1 = 15
  14 - 1 = 13
  15 + 1 = 16
  15 - 1 = 14