🚀 The feature, motivation and pitch

Currently, we have 2 versions of benchmark scripts (e.g. benchmark_serving.py and serve.py). Most of their code are simply duplicated, and sometime it's confusing which should be treated as the source of truth.

Discussed with @simon-mo @ywang96 @yeqcharlotte in #21108 (comment), we all agreed that we should ultimately deprecate benchmark_serving.py and only maintain serve.py as the golden benchmark script.

A few action items to address the issue.

• Stop refering benchmark_serving.py in all documentations
• Delete duplicated code in benchmark_serving.py, and reuse serve.py code as much as possible
• [Stretched] Delete benchmark_serving.py

Alternatives

No response

Additional context

No response

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.