This repository was archived by the owner on Aug 7, 2025. It is now read-only.
update deepspeed opt example #2614
Description
🚀 The feature
- Integrate with TextIteratorStreamerBatch, provide streaming response
- update https://pytorch.org/serve/large_model_inference.html#, introduce E2E streaming response example
Motivation, pitch
provide streaming response example
Alternatives
No response
Additional context
No response