[Docs] Fix the example code of streaming chat completions in reasoning

[hsliuustc0106]

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

Purpose

Fix the example code in reasoning_outputs.md described in #21736

Test Plan

serve command:

vllm serve /workspace/models/granite-3.2-8b-instruct     --reasoning-parser granite --served-model-name ' ibm-granite/granite-3.2-8b-instruct'

code:

from openai import OpenAI

# Modify OpenAI's API key and API base to use vLLM's API server.
openai_api_key = "EMPTY"
openai_api_base = "http://localhost:8000/v1"

messages = [{"role": "user", "content": "9.11 and 9.8, which is greater?"}]


def main():
    client = OpenAI(
        api_key=openai_api_key,
        base_url=openai_api_base,
    )

    models = client.models.list()
    model = models.data[0].id

    # ruff: noqa: E501
    # For granite: add: `extra_body={"chat_template_kwargs": {"thinking": True}}`
    stream = client.chat.completions.create(model=model, messages=messages, stream=True, extra_body={"chat_template_kwargs": {"thinking": True}})

    print("client: Start streaming chat completions...")
    printed_reasoning_content = False
    printed_content = False

    for chunk in stream:
        reasoning_content = None
        content = None
        # Check the content is reasoning_content or content
        if (
            hasattr(chunk.choices[0].delta, "reasoning_content")
            and chunk.choices[0].delta.reasoning_content
        ):
            reasoning_content = chunk.choices[0].delta.reasoning_content
        elif (
            hasattr(chunk.choices[0].delta, "content")
            and chunk.choices[0].delta.content
        ):
            content = chunk.choices[0].delta.content

        if reasoning_content is not None:
            if not printed_reasoning_content:
                printed_reasoning_content = True
                print("reasoning_content:", end="", flush=True)
            print(reasoning_content, end="", flush=True)
        elif content is not None:
            if not printed_content:
                printed_content = True
                print("\ncontent:", end="", flush=True)
            # Extract and print the content
            print(content, end="", flush=True)


if __name__ == "__main__":
    main()

Test Result

output:

client: Start streaming chat completions...
reasoning_content:
This question is asking to compare two specific numbers, 9.11 and 9.8. It's a simple numerical comparison that involves understanding decimal places. 

1. Recognize that both numbers are close but 9.8 has a higher whole number part (9) than 9.11.
2. Consider the decimal places: 9.11 is closer to 9 but has a slightly larger fractional part.
3. Conclude that 9.8 is greater because its whole number component (9) is greater than 9.11's (9), and the decimal portion doesn't change this. 


content:

9.8 is greater than 9.11. 

Here’s the reasoning:

1. **Whole Number Comparison**: Both numbers start with '9', but 9.8 has a whole number component of 9, whereas 9.11 has 9.1. 
2. Since the whole numbers are equal, we look to the decimal parts. 
3. 9.8's decimal part is .8, and 9.11's is .11. 
4. Any number with a higher decimal value is greater than a number with a lower decimal value, even when the whole numbers are the same. 

Therefore, 9.8 > 9.11.

(Optional) Documentation Update

Update the example code of streaming chat completions in reasoning_outputs.md

[gemini-code-assist]

Code Review

This pull request correctly fixes a bug in the example code for streaming chat completions with reasoning, where empty reasoning_content could cause the actual content to be missed. The proposed change is logically sound and addresses the issue.

[gemini-code-assist]

The added checks for chunk.choices[0].delta.reasoning_content and chunk.choices[0].delta.content are necessary to avoid errors when these attributes are not present in the response. This ensures the code handles different response formats gracefully.

Suggested change

	if (
	hasattr(chunk.choices[0].delta, "reasoning_content")
	and chunk.choices[0].delta.reasoning_content
	if (
	hasattr(chunk.choices[0].delta, "reasoning_content")
	and chunk.choices[0].delta.reasoning_content
	):

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

[hmellor]

Suggested change

	reasoning_content = None
	content = None
	# Check the content is reasoning_content or content
	if hasattr(chunk.choices[0].delta, "reasoning_content"):
	if (
	hasattr(chunk.choices[0].delta, "reasoning_content")
	and chunk.choices[0].delta.reasoning_content
	):
	reasoning_content = chunk.choices[0].delta.reasoning_content
	elif hasattr(chunk.choices[0].delta, "content"):
	elif (
	hasattr(chunk.choices[0].delta, "content")
	and chunk.choices[0].delta.content
	):
	reasoning_content = getattr(chunk.choices[0].delta, "reasoning_content", None) or None
	content = getattr(chunk.choices[0].delta, "content", None) or None

[BruceW-07]

Thanks for the advice! The code has been simplified.

[hmellor]

LGTM, thanks for the improvement!