[trainer] fix: dump all outputs in validation in main_ppo_sync

[guillemgt]

What does this PR do?

In _validate in main_ppo_sync.py, when validation_data_dir is set, the dump only included the final output per session. For multi-turn or multi-output rollouts, all intermediate and alternative outputs were silently discarded. This PR fixes _validate to fetch prompts/responses for all keys and include every output in the dump, while mapping each entry back to its session's final score/ground-truth. Follows up on #6101.

Checklist Before Starting

• Search for similar PRs. Paste at least one query link here: https://github.com/verl-project/verl/pulls?q=all+outputs+in+validation https://github.com/verl-project/verl/pulls?q=main_ppo_sync+validation+outputs
• Format the PR title as [{modules}] {type}: {description} (This will be checked by the CI)
  • {modules} include fsdp, megatron, veomni, sglang, vllm, rollout, trainer, ci, training_utils, recipe, hardware, deployment, ray, worker, single_controller, misc, perf, model, algo, env, tool, ckpt, doc, data, cfg, reward, fully_async, one_step_off
  • If this PR involves multiple modules, separate them with , like [megatron, fsdp, doc]
  • {type} is in feat, fix, refactor, chore, test
  • If this PR breaks any API (CLI arguments, config, function signature, etc.), add [BREAKING] to the beginning of the title.
  • Example: [BREAKING][fsdp, megatron] feat: dynamic batching

Test

This change only affects the file dump path (when trainer.validation_data_dir is set). No CI test covers this code path — it requires a full training run with multi-output rollouts.

API and Usage Example

No API changes.

# Add code snippet or script demonstrating how to use this

Design & Code Changes

• Fetch prompts/responses for all batch keys (not just final ones per session) using a separate kv_batch_get call, so every output is captured in the dump.
• dump_all_inputs, dump_all_outputs, dump_all_keys accumulate all entries across batches.
• session_to_sample_idx maps session keys to their position in the per-session lists, so scores/ground-truths can be looked up correctly when writing the dump.
• Remove prompts/responses from the fields list of the final-key kv_batch_get (fetched separately above).
• Sort the dump by (uid, session_id, index) so outputs from the same rollout are grouped together.

Checklist Before Submitting

Important

Please check all the following items before requesting a review, otherwise the reviewer might deprioritize this PR for review.

• Read the Contribute Guide.
• Apply pre-commit checks: pre-commit install && pre-commit run --all-files --show-diff-on-failure --color=always — passed (naming convention failures are pre-existing in third-party packages, not in changed files).
• Add / Update the documentation. No docs needed — internal dump behavior only.
• Add unit or end-to-end test(s) to the CI workflow to cover all the code. If not feasible, explain why: Not feasible — the validation dump path has no CI coverage and requires a full training run with multi-output rollouts to exercise.
• Once your PR is ready for CI, send a message in the ci-request channel in the verl Slack workspace. (If not accessible, please try the Feishu group (飞书群).)
• If your PR is related to the recipe submodule, please also update the reference to the submodule commit via git submodule update --remote or cd recipe && git pull origin main. Not applicable.

[gemini-code-assist]

Code Review

This pull request updates the validation process in verl/trainer/main_ppo_sync.py to collect and dump all inputs and outputs, rather than just the final samples per session. It introduces new tracking for all keys and sessions, ensuring that the dumped data is sorted by UID, session ID, and index. Feedback was provided regarding the sorting logic, which assumes specific key formats for integer conversion and could lead to a crash if unexpected formats are encountered.

[gemini-code-assist]

The sorting logic assumes that parts[1] (session_id) and parts[2] (index) are always convertible to integers. While this is generally true for keys generated by AgentLoopWorkerTQ, any unexpected key format in dump_all_keys will cause a ValueError during the validation dump. It is safer to wrap the integer conversion in a try-except block or use a more robust parsing method to avoid crashing the trainer at the end of a validation run.