Problem
WhisperReplica::generate currently limits decoding with:
decoding_options.max_length = std::min(total_max_length / 2, total_max_length - start_step);
This applies the original 224-token sampling limit even when the prompt ends in <|notimestamps|>. For languages that rely heavily on Whisper's byte-level fallback, 224 tokens can be exhausted by a short audio clip even though the model has a 448-position text context.
Reproduction
Using faster-whisper with large-v3, vad_filter=False, and without_timestamps=True, both an Armenian and a Georgian sample stopped at exactly 224 re-tokenized visible-text tokens. One output ended with the Unicode replacement character because generation stopped mid-byte sequence.
The same audio with timestamp decoding used more than 224 visible-text tokens because timestamp boundaries allow faster-whisper to continue the transcription loop, but disabling timestamps produces one decode for the complete audio window.
Proposed behavior
When the task prompt ends with <|notimestamps|>, allow decoding to use the remaining text context:
total_max_length - start_step
Keep the existing half-context limit for timestamped decoding. This preserves the standard timestamp path while allowing explicit no-timestamp callers to use the model's remaining decoder positions.
Problem
WhisperReplica::generatecurrently limits decoding with:decoding_options.max_length = std::min(total_max_length / 2, total_max_length - start_step);This applies the original 224-token sampling limit even when the prompt ends in
<|notimestamps|>. For languages that rely heavily on Whisper's byte-level fallback, 224 tokens can be exhausted by a short audio clip even though the model has a 448-position text context.Reproduction
Using
faster-whisperwithlarge-v3,vad_filter=False, andwithout_timestamps=True, both an Armenian and a Georgian sample stopped at exactly 224 re-tokenized visible-text tokens. One output ended with the Unicode replacement character because generation stopped mid-byte sequence.The same audio with timestamp decoding used more than 224 visible-text tokens because timestamp boundaries allow faster-whisper to continue the transcription loop, but disabling timestamps produces one decode for the complete audio window.
Proposed behavior
When the task prompt ends with
<|notimestamps|>, allow decoding to use the remaining text context:Keep the existing half-context limit for timestamped decoding. This preserves the standard timestamp path while allowing explicit no-timestamp callers to use the model's remaining decoder positions.