Feature request: reliable pause/silence control for Voice Design mode

[ergut]

Feature Request: Reliable pause/silence control in Voice Design mode

Context

When using VoxCPM2 in Voice Design mode (text description in parentheses, no reference audio) for long-form narration, sentence-to-sentence pacing is too tight — the next sentence begins almost immediately after a period, which is fatiguing for the listener.

I tried the following approaches to control pause duration. None is reliably effective:

What doesn't work

1. Explicit duration instructions in the Voice Design prompt — adding phrases like "take a one-second pause after every period" or "long pauses between sentences" into the parenthetical description does not produce measurable silence between sentences. The model does not interpret timing instructions semantically.

2. Inline parenthetical pause commands mid-text — inserting (pause for 1 second) or similar in the middle of the input text causes those words to be spoken verbatim. Parentheses are only parsed as a style descriptor at the very start of the input, not as inline control tokens.

3. Ellipsis + em-dash (... — ...) between sentences produces slightly longer pauses than a plain period — better than nothing, but the difference is subtle and not nearly enough for comfortable narration pacing. It is also undocumented and pollutes the text for any downstream consumer (subtitles, transcripts, etc.).

Related existing issues

• Speed parameter? #56 — "Speed parameter?" (open, maintainer: "we are considering adding this parameter")
• Style control instructions ignored #210 — "Style control instructions ignored" (closed; clarified that Ultimate Cloning ignores style prompts — but this issue is about Voice Design, which is the style-controlled mode)
• Support word and punctuation timestamps #117 — "Support word and punctuation timestamps" (about output timestamps, not input pause control)

To my knowledge, no existing issue covers input-side pause duration control.

Proposal

Any of the following would resolve this use case — in rough order of preference:

1. SSML-style inline break tokens, e.g. <break time="1000ms"/> or [pause:1s]. Deterministic, composable, and familiar from other TTS systems (Amazon Polly, Azure, ElevenLabs).

2. Configurable punctuation→pause mapping on model.generate(...), e.g. pause_map={".": 0.8, ",": 0.3, ";": 0.5}. Even if implemented as post-processing on the output waveform, this would unblock most users.

3. Documentation on which input patterns reliably produce longer silences in the current model, so users can make informed choices without reverse-engineering.

Environment

• VoxCPM2 (voxcpm==2.0.2)
• macOS 15.x, Apple M4 Pro
• PyTorch 2.11.0, MPS backend, bfloat16
• Python 3.12

[RyannLLM]

😆 Laughter and Sighs: [laughing] (laughter), [sigh] (sighing)
🤫 Pauses, Breathing, and Filler Words: [breath] (breathing/inhaling), [Uhm] (hesitation/thinking), [Shh] (hushing)
❓ Questions and Confirmations: [Question-ah], [Question-ei], [Question-en], [Question-oh], [Confirmation-en] (affirmative tone/agreement)
❗ Surprise and Subtle Emotions: [Surprise-wa] (wow!), [Surprise-yo] (hey!), [Surprise-ah], [Surprise-oh], [Dissatisfaction-hnn] (disgruntled grunt/humph)

Thank you for the detailed feature request and for providing your test results.

As a current workaround, you can try inserting the [breath] tag between sentences. While the model does not yet support semantic timing instructions (like "1 second"), the [breath] tag prompts the model to generate a natural inhalation, which effectively creates a realistic pause and breaks up the tight pacing you described.

We are aware that more deterministic control (like SSML or a pause map) is highly desirable for long-form narration. We are looking into your suggestions, and we plan to introduce more robust pause and silence control features in upcoming updates.

[prixa-yushan]

Hello @RyannLLM,

I fine-tuned a base model (LoRA) using my own voice data (approximately 100 samples) for testing, training it for around 2,000 steps. The resulting model produces fairly natural and high-quality speech.

However, I am encountering two issues:

1. Voice design controls do not appear to function as expected in the fine-tuned model.
2. Inserting expressive tags such as [breath], [laughing], or [Surprise-yo] within the text does not produce the intended effects during synthesis.

I would like to understand the possible reasons for these limitations and whether they stem from the dataset size, training configuration, or model constraints.

Environment :

• voxcpm == 2.0.2
• torch == 2.6.0+cu124
• python == 3.11.7

[RyannLLM]

Hi @prixa-yushan this is caused by overfitting and catastrophic forgetting. Training for 2,000 steps on 100 samples makes the LoRA forget the base model's expressive features. To fix this, try reducing training steps (to 500-1000) or lowering the LoRA scale during inference to allow the base model's capabilities to shine through.