Skip to content

[BUG] banned_strings issue with some strings containing ASCII period #412

@mfiano

Description

@mfiano

OS

Linux

GPU Library

CUDA 12.x

Python version

3.12

Describe the bug

banned_strings=[] OK
banned_strings=["."] OK
banned_strings=[".."] OK
banned_strings=["..."] breaks
banned_strings=[".foo."] breaks
banned_strings=["---"] OK

It seems that any literal string of at least 3 characters which includes a . at in position 0 corrupts the model's output (usually just emitting the stop token, but sometimes gibberish or an incomplete message).

Reproduction steps

Pass banned_strings="..." for a simple reproduction.

I tested with EXL3 and Mistral-Small 24B.

Expected behavior

Expected behavior would be for literal strings to match verbatim instead of getting confused with regex or backtracking.

Logs

No response

Additional context

No response

Acknowledgements

  • I have looked for similar issues before submitting this one.
  • I have read the disclaimer, and this issue is related to a code bug. If I have a question, I will use the Discord server.
  • I understand that the developers have lives and my issue will be answered when possible.
  • I understand the developers of this program are human, and I will ask my questions politely.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions