-
-
Notifications
You must be signed in to change notification settings - Fork 144
Open
Labels
bugSomething isn't workingSomething isn't working
Description
OS
Linux
GPU Library
CUDA 12.x
Python version
3.12
Describe the bug
banned_strings=[] OK
banned_strings=["."] OK
banned_strings=[".."] OK
banned_strings=["..."] breaks
banned_strings=[".foo."] breaks
banned_strings=["---"] OK
It seems that any literal string of at least 3 characters which includes a . at in position 0 corrupts the model's output (usually just emitting the stop token, but sometimes gibberish or an incomplete message).
Reproduction steps
Pass banned_strings="..." for a simple reproduction.
I tested with EXL3 and Mistral-Small 24B.
Expected behavior
Expected behavior would be for literal strings to match verbatim instead of getting confused with regex or backtracking.
Logs
No response
Additional context
No response
Acknowledgements
- I have looked for similar issues before submitting this one.
- I have read the disclaimer, and this issue is related to a code bug. If I have a question, I will use the Discord server.
- I understand that the developers have lives and my issue will be answered when possible.
- I understand the developers of this program are human, and I will ask my questions politely.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't working