Supporting Velvet model #11716
Supporting Velvet model #11716
Conversation
src/llama-chat.h
Outdated
| @@ -39,6 +39,7 @@ enum llm_chat_template { | |||
| LLM_CHAT_TEMPLATE_GIGACHAT, | |||
| LLM_CHAT_TEMPLATE_MEGREZ, | |||
| LLM_CHAT_TEMPLATE_UNKNOWN, | |||
| LLM_CHAT_TEMPLATE_VELVET | |||
There was a problem hiding this comment.
It's also recommended to add a test for this chat template, see test-chat-template.cpp
There was a problem hiding this comment.
btw, this enum value is in the incorrect place. everything must come before LLM_CHAT_TEMPLATE_UNKNOWN
please also add a test case in test-chat-template.cpp. Otherwise you will expect it to be broken in the future
There was a problem hiding this comment.
I added the test case for chat template as you suggested and I did the requested changes. Let me know if I still missing something. Thanks.
Co-authored-by: Xuan-Son Nguyen <[email protected]>
Co-authored-by: Xuan-Son Nguyen <[email protected]>
Co-authored-by: Xuan-Son Nguyen <[email protected]>
Co-authored-by: Xuan-Son Nguyen <[email protected]>
src/llama-chat.h
Outdated
| @@ -39,6 +39,7 @@ enum llm_chat_template { | |||
| LLM_CHAT_TEMPLATE_GIGACHAT, | |||
| LLM_CHAT_TEMPLATE_MEGREZ, | |||
| LLM_CHAT_TEMPLATE_UNKNOWN, | |||
| LLM_CHAT_TEMPLATE_VELVET | |||
There was a problem hiding this comment.
btw, this enum value is in the incorrect place. everything must come before LLM_CHAT_TEMPLATE_UNKNOWN
please also add a test case in test-chat-template.cpp. Otherwise you will expect it to be broken in the future
|
I pause the pull request for more tests. |
|
Sorry I've been busy recently, feel free to ping when you're ready. |
|
would be nice having this integrated. I have tried the fork at https://github.com/fbuciuni90/llama.cpp but it did not work for the 2B ... not sure why. |
I've aligned the pull request content with the latest release. Could you let me know if I'm still missing anything before it can be merged? |
I'm adding support to Velvet models (see https://huggingface.co/Almawave/Velvet-14B)