FEAT replace fastchat in GCG #965
Description
Is your feature request related to a problem? Please describe.
fastchat hasn't had releases for 15 months and we rely on it to provide conversation templates. This task is about investigating alternatives since we use relatively little of their functionality:
from fastchat.conversation import Conversation, SeparatorStyle
from fastchat.model import get_conversation_template
and
for template in params.conversation_templates:
if template in ["llama-2", "mistral", "llama-3-8b", "vicuna"]:
raw_conv_templates.append(get_conversation_template(template)),
elif template in ["phi-3-mini"]:
conv_template = Conversation(
name="phi-3-mini",
system_template="<|system|>\n{system_message}",
system_message="",
roles=("<|user|>", "<|assistant|>"),
sep_style=SeparatorStyle.CHATML,
sep="<|end|>",
stop_token_ids=[32000, 32001, 32007],
)
raw_conv_templates.append(conv_template)
else:
raise ValueError("Conversation template not recognized")
We should be able to expand to more models. Currently, we can work with
"mistral", "llama_2", "llama_3", "vicuna", "phi_3_mini", "all_models"
where the last one means all of the before.
I would like a much expanded set of models. We need to explore how that can be facilitated. Perhaps there are up-to-date packages now that make this a lot easier.