How to SFT llama3 with a labelled dataset

[kandeng]

Hi, there,

We want to SFT a llama3 model with a dataset, with following format,

[
   {
      "question": "the content of question 1 ...", 
      "answer":  "the content of answer 1 ...", 
      "label":  either "good" or "bad", to evaluate the answer. 
   },
   ...
]

Our questions are,

1. How to convert this dataset's format into the format that is acceptable by Llama3?

2. Out of curiosity, what will happen inside Llama3 during training, if we convert the dateset into the following format?

   <|begin_of_text|>
         <|start_header_id|>question<|end_header_id|>
         Here is my first question ...
   
         <|start_header_id|>answer<|end_header_id|>
         Here is the LLM's answer to the first question ...
   
         <|start_header_id|>system<|end_header_id|>
         good
      <|eot_id|>
         ...
   <|end_of_text|>
   

   As SFT, for each message, will Llama3 take the question as prompt, and start Llama3's prediction from answer? If so, what will happen to eval that is either "good" or "bad"?

Many thanks,
Kan