File tree Expand file tree Collapse file tree 2 files changed +3
-3
lines changed Expand file tree Collapse file tree 2 files changed +3
-3
lines changed Original file line number Diff line number Diff line change @@ -19,7 +19,7 @@ Let's think about this step by step.<|im_end|>
19
19
20
20
## Answer Extraction
21
21
``` python
22
- # Remove reasoning, extract final answer
22
+ # Extract final answer
23
23
output = full_output.split(' <|/thinking|>' )[- 1 ] if ' <|/thinking|>' in full_output else full_output
24
24
answer = re.search(r ' ####\s * ( \d + ) ' , output)
25
25
final_answer = answer.group(1 ) if answer else output.strip()
Original file line number Diff line number Diff line change 2
2
3
3
## Tokenization
4
4
``` python
5
- tokenizer = AutoTokenizer.from_pretrained(model_path) # Use model's tokenizer
5
+ tokenizer = AutoTokenizer.from_pretrained(model_path)
6
6
max_length = 2048
7
7
```
8
8
@@ -20,6 +20,6 @@ Summarize this article:
20
20
21
21
## Verification
22
22
``` bash
23
- # Check first token is 128000 (begin_of_text)
23
+ # Verify first token
24
24
python -c " print(tokenizer.encode('<|begin_of_text|>')[0])"
25
25
```
You can’t perform that action at this time.
0 commit comments