File tree Expand file tree Collapse file tree 2 files changed +3
-3
lines changed Expand file tree Collapse file tree 2 files changed +3
-3
lines changed Original file line number Diff line number Diff line change @@ -19,7 +19,7 @@ Let's think about this step by step.<|im_end|>
1919
2020## Answer Extraction  
2121``` python 
22- #  Remove reasoning, extract  final answer
22+ #  Extract  final answer
2323output =  full_output.split(' <|/thinking|>' - 1 ] if  ' <|/thinking|>' in  full_output else  full_output
2424answer =  re.search(r ' ####\s * ( \d + ) ' 
2525final_answer =  answer.group(1 ) if  answer else  output.strip()
Original file line number Diff line number Diff line change 22
33## Tokenization  
44``` python 
5- tokenizer =  AutoTokenizer.from_pretrained(model_path)   #  Use model's tokenizer 
5+ tokenizer =  AutoTokenizer.from_pretrained(model_path)
66max_length =  2048 
77``` 
88
@@ -20,6 +20,6 @@ Summarize this article:
2020
2121## Verification  
2222``` bash 
23- #  Check  first token is 128000 (begin_of_text) 
23+ #  Verify  first token
2424python -c " print(tokenizer.encode('<|begin_of_text|>')[0])" 
2525``` 
 
 
   
 
     
   
   
          
    
    
     
    
      
     
     
    You can’t perform that action at this time.
  
 
    
  
    
      
        
     
       
      
     
   
 
    
    
  
 
  
 
     
    
0 commit comments