mylesgoose
commited on
Commit
•
4eb7274
1
Parent(s):
a75702a
Update README.md
Browse files
README.md
CHANGED
@@ -390,8 +390,9 @@ Dapper and dignified
|
|
390 |
Country cottage charm<|eot_id|>
|
391 |
|
392 |
and so the model learns to do this in its first output:
|
|
|
393 |
<|eot_id|><|start_header_id|>assistant<|end_header_id|>
|
394 |
-
which naturally messes up the training.
|
395 |
<|begin_of_text|><|start_header_id|>user<|end_header_id|>
|
396 |
|
397 |
<|image|>If I had to write a haiku for this one, it would be: <|eot_id|>
|
@@ -411,8 +412,8 @@ If I had to write a haiku for this one, it would be:<|eot_id|>#notice that this
|
|
411 |
<|start_header_id|>assistant<|end_header_id|>
|
412 |
|
413 |
['A rabbit on a sunny day']
|
414 |
-
this is an example of the 3.1 models chat template
|
415 |
-
|
416 |
|
417 |
Llama 3.2 Version Release Date: September 25, 2024
|
418 |
|
|
|
390 |
Country cottage charm<|eot_id|>
|
391 |
|
392 |
and so the model learns to do this in its first output:
|
393 |
+
|
394 |
<|eot_id|><|start_header_id|>assistant<|end_header_id|>
|
395 |
+
which naturally messes up the training. so i put a new line character after the eot_id or prior to the start header id in the chat template: so that the format is like so :
|
396 |
<|begin_of_text|><|start_header_id|>user<|end_header_id|>
|
397 |
|
398 |
<|image|>If I had to write a haiku for this one, it would be: <|eot_id|>
|
|
|
412 |
<|start_header_id|>assistant<|end_header_id|>
|
413 |
|
414 |
['A rabbit on a sunny day']
|
415 |
+
this is an example of the 3.1 models chat template.. to prevent the model learning that eot comes first there need to be a clearer distinction made with a \n
|
416 |
+
which Is done in this template
|
417 |
|
418 |
Llama 3.2 Version Release Date: September 25, 2024
|
419 |
|