mylesgoose
commited on
Commit
•
67f08b8
1
Parent(s):
9c0d0a8
Update README.md
Browse files
README.md
CHANGED
@@ -4,8 +4,8 @@ license_name: other
|
|
4 |
license_link: https://ai.meta.com/llama/license
|
5 |
---
|
6 |
Repairing the chat template for the model.
|
7 |
-
There is a slight problem with the
|
8 |
-
|
9 |
the model learns to see this:
|
10 |
|
11 |
<|begin_of_text|><|start_header_id|>user<|end_header_id|>
|
|
|
4 |
license_link: https://ai.meta.com/llama/license
|
5 |
---
|
6 |
Repairing the chat template for the model.
|
7 |
+
There is a slight problem with the original llama 3.1 3.2 chat template. If you train a model with that current chat template and if the training script builds the prompts
|
8 |
+
from a json file with the chat tempalte the model starts to output as its first token <|eot_id|><|start_header_id|>assistant<|end_header_id|> and naturally the script will then halt generation.
|
9 |
the model learns to see this:
|
10 |
|
11 |
<|begin_of_text|><|start_header_id|>user<|end_header_id|>
|