yinsong1986
commited on
Commit
•
2ae0e2b
1
Parent(s):
66d630e
Update README.md
Browse files
README.md
CHANGED
@@ -9,7 +9,7 @@ MistralLite is a fine-tuned [Mistral-7B-v0.1](https://huggingface.co/mistralai/M
|
|
9 |
|
10 |
MistralLight evolves from [Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1), and their similarities and differences are summarized below:
|
11 |
|Model|Fine-tuned on long contexts| Max context length| RotaryEmbedding adaptation| Sliding Window Size|
|
12 |
-
|
13 |
| Mistral-7B-v0.1 | No | 32K | rope_theta = 10000 | 4096 |
|
14 |
| MistralLite | Yes | 32K | **rope_theta = 1000000** | **16384** |
|
15 |
|
|
|
9 |
|
10 |
MistralLight evolves from [Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1), and their similarities and differences are summarized below:
|
11 |
|Model|Fine-tuned on long contexts| Max context length| RotaryEmbedding adaptation| Sliding Window Size|
|
12 |
+
|----------|-------------:|------------:|-----------:|-----------:|
|
13 |
| Mistral-7B-v0.1 | No | 32K | rope_theta = 10000 | 4096 |
|
14 |
| MistralLite | Yes | 32K | **rope_theta = 1000000** | **16384** |
|
15 |
|