Update README.md (#3)
Browse files- Update README.md (c4ec16a65abe020b1e85dd5e2f0618984fe5b36f)
Co-authored-by: Ariel Lee <[email protected]>
README.md
CHANGED
@@ -13,7 +13,7 @@ metrics:
|
|
13 |
|
14 |
# 🥳 Platypus-30B has arrived!
|
15 |
|
16 |
-
Platypus-30B is an instruction fine-tuned model based on the LLaMA-
|
17 |
|
18 |
| Metric | Value |
|
19 |
|-----------------------|-------|
|
@@ -21,18 +21,11 @@ Platypus-30B is an instruction fine-tuned model based on the LLaMA-30b transform
|
|
21 |
| ARC (25-shot) | 64.6 |
|
22 |
| HellaSwag (10-shot) | 84.3 |
|
23 |
| TruthfulQA (0-shot) | 45.8 |
|
24 |
-
|
25 |
-
| Avg. | 65 | 💥
|
26 |
-
|
27 |
-
## Usage
|
28 |
-
|
29 |
-
```sh
|
30 |
-
ADD
|
31 |
-
```
|
32 |
|
33 |
## Model Details
|
34 |
|
35 |
-
* **Trained by**:
|
36 |
* **Model type:** **Platypus-30B** is an auto-regressive language model based on the LLaMA transformer architecture.
|
37 |
* **Language(s)**: English
|
38 |
* **License for base weights**: License for the base LLaMA model's weights is Meta's [non-commercial bespoke license](https://github.com/facebookresearch/llama/blob/main/MODEL_CARD.md).
|
@@ -50,21 +43,7 @@ Dataset of highly filtered and curated question and answer pairs. Release TBD.
|
|
50 |
|
51 |
## Training Procedure
|
52 |
|
53 |
-
`lilloukas/Platypus-
|
54 |
-
|
55 |
-
| Hyperparameter | Value |
|
56 |
-
|---------------------|-------|
|
57 |
-
| learning_rate | --- |
|
58 |
-
| batch_size | --- |
|
59 |
-
| microbatch_size | --- |
|
60 |
-
| warmup_steps | --- |
|
61 |
-
| epochs | --- |
|
62 |
-
| weight_decay | --- |
|
63 |
-
| optimizer | --- |
|
64 |
-
| weight_decay | --- |
|
65 |
-
| cutoff_len | --- |
|
66 |
-
| lora_target_modules | --- |
|
67 |
-
|
68 |
|
69 |
## Limitations and bias
|
70 |
|
|
|
13 |
|
14 |
# 🥳 Platypus-30B has arrived!
|
15 |
|
16 |
+
Platypus-30B is an instruction fine-tuned model based on the LLaMA-30B transformer architecture and takes advantage of [LoRA]([LoRA](https://arxiv.org/pdf/2106.09685.pdf).
|
17 |
|
18 |
| Metric | Value |
|
19 |
|-----------------------|-------|
|
|
|
21 |
| ARC (25-shot) | 64.6 |
|
22 |
| HellaSwag (10-shot) | 84.3 |
|
23 |
| TruthfulQA (0-shot) | 45.8 |
|
24 |
+
| Avg. | 65 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
25 |
|
26 |
## Model Details
|
27 |
|
28 |
+
* **Trained by**: Cole Hunter & Ariel Lee
|
29 |
* **Model type:** **Platypus-30B** is an auto-regressive language model based on the LLaMA transformer architecture.
|
30 |
* **Language(s)**: English
|
31 |
* **License for base weights**: License for the base LLaMA model's weights is Meta's [non-commercial bespoke license](https://github.com/facebookresearch/llama/blob/main/MODEL_CARD.md).
|
|
|
43 |
|
44 |
## Training Procedure
|
45 |
|
46 |
+
`lilloukas/Platypus-30B` was instruction fine-tuned using LoRA on 4 A100 80GB. For training details and inference instructions please see the [Platypus-30B](https://github.com/arielnlee/Platypus-30B.git) GitHub repo.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
47 |
|
48 |
## Limitations and bias
|
49 |
|