bleysg commited on
Commit
1f28455
1 Parent(s): 6e790d3

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -16,7 +16,7 @@ datasets:
16
 
17
  # OpenOrca x OpenChat - Preview2 - 13B
18
 
19
- We have used our own [OpenOrca dataset](https://huggingface.co/datasets/Open-Orca/OpenOrca) to fine-tune Llama2-13B using [OpenChat](https://huggingface.co/openchat) packing and conditional behavior cloning.
20
  This dataset is our attempt to reproduce the dataset generated for Microsoft Research's [Orca Paper](https://arxiv.org/abs/2306.02707).
21
 
22
  This second preview release is trained on a curated filtered subset of most of our GPT-4 augmented data.
@@ -28,7 +28,7 @@ As well, this is done with <1/10th the compute requirement and using <20% of the
28
  We have run extensive evaluations internally and expect this model to **place number 1** on both the HuggingFaceH4 Open LLM Leaderboard and the GPT4ALL Leaderboard for 13B models.
29
 
30
  "One" of [OpenChat](https://huggingface.co/openchat) has joined our team, and we'd like to provide special thanks for their training of this model!
31
- We have utilized OpenChat conditional behavior cloning and [MultiPack algorithm](https://github.com/imoneoi/multipack_sampler) which achieves 99.85% bin-packing efficiency on our dataset.
32
  This has significantly reduced training time, with efficiency improvement of 3-10X over traditional methods.
33
 
34
 
 
16
 
17
  # OpenOrca x OpenChat - Preview2 - 13B
18
 
19
+ We have used our own [OpenOrca dataset](https://huggingface.co/datasets/Open-Orca/OpenOrca) to fine-tune Llama2-13B using [OpenChat](https://huggingface.co/openchat) packing.
20
  This dataset is our attempt to reproduce the dataset generated for Microsoft Research's [Orca Paper](https://arxiv.org/abs/2306.02707).
21
 
22
  This second preview release is trained on a curated filtered subset of most of our GPT-4 augmented data.
 
28
  We have run extensive evaluations internally and expect this model to **place number 1** on both the HuggingFaceH4 Open LLM Leaderboard and the GPT4ALL Leaderboard for 13B models.
29
 
30
  "One" of [OpenChat](https://huggingface.co/openchat) has joined our team, and we'd like to provide special thanks for their training of this model!
31
+ We have utilized OpenChat [MultiPack algorithm](https://github.com/imoneoi/multipack_sampler) which achieves 99.85% bin-packing efficiency on our dataset.
32
  This has significantly reduced training time, with efficiency improvement of 3-10X over traditional methods.
33
 
34