mlinmg commited on
Commit
799e908
1 Parent(s): 3ef7a8c

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +54 -0
README.md CHANGED
@@ -2,4 +2,58 @@
2
  license: other
3
  license_name: yi
4
  license_link: https://huggingface.co/01-ai/Yi-34B-Chat/blob/main/LICENSE
 
 
 
 
5
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2
  license: other
3
  license_name: yi
4
  license_link: https://huggingface.co/01-ai/Yi-34B-Chat/blob/main/LICENSE
5
+ language:
6
+ - en,
7
+ pipeline_tag: conversational
8
+ library_name: adapter-transformers
9
  ---
10
+ # SG Raccoon orion-to-dolph 66B
11
+
12
+ An auto-regressive causal LM created by combining 2x finetuned [Yi 34b](https://huggingface.co/01-ai/Yi-34B) into one.
13
+
14
+
15
+ # Prompting Format
16
+
17
+ chat format:
18
+
19
+ single-turn: <|startoftext|>Human: Hello!\n\nAssistant: <|endoftext|>
20
+
21
+ multi-turn: <|startoftext|>Human: Hello!\n\nAssistant: <|endoftext|>Hi!<|endoftext|>Human: How are you?\n\nAssistant: <|endoftext|>target2<|endoftext|>
22
+
23
+ # Merge process
24
+
25
+ The models used in the merge are [Xwin](https://huggingface.co/Xwin-LM/Xwin-LM-70B-V0.1) and [Euryale](https://huggingface.co/Sao10K/Euryale-1.3-L2-70B).
26
+
27
+ The layer ranges used are as follows:
28
+
29
+ ```yaml
30
+ - range 0, 16
31
+ Xwin
32
+ - range 8, 24
33
+ Euryale
34
+ - range 17, 32
35
+ Xwin
36
+ - range 25, 40
37
+ Euryale
38
+ - range 33, 48
39
+ Xwin
40
+ - range 41, 56
41
+ Euryale
42
+ - range 49, 64
43
+ Xwin
44
+ - range 57, 72
45
+ Euryale
46
+ - range 65, 80
47
+ Xwin
48
+ ```
49
+
50
+
51
+ # Benchmarks
52
+ Coming soon.
53
+
54
+ # Acknowledgements
55
+ Credits goes to [@chargoddard](https://huggingface.co/chargoddard) for developing the framework used to merge the model - [mergekit](https://github.com/cg123/mergekit).
56
+
57
+ Special thanks to [@Undi95](https://huggingface.co/Undi95).
58
+
59
+ Also credits to the [01-ai](https://huggingface.co/01-ai) team for their amazing model