kidyu commited on
Commit
3131977
1 Parent(s): 9cbbc84

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +26 -3
README.md CHANGED
@@ -15,15 +15,27 @@ tags:
15
  - mergekit
16
  - merge
17
 
 
 
18
  ---
19
  # Moza-7B-v1.0
20
 
21
- This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
 
 
 
 
22
 
23
- ## Merge Details
24
  ### Merge Method
25
 
26
- This model was merged using the [DARE](https://arxiv.org/abs/2311.03099) [TIES](https://arxiv.org/abs/2306.01708) merge method using [mistralai/Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1) as a base.
 
 
 
 
 
 
27
 
28
  ### Models Merged
29
 
@@ -38,6 +50,17 @@ The following models were included in the merge:
38
  * [meta-math/MetaMath-Mistral-7B](https://huggingface.co/meta-math/MetaMath-Mistral-7B)
39
  * [Intel/neural-chat-7b-v3-3](https://huggingface.co/Intel/neural-chat-7b-v3-3)
40
 
 
 
 
 
 
 
 
 
 
 
 
41
  ### Configuration
42
 
43
  The following YAML configuration was used to produce this model:
 
15
  - mergekit
16
  - merge
17
 
18
+
19
+
20
  ---
21
  # Moza-7B-v1.0
22
 
23
+ ![image/png](https://cdn-uploads.huggingface.co/production/uploads/63474d73511cd17d2c790ed7/e7hw2xIzfpUseCFEOINg7.png)
24
+
25
+ This is a [meme-merge](https://en.wikipedia.org/wiki/Joke) of pre-trained language models,
26
+ created using [mergekit](https://github.com/cg123/mergekit).
27
+ Use at your own risk.
28
 
29
+ ## Details
30
  ### Merge Method
31
 
32
+ This model was merged using the [DARE](https://arxiv.org/abs/2311.03099) [TIES](https://arxiv.org/abs/2306.01708) merge method,
33
+ using [mistralai/Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1) as a base.
34
+
35
+ The value for `density` are from some blog I found,
36
+ and the weight was randomly generated and then assigned to the models,
37
+ with priority (of using the bigger weight) to `NeuralHermes`, `OpenOrca`, and `neural-chat`.
38
+ The models themselves are chosen by "vibes".
39
 
40
  ### Models Merged
41
 
 
50
  * [meta-math/MetaMath-Mistral-7B](https://huggingface.co/meta-math/MetaMath-Mistral-7B)
51
  * [Intel/neural-chat-7b-v3-3](https://huggingface.co/Intel/neural-chat-7b-v3-3)
52
 
53
+ ### Prompt Format
54
+
55
+ You (probably) can use `Alpaca` formatting for inference
56
+ (I mean it's not like I actually tested it in the first place :joy:)
57
+
58
+ ```
59
+ ### Instruction:
60
+
61
+ ### Response:
62
+ ```
63
+
64
  ### Configuration
65
 
66
  The following YAML configuration was used to produce this model: