BlueNipples commited on
Commit
1f6aa2f
1 Parent(s): 57fb995

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +41 -0
README.md CHANGED
@@ -1,3 +1,44 @@
1
  ---
2
  license: apache-2.0
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: apache-2.0
3
+ tags:
4
+ - Roleplay
5
+ - Solar
6
+ - Mistral
7
+ - Text Generation
8
  ---
9
+ ![SnowLotus Logo](https://cdn-uploads.huggingface.co/production/uploads/64bb1109aaccfd28b023bcec/gTQtPK46laLIFg0RTAv73.png)
10
+
11
+ ### Premise
12
+
13
+ So this is a basic slerp merge between a smart model and a good prose model. Prose and smarts. What we all want in an uncensored RP model right? I feel like Solar has untapped potential, in any case.
14
+
15
+ Sao10K's Frostwind finetune is a key component of the mixture, its smarts are impressive. NyxKrage's Frostmaid experiment, which merges Frostwind with a frankenmerge of Noromaid and a mystery medical model, delivers quite impressive prose. His model creatively incorporates long-range context and instructions too, despite being slightly incoherent due to the fraken merging.
16
+
17
+ So those are the main ingredients. Thanks to Nyx for sorting out the pytorch files btw.
18
+
19
+ ### Recipe
20
+
21
+ So, the recipe. I added solardoc by Nyx to frostwind at a 0.15 weight, and the gradient SLERP'd Frostwind (+solardoc) into Frostmaid with these params:
22
+
23
+ - filter: self_attn
24
+ value: [0.9, 0.4, 0.1, 0, 0]
25
+ - filter: mlp
26
+ value: [0.05, 0.95]
27
+ - value: 0.45
28
+
29
+
30
+ ### Tentative Dozen or So Test Conclusion
31
+
32
+ This model seems to have better prose, less GPT-ish language and no degredation in coherency from the last version.
33
+
34
+ Cheers to all the finetuners, mergers and developers without which open source models wouldn't be half of what they are.
35
+
36
+ Resources used:
37
+
38
+ https://huggingface.co/NyxKrage/FrostMaid-10.7B-TESTING-pt
39
+
40
+ https://huggingface.co/Sao10K/Frostwind-10.7B-v1
41
+
42
+ https://huggingface.co/NyxKrage/Solar-Doc-10.7B-Lora
43
+
44
+ https://github.com/cg123/mergekit/tree/main