Heralax commited on
Commit
c529f7b
1 Parent(s): 1be6377

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -3,7 +3,7 @@ license: llama2
3
  ---
4
  # Version 1.50 B -- coherency fixes! The model should be good now.
5
 
6
- **What this update is: after some early feedback, and some internal testing that confirmed it, I discovered that the first version of Augmental-13b was undercooked and had hyperparamter issues. This version corrects those and also uses the same trick that MythoMakise did to ensure greater stability: merging the base model (MythoMax) back in at .33% weighting. The result is that this model stays more sane and in character while also still having its own unique flair.**
7
 
8
  So why 1.50 version A and version B? Version B is the original Augmental-13b with MythoMax merged back into it at .33% weighting; version A is a new version of Augmental trained with different hyperparameters, meant to fix the undertraining issue -- which then had MythoMax merged back into it at .33% weighting. The difference? From my testing, Augmental-13b-v1.50 B is a more distinct model from MythoMax, while Augmental-13b-v1.50A is closer to the base model (this makes sense, as the difference between the two is a lower LoRA rank for version A, which means fewer parameters were trained and less-complex new patterns were learned by the model).
9
 
 
3
  ---
4
  # Version 1.50 B -- coherency fixes! The model should be good now.
5
 
6
+ **What this update is: after some early feedback, and some internal testing that confirmed it, I discovered that the first version of Augmental-13b was a bit too inconsistent and incoherent. This version corrects that by using the same trick that MythoMakise did to ensure greater stability: merging the base model (MythoMax) back in at .33% weighting. The result is that this model stays more sane and in character while also still having its own unique flair.**
7
 
8
  So why 1.50 version A and version B? Version B is the original Augmental-13b with MythoMax merged back into it at .33% weighting; version A is a new version of Augmental trained with different hyperparameters, meant to fix the undertraining issue -- which then had MythoMax merged back into it at .33% weighting. The difference? From my testing, Augmental-13b-v1.50 B is a more distinct model from MythoMax, while Augmental-13b-v1.50A is closer to the base model (this makes sense, as the difference between the two is a lower LoRA rank for version A, which means fewer parameters were trained and less-complex new patterns were learned by the model).
9