--- tags: - merge - mergekit - lazymergekit - not-for-all-audiences - nsfw - rp - roleplay - role-play license: llama3 language: - en library_name: transformers pipeline_tag: text-generation base_model: - Sao10K/L3-8B-Stheno-v3.3-32K - Hastagaras/Jamet-8B-L3-MK.V-Blackroot - grimjim/Llama-3-Oasis-v1-OAS-8B - Casual-Autopsy/SOVL-MopeyMule-8B - Casual-Autopsy/MopeyMule-Blackroot-8B - ResplendentAI/Theory_of_Mind_Llama3 - ResplendentAI/RP_Format_QuoteAsterisk_Llama3 - ResplendentAI/Smarts_Llama3 - Casual-Autopsy/L3-Umbral-Mind-RP-v0.3-8B - Casual-Autopsy/Llama-3-Mopeyfied-Psychology-8B - Hastagaras/Halu-8B-Llama3-Blackroot --- Image by ろ47 **Highest ranked 8B model on the [UGI Leaderboard](https://huggingface.co/spaces/DontPlanToEnd/UGI-Leaderboard) as of writing this!** # Merge This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit). ## Merge Details **Stheno 3.3 seems to have a problem with quality when qaunted, but I will keep this up for archival perposes** The goal of this merge was to make an RP model better suited for role-plays with heavy themes such as but not limited to: - Mental illness - Self-harm - Trauma - Suicide I hated how RP models tended to be overly positive and hopeful with role-plays involving such themes, but thanks to [failspy/Llama-3-8B-Instruct-MopeyMule](https://huggingface.co/failspy/Llama-3-8B-Instruct-MopeyMule) this problem has been lessened considerably. If you're an enjoyer of savior/reverse savior type role-plays like myself, then this model is for you. ### Usage Info This model is meant to be used with asterisks/quotes RPing formats, any other format that isn't asterisks/quotes is likely to cause issues ### Quants ### Merge Method This model was merged using several Task Arithmetic merges and then tied together with a Model Stock merge, followed by another Task Arithmetic merge with a model containing psychology data. ### Models Merged The following models were included in the merge: * [Sao10K/L3-8B-Stheno-v3.3-32K](https://huggingface.co/Sao10K/L3-8B-Stheno-v3.3-32K) * [Hastagaras/Halu-8B-Llama3-Blackroot](Hastagaras/Halu-8B-Llama3-Blackroot) * [Casual-Autopsy/Llama-3-Mopeyfied-Psychology-8B](https://huggingface.co/Casual-Autopsy/Llama-3-Mopeyfied-Psychology-8B) * [Casual-Autopsy/L3-Umbral-Mind-RP-v0.3-8B](https://huggingface.co/Casual-Autopsy/L3-Umbral-Mind-RP-v0.3-8B) * Casual-Autopsy/Umbral-v3-1 + [ResplendentAI/Theory_of_Mind_Llama3](https://huggingface.co/ResplendentAI/Theory_of_Mind_Llama3) * [Sao10K/L3-8B-Stheno-v3.3-32K](https://huggingface.co/Sao10K/L3-8B-Stheno-v3.3-32K) * [Casual-Autopsy/SOVL-MopeyMule-8B](https://huggingface.co/Casual-Autopsy/SOVL-MopeyMule-8B) * [Casual-Autopsy/MopeyMule-Blackroot-8B](https://huggingface.co/Casual-Autopsy/MopeyMule-Blackroot-8B) * Casual-Autopsy/Umbral-v3-2 + [ResplendentAI/Smarts_Llama3](https://huggingface.co/ResplendentAI/Smarts_Llama3) * [Hastagaras/Jamet-8B-L3-MK.V-Blackroot](https://huggingface.co/Hastagaras/Jamet-8B-L3-MK.V-Blackroot) * [Casual-Autopsy/SOVL-MopeyMule-8B](https://huggingface.co/Casual-Autopsy/SOVL-MopeyMule-8B) * [Casual-Autopsy/MopeyMule-Blackroot-8B](https://huggingface.co/Casual-Autopsy/MopeyMule-Blackroot-8B) * Casual-Autopsy/Umbral-v3-3 + [ResplendentAI/RP_Format_QuoteAsterisk_Llama3](https://huggingface.co/ResplendentAI/RP_Format_QuoteAsterisk_Llama3) * [grimjim/Llama-3-Oasis-v1-OAS-8B](https://huggingface.co/grimjim/Llama-3-Oasis-v1-OAS-8B) * [Casual-Autopsy/SOVL-MopeyMule-8B](https://huggingface.co/Casual-Autopsy/SOVL-MopeyMule-8B) * [Casual-Autopsy/MopeyMule-Blackroot-8B](https://huggingface.co/Casual-Autopsy/MopeyMule-Blackroot-8B) ## Secret Sauce The following YAML configuration was used to produce this model: ### Umbral-1 ```yaml slices: - sources: - model: Sao10K/L3-8B-Stheno-v3.3-32K layer_range: [0, 32] parameters: weight: 0.65 - model: Casual-Autopsy/SOVL-MopeyMule-8B layer_range: [0, 32] parameters: weight: 0.25 - model: Casual-Autopsy/MopeyMule-Blackroot-8B layer_range: [0, 32] parameters: weight: 0.1 merge_method: task_arithmetic base_model: Sao10K/L3-8B-Stheno-v3.2 normalize: False dtype: bfloat16 ``` ### Umbral-2 ```yaml slices: - sources: - model: Hastagaras/Jamet-8B-L3-MK.V-Blackroot layer_range: [0, 32] parameters: weight: 0.75 - model: Casual-Autopsy/SOVL-MopeyMule-8B layer_range: [0, 32] parameters: weight: 0.15 - model: Casual-Autopsy/MopeyMule-Blackroot-8B layer_range: [0, 32] parameters: weight: 0.1 merge_method: task_arithmetic base_model: Hastagaras/Jamet-8B-L3-MK.V-Blackroot normalize: False dtype: bfloat16 ``` ### Umbral-3 ```yaml slices: - sources: - model: grimjim/Llama-3-Oasis-v1-OAS-8B layer_range: [0, 32] parameters: weight: 0.55 - model: Casual-Autopsy/SOVL-MopeyMule-8B layer_range: [0, 32] parameters: weight: 0.35 - model: Casual-Autopsy/MopeyMule-Blackroot-8B layer_range: [0, 32] parameters: weight: 0.1 merge_method: task_arithmetic base_model: grimjim/Llama-3-Oasis-v1-OAS-8B normalize: False dtype: bfloat16 ``` ### Umbral-Mind ```yaml models: - model: Casual-Autopsy/Umbral-1+ResplendentAI/Theory_of_Mind_Llama3 - model: Casual-Autopsy/Umbral-2+ResplendentAI/Smarts_Llama3 - model: Casual-Autopsy/Umbral-3+ResplendentAI/RP_Format_QuoteAsterisk_Llama3 merge_method: model_stock base_model: Casual-Autopsy/Umbral-1 dtype: bfloat16 ``` ### L3-Umbral-Mind-RP-v1.0.1-8B ```yaml slices: - sources: - model: Casual-Autopsy/Umbral-Mind layer_range: [0, 32] - model: Casual-Autopsy/Llama-3-Mopeyfied-Psychology-8B layer_range: [0, 32] parameters: weight: 0.14 - model: Sao10K/L3-8B-Stheno-v3.3-32K layer_range: [0, 32] parameters: weight: 0.03 - model: Hastagaras/Halu-8B-Llama3-Blackroot layer_range: [0, 32] parameters: weight: 0.03 merge_method: task_arithmetic base_model: Casual-Autopsy/Umbral-Mind dtype: bfloat16 ```