metadata
language:
- en
- ja
license: cc-by-nc-4.0
library_name: transformers
tags:
- nsfw
- Visual novel
- roleplay
- mergekit
- merge
base_model:
- mistralai/Mistral-Small-Instruct-2409
datasets:
- roleplay4fun/aesir-v1.1
- kalomaze/Opus_Instruct_3k
- Gryphe/Sonnet3.5-SlimOrcaDedupCleaned
- Aratako/Synthetic-Japanese-Roleplay-gpt-4o-mini-39.6k-formatted
- Aratako/Synthetic-Japanese-Roleplay-NSFW-Claude-3.5s-15.3k-formatted
- SkunkworksAI/reasoning-0.01
pipeline_tag: text-generation
model-index:
- name: ChatWaifu_22B_v2.0_preview
results:
- task:
type: text-generation
name: Text Generation
dataset:
name: IFEval (0-Shot)
type: HuggingFaceH4/ifeval
args:
num_few_shot: 0
metrics:
- type: inst_level_strict_acc and prompt_level_strict_acc
value: 67.45
name: strict accuracy
source:
url: >-
https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=spow12/ChatWaifu_22B_v2.0_preview
name: Open LLM Leaderboard
- task:
type: text-generation
name: Text Generation
dataset:
name: BBH (3-Shot)
type: BBH
args:
num_few_shot: 3
metrics:
- type: acc_norm
value: 45.49
name: normalized accuracy
source:
url: >-
https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=spow12/ChatWaifu_22B_v2.0_preview
name: Open LLM Leaderboard
- task:
type: text-generation
name: Text Generation
dataset:
name: MATH Lvl 5 (4-Shot)
type: hendrycks/competition_math
args:
num_few_shot: 4
metrics:
- type: exact_match
value: 16.31
name: exact match
source:
url: >-
https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=spow12/ChatWaifu_22B_v2.0_preview
name: Open LLM Leaderboard
- task:
type: text-generation
name: Text Generation
dataset:
name: GPQA (0-shot)
type: Idavidrein/gpqa
args:
num_few_shot: 0
metrics:
- type: acc_norm
value: 8.72
name: acc_norm
source:
url: >-
https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=spow12/ChatWaifu_22B_v2.0_preview
name: Open LLM Leaderboard
- task:
type: text-generation
name: Text Generation
dataset:
name: MuSR (0-shot)
type: TAUR-Lab/MuSR
args:
num_few_shot: 0
metrics:
- type: acc_norm
value: 3.53
name: acc_norm
source:
url: >-
https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=spow12/ChatWaifu_22B_v2.0_preview
name: Open LLM Leaderboard
- task:
type: text-generation
name: Text Generation
dataset:
name: MMLU-PRO (5-shot)
type: TIGER-Lab/MMLU-Pro
config: main
split: test
args:
num_few_shot: 5
metrics:
- type: acc
value: 33.2
name: accuracy
source:
url: >-
https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=spow12/ChatWaifu_22B_v2.0_preview
name: Open LLM Leaderboard
Model Card for Model ID
Merged model using mergekit
This model aimed to act like visual novel character.
Merge Format
models:
- model: mistralai/Mistral-Small-Instruct-2409_SFT
layer_range: [0, 56]
- model: mistralai/Mistral-Small-Instruct-2409
layer_range: [0, 56]
merge_method: slerp
base_model: mistralai/Mistral-Small-Instruct-2409_SFT
parameters:
t:
- filter: self_attn
value: [0, 0.5, 0.3, 0.7, 1]
- filter: mlp
value: [1, 0.5, 0.7, 0.3, 0]
- value: 0.5 # fallback for rest of tensors
dtype: bfloat16
WaifuModel Collections
Unified demo
Update 2.0
- 2024.09.23 Update 22B, Ver 2.0
Model Details
Model Description
- Developed by: spow12(yw_nam)
- Shared by : spow12(yw_nam)
- Model type: CausalLM
- Language(s) (NLP): japanese. English
- Finetuned from model : mistralai/Mistral-Small-Instruct-2409
Currently, chatbot has below personality.
| character | visual_novel |
|---|---|
| ムラサメ | Senren＊Banka |
| 茉子 | Senren＊Banka |
| 芳乃 | Senren＊Banka |
| レナ | Senren＊Banka |
| 千咲 | Senren＊Banka |
| 芦花 | Senren＊Banka |
| 愛衣 | Café Stella and the Reaper's Butterflies |
| 栞那 | Café Stella and the Reaper's Butterflies |
| ナツメ | Café Stella and the Reaper's Butterflies |
| 希 | Café Stella and the Reaper's Butterflies |
| 涼音 | Café Stella and the Reaper's Butterflies |
| あやせ | Riddle Joker |
| 七海 | Riddle Joker |
| 羽月 | Riddle Joker |
| 茉優 | Riddle Joker |
| 小春 | Riddle Joker |
But you can chat your own Character with persona text.
Feel free to test.
Your feedback will be helpful for improving model.
Dataset
Riddle Joker(Prviate)
Café Stella and the Reaper's Butterflies(Private)
Senren＊Banka(Private)
roleplay4fun/aesir-v1.1
kalomaze/Opus_Instruct_3k
Gryphe/Sonnet3.5-SlimOrcaDedupCleaned
Aratako/Synthetic-JP-EN-Coding-Dataset-567k (only using 50000 sample)
Aratako/Synthetic-Japanese-Roleplay-gpt-4o-mini-39.6k-formatted
Aratako/Synthetic-Japanese-Roleplay-NSFW-Claude-3.5s-15.3k-formatted
SkunkworksAI/reasoning-0.01
Feature
- Fluent Chat performance
- Reduce repetition problem when generate with many turn(over 20~30)
- Zero Shot character persona using description of character.
- 128k context window
- Memory ability that does not forget even after long-context generation
Demo
You can use Demo in google colab.
Check Here
Bias, Risks, and Limitations
This model can generate NSFW content.
Use & Credit
This model is currently available for non-commercial & Research purpose only.
Also, since I'm not detailed in licensing, I hope you use this model responsibly.
By sharing this model, I hope to contribute to the research efforts of our community (the open-source community and Waifu Lovers).
Citation
@misc {ChatWaifu_22B_v2.0
author = { YoungWoo Nam },
title = { ChatWaifu_22B_v2.0_preview },
year = 2024,
url = { https://huggingface.co/spow12/ChatWaifu_22B_v2.0_preview },
publisher = { Hugging Face }
}
Open LLM Leaderboard Evaluation Results
Detailed results can be found here
| Metric | Value |
|---|---|
| Avg. | 29.12 |
| IFEval (0-Shot) | 67.45 |
| BBH (3-Shot) | 45.49 |
| MATH Lvl 5 (4-Shot) | 16.31 |
| GPQA (0-shot) | 8.72 |
| MuSR (0-shot) | 3.53 |
| MMLU-PRO (5-shot) | 33.20 |