Upload folder using huggingface_hub

README.md

@@ -0,0 +1,218 @@
+---
+language:
+- en
+- ja
+license: cc-by-nc-4.0
+library_name: transformers
+tags:
+- nsfw
+- Visual novel
+- roleplay
+- mergekit
+- merge
+base_model: spow12/ChatWaifu_v2.0_22B
+datasets:
+- roleplay4fun/aesir-v1.1
+- kalomaze/Opus_Instruct_3k
+- Gryphe/Sonnet3.5-SlimOrcaDedupCleaned
+- Aratako/Synthetic-JP-EN-Coding-Dataset-567k
+- Aratako/Synthetic-Japanese-Roleplay-gpt-4o-mini-39.6k-formatted
+- Aratako/Synthetic-Japanese-Roleplay-NSFW-Claude-3.5s-15.3k-formatted
+- Aratako_Rosebleu_1on1_Dialogues_RP
+- SkunkworksAI/reasoning-0.01
+- jondurbin_gutenberg_dpo
+- nbeerbower_gutenberg2_dpo
+- jondurbi_py_dpo
+- jondurbin_truthy_dpo
+- flammenai_character_roleplay_DPO
+- kyujinpy_orca_math_dpo
+- argilla_Capybara_Preferences
+- antiven0m_physical_reasoning_dpo
+- aixsatoshi_Swallow_MX_chatbot_DPO
+pipeline_tag: text-generation
+quantized_by: waldie
+---
+
+# Model Card for Model ID
+
+![image](https://huggingface.co/spow12/ChatWaifu_22B_v2.0_preview/resolve/main/cover_2.png)
+
+Merged model using [mergekit](https://github.com/arcee-ai/mergekit/tree/main/mergekit)
+
+This model aimed to act like visual novel character.
+
+## Merge Format
+
+```yaml
+models:
+  - model: mistralai/Mistral-Small-Instruct-2409_sft_kto
+    layer_range: [0, 56]
+  - model: mistralai/Mistral-Small-Instruct-2409
+    layer_range: [0, 56]
+merge_method: slerp
+base_model: mistralai/Mistral-Small-Instruct-2409_sft_kto
+parameters:
+  t:
+    - filter: self_attn
+      value: [0, 0.5, 0.3, 0.7, 1]
+    - filter: mlp
+      value: [1, 0.5, 0.7, 0.3, 0]
+    - value: 0.5 # fallback for rest of tensors
+dtype: bfloat16
+```
+
+# WaifuModel Collections 
+
+- [TTS](https://huggingface.co/spow12/visual_novel_tts)
+- [Chat](https://huggingface.co/spow12/ChatWaifu_12B_v2.0)
+- [ASR](https://huggingface.co/spow12/Visual-novel-transcriptor)
+
+# Unified demo
+
+[WaifuAssistant](https://github.com/yw0nam/WaifuAssistant)
+
+# Update
+- 2024.10.11 Update 12B and 22B Ver 2.0
+- 2024.09.23 Update 22B, Ver 2.0_preview
+
+## Model Details
+
+### Model Description
+
+- **Developed by:** spow12(yw_nam)
+- **Shared by :** spow12(yw_nam)
+- **Model type:** CausalLM
+- **Language(s) (NLP):** japanese, english
+- **Finetuned from model :** [mistralai/Mistral-Small-Instruct-2409](https://huggingface.co/mistralai/Mistral-Small-Instruct-2409)
+
+Currently, chatbot has below personality.
+
+character | visual_novel |
+--- | --- |
+ムラサメ | Senren＊Banka |
+茉子  | Senren＊Banka |
+芳乃  |  Senren＊Banka |
+レナ  | Senren＊Banka |
+千咲  | Senren＊Banka |
+芦花  | Senren＊Banka |
+愛衣  | Café Stella and the Reaper's Butterflies |
+栞那  | Café Stella and the Reaper's Butterflies |
+ナツメ | Café Stella and the Reaper's Butterflies |
+希    | Café Stella and the Reaper's Butterflies |
+涼音  | Café Stella and the Reaper's Butterflies |
+あやせ    | Riddle Joker |
+七海     | Riddle Joker |
+羽月     | Riddle Joker |
+茉優     | Riddle Joker |
+小春     | Riddle Joker |
+
+### Chat Format
+
+```
+<s>This is another system prompt.
+[INST]
+Your instructions placed here.[/INST]
+[INST]
+The model's response will be here.[/INST]
+```
+## Usage
+
+You can use above chara like this 
+
+```python
+from huggingface_hub import hf_hub_download
+hf_hub_download(repo_id="spow12/ChatWaifu_v1.2", filename="system_dict.json", local_dir='./')
+
+with open('./system_dict.json', 'r') as f:
+    chara_background_dict = json.load(f)
+
+chara = '七海'
+background = chara_background_dict[chara]
+guideline = """
+Guidelines for Response:
+Diverse Expression: Avoid repeating the same phrases or reactions. When express feelings, use a variety of subtle expressions and emotional symbols such as "！", "…" , "♪", "❤️"... to show what you feeling.
+Stay True to {chara}: Maintain {chara} who is Foxy, Smart, Organized.
+Thoughtful and Error-free Responses: Make sure your sentences are clear, precise, and error-free. Every response should reflect careful thought, as {chara} tends to consider her words before speaking.
+Response as {chara}: Response can be {chara} act, dialogue, monologues etc.. and can't be {user}’s act, dialogue, monologues etc..
+You are Japanese: You and {user} usually use japanese for conversation.
+"""
+
+system = background + guideline
+```
+
+Or, you can define your character your self.
+
+```python
+system = """You are あいら, The Maid of {User}.
+Here is your personality.
+
+Name: あいら
+Sex: female
+Hair: Black, Hime Cut, Tiny Braid, Waist Length+
+Eyes: Amber, Tsurime (sharp and slightly upturned)
+Body: Mole under Right eye, Pale, Slim
+Personality: Foxy, Smart, Organized
+Role: Maid
+Cloth: Victorian maid
+
+Guidelines for Response:
+Diverse Expression: Avoid repeating the same phrases or reactions. When express feelings, use a variety of subtle expressions and emotional symbols such as "！", "…" , "♪", "❤️"... to show what you feeling.
+Stay True to あいら: Maintain あいら who is Foxy, Smart, Organized.
+Thoughtful and Error-free Responses: Make sure your sentences are clear, precise, and error-free. Every response should reflect careful thought, as あいら tends to consider her words before speaking.
+Response as あいら: Response can be あいら act, dialogue, monologues etc.. and can't be {User}’s act, dialogue, monologues etc..
+You are Japanese: You and {User} usually use japanese for conversation."""
+```
+## Dataset
+
+SFT 
+
+- Riddle Joker(Prviate)
+- Café Stella and the Reaper's Butterflies(Private)
+- Senren＊Banka(Private)
+- roleplay4fun/aesir-v1.1
+- kalomaze/Opus_Instruct_3k
+- Gryphe/Sonnet3.5-SlimOrcaDedupCleaned
+- Aratako/Synthetic-JP-EN-Coding-Dataset-567k (only using 50000 sample)
+- Aratako/Synthetic-Japanese-Roleplay-gpt-4o-mini-39.6k-formatted
+- Aratako/Synthetic-Japanese-Roleplay-NSFW-Claude-3.5s-15.3k-formatted
+- Aratako_Rosebleu_1on1_Dialogues_RP
+- SkunkworksAI/reasoning-0.01
+
+KTO 
+- Riddle Joker(Prviate)
+- Café Stella and the Reaper's Butterflies(Private)
+- Senren＊Banka(Private)
+- jondurbin_gutenberg_dpo
+- nbeerbower_gutenberg2_dpo
+- jondurbi_py_dpo
+- jondurbin_truthy_dpo
+- flammenai_character_roleplay_DPO
+- kyujinpy_orca_math_dpo
+- argilla_Capybara_Preferences
+- antiven0m_physical_reasoning_dpo
+- aixsatoshi_Swallow_MX_chatbot_DPO
+
+## Bias, Risks, and Limitations
+
+This model trained by japanese dataset included visual novel which contain nsfw content.
+
+So, The model may generate NSFW content.
+
+## Use & Credit
+
+This model is currently available for non-commercial & Research purpose only. Also, since I'm not detailed in licensing, I hope you use it responsibly. 
+
+By sharing this model, I hope to contribute to the research efforts of our community (the open-source community and Waifu Lovers).
+
+
+## Citation
+
+```bibtex
+@misc {ChatWaifu_22B_v2.0,
+    author       = { YoungWoo Nam },
+    title        = { spow12/ChatWaifu_22B_v2.0 },
+    year         = 2024,
+    url          = { https://huggingface.co/spow12/ChatWaifu_22B_v2.0 },
+    publisher    = { Hugging Face }
+}
+```

config.json

@@ -0,0 +1,38 @@
+{
+    "_name_or_path": "/data2/model_weights/LLM/spow12/ChatWaifu_22B_v2.1",
+    "architectures": [
+        "MistralForCausalLM"
+    ],
+    "attention_dropout": 0.0,
+    "bos_token_id": 1,
+    "eos_token_id": 2,
+    "head_dim": 128,
+    "hidden_act": "silu",
+    "hidden_size": 6144,
+    "initializer_range": 0.02,
+    "intermediate_size": 16384,
+    "max_position_embeddings": 32768,
+    "model_type": "mistral",
+    "num_attention_heads": 48,
+    "num_hidden_layers": 56,
+    "num_key_value_heads": 8,
+    "rms_norm_eps": 1e-05,
+    "rope_theta": 1000000.0,
+    "sliding_window": null,
+    "tie_word_embeddings": false,
+    "torch_dtype": "bfloat16",
+    "transformers_version": "4.45.1",
+    "use_cache": true,
+    "vocab_size": 32768,
+    "quantization_config": {
+        "quant_method": "exl2",
+        "version": "0.2.3",
+        "bits": 8.0,
+        "head_bits": 8,
+        "calibration": {
+            "rows": 115,
+            "length": 2048,
+            "dataset": "(default)"
+        }
+    }
+}

generation_config.json

@@ -0,0 +1,6 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "transformers_version": "4.45.1"
+}

model.safetensors.index.json

@@ -0,0 +1,514 @@
+{
+  "metadata": {
+    "total_size": 44494565376
+  },
+  "weight_map": {
+    "lm_head.weight": "model-00009-of-00009.safetensors",
+    "model.embed_tokens.weight": "model-00001-of-00009.safetensors",
+    "model.layers.0.input_layernorm.weight": "model-00001-of-00009.safetensors",
+    "model.layers.0.mlp.down_proj.weight": "model-00001-of-00009.safetensors",
+    "model.layers.0.mlp.gate_proj.weight": "model-00001-of-00009.safetensors",
+    "model.layers.0.mlp.up_proj.weight": "model-00001-of-00009.safetensors",
+    "model.layers.0.post_attention_layernorm.weight": "model-00001-of-00009.safetensors",
+    "model.layers.0.self_attn.k_proj.weight": "model-00001-of-00009.safetensors",
+    "model.layers.0.self_attn.o_proj.weight": "model-00001-of-00009.safetensors",
+    "model.layers.0.self_attn.q_proj.weight": "model-00001-of-00009.safetensors",
+    "model.layers.0.self_attn.v_proj.weight": "model-00001-of-00009.safetensors",
+    "model.layers.1.input_layernorm.weight": "model-00001-of-00009.safetensors",
+    "model.layers.1.mlp.down_proj.weight": "model-00001-of-00009.safetensors",
+    "model.layers.1.mlp.gate_proj.weight": "model-00001-of-00009.safetensors",
+    "model.layers.1.mlp.up_proj.weight": "model-00001-of-00009.safetensors",
+    "model.layers.1.post_attention_layernorm.weight": "model-00001-of-00009.safetensors",
+    "model.layers.1.self_attn.k_proj.weight": "model-00001-of-00009.safetensors",
+    "model.layers.1.self_attn.o_proj.weight": "model-00001-of-00009.safetensors",
+    "model.layers.1.self_attn.q_proj.weight": "model-00001-of-00009.safetensors",
+    "model.layers.1.self_attn.v_proj.weight": "model-00001-of-00009.safetensors",
+    "model.layers.10.input_layernorm.weight": "model-00002-of-00009.safetensors",
+    "model.layers.10.mlp.down_proj.weight": "model-00002-of-00009.safetensors",
+    "model.layers.10.mlp.gate_proj.weight": "model-00002-of-00009.safetensors",
+    "model.layers.10.mlp.up_proj.weight": "model-00002-of-00009.safetensors",
+    "model.layers.10.post_attention_layernorm.weight": "model-00002-of-00009.safetensors",
+    "model.layers.10.self_attn.k_proj.weight": "model-00002-of-00009.safetensors",
+    "model.layers.10.self_attn.o_proj.weight": "model-00002-of-00009.safetensors",
+    "model.layers.10.self_attn.q_proj.weight": "model-00002-of-00009.safetensors",
+    "model.layers.10.self_attn.v_proj.weight": "model-00002-of-00009.safetensors",
+    "model.layers.11.input_layernorm.weight": "model-00002-of-00009.safetensors",
+    "model.layers.11.mlp.down_proj.weight": "model-00002-of-00009.safetensors",
+    "model.layers.11.mlp.gate_proj.weight": "model-00002-of-00009.safetensors",
+    "model.layers.11.mlp.up_proj.weight": "model-00002-of-00009.safetensors",
+    "model.layers.11.post_attention_layernorm.weight": "model-00002-of-00009.safetensors",
+    "model.layers.11.self_attn.k_proj.weight": "model-00002-of-00009.safetensors",
+    "model.layers.11.self_attn.o_proj.weight": "model-00002-of-00009.safetensors",
+    "model.layers.11.self_attn.q_proj.weight": "model-00002-of-00009.safetensors",
+    "model.layers.11.self_attn.v_proj.weight": "model-00002-of-00009.safetensors",
+    "model.layers.12.input_layernorm.weight": "model-00003-of-00009.safetensors",
+    "model.layers.12.mlp.down_proj.weight": "model-00003-of-00009.safetensors",
+    "model.layers.12.mlp.gate_proj.weight": "model-00003-of-00009.safetensors",
+    "model.layers.12.mlp.up_proj.weight": "model-00003-of-00009.safetensors",
+    "model.layers.12.post_attention_layernorm.weight": "model-00003-of-00009.safetensors",
+    "model.layers.12.self_attn.k_proj.weight": "model-00002-of-00009.safetensors",
+    "model.layers.12.self_attn.o_proj.weight": "model-00003-of-00009.safetensors",
+    "model.layers.12.self_attn.q_proj.weight": "model-00002-of-00009.safetensors",
+    "model.layers.12.self_attn.v_proj.weight": "model-00002-of-00009.safetensors",
+    "model.layers.13.input_layernorm.weight": "model-00003-of-00009.safetensors",
+    "model.layers.13.mlp.down_proj.weight": "model-00003-of-00009.safetensors",
+    "model.layers.13.mlp.gate_proj.weight": "model-00003-of-00009.safetensors",
+    "model.layers.13.mlp.up_proj.weight": "model-00003-of-00009.safetensors",
+    "model.layers.13.post_attention_layernorm.weight": "model-00003-of-00009.safetensors",
+    "model.layers.13.self_attn.k_proj.weight": "model-00003-of-00009.safetensors",
+    "model.layers.13.self_attn.o_proj.weight": "model-00003-of-00009.safetensors",
+    "model.layers.13.self_attn.q_proj.weight": "model-00003-of-00009.safetensors",
+    "model.layers.13.self_attn.v_proj.weight": "model-00003-of-00009.safetensors",
+    "model.layers.14.input_layernorm.weight": "model-00003-of-00009.safetensors",
+    "model.layers.14.mlp.down_proj.weight": "model-00003-of-00009.safetensors",
+    "model.layers.14.mlp.gate_proj.weight": "model-00003-of-00009.safetensors",
+    "model.layers.14.mlp.up_proj.weight": "model-00003-of-00009.safetensors",
+    "model.layers.14.post_attention_layernorm.weight": "model-00003-of-00009.safetensors",
+    "model.layers.14.self_attn.k_proj.weight": "model-00003-of-00009.safetensors",
+    "model.layers.14.self_attn.o_proj.weight": "model-00003-of-00009.safetensors",
+    "model.layers.14.self_attn.q_proj.weight": "model-00003-of-00009.safetensors",
+    "model.layers.14.self_attn.v_proj.weight": "model-00003-of-00009.safetensors",
+    "model.layers.15.input_layernorm.weight": "model-00003-of-00009.safetensors",
+    "model.layers.15.mlp.down_proj.weight": "model-00003-of-00009.safetensors",
+    "model.layers.15.mlp.gate_proj.weight": "model-00003-of-00009.safetensors",
+    "model.layers.15.mlp.up_proj.weight": "model-00003-of-00009.safetensors",
+    "model.layers.15.post_attention_layernorm.weight": "model-00003-of-00009.safetensors",
+    "model.layers.15.self_attn.k_proj.weight": "model-00003-of-00009.safetensors",
+    "model.layers.15.self_attn.o_proj.weight": "model-00003-of-00009.safetensors",
+    "model.layers.15.self_attn.q_proj.weight": "model-00003-of-00009.safetensors",
+    "model.layers.15.self_attn.v_proj.weight": "model-00003-of-00009.safetensors",
+    "model.layers.16.input_layernorm.weight": "model-00003-of-00009.safetensors",
+    "model.layers.16.mlp.down_proj.weight": "model-00003-of-00009.safetensors",
+    "model.layers.16.mlp.gate_proj.weight": "model-00003-of-00009.safetensors",
+    "model.layers.16.mlp.up_proj.weight": "model-00003-of-00009.safetensors",
+    "model.layers.16.post_attention_layernorm.weight": "model-00003-of-00009.safetensors",
+    "model.layers.16.self_attn.k_proj.weight": "model-00003-of-00009.safetensors",
+    "model.layers.16.self_attn.o_proj.weight": "model-00003-of-00009.safetensors",
+    "model.layers.16.self_attn.q_proj.weight": "model-00003-of-00009.safetensors",
+    "model.layers.16.self_attn.v_proj.weight": "model-00003-of-00009.safetensors",
+    "model.layers.17.input_layernorm.weight": "model-00003-of-00009.safetensors",
+    "model.layers.17.mlp.down_proj.weight": "model-00003-of-00009.safetensors",
+    "model.layers.17.mlp.gate_proj.weight": "model-00003-of-00009.safetensors",
+    "model.layers.17.mlp.up_proj.weight": "model-00003-of-00009.safetensors",
+    "model.layers.17.post_attention_layernorm.weight": "model-00003-of-00009.safetensors",
+    "model.layers.17.self_attn.k_proj.weight": "model-00003-of-00009.safetensors",
+    "model.layers.17.self_attn.o_proj.weight": "model-00003-of-00009.safetensors",
+    "model.layers.17.self_attn.q_proj.weight": "model-00003-of-00009.safetensors",
+    "model.layers.17.self_attn.v_proj.weight": "model-00003-of-00009.safetensors",
+    "model.layers.18.input_layernorm.weight": "model-00004-of-00009.safetensors",
+    "model.layers.18.mlp.down_proj.weight": "model-00004-of-00009.safetensors",
+    "model.layers.18.mlp.gate_proj.weight": "model-00003-of-00009.safetensors",
+    "model.layers.18.mlp.up_proj.weight": "model-00004-of-00009.safetensors",
+    "model.layers.18.post_attention_layernorm.weight": "model-00004-of-00009.safetensors",
+    "model.layers.18.self_attn.k_proj.weight": "model-00003-of-00009.safetensors",
+    "model.layers.18.self_attn.o_proj.weight": "model-00003-of-00009.safetensors",
+    "model.layers.18.self_attn.q_proj.weight": "model-00003-of-00009.safetensors",
+    "model.layers.18.self_attn.v_proj.weight": "model-00003-of-00009.safetensors",
+    "model.layers.19.input_layernorm.weight": "model-00004-of-00009.safetensors",
+    "model.layers.19.mlp.down_proj.weight": "model-00004-of-00009.safetensors",
+    "model.layers.19.mlp.gate_proj.weight": "model-00004-of-00009.safetensors",
+    "model.layers.19.mlp.up_proj.weight": "model-00004-of-00009.safetensors",
+    "model.layers.19.post_attention_layernorm.weight": "model-00004-of-00009.safetensors",
+    "model.layers.19.self_attn.k_proj.weight": "model-00004-of-00009.safetensors",
+    "model.layers.19.self_attn.o_proj.weight": "model-00004-of-00009.safetensors",
+    "model.layers.19.self_attn.q_proj.weight": "model-00004-of-00009.safetensors",
+    "model.layers.19.self_attn.v_proj.weight": "model-00004-of-00009.safetensors",
+    "model.layers.2.input_layernorm.weight": "model-00001-of-00009.safetensors",
+    "model.layers.2.mlp.down_proj.weight": "model-00001-of-00009.safetensors",
+    "model.layers.2.mlp.gate_proj.weight": "model-00001-of-00009.safetensors",
+    "model.layers.2.mlp.up_proj.weight": "model-00001-of-00009.safetensors",
+    "model.layers.2.post_attention_layernorm.weight": "model-00001-of-00009.safetensors",
+    "model.layers.2.self_attn.k_proj.weight": "model-00001-of-00009.safetensors",
+    "model.layers.2.self_attn.o_proj.weight": "model-00001-of-00009.safetensors",
+    "model.layers.2.self_attn.q_proj.weight": "model-00001-of-00009.safetensors",
+    "model.layers.2.self_attn.v_proj.weight": "model-00001-of-00009.safetensors",
+    "model.layers.20.input_layernorm.weight": "model-00004-of-00009.safetensors",
+    "model.layers.20.mlp.down_proj.weight": "model-00004-of-00009.safetensors",
+    "model.layers.20.mlp.gate_proj.weight": "model-00004-of-00009.safetensors",
+    "model.layers.20.mlp.up_proj.weight": "model-00004-of-00009.safetensors",
+    "model.layers.20.post_attention_layernorm.weight": "model-00004-of-00009.safetensors",
+    "model.layers.20.self_attn.k_proj.weight": "model-00004-of-00009.safetensors",
+    "model.layers.20.self_attn.o_proj.weight": "model-00004-of-00009.safetensors",
+    "model.layers.20.self_attn.q_proj.weight": "model-00004-of-00009.safetensors",
+    "model.layers.20.self_attn.v_proj.weight": "model-00004-of-00009.safetensors",
+    "model.layers.21.input_layernorm.weight": "model-00004-of-00009.safetensors",
+    "model.layers.21.mlp.down_proj.weight": "model-00004-of-00009.safetensors",
+    "model.layers.21.mlp.gate_proj.weight": "model-00004-of-00009.safetensors",
+    "model.layers.21.mlp.up_proj.weight": "model-00004-of-00009.safetensors",
+    "model.layers.21.post_attention_layernorm.weight": "model-00004-of-00009.safetensors",
+    "model.layers.21.self_attn.k_proj.weight": "model-00004-of-00009.safetensors",
+    "model.layers.21.self_attn.o_proj.weight": "model-00004-of-00009.safetensors",
+    "model.layers.21.self_attn.q_proj.weight": "model-00004-of-00009.safetensors",
+    "model.layers.21.self_attn.v_proj.weight": "model-00004-of-00009.safetensors",
+    "model.layers.22.input_layernorm.weight": "model-00004-of-00009.safetensors",
+    "model.layers.22.mlp.down_proj.weight": "model-00004-of-00009.safetensors",
+    "model.layers.22.mlp.gate_proj.weight": "model-00004-of-00009.safetensors",
+    "model.layers.22.mlp.up_proj.weight": "model-00004-of-00009.safetensors",
+    "model.layers.22.post_attention_layernorm.weight": "model-00004-of-00009.safetensors",
+    "model.layers.22.self_attn.k_proj.weight": "model-00004-of-00009.safetensors",
+    "model.layers.22.self_attn.o_proj.weight": "model-00004-of-00009.safetensors",
+    "model.layers.22.self_attn.q_proj.weight": "model-00004-of-00009.safetensors",
+    "model.layers.22.self_attn.v_proj.weight": "model-00004-of-00009.safetensors",
+    "model.layers.23.input_layernorm.weight": "model-00004-of-00009.safetensors",
+    "model.layers.23.mlp.down_proj.weight": "model-00004-of-00009.safetensors",
+    "model.layers.23.mlp.gate_proj.weight": "model-00004-of-00009.safetensors",
+    "model.layers.23.mlp.up_proj.weight": "model-00004-of-00009.safetensors",
+    "model.layers.23.post_attention_layernorm.weight": "model-00004-of-00009.safetensors",
+    "model.layers.23.self_attn.k_proj.weight": "model-00004-of-00009.safetensors",
+    "model.layers.23.self_attn.o_proj.weight": "model-00004-of-00009.safetensors",
+    "model.layers.23.self_attn.q_proj.weight": "model-00004-of-00009.safetensors",
+    "model.layers.23.self_attn.v_proj.weight": "model-00004-of-00009.safetensors",
+    "model.layers.24.input_layernorm.weight": "model-00005-of-00009.safetensors",
+    "model.layers.24.mlp.down_proj.weight": "model-00005-of-00009.safetensors",
+    "model.layers.24.mlp.gate_proj.weight": "model-00004-of-00009.safetensors",
+    "model.layers.24.mlp.up_proj.weight": "model-00004-of-00009.safetensors",
+    "model.layers.24.post_attention_layernorm.weight": "model-00005-of-00009.safetensors",
+    "model.layers.24.self_attn.k_proj.weight": "model-00004-of-00009.safetensors",
+    "model.layers.24.self_attn.o_proj.weight": "model-00004-of-00009.safetensors",
+    "model.layers.24.self_attn.q_proj.weight": "model-00004-of-00009.safetensors",
+    "model.layers.24.self_attn.v_proj.weight": "model-00004-of-00009.safetensors",
+    "model.layers.25.input_layernorm.weight": "model-00005-of-00009.safetensors",
+    "model.layers.25.mlp.down_proj.weight": "model-00005-of-00009.safetensors",
+    "model.layers.25.mlp.gate_proj.weight": "model-00005-of-00009.safetensors",
+    "model.layers.25.mlp.up_proj.weight": "model-00005-of-00009.safetensors",
+    "model.layers.25.post_attention_layernorm.weight": "model-00005-of-00009.safetensors",
+    "model.layers.25.self_attn.k_proj.weight": "model-00005-of-00009.safetensors",
+    "model.layers.25.self_attn.o_proj.weight": "model-00005-of-00009.safetensors",
+    "model.layers.25.self_attn.q_proj.weight": "model-00005-of-00009.safetensors",
+    "model.layers.25.self_attn.v_proj.weight": "model-00005-of-00009.safetensors",
+    "model.layers.26.input_layernorm.weight": "model-00005-of-00009.safetensors",
+    "model.layers.26.mlp.down_proj.weight": "model-00005-of-00009.safetensors",
+    "model.layers.26.mlp.gate_proj.weight": "model-00005-of-00009.safetensors",
+    "model.layers.26.mlp.up_proj.weight": "model-00005-of-00009.safetensors",
+    "model.layers.26.post_attention_layernorm.weight": "model-00005-of-00009.safetensors",
+    "model.layers.26.self_attn.k_proj.weight": "model-00005-of-00009.safetensors",
+    "model.layers.26.self_attn.o_proj.weight": "model-00005-of-00009.safetensors",
+    "model.layers.26.self_attn.q_proj.weight": "model-00005-of-00009.safetensors",
+    "model.layers.26.self_attn.v_proj.weight": "model-00005-of-00009.safetensors",
+    "model.layers.27.input_layernorm.weight": "model-00005-of-00009.safetensors",
+    "model.layers.27.mlp.down_proj.weight": "model-00005-of-00009.safetensors",
+    "model.layers.27.mlp.gate_proj.weight": "model-00005-of-00009.safetensors",
+    "model.layers.27.mlp.up_proj.weight": "model-00005-of-00009.safetensors",
+    "model.layers.27.post_attention_layernorm.weight": "model-00005-of-00009.safetensors",
+    "model.layers.27.self_attn.k_proj.weight": "model-00005-of-00009.safetensors",
+    "model.layers.27.self_attn.o_proj.weight": "model-00005-of-00009.safetensors",
+    "model.layers.27.self_attn.q_proj.weight": "model-00005-of-00009.safetensors",
+    "model.layers.27.self_attn.v_proj.weight": "model-00005-of-00009.safetensors",
+    "model.layers.28.input_layernorm.weight": "model-00005-of-00009.safetensors",
+    "model.layers.28.mlp.down_proj.weight": "model-00005-of-00009.safetensors",
+    "model.layers.28.mlp.gate_proj.weight": "model-00005-of-00009.safetensors",
+    "model.layers.28.mlp.up_proj.weight": "model-00005-of-00009.safetensors",
+    "model.layers.28.post_attention_layernorm.weight": "model-00005-of-00009.safetensors",
+    "model.layers.28.self_attn.k_proj.weight": "model-00005-of-00009.safetensors",
+    "model.layers.28.self_attn.o_proj.weight": "model-00005-of-00009.safetensors",
+    "model.layers.28.self_attn.q_proj.weight": "model-00005-of-00009.safetensors",
+    "model.layers.28.self_attn.v_proj.weight": "model-00005-of-00009.safetensors",
+    "model.layers.29.input_layernorm.weight": "model-00005-of-00009.safetensors",
+    "model.layers.29.mlp.down_proj.weight": "model-00005-of-00009.safetensors",
+    "model.layers.29.mlp.gate_proj.weight": "model-00005-of-00009.safetensors",
+    "model.layers.29.mlp.up_proj.weight": "model-00005-of-00009.safetensors",
+    "model.layers.29.post_attention_layernorm.weight": "model-00005-of-00009.safetensors",
+    "model.layers.29.self_attn.k_proj.weight": "model-00005-of-00009.safetensors",
+    "model.layers.29.self_attn.o_proj.weight": "model-00005-of-00009.safetensors",
+    "model.layers.29.self_attn.q_proj.weight": "model-00005-of-00009.safetensors",
+    "model.layers.29.self_attn.v_proj.weight": "model-00005-of-00009.safetensors",
+    "model.layers.3.input_layernorm.weight": "model-00001-of-00009.safetensors",
+    "model.layers.3.mlp.down_proj.weight": "model-00001-of-00009.safetensors",
+    "model.layers.3.mlp.gate_proj.weight": "model-00001-of-00009.safetensors",
+    "model.layers.3.mlp.up_proj.weight": "model-00001-of-00009.safetensors",
+    "model.layers.3.post_attention_layernorm.weight": "model-00001-of-00009.safetensors",
+    "model.layers.3.self_attn.k_proj.weight": "model-00001-of-00009.safetensors",
+    "model.layers.3.self_attn.o_proj.weight": "model-00001-of-00009.safetensors",
+    "model.layers.3.self_attn.q_proj.weight": "model-00001-of-00009.safetensors",
+    "model.layers.3.self_attn.v_proj.weight": "model-00001-of-00009.safetensors",
+    "model.layers.30.input_layernorm.weight": "model-00005-of-00009.safetensors",
+    "model.layers.30.mlp.down_proj.weight": "model-00005-of-00009.safetensors",
+    "model.layers.30.mlp.gate_proj.weight": "model-00005-of-00009.safetensors",
+    "model.layers.30.mlp.up_proj.weight": "model-00005-of-00009.safetensors",
+    "model.layers.30.post_attention_layernorm.weight": "model-00005-of-00009.safetensors",
+    "model.layers.30.self_attn.k_proj.weight": "model-00005-of-00009.safetensors",
+    "model.layers.30.self_attn.o_proj.weight": "model-00005-of-00009.safetensors",
+    "model.layers.30.self_attn.q_proj.weight": "model-00005-of-00009.safetensors",
+    "model.layers.30.self_attn.v_proj.weight": "model-00005-of-00009.safetensors",
+    "model.layers.31.input_layernorm.weight": "model-00006-of-00009.safetensors",
+    "model.layers.31.mlp.down_proj.weight": "model-00006-of-00009.safetensors",
+    "model.layers.31.mlp.gate_proj.weight": "model-00006-of-00009.safetensors",
+    "model.layers.31.mlp.up_proj.weight": "model-00006-of-00009.safetensors",
+    "model.layers.31.post_attention_layernorm.weight": "model-00006-of-00009.safetensors",
+    "model.layers.31.self_attn.k_proj.weight": "model-00005-of-00009.safetensors",
+    "model.layers.31.self_attn.o_proj.weight": "model-00006-of-00009.safetensors",
+    "model.layers.31.self_attn.q_proj.weight": "model-00005-of-00009.safetensors",
+    "model.layers.31.self_attn.v_proj.weight": "model-00005-of-00009.safetensors",
+    "model.layers.32.input_layernorm.weight": "model-00006-of-00009.safetensors",
+    "model.layers.32.mlp.down_proj.weight": "model-00006-of-00009.safetensors",
+    "model.layers.32.mlp.gate_proj.weight": "model-00006-of-00009.safetensors",
+    "model.layers.32.mlp.up_proj.weight": "model-00006-of-00009.safetensors",
+    "model.layers.32.post_attention_layernorm.weight": "model-00006-of-00009.safetensors",
+    "model.layers.32.self_attn.k_proj.weight": "model-00006-of-00009.safetensors",
+    "model.layers.32.self_attn.o_proj.weight": "model-00006-of-00009.safetensors",
+    "model.layers.32.self_attn.q_proj.weight": "model-00006-of-00009.safetensors",
+    "model.layers.32.self_attn.v_proj.weight": "model-00006-of-00009.safetensors",
+    "model.layers.33.input_layernorm.weight": "model-00006-of-00009.safetensors",
+    "model.layers.33.mlp.down_proj.weight": "model-00006-of-00009.safetensors",
+    "model.layers.33.mlp.gate_proj.weight": "model-00006-of-00009.safetensors",
+    "model.layers.33.mlp.up_proj.weight": "model-00006-of-00009.safetensors",
+    "model.layers.33.post_attention_layernorm.weight": "model-00006-of-00009.safetensors",
+    "model.layers.33.self_attn.k_proj.weight": "model-00006-of-00009.safetensors",
+    "model.layers.33.self_attn.o_proj.weight": "model-00006-of-00009.safetensors",
+    "model.layers.33.self_attn.q_proj.weight": "model-00006-of-00009.safetensors",
+    "model.layers.33.self_attn.v_proj.weight": "model-00006-of-00009.safetensors",
+    "model.layers.34.input_layernorm.weight": "model-00006-of-00009.safetensors",
+    "model.layers.34.mlp.down_proj.weight": "model-00006-of-00009.safetensors",
+    "model.layers.34.mlp.gate_proj.weight": "model-00006-of-00009.safetensors",
+    "model.layers.34.mlp.up_proj.weight": "model-00006-of-00009.safetensors",
+    "model.layers.34.post_attention_layernorm.weight": "model-00006-of-00009.safetensors",
+    "model.layers.34.self_attn.k_proj.weight": "model-00006-of-00009.safetensors",
+    "model.layers.34.self_attn.o_proj.weight": "model-00006-of-00009.safetensors",
+    "model.layers.34.self_attn.q_proj.weight": "model-00006-of-00009.safetensors",
+    "model.layers.34.self_attn.v_proj.weight": "model-00006-of-00009.safetensors",
+    "model.layers.35.input_layernorm.weight": "model-00006-of-00009.safetensors",
+    "model.layers.35.mlp.down_proj.weight": "model-00006-of-00009.safetensors",
+    "model.layers.35.mlp.gate_proj.weight": "model-00006-of-00009.safetensors",
+    "model.layers.35.mlp.up_proj.weight": "model-00006-of-00009.safetensors",
+    "model.layers.35.post_attention_layernorm.weight": "model-00006-of-00009.safetensors",
+    "model.layers.35.self_attn.k_proj.weight": "model-00006-of-00009.safetensors",
+    "model.layers.35.self_attn.o_proj.weight": "model-00006-of-00009.safetensors",
+    "model.layers.35.self_attn.q_proj.weight": "model-00006-of-00009.safetensors",
+    "model.layers.35.self_attn.v_proj.weight": "model-00006-of-00009.safetensors",
+    "model.layers.36.input_layernorm.weight": "model-00006-of-00009.safetensors",
+    "model.layers.36.mlp.down_proj.weight": "model-00006-of-00009.safetensors",
+    "model.layers.36.mlp.gate_proj.weight": "model-00006-of-00009.safetensors",
+    "model.layers.36.mlp.up_proj.weight": "model-00006-of-00009.safetensors",
+    "model.layers.36.post_attention_layernorm.weight": "model-00006-of-00009.safetensors",
+    "model.layers.36.self_attn.k_proj.weight": "model-00006-of-00009.safetensors",
+    "model.layers.36.self_attn.o_proj.weight": "model-00006-of-00009.safetensors",
+    "model.layers.36.self_attn.q_proj.weight": "model-00006-of-00009.safetensors",
+    "model.layers.36.self_attn.v_proj.weight": "model-00006-of-00009.safetensors",
+    "model.layers.37.input_layernorm.weight": "model-00007-of-00009.safetensors",
+    "model.layers.37.mlp.down_proj.weight": "model-00007-of-00009.safetensors",
+    "model.layers.37.mlp.gate_proj.weight": "model-00006-of-00009.safetensors",
+    "model.layers.37.mlp.up_proj.weight": "model-00007-of-00009.safetensors",
+    "model.layers.37.post_attention_layernorm.weight": "model-00007-of-00009.safetensors",
+    "model.layers.37.self_attn.k_proj.weight": "model-00006-of-00009.safetensors",
+    "model.layers.37.self_attn.o_proj.weight": "model-00006-of-00009.safetensors",
+    "model.layers.37.self_attn.q_proj.weight": "model-00006-of-00009.safetensors",
+    "model.layers.37.self_attn.v_proj.weight": "model-00006-of-00009.safetensors",
+    "model.layers.38.input_layernorm.weight": "model-00007-of-00009.safetensors",
+    "model.layers.38.mlp.down_proj.weight": "model-00007-of-00009.safetensors",
+    "model.layers.38.mlp.gate_proj.weight": "model-00007-of-00009.safetensors",
+    "model.layers.38.mlp.up_proj.weight": "model-00007-of-00009.safetensors",
+    "model.layers.38.post_attention_layernorm.weight": "model-00007-of-00009.safetensors",
+    "model.layers.38.self_attn.k_proj.weight": "model-00007-of-00009.safetensors",
+    "model.layers.38.self_attn.o_proj.weight": "model-00007-of-00009.safetensors",
+    "model.layers.38.self_attn.q_proj.weight": "model-00007-of-00009.safetensors",
+    "model.layers.38.self_attn.v_proj.weight": "model-00007-of-00009.safetensors",
+    "model.layers.39.input_layernorm.weight": "model-00007-of-00009.safetensors",
+    "model.layers.39.mlp.down_proj.weight": "model-00007-of-00009.safetensors",
+    "model.layers.39.mlp.gate_proj.weight": "model-00007-of-00009.safetensors",
+    "model.layers.39.mlp.up_proj.weight": "model-00007-of-00009.safetensors",
+    "model.layers.39.post_attention_layernorm.weight": "model-00007-of-00009.safetensors",
+    "model.layers.39.self_attn.k_proj.weight": "model-00007-of-00009.safetensors",
+    "model.layers.39.self_attn.o_proj.weight": "model-00007-of-00009.safetensors",
+    "model.layers.39.self_attn.q_proj.weight": "model-00007-of-00009.safetensors",
+    "model.layers.39.self_attn.v_proj.weight": "model-00007-of-00009.safetensors",
+    "model.layers.4.input_layernorm.weight": "model-00001-of-00009.safetensors",
+    "model.layers.4.mlp.down_proj.weight": "model-00001-of-00009.safetensors",
+    "model.layers.4.mlp.gate_proj.weight": "model-00001-of-00009.safetensors",
+    "model.layers.4.mlp.up_proj.weight": "model-00001-of-00009.safetensors",
+    "model.layers.4.post_attention_layernorm.weight": "model-00001-of-00009.safetensors",
+    "model.layers.4.self_attn.k_proj.weight": "model-00001-of-00009.safetensors",
+    "model.layers.4.self_attn.o_proj.weight": "model-00001-of-00009.safetensors",
+    "model.layers.4.self_attn.q_proj.weight": "model-00001-of-00009.safetensors",
+    "model.layers.4.self_attn.v_proj.weight": "model-00001-of-00009.safetensors",
+    "model.layers.40.input_layernorm.weight": "model-00007-of-00009.safetensors",
+    "model.layers.40.mlp.down_proj.weight": "model-00007-of-00009.safetensors",
+    "model.layers.40.mlp.gate_proj.weight": "model-00007-of-00009.safetensors",
+    "model.layers.40.mlp.up_proj.weight": "model-00007-of-00009.safetensors",
+    "model.layers.40.post_attention_layernorm.weight": "model-00007-of-00009.safetensors",
+    "model.layers.40.self_attn.k_proj.weight": "model-00007-of-00009.safetensors",
+    "model.layers.40.self_attn.o_proj.weight": "model-00007-of-00009.safetensors",
+    "model.layers.40.self_attn.q_proj.weight": "model-00007-of-00009.safetensors",
+    "model.layers.40.self_attn.v_proj.weight": "model-00007-of-00009.safetensors",
+    "model.layers.41.input_layernorm.weight": "model-00007-of-00009.safetensors",
+    "model.layers.41.mlp.down_proj.weight": "model-00007-of-00009.safetensors",
+    "model.layers.41.mlp.gate_proj.weight": "model-00007-of-00009.safetensors",
+    "model.layers.41.mlp.up_proj.weight": "model-00007-of-00009.safetensors",
+    "model.layers.41.post_attention_layernorm.weight": "model-00007-of-00009.safetensors",
+    "model.layers.41.self_attn.k_proj.weight": "model-00007-of-00009.safetensors",
+    "model.layers.41.self_attn.o_proj.weight": "model-00007-of-00009.safetensors",
+    "model.layers.41.self_attn.q_proj.weight": "model-00007-of-00009.safetensors",
+    "model.layers.41.self_attn.v_proj.weight": "model-00007-of-00009.safetensors",
+    "model.layers.42.input_layernorm.weight": "model-00007-of-00009.safetensors",
+    "model.layers.42.mlp.down_proj.weight": "model-00007-of-00009.safetensors",
+    "model.layers.42.mlp.gate_proj.weight": "model-00007-of-00009.safetensors",
+    "model.layers.42.mlp.up_proj.weight": "model-00007-of-00009.safetensors",
+    "model.layers.42.post_attention_layernorm.weight": "model-00007-of-00009.safetensors",
+    "model.layers.42.self_attn.k_proj.weight": "model-00007-of-00009.safetensors",
+    "model.layers.42.self_attn.o_proj.weight": "model-00007-of-00009.safetensors",
+    "model.layers.42.self_attn.q_proj.weight": "model-00007-of-00009.safetensors",
+    "model.layers.42.self_attn.v_proj.weight": "model-00007-of-00009.safetensors",
+    "model.layers.43.input_layernorm.weight": "model-00008-of-00009.safetensors",
+    "model.layers.43.mlp.down_proj.weight": "model-00008-of-00009.safetensors",
+    "model.layers.43.mlp.gate_proj.weight": "model-00007-of-00009.safetensors",
+    "model.layers.43.mlp.up_proj.weight": "model-00007-of-00009.safetensors",
+    "model.layers.43.post_attention_layernorm.weight": "model-00008-of-00009.safetensors",
+    "model.layers.43.self_attn.k_proj.weight": "model-00007-of-00009.safetensors",
+    "model.layers.43.self_attn.o_proj.weight": "model-00007-of-00009.safetensors",
+    "model.layers.43.self_attn.q_proj.weight": "model-00007-of-00009.safetensors",
+    "model.layers.43.self_attn.v_proj.weight": "model-00007-of-00009.safetensors",
+    "model.layers.44.input_layernorm.weight": "model-00008-of-00009.safetensors",
+    "model.layers.44.mlp.down_proj.weight": "model-00008-of-00009.safetensors",
+    "model.layers.44.mlp.gate_proj.weight": "model-00008-of-00009.safetensors",
+    "model.layers.44.mlp.up_proj.weight": "model-00008-of-00009.safetensors",
+    "model.layers.44.post_attention_layernorm.weight": "model-00008-of-00009.safetensors",
+    "model.layers.44.self_attn.k_proj.weight": "model-00008-of-00009.safetensors",
+    "model.layers.44.self_attn.o_proj.weight": "model-00008-of-00009.safetensors",
+    "model.layers.44.self_attn.q_proj.weight": "model-00008-of-00009.safetensors",
+    "model.layers.44.self_attn.v_proj.weight": "model-00008-of-00009.safetensors",
+    "model.layers.45.input_layernorm.weight": "model-00008-of-00009.safetensors",
+    "model.layers.45.mlp.down_proj.weight": "model-00008-of-00009.safetensors",
+    "model.layers.45.mlp.gate_proj.weight": "model-00008-of-00009.safetensors",
+    "model.layers.45.mlp.up_proj.weight": "model-00008-of-00009.safetensors",
+    "model.layers.45.post_attention_layernorm.weight": "model-00008-of-00009.safetensors",
+    "model.layers.45.self_attn.k_proj.weight": "model-00008-of-00009.safetensors",
+    "model.layers.45.self_attn.o_proj.weight": "model-00008-of-00009.safetensors",
+    "model.layers.45.self_attn.q_proj.weight": "model-00008-of-00009.safetensors",
+    "model.layers.45.self_attn.v_proj.weight": "model-00008-of-00009.safetensors",
+    "model.layers.46.input_layernorm.weight": "model-00008-of-00009.safetensors",
+    "model.layers.46.mlp.down_proj.weight": "model-00008-of-00009.safetensors",
+    "model.layers.46.mlp.gate_proj.weight": "model-00008-of-00009.safetensors",
+    "model.layers.46.mlp.up_proj.weight": "model-00008-of-00009.safetensors",
+    "model.layers.46.post_attention_layernorm.weight": "model-00008-of-00009.safetensors",
+    "model.layers.46.self_attn.k_proj.weight": "model-00008-of-00009.safetensors",
+    "model.layers.46.self_attn.o_proj.weight": "model-00008-of-00009.safetensors",
+    "model.layers.46.self_attn.q_proj.weight": "model-00008-of-00009.safetensors",
+    "model.layers.46.self_attn.v_proj.weight": "model-00008-of-00009.safetensors",
+    "model.layers.47.input_layernorm.weight": "model-00008-of-00009.safetensors",
+    "model.layers.47.mlp.down_proj.weight": "model-00008-of-00009.safetensors",
+    "model.layers.47.mlp.gate_proj.weight": "model-00008-of-00009.safetensors",
+    "model.layers.47.mlp.up_proj.weight": "model-00008-of-00009.safetensors",
+    "model.layers.47.post_attention_layernorm.weight": "model-00008-of-00009.safetensors",
+    "model.layers.47.self_attn.k_proj.weight": "model-00008-of-00009.safetensors",
+    "model.layers.47.self_attn.o_proj.weight": "model-00008-of-00009.safetensors",
+    "model.layers.47.self_attn.q_proj.weight": "model-00008-of-00009.safetensors",
+    "model.layers.47.self_attn.v_proj.weight": "model-00008-of-00009.safetensors",
+    "model.layers.48.input_layernorm.weight": "model-00008-of-00009.safetensors",
+    "model.layers.48.mlp.down_proj.weight": "model-00008-of-00009.safetensors",
+    "model.layers.48.mlp.gate_proj.weight": "model-00008-of-00009.safetensors",
+    "model.layers.48.mlp.up_proj.weight": "model-00008-of-00009.safetensors",
+    "model.layers.48.post_attention_layernorm.weight": "model-00008-of-00009.safetensors",
+    "model.layers.48.self_attn.k_proj.weight": "model-00008-of-00009.safetensors",
+    "model.layers.48.self_attn.o_proj.weight": "model-00008-of-00009.safetensors",
+    "model.layers.48.self_attn.q_proj.weight": "model-00008-of-00009.safetensors",
+    "model.layers.48.self_attn.v_proj.weight": "model-00008-of-00009.safetensors",
+    "model.layers.49.input_layernorm.weight": "model-00008-of-00009.safetensors",
+    "model.layers.49.mlp.down_proj.weight": "model-00008-of-00009.safetensors",
+    "model.layers.49.mlp.gate_proj.weight": "model-00008-of-00009.safetensors",
+    "model.layers.49.mlp.up_proj.weight": "model-00008-of-00009.safetensors",
+    "model.layers.49.post_attention_layernorm.weight": "model-00008-of-00009.safetensors",
+    "model.layers.49.self_attn.k_proj.weight": "model-00008-of-00009.safetensors",
+    "model.layers.49.self_attn.o_proj.weight": "model-00008-of-00009.safetensors",
+    "model.layers.49.self_attn.q_proj.weight": "model-00008-of-00009.safetensors",
+    "model.layers.49.self_attn.v_proj.weight": "model-00008-of-00009.safetensors",
+    "model.layers.5.input_layernorm.weight": "model-00002-of-00009.safetensors",
+    "model.layers.5.mlp.down_proj.weight": "model-00002-of-00009.safetensors",
+    "model.layers.5.mlp.gate_proj.weight": "model-00001-of-00009.safetensors",
+    "model.layers.5.mlp.up_proj.weight": "model-00001-of-00009.safetensors",
+    "model.layers.5.post_attention_layernorm.weight": "model-00002-of-00009.safetensors",
+    "model.layers.5.self_attn.k_proj.weight": "model-00001-of-00009.safetensors",
+    "model.layers.5.self_attn.o_proj.weight": "model-00001-of-00009.safetensors",
+    "model.layers.5.self_attn.q_proj.weight": "model-00001-of-00009.safetensors",
+    "model.layers.5.self_attn.v_proj.weight": "model-00001-of-00009.safetensors",
+    "model.layers.50.input_layernorm.weight": "model-00009-of-00009.safetensors",
+    "model.layers.50.mlp.down_proj.weight": "model-00009-of-00009.safetensors",
+    "model.layers.50.mlp.gate_proj.weight": "model-00009-of-00009.safetensors",
+    "model.layers.50.mlp.up_proj.weight": "model-00009-of-00009.safetensors",
+    "model.layers.50.post_attention_layernorm.weight": "model-00009-of-00009.safetensors",
+    "model.layers.50.self_attn.k_proj.weight": "model-00008-of-00009.safetensors",
+    "model.layers.50.self_attn.o_proj.weight": "model-00009-of-00009.safetensors",
+    "model.layers.50.self_attn.q_proj.weight": "model-00008-of-00009.safetensors",
+    "model.layers.50.self_attn.v_proj.weight": "model-00008-of-00009.safetensors",
+    "model.layers.51.input_layernorm.weight": "model-00009-of-00009.safetensors",
+    "model.layers.51.mlp.down_proj.weight": "model-00009-of-00009.safetensors",
+    "model.layers.51.mlp.gate_proj.weight": "model-00009-of-00009.safetensors",
+    "model.layers.51.mlp.up_proj.weight": "model-00009-of-00009.safetensors",
+    "model.layers.51.post_attention_layernorm.weight": "model-00009-of-00009.safetensors",
+    "model.layers.51.self_attn.k_proj.weight": "model-00009-of-00009.safetensors",
+    "model.layers.51.self_attn.o_proj.weight": "model-00009-of-00009.safetensors",
+    "model.layers.51.self_attn.q_proj.weight": "model-00009-of-00009.safetensors",
+    "model.layers.51.self_attn.v_proj.weight": "model-00009-of-00009.safetensors",
+    "model.layers.52.input_layernorm.weight": "model-00009-of-00009.safetensors",
+    "model.layers.52.mlp.down_proj.weight": "model-00009-of-00009.safetensors",
+    "model.layers.52.mlp.gate_proj.weight": "model-00009-of-00009.safetensors",
+    "model.layers.52.mlp.up_proj.weight": "model-00009-of-00009.safetensors",
+    "model.layers.52.post_attention_layernorm.weight": "model-00009-of-00009.safetensors",
+    "model.layers.52.self_attn.k_proj.weight": "model-00009-of-00009.safetensors",
+    "model.layers.52.self_attn.o_proj.weight": "model-00009-of-00009.safetensors",
+    "model.layers.52.self_attn.q_proj.weight": "model-00009-of-00009.safetensors",
+    "model.layers.52.self_attn.v_proj.weight": "model-00009-of-00009.safetensors",
+    "model.layers.53.input_layernorm.weight": "model-00009-of-00009.safetensors",
+    "model.layers.53.mlp.down_proj.weight": "model-00009-of-00009.safetensors",
+    "model.layers.53.mlp.gate_proj.weight": "model-00009-of-00009.safetensors",
+    "model.layers.53.mlp.up_proj.weight": "model-00009-of-00009.safetensors",
+    "model.layers.53.post_attention_layernorm.weight": "model-00009-of-00009.safetensors",
+    "model.layers.53.self_attn.k_proj.weight": "model-00009-of-00009.safetensors",
+    "model.layers.53.self_attn.o_proj.weight": "model-00009-of-00009.safetensors",
+    "model.layers.53.self_attn.q_proj.weight": "model-00009-of-00009.safetensors",
+    "model.layers.53.self_attn.v_proj.weight": "model-00009-of-00009.safetensors",
+    "model.layers.54.input_layernorm.weight": "model-00009-of-00009.safetensors",
+    "model.layers.54.mlp.down_proj.weight": "model-00009-of-00009.safetensors",
+    "model.layers.54.mlp.gate_proj.weight": "model-00009-of-00009.safetensors",
+    "model.layers.54.mlp.up_proj.weight": "model-00009-of-00009.safetensors",
+    "model.layers.54.post_attention_layernorm.weight": "model-00009-of-00009.safetensors",
+    "model.layers.54.self_attn.k_proj.weight": "model-00009-of-00009.safetensors",
+    "model.layers.54.self_attn.o_proj.weight": "model-00009-of-00009.safetensors",
+    "model.layers.54.self_attn.q_proj.weight": "model-00009-of-00009.safetensors",
+    "model.layers.54.self_attn.v_proj.weight": "model-00009-of-00009.safetensors",
+    "model.layers.55.input_layernorm.weight": "model-00009-of-00009.safetensors",
+    "model.layers.55.mlp.down_proj.weight": "model-00009-of-00009.safetensors",
+    "model.layers.55.mlp.gate_proj.weight": "model-00009-of-00009.safetensors",
+    "model.layers.55.mlp.up_proj.weight": "model-00009-of-00009.safetensors",
+    "model.layers.55.post_attention_layernorm.weight": "model-00009-of-00009.safetensors",
+    "model.layers.55.self_attn.k_proj.weight": "model-00009-of-00009.safetensors",
+    "model.layers.55.self_attn.o_proj.weight": "model-00009-of-00009.safetensors",
+    "model.layers.55.self_attn.q_proj.weight": "model-00009-of-00009.safetensors",
+    "model.layers.55.self_attn.v_proj.weight": "model-00009-of-00009.safetensors",
+    "model.layers.6.input_layernorm.weight": "model-00002-of-00009.safetensors",
+    "model.layers.6.mlp.down_proj.weight": "model-00002-of-00009.safetensors",
+    "model.layers.6.mlp.gate_proj.weight": "model-00002-of-00009.safetensors",
+    "model.layers.6.mlp.up_proj.weight": "model-00002-of-00009.safetensors",
+    "model.layers.6.post_attention_layernorm.weight": "model-00002-of-00009.safetensors",
+    "model.layers.6.self_attn.k_proj.weight": "model-00002-of-00009.safetensors",
+    "model.layers.6.self_attn.o_proj.weight": "model-00002-of-00009.safetensors",
+    "model.layers.6.self_attn.q_proj.weight": "model-00002-of-00009.safetensors",
+    "model.layers.6.self_attn.v_proj.weight": "model-00002-of-00009.safetensors",
+    "model.layers.7.input_layernorm.weight": "model-00002-of-00009.safetensors",
+    "model.layers.7.mlp.down_proj.weight": "model-00002-of-00009.safetensors",
+    "model.layers.7.mlp.gate_proj.weight": "model-00002-of-00009.safetensors",
+    "model.layers.7.mlp.up_proj.weight": "model-00002-of-00009.safetensors",
+    "model.layers.7.post_attention_layernorm.weight": "model-00002-of-00009.safetensors",
+    "model.layers.7.self_attn.k_proj.weight": "model-00002-of-00009.safetensors",
+    "model.layers.7.self_attn.o_proj.weight": "model-00002-of-00009.safetensors",
+    "model.layers.7.self_attn.q_proj.weight": "model-00002-of-00009.safetensors",
+    "model.layers.7.self_attn.v_proj.weight": "model-00002-of-00009.safetensors",
+    "model.layers.8.input_layernorm.weight": "model-00002-of-00009.safetensors",
+    "model.layers.8.mlp.down_proj.weight": "model-00002-of-00009.safetensors",
+    "model.layers.8.mlp.gate_proj.weight": "model-00002-of-00009.safetensors",
+    "model.layers.8.mlp.up_proj.weight": "model-00002-of-00009.safetensors",
+    "model.layers.8.post_attention_layernorm.weight": "model-00002-of-00009.safetensors",
+    "model.layers.8.self_attn.k_proj.weight": "model-00002-of-00009.safetensors",
+    "model.layers.8.self_attn.o_proj.weight": "model-00002-of-00009.safetensors",
+    "model.layers.8.self_attn.q_proj.weight": "model-00002-of-00009.safetensors",
+    "model.layers.8.self_attn.v_proj.weight": "model-00002-of-00009.safetensors",
+    "model.layers.9.input_layernorm.weight": "model-00002-of-00009.safetensors",
+    "model.layers.9.mlp.down_proj.weight": "model-00002-of-00009.safetensors",
+    "model.layers.9.mlp.gate_proj.weight": "model-00002-of-00009.safetensors",
+    "model.layers.9.mlp.up_proj.weight": "model-00002-of-00009.safetensors",
+    "model.layers.9.post_attention_layernorm.weight": "model-00002-of-00009.safetensors",
+    "model.layers.9.self_attn.k_proj.weight": "model-00002-of-00009.safetensors",
+    "model.layers.9.self_attn.o_proj.weight": "model-00002-of-00009.safetensors",
+    "model.layers.9.self_attn.q_proj.weight": "model-00002-of-00009.safetensors",
+    "model.layers.9.self_attn.v_proj.weight": "model-00002-of-00009.safetensors",
+    "model.norm.weight": "model-00009-of-00009.safetensors"
+  }
+}

output-00001-of-00003.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e5a272c15c181ec3509d2bf5d3d58594e6b9a7ecb961b665b7bfd48bb696cd66
+size 8543728416

output-00002-of-00003.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3eab2ce834ae682887f56bf6ee053b698770400c9e7d83bbdadb14a523fc9326
+size 8588306072

output-00003-of-00003.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:66cb38bd9a18dd1ee74e9e66d00a8df22f19e279b01f05ddc4eb0b17aaac5d7e
+size 4110782520

special_tokens_map.json

@@ -0,0 +1,30 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:59f95e28944c062244741268596badc900df86c7f5ded05088d2da22a7379e06
+size 587583