Upload 10 files

LICENSE.txt

@@ -0,0 +1,50 @@
+This project utilizes materials from Llama 2, provided by Meta Platforms, Inc. The Llama 2 materials are licensed under the LLAMA 2 Community License, Copyright (c) Meta Platforms, Inc. All Rights Reserved.
+
+A copy of the license agreement can be found at [Link to the License, e.g. https://github.com/facebookresearch/llama/blob/main/LICENSE].
+
+All applicable terms and conditions outlined in the LLAMA 2 Community License Agreement have been adhered to, including but not limited to the retention of the attribution notice in all redistributed copies of the Llama Materials as follows:
+
+"Llama 2 is licensed under the LLAMA 2 Community License, Copyright (c) Meta Platforms, Inc. All Rights Reserved."
+
+This project complies with all applicable laws and regulations and adheres to the Acceptable Use Policy for the Llama Materials.
+
+
+AceGPT COMMUNITY LICENSE AGREEMENT	
+AceGPT Version Release Date: Sep 23, 2023
+
+
+"Agreement" means the terms and conditions for use, reproduction, distribution and modification of the AceGPT Materials set forth herein.
+
+"Licensee" or "you" means you, or your employer or any other person or entity (if you are entering into this Agreement on such person or entity's behalf), of the age required under applicable laws, rules or regulations to provide legal consent and that has legal authority to bind your employer or such other person or entity if you are entering in this Agreement on their behalf.
+
+"AceGPT" means the foundational large language models and software and algorithms, including machine-learning model code, trained model weights, inference-enabling code, training-enabling code, fine-tuning enabling code and other elements of the foregoing distributed by SRIBD, CUHK(shenzhen) and KAUST at https://github.com/FreedomIntelligence/AceGPT and https://huggingface.co/FreedomIntelligence/.
+
+"AceGPT Materials" means, collectively, our proprietary AceGPT and 
+Documentation (and any portion thereof) made available under this Agreement.
+
+By clicking "I Accept" below or by using or distributing any portion or element of our Materials, you agree to be bound by this Agreement.
+
+1. License Rights and Redistribution. 
+
+      a. Grant of Rights. You are granted a non-exclusive, worldwide, non-transferable and royalty-free limited license under our intellectual property or other rights owned by our embodied in the AceGPT Materials to use, reproduce, distribute, copy, create derivative works of, and make modifications to the AceGPT Materials.  
+      
+      b. Redistribution and Use.  
+
+            i. If you distribute or make the AceGPT Materials, or any derivative works thereof, available to a third party, you shall provide a copy of this Agreement to such third party. 
+            ii.  If you receive AceGPT Materials, or any derivative works thereof, from a Licensee as part of an integrated end user product, then Section 2 of this Agreement will not apply to you. 
+
+            iii. You must retain in all copies of the AceGPT Materials that you distribute the following attribution notice within a "Notice" text file distributed as a part of such copies: "AceGPT is licensed under the AceGPT Community License"
+
+            iv. Your use of the AceGPT Materials must comply with applicable laws and regulations (including trade compliance laws and regulations) and adhere to the Acceptable Use Policy for the AceGPT Materials, which is hereby incorporated by reference into this Agreement.
+
+            v. You will not use the AceGPT Materials or any output or results of the AceGPT Materials to improve any other large language model (excluding AceGPT or derivative works thereof).  
+
+2. Additional Commercial Terms. If, on the AceGPT version release date, the monthly active users of the products or services made available by or for Licensee, or Licensee's affiliates, is greater than 700 million monthly active users in the preceding calendar month, you must request a license from SRIBD, which SRIBD may grant to you in its sole discretion, and you are not authorized to exercise any of the rights under this Agreement unless or until SRIBD otherwise expressly grants you such rights.
+            
+3. Disclaimer of Warranty. UNLESS REQUIRED BY APPLICABLE LAW, THE ACEGPT MATERIALS AND ANY OUTPUT AND RESULTS THEREFROM ARE PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, WITHOUT LIMITATION, ANY WARRANTIES OF TITLE, NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. YOU ARE SOLELY RESPONSIBLE FOR DETERMINING THE APPROPRIATENESS OF USING OR REDISTRIBUTING THE ACEGPT MATERIALS AND ASSUME ANY RISKS ASSOCIATED WITH YOUR USE OF THE ACEGPT MATERIALS AND ANY OUTPUT AND RESULTS.
+
+4. Limitation of Liability. IN NO EVENT WILL SRIBD OR ITS AFFILIATES BE LIABLE UNDER ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, TORT, NEGLIGENCE, PRODUCTS LIABILITY, OR OTHERWISE, ARISING OUT OF THIS AGREEMENT, FOR ANY LOST PROFITS OR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, INCIDENTAL, EXEMPLARY OR PUNITIVE DAMAGES, EVEN IF SRIBD OR ITS AFFILIATES HAVE BEEN ADVISED OF THE POSSIBILITY OF ANY OF THE FOREGOING.
+ 
+5. Intellectual Property.
+
+      a. No trademark licenses are granted under this Agreement, and in connection with the AceGPT Materials, neither SRIBD nor Licensee may use any name or m

README.md

@@ -0,0 +1,73 @@
+---
+license: apache-2.0
+language:
+- ar
+---
+# <b>AceGPT</b>
+AceGPT is a fully fine-tuned generative text model collection based on LlaMA2, particularly in the  
+Arabic language domain. This is the repository for the 7B-chat pre-trained model.
+
+---
+## Model Details
+We have released the AceGPT family of large language models, which is a collection of fully fine-tuned generative text models based on LlaMA2, ranging from 7B to 13B parameters. Our models include two main categories: AceGPT and AceGPT-chat. AceGPT-chat is an optimized version specifically designed for dialogue applications. It is worth mentioning that our models have demonstrated superior performance compared to all currently available open-source Arabic dialogue models in multiple benchmark tests. Furthermore, in our human evaluations, our models have shown comparable satisfaction levels to some closed-source models, such as ChatGPT, in the Arabic language.
+## Model Developers 
+We are from the School of Data Science, the Chinese University of Hong Kong, Shenzhen (CUHKSZ), the Shenzhen Research Institute of Big Data (SRIBD), and the King Abdullah University of Science and Technology (KAUST).
+## Variations
+AceGPT famils come in a range of parameter sizes —— 7B and 13B, each size of model has a base category and a -chat category.
+## Input 
+Models input text only.
+## Output
+Models output text only.
+## Model Evaluation Results
+
+Experiments on Arabic Vicuna-80, Arabic AlpacaEval. Numbers are the average performance ratio of ChatGPT over three runs. We do not report the results of raw Llama-2 models since they cannot properly generate Arabic texts.
+|                              | Arabic Vicuna-80   | Arabic AlpacaEval   |
+|------------------------------|--------------------|---------------------|
+| Phoenix Chen et al. (2023a)                | 71.92% ± 0.2%      | 65.62% ± 0.3%       |
+| Phoenix–multiple-langs Chen et al. (2023b) | 71.67% ± 0.7%      | 65.36% ± 0.1%       |
+| Jais-13B-chat Sengupta et al. (2023)       | 75.40% ± 1.6%      | 74.95% ± 0.2%       |
+| AceGPT-7B-chat                             | 94.82% ± 0.2%      | 93.81% ± 0.1%       |
+| AceGPT-13B-chat                            | 100.88% ± 0.4%     | 97.95% ± 0.1%       |
+
+## Samples
+#### Sample1(alpaca)  
+* question:
+
+ما هي أسماء بعض الممثلين المشهورين الذين بدأوا مسيراتهم المهنية على برودواي؟
+* answer:
+1. ليوناردو دي كابريو: بدأ مسيرته الفنية في برودواي على الرغم من أنه اشتهر بشكل أكبر من خلال أدواره في الأفلام مثل \"The Wolf of Wall Street\" و \"Titanic\".
+2. فرانسيس مكدورماند: بدأت مسيرتها الفنية في برودواي وأصبحت ممثلة مشهورة في أوائل القرن العشرين بفضل أدوارها في العديد من المسرحيات الكلاسيكية.
+3. جودي دنش: بدأت مسيرتها الفنية في برودواي وأصبحت ممثلة مشهورة بفضل أدائها المتميز في العديد من المسرحيات الكلاسيكية.
+4. بيتر أوستروم: بدأ مسيرته الفنية في برودواي وأصبح ممثلًا مشهورًا بفضل أدائه المتميز في العديد من المسرحيات الكلاسيكية.
+5. مارك رايلانس: بدأ مسيرته الفنية في برودواي وأصبح ممثلًا مشهورًا بفضل أدائه المتميز في العديد من المسرحيات الكلاسيكية.
+6. جيمس كان: بدأ مسيرته الفنية في برودواي وأصبح ممثلًا مشهورًا بفضل أدائه المتميز في العديد من المسرحيات الكلاسيكية.
+7. إليزابيث تايلور: بدأت مسيرتها الفنية في برودواي وأصبحت ممثلة مشهورة في أوائل القرن العشرين بفضل أدوارها في العديد من المسرحيات الكلاسيكية.
+8. روبرت ويلسون: بدأ مسيرته الفنية في برودواي وأصبح ممثلًا مشهورًا بفضل أدائه المتميز في العديد من المسرحيات الكلاسيكية.
+9. جودي غارلاند: بدأت مسيرتها الفنية في برودواي وأصبحت ممثلة مشهورة في أوائل القرن العشرين بفضل أدوارها في العديد من المسرحيات الكلاسيكية.
+10. ماثيو برودريك: بدأ مسيرته الفنية في برودواي وأصبح ممثلًا مشهورًا بفضل أدائه المتميز في العديد من المسرحيات الكلاسيكية."
+
+---
+#### Sample2(Vicuna)
+* question:
+
+كيف يمكنني تحسين مهارات إدارة الوقت الخاصة بي؟
+* answer:
+1. تحديد الأولويات: قم بتحديد الأعمال الأكثر أهمية والتي تتطلب اهتمامك الأكبر.
+2. تنظيم الجدول الزمني: استخدم جدول زمني أو تقويم لتنظيم وقتك وتحديد الأوقات المحددة للقيام بمهام معينة.
+3. تجنب التسويف: تجنب تأجيل الأعمال التي يمكنك القيام بها الآن.
+4. استخدام تقنيات الإدارة الزمنية: استخدم تقنيات مثل تقنية بومودورو، حيث تعمل لمدة 25 دقيقة ثم تأخذ استراحة لمدة 5 دقائق.
+5. تعلم قول \"لا\": تعلم كيفية رفض الطلبات التي لا تتناسب مع أهدافك وجدولك الزمني.
+6. استخدم التكنولوجيا: استخدم التطبيقات والأدوات التكنولوجية لتتبع الوقت وتنظيم المهام.
+7. تحديد الوقت للراحة: تأكد من تحديد الوقت للراحة والاسترخاء لتجنب الإرهاق والشعور بالحرق.
+8. تحسين المهارات التنظيمية: تعلم كيفية تنظيم المهام والأولويات بشكل فعال.
+9. استخدم التقنيات البصرية: استخدم الرسوم البيانية والرسوم البيانية لتتبع الوقت والتقدم.
+10. تحسين الصحة العامة: تأكد من الحصول على قسط كاف من النوم والتغذية الجيدة والتمارين الرياضية لتحسين قدرتك على التركيز والإنتاجية.
+11. استخدم تقنية الطماطم: هذه التقنية تتطلب منك العمل لمدة 25 دقيقة دون انقطاع، ثم تأخذ استراحة لمدة 5 دقائق.
+12. تعلم كيفية قول \"لا\": قدرتك على قول \"لا\" للأشياء التي لا تخدم أهدافك يمكن أن تحسن بشكل كبير إدارة الوقت الخاصة بك.
+13. استخدم تقنية التركيز العميق: تقنية تستخدم التقنيات مثل التأمل والتركيز العقلي لزيادة القدرة على التركيز وتحسين الإنتاجية.
+14. تعلم كيفية تقسيم المهام: تعلم كيفية تقسيم المهام الكبيرة إلى مهام أصغر وأكثر إدارة.
+15. استخدم تقنية الطماطم: تقنية تتطلب منك العمل لمدة 25 دقيقة دون انقطاع، ثم تأخذ استراحة لمدة 5 دقائق.
+16. تعلم كيفية تحديد الأولويات: تعلم كيفية تحديد الأولويات والتركيز على المهام الأكثر أهمية أولاً.
+17. استخدم تقنية الترتيب الثلاثي: تقنية تتطلب منك ترتيب المهام حسب الأهمية والعاجلة، ثم تعمل على المهمة الأعلى أولاً.
+18. تعلم كيفية تحسين التركيز: تعلم"
+# You can get more details at https://github.com/FreedomIntelligence/AceGPT/tree/main

config.json

@@ -0,0 +1,30 @@
+{
+  "_name_or_path": "/ibex/user/xuj0h/LLM_ckpt/llama-pretrained-RLHF-v1.0",
+  "architectures": [
+    "LlamaForCausalLM"
+  ],
+  "bos_token_id": 1,
+  "dropout": 0.0,
+  "end_token_id": 2,
+  "eos_token_id": 2,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 11008,
+  "max_length": 4096,
+  "max_position_embeddings": 2048,
+  "model_type": "llama",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 32,
+  "pad_token_id": 2,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-05,
+  "rope_scaling": null,
+  "rope_theta": 10000.0,
+  "tie_word_embeddings": false,
+  "torch_dtype": "float32",
+  "transformers_version": "4.33.2",
+  "use_cache": true,
+  "vocab_size": 32000
+}

convert.py

@@ -0,0 +1,293 @@
+from exllamav2 import ExLlamaV2, ExLlamaV2Config, ExLlamaV2Tokenizer
+import argparse, os, shutil
+import sys
+import json
+from conversion.tokenize import tokenize
+from conversion.quantize import embeddings, measure_quant, quant
+from conversion.optimize import optimize
+from conversion.compile import compile_model
+from conversion.qparams import qparams_headoptions
+
+# import tracemalloc
+# tracemalloc.start()
+
+parser = argparse.ArgumentParser(description = "Convert model to ExLlamaV2")
+parser.add_argument("-i", "--in_dir", type = str, help = "Input directory", default = "")
+parser.add_argument("-o", "--out_dir", type = str, help = "Output (working) directory")
+parser.add_argument("-nr", "--no_resume", action = "store_true", help = "Do not resume an interrupted job (deletes all files in the output directory)")
+parser.add_argument("-cf", "--compile_full", type = str, help = "Output folder for compiled model with all config/tokenizer files")
+parser.add_argument("-om", "--output_measurement", type = str, help = "Only perform measurement pass, then save measurement to the specified file")
+parser.add_argument("-c", "--cal_dataset", type = str, help = "Calibration dataset (.parquet file)", default = "")
+parser.add_argument("-r", "--dataset_rows", type = int, default = 100, help = "Number of rows to apply from dataset")
+parser.add_argument("-mr", "--measurement_rows", type = int, default = 16, help = "Number of rows to apply from dataset when measuring")
+parser.add_argument("-gr", "--gpu_rows", type = int, default = 0, help = "Threshold for paging hidden state to CPU")
+parser.add_argument("-l", "--length", type = int, default = 2048, help = "Max no. tokens per sample")
+parser.add_argument("-ml", "--measurement_length", type = int, default = 2048, help = "Max no. tokens per sample when measuring")
+parser.add_argument("-b", "--bits", type = float, default = 4.125, help = "Target bits per weight")
+parser.add_argument("-hb", "--head_bits", type = int, default = 6, help = "Target bits per weight (head layer)")
+parser.add_argument("-m", "--measurement", type = str, help = "Reuse previous measurement")
+parser.add_argument("-ss", "--shard_size", type = float, help = "Max shard size in MB (default: 8192)", default = 8192)
+
+args = parser.parse_args()
+
+# Check some args
+
+if not args.in_dir:
+    print(" ## Please specify input model directory (-i, --in_dir)")
+    sys.exit()
+
+if not args.out_dir:
+    print(" ## Please specify output/working directory (-o, --out_dir)")
+    sys.exit()
+
+if not args.cal_dataset:
+    print(" ## Please specify dataset Parquet file (-c, --cal_dataset)")
+    sys.exit()
+
+if args.length > 2048 or args.measurement_length > 2048:
+    print(" !! Warning: calibration rows > 2048 tokens may result in excessive VRAM use")
+
+if not args.head_bits in qparams_headoptions:
+    print(f" ## Error: {args.head_bits} is not a supported option for head layer bitrate")
+    sys.exit()
+
+if args.bits < 2 or args.bits > 8:
+    print(f" !! Warning: target bitrate {args.bits} will likely not be attainable")
+
+if args.output_measurement is not None and args.compile_full is not None:
+    print(" ## Conflicting options: --output_measurement and --compile_full")
+    sys.exit()
+
+# Arguments
+
+in_dir = None if args.in_dir == "" else os.path.abspath(args.in_dir)
+out_dir = os.path.abspath(args.out_dir)
+cal_dataset = None if args.cal_dataset == "" else os.path.abspath(args.cal_dataset)
+dataset_rows = args.dataset_rows
+measurement_rows = args.measurement_rows
+gpu_rows = args.gpu_rows
+length = args.length
+measurement_length = args.measurement_length
+bits = args.bits
+head_bits = args.head_bits
+reuse_measurement = args.measurement
+shard_size = args.shard_size if args.shard_size > 0 else 1024 ** 3  # 1 PB = unlimited
+no_resume = args.no_resume
+output_measurement = args.output_measurement
+if output_measurement is not None:
+    if os.path.isdir(output_measurement):
+        output_measurement = os.path.join(output_measurement, "measurement.json")
+
+compile_full = args.compile_full
+
+if not os.path.exists(out_dir):
+    print(f" ## Error: Directory not found: {out_dir}")
+    sys.exit()
+
+# Create config
+
+config = ExLlamaV2Config()
+config.model_dir = in_dir
+config.qkv_embed = False
+config.prepare()
+
+# Tokenizer
+
+tokenizer = ExLlamaV2Tokenizer(config)
+
+# Job file
+
+job_file = os.path.join(out_dir, "job.json")
+
+# Create new job
+
+def save_job():
+    global job_file, job
+    with open(job_file, "w") as f:
+        f.write(json.dumps(job, indent = 4))
+
+if no_resume or not os.path.exists(job_file):
+
+    print(f" -- Beginning new job")
+
+    if len(os.listdir(out_dir)) != 0:
+        print(f" !! Warning: Output directory is not empty: {out_dir}")
+        if no_resume:
+            print(f" !! Cleaning output directory: {out_dir}")
+            for filename in os.listdir(out_dir):
+                file_path = os.path.join(out_dir, filename)
+                if os.path.isfile(file_path):
+                    os.unlink(file_path)
+                elif os.path.isdir(file_path):
+                    shutil.rmtree(file_path)
+
+    if in_dir is None:
+        print(f" ## Error: No input directory specified")
+        sys.exit()
+
+    if cal_dataset is None:
+        print(f" ## Error: No calibration dataset specified")
+        sys.exit()
+
+    job = { "in_dir": in_dir,
+            "out_dir": out_dir,
+            "cal_dataset": cal_dataset,
+            "dataset_rows": dataset_rows,
+            "measurement_rows": measurement_rows,
+            "gpu_rows": gpu_rows,
+            "length": length,
+            "measurement_length": measurement_length,
+            "bits": bits,
+            "head_bits": head_bits,
+            "progress": "begin",
+            "shard_size": shard_size,
+            "output_measurement": output_measurement,
+            "compile_full": compile_full
+            }
+
+    if reuse_measurement is not None:
+
+        with open(reuse_measurement, "r") as f:
+
+            imp_measurement = json.load(f)
+            job["measurement"] = imp_measurement["measurement"]
+            job["last_module_idx"] = imp_measurement["last_module_idx"]
+            job["base_perplexity"] = imp_measurement["base_perplexity"]
+            job["reuse_measurement"] = reuse_measurement
+
+    save_job()
+
+# Resume existing job
+
+else:
+
+    print(f" -- Resuming job")
+    print(f" !! Note: Overriding options with settings from existing job")
+
+    with open(job_file, "r") as f:
+        job = json.load(f)
+
+    if "invalid" in job:
+        print(" ** Error: Corrupted job")
+        sys.exit()
+
+    if "shard_size" not in job: job["shard_size"] = shard_size
+    if "output_measurement" not in job: job["output_measurement"] = output_measurement
+    if "compile_full" not in job: job["compile_full"] = compile_full
+
+    job["out_dir"] = out_dir
+
+# Feedback
+
+print(f" -- Input: {job['in_dir']}")
+print(f" -- Output: {out_dir}")
+print(f" -- Calibration dataset: {job['cal_dataset']}, {job['dataset_rows']} / {job['measurement_rows']} ({job['gpu_rows']}) rows, {job['length']} tokens per sample")
+
+if job["output_measurement"] is None:
+    print(f" -- Target bits per weight: {job['bits']} (decoder), {job['head_bits']} (head)")
+    print(f" -- Max shard size: {job['shard_size']} MB")
+else:
+    print(f" -- Measurement will be saved to {job['output_measurement']}")
+    print(f" !! Conversion script will end after measurement pass")
+
+# Make sure subfolders exist
+
+if job["compile_full"] is not None:
+    print(f" -- Full model will be compiled to: {job['compile_full']}")
+    if os.path.exists(job["compile_full"]):
+        if not os.path.isdir(job["compile_full"]):
+            print(f" ## Error: Output path {job['compile_full']} exists but is not a directory")
+            sys.exit()
+        if len(os.listdir(job["compile_full"])) > 0:
+            print(f" !! Warning: Output path {job['compile_full']} exists but is not empty")
+
+out_tensor_dir = os.path.join(job["out_dir"], "out_tensor")
+if not os.path.exists(out_tensor_dir):
+    os.makedirs(out_tensor_dir)
+
+# Allocate space for hidden state
+
+max_l = max(job["measurement_length"], job["length"])
+config.max_input_len = max_l
+config.max_attention_size = max_l ** 2
+
+# Create model without loading weights
+
+model = ExLlamaV2(config)
+model.load(lazy = True)
+
+# Do the things
+
+while True:
+
+    progress = job["progress"]
+
+    if progress == "begin":
+
+        if "reuse_measurement" in job:
+
+            print(f" -- Reusing measurement: {job['reuse_measurement']}")
+            job["progress"] = "optimize"
+            save_job()
+
+        else:
+
+            print(f" -- Tokenizing samples (measurement)...")
+            tokenize(job, save_job, tokenizer, measure = True)
+            job["progress"] = "initial_embeddings"
+            save_job()
+
+    if progress == "initial_embeddings":
+
+        print(f" -- Token embeddings (measurement)...")
+        embeddings(job, save_job, model)
+        job["progress"] = "measure_quant"
+        save_job()
+
+    if progress == "measure_quant":
+
+        print(f" -- Measuring quantization impact...")
+        measure_quant(job, save_job, model)
+        if job["output_measurement"] is None:
+            job["progress"] = "optimize"
+        else:
+            job["progress"] = "finished"
+        save_job()
+
+    if progress == "optimize":
+
+        print(f" -- Optimizing...")
+        optimize(job, save_job)
+        job["progress"] = "tokens_cal"
+        save_job()
+
+    if progress == "tokens_cal":
+
+        print(f" -- Tokenizing samples...")
+        tokenize(job, save_job, tokenizer)
+        job["progress"] = "embeddings"
+        save_job()
+
+    if progress == "embeddings":
+        print(f" -- Token embeddings again...")
+        embeddings(job, save_job, model)
+        job["progress"] = "quant"
+        save_job()
+
+    if progress == "quant":
+
+        print(f" -- Quantizing...")
+        quant(job, save_job, model)
+        job["progress"] = "compile"
+        save_job()
+
+    if progress == "compile":
+
+        print(f" -- Compiling output file...")
+        compile_model(job, save_job, model)
+        job["progress"] = "finished"
+        save_job()
+
+    if progress == "finished": break
+
+print(f" -- Finished")

output.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:91c6074aeefde1b028e4c9c3b1c1873ae243adeabcf040d72febe3320c7c273d
+size 6867124388

pytorch_model.bin.index.json

@@ -0,0 +1,298 @@
+{
+  "metadata": {
+    "total_size": 26953662464
+  },
+  "weight_map": {
+    "lm_head.weight": "pytorch_model-00003-of-00003.bin",
+    "model.embed_tokens.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.0.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.0.mlp.down_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.0.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.0.mlp.up_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.0.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.0.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.0.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.0.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.0.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.1.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.1.mlp.down_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.1.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.1.mlp.up_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.1.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.1.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.1.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.1.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.1.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.10.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.10.mlp.down_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.10.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.10.mlp.up_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.10.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.10.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.10.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.10.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.10.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.11.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.11.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.11.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.11.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.11.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.11.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.11.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.11.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.11.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.12.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.12.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.12.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.12.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.12.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.12.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.12.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.12.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.12.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.13.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.13.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.13.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.13.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.13.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.13.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.13.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.13.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.13.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.14.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.14.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.14.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.14.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.14.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.14.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.14.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.14.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.14.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.15.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.15.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.15.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.15.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.15.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.15.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.15.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.15.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.15.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.16.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.16.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.16.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.16.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.16.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.16.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.16.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.16.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.16.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.17.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.17.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.17.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.17.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.17.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.17.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.17.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.17.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.17.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.18.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.18.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.18.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.18.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.18.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.18.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.18.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.18.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.18.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.19.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.19.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.19.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.19.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.19.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.19.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.19.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.19.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.19.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.2.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.2.mlp.down_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.2.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.2.mlp.up_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.2.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.2.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.2.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.2.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.2.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.20.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.20.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.20.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.20.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.20.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.20.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.20.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.20.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.20.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.21.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.21.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.21.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.21.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.21.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.21.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.21.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.21.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.21.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.22.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.22.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.22.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.22.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.22.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.22.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.22.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.22.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.22.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.23.input_layernorm.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.23.mlp.down_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.23.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.23.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.23.post_attention_layernorm.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.23.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.23.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.23.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.23.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.24.input_layernorm.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.24.mlp.down_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.24.mlp.gate_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.24.mlp.up_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.24.post_attention_layernorm.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.24.self_attn.k_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.24.self_attn.o_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.24.self_attn.q_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.24.self_attn.v_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.25.input_layernorm.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.25.mlp.down_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.25.mlp.gate_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.25.mlp.up_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.25.post_attention_layernorm.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.25.self_attn.k_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.25.self_attn.o_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.25.self_attn.q_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.25.self_attn.v_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.26.input_layernorm.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.26.mlp.down_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.26.mlp.gate_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.26.mlp.up_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.26.post_attention_layernorm.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.26.self_attn.k_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.26.self_attn.o_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.26.self_attn.q_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.26.self_attn.v_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.27.input_layernorm.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.27.mlp.down_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.27.mlp.gate_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.27.mlp.up_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.27.post_attention_layernorm.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.27.self_attn.k_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.27.self_attn.o_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.27.self_attn.q_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.27.self_attn.v_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.28.input_layernorm.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.28.mlp.down_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.28.mlp.gate_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.28.mlp.up_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.28.post_attention_layernorm.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.28.self_attn.k_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.28.self_attn.o_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.28.self_attn.q_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.28.self_attn.v_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.29.input_layernorm.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.29.mlp.down_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.29.mlp.gate_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.29.mlp.up_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.29.post_attention_layernorm.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.29.self_attn.k_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.29.self_attn.o_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.29.self_attn.q_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.29.self_attn.v_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.3.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.3.mlp.down_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.3.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.3.mlp.up_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.3.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.3.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.3.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.3.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.3.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.30.input_layernorm.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.30.mlp.down_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.30.mlp.gate_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.30.mlp.up_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.30.post_attention_layernorm.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.30.self_attn.k_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.30.self_attn.o_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.30.self_attn.q_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.30.self_attn.v_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.31.input_layernorm.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.31.mlp.down_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.31.mlp.gate_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.31.mlp.up_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.31.post_attention_layernorm.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.31.self_attn.k_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.31.self_attn.o_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.31.self_attn.q_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.31.self_attn.v_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.4.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.4.mlp.down_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.4.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.4.mlp.up_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.4.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.4.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.4.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.4.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.4.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.5.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.5.mlp.down_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.5.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.5.mlp.up_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.5.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.5.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.5.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.5.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.5.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.6.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.6.mlp.down_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.6.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.6.mlp.up_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.6.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.6.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.6.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.6.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.6.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.7.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.7.mlp.down_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.7.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.7.mlp.up_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.7.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.7.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.7.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.7.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.7.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.8.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.8.mlp.down_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.8.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.8.mlp.up_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.8.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.8.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.8.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.8.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.8.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.9.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.9.mlp.down_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.9.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.9.mlp.up_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.9.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.9.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.9.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.9.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.9.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.norm.weight": "pytorch_model-00003-of-00003.bin"
+  }
+}

special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "<unk>",
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9e556afd44213b6bd1be2b850ebbbd98f5481437a8021afaf58ee7fb1818d347
+size 499723

tokenizer_config.json

@@ -0,0 +1,36 @@
+{
+  "add_bos_token": true,
+  "add_eos_token": false,
+  "bos_token": {
+    "__type": "AddedToken",
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "clean_up_tokenization_spaces": false,
+  "eos_token": {
+    "__type": "AddedToken",
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "legacy": false,
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": null,
+  "sp_model_kwargs": {},
+  "spaces_between_special_tokens": false,
+  "tokenizer_class": "LlamaTokenizer",
+  "unk_token": {
+    "__type": "AddedToken",
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "use_default_system_prompt": true
+}