Add files using upload-large-folder tool

README.md

@@ -0,0 +1,216 @@
+---
+license: apache-2.0
+model-index:
+- name: WizardLM-2-8x22B
+  results:
+  - task:
+      type: text-generation
+      name: Text Generation
+    dataset:
+      name: IFEval (0-Shot)
+      type: HuggingFaceH4/ifeval
+      args:
+        num_few_shot: 0
+    metrics:
+    - type: inst_level_strict_acc and prompt_level_strict_acc
+      value: 52.72
+      name: strict accuracy
+    source:
+      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=alpindale/WizardLM-2-8x22B
+      name: Open LLM Leaderboard
+  - task:
+      type: text-generation
+      name: Text Generation
+    dataset:
+      name: BBH (3-Shot)
+      type: BBH
+      args:
+        num_few_shot: 3
+    metrics:
+    - type: acc_norm
+      value: 48.58
+      name: normalized accuracy
+    source:
+      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=alpindale/WizardLM-2-8x22B
+      name: Open LLM Leaderboard
+  - task:
+      type: text-generation
+      name: Text Generation
+    dataset:
+      name: MATH Lvl 5 (4-Shot)
+      type: hendrycks/competition_math
+      args:
+        num_few_shot: 4
+    metrics:
+    - type: exact_match
+      value: 22.28
+      name: exact match
+    source:
+      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=alpindale/WizardLM-2-8x22B
+      name: Open LLM Leaderboard
+  - task:
+      type: text-generation
+      name: Text Generation
+    dataset:
+      name: GPQA (0-shot)
+      type: Idavidrein/gpqa
+      args:
+        num_few_shot: 0
+    metrics:
+    - type: acc_norm
+      value: 17.56
+      name: acc_norm
+    source:
+      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=alpindale/WizardLM-2-8x22B
+      name: Open LLM Leaderboard
+  - task:
+      type: text-generation
+      name: Text Generation
+    dataset:
+      name: MuSR (0-shot)
+      type: TAUR-Lab/MuSR
+      args:
+        num_few_shot: 0
+    metrics:
+    - type: acc_norm
+      value: 14.54
+      name: acc_norm
+    source:
+      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=alpindale/WizardLM-2-8x22B
+      name: Open LLM Leaderboard
+  - task:
+      type: text-generation
+      name: Text Generation
+    dataset:
+      name: MMLU-PRO (5-shot)
+      type: TIGER-Lab/MMLU-Pro
+      config: main
+      split: test
+      args:
+        num_few_shot: 5
+    metrics:
+    - type: acc
+      value: 39.96
+      name: accuracy
+    source:
+      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=alpindale/WizardLM-2-8x22B
+      name: Open LLM Leaderboard
+---
+
+
+
+<p style="font-size:20px;" align="center">
+🏠 <a href="https://wizardlm.github.io/WizardLM2" target="_blank">WizardLM-2 Release Blog</a> </p>
+<p align="center">
+🤗 <a href="https://huggingface.co/collections/microsoft/wizardlm-2-661d403f71e6c8257dbd598a" target="_blank">HF Repo</a>  •🐱 <a href="https://github.com/victorsungo/WizardLM/tree/main/WizardLM-2" target="_blank">Github Repo</a>  • 🐦 <a href="https://twitter.com/WizardLM_AI" target="_blank">Twitter</a> • 📃 <a href="https://arxiv.org/abs/2304.12244" target="_blank">[WizardLM]</a>  • 📃 <a href="https://arxiv.org/abs/2306.08568" target="_blank">[WizardCoder]</a>   • 📃 <a href="https://arxiv.org/abs/2308.09583" target="_blank">[WizardMath]</a>  <br>
+</p>
+<p align="center">
+    👋 Join our <a href="https://discord.gg/VZjjHtWrKs" target="_blank">Discord</a>
+</p>
+
+## See [here](https://huggingface.co/lucyknada/microsoft_WizardLM-2-7B) for the WizardLM-2-7B re-upload.
+
+## News 🔥🔥🔥 [2024/04/15]
+
+We introduce and opensource WizardLM-2, our next generation state-of-the-art large language models, 
+which have improved performance on complex chat, multilingual, reasoning and agent. 
+New family includes three cutting-edge models: WizardLM-2 8x22B, WizardLM-2 70B, and WizardLM-2 7B.
+
+- WizardLM-2 8x22B is our most advanced model, demonstrates highly competitive performance compared to those leading proprietary works 
+and consistently outperforms all the existing state-of-the-art opensource models.
+- WizardLM-2 70B reaches top-tier reasoning capabilities and is the first choice in the same size. 
+- WizardLM-2 7B is the fastest and achieves comparable performance with existing 10x larger opensource leading models.
+
+For more  details of WizardLM-2 please read our [release blog post](https://web.archive.org/web/20240415221214/https://wizardlm.github.io/WizardLM2/) and  upcoming paper.
+
+
+## Model Details
+
+* **Model name**: WizardLM-2 8x22B
+* **Developed by**: WizardLM@Microsoft AI
+* **Model type**: Mixture of Experts (MoE)
+* **Base model**: [mistral-community/Mixtral-8x22B-v0.1](https://huggingface.co/mistral-community/Mixtral-8x22B-v0.1)
+* **Parameters**: 141B
+* **Language(s)**: Multilingual
+* **Blog**: [Introducing WizardLM-2](https://web.archive.org/web/20240415221214/https://wizardlm.github.io/WizardLM2/)
+* **Repository**: [https://github.com/nlpxucan/WizardLM](https://github.com/nlpxucan/WizardLM)
+* **Paper**: WizardLM-2 (Upcoming)
+* **License**: Apache2.0
+
+
+## Model Capacities
+
+
+**MT-Bench**
+
+We also adopt the automatic MT-Bench evaluation framework based on GPT-4 proposed by lmsys to assess the performance of models. 
+The WizardLM-2 8x22B even demonstrates highly competitive performance compared to the most advanced proprietary models. 
+Meanwhile, WizardLM-2 7B and WizardLM-2 70B are all the top-performing models among the other leading baselines at 7B to 70B model scales.
+
+<p align="center" width="100%">
+<a ><img src="https://web.archive.org/web/20240415175608im_/https://wizardlm.github.io/WizardLM2/static/images/mtbench.png" alt="MTBench" style="width: 96%; min-width: 300px; display: block; margin: auto;"></a>
+</p>
+
+
+**Human Preferences Evaluation**
+
+We carefully collected a complex and challenging set consisting of real-world instructions, which includes main requirements of humanity, such as writing, coding, math, reasoning, agent, and multilingual. 
+We report the win:loss rate without tie:
+
+- WizardLM-2 8x22B is just slightly falling behind GPT-4-1106-preview, and significantly stronger than Command R Plus and GPT4-0314.
+- WizardLM-2 70B is better than GPT4-0613, Mistral-Large, and Qwen1.5-72B-Chat.
+- WizardLM-2 7B is comparable with Qwen1.5-32B-Chat, and surpasses Qwen1.5-14B-Chat and Starling-LM-7B-beta.
+
+<p align="center" width="100%">
+<a ><img src="https://web.archive.org/web/20240415163303im_/https://wizardlm.github.io/WizardLM2/static/images/winall.png" alt="Win" style="width: 96%; min-width: 300px; display: block; margin: auto;"></a>
+</p>
+
+
+
+
+
+## Method Overview
+We built a **fully AI powered synthetic training system** to train WizardLM-2 models, please refer to our [blog](https://web.archive.org/web/20240415221214/https://wizardlm.github.io/WizardLM2/) for more details of this system.
+
+<p align="center" width="100%">
+<a ><img src="https://web.archive.org/web/20240415163303im_/https://wizardlm.github.io/WizardLM2/static/images/exp_1.png" alt="Method" style="width: 96%; min-width: 300px; display: block; margin: auto;"></a>
+</p>
+
+
+
+
+
+## Usage
+
+❗<b>Note for model system prompts usage:</b>
+
+
+<b>WizardLM-2</b>  adopts the prompt format from <b>Vicuna</b> and supports **multi-turn** conversation. The prompt should be as following:
+
+```
+A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, 
+detailed, and polite answers to the user's questions. USER: Hi ASSISTANT: Hello.</s>
+USER: Who are you? ASSISTANT: I am WizardLM.</s>......
+```
+
+<b> Inference WizardLM-2 Demo Script</b>
+
+We provide a WizardLM-2 inference demo [code](https://github.com/nlpxucan/WizardLM/tree/main/demo) on our github.
+
+
+
+
+
+# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
+Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_alpindale__WizardLM-2-8x22B)
+
+|      Metric       |Value|
+|-------------------|----:|
+|Avg.               |32.61|
+|IFEval (0-Shot)    |52.72|
+|BBH (3-Shot)       |48.58|
+|MATH Lvl 5 (4-Shot)|22.28|
+|GPQA (0-shot)      |17.56|
+|MuSR (0-shot)      |14.54|
+|MMLU-PRO (5-shot)  |39.96|
+

config.json

@@ -0,0 +1,42 @@
+{
+    "_name_or_path": "",
+    "architectures": [
+        "MixtralForCausalLM"
+    ],
+    "attention_dropout": 0.0,
+    "bos_token_id": 1,
+    "eos_token_id": 2,
+    "hidden_act": "silu",
+    "hidden_size": 6144,
+    "initializer_range": 0.02,
+    "intermediate_size": 16384,
+    "max_position_embeddings": 65536,
+    "model_type": "mixtral",
+    "num_attention_heads": 48,
+    "num_experts_per_tok": 2,
+    "num_hidden_layers": 56,
+    "num_key_value_heads": 8,
+    "num_local_experts": 8,
+    "output_router_logits": false,
+    "rms_norm_eps": 1e-05,
+    "rope_theta": 1000000,
+    "router_aux_loss_coef": 0.001,
+    "router_jitter_noise": 0.0,
+    "sliding_window": null,
+    "tie_word_embeddings": false,
+    "torch_dtype": "bfloat16",
+    "transformers_version": "4.36.2",
+    "use_cache": false,
+    "vocab_size": 32000,
+    "quantization_config": {
+        "quant_method": "exl2",
+        "version": "0.2.1",
+        "bits": 2.6,
+        "head_bits": 6,
+        "calibration": {
+            "rows": 115,
+            "length": 2048,
+            "dataset": "(default)"
+        }
+    }
+}

config.yml

@@ -0,0 +1,212 @@
+# Sample YAML file for configuration.
+# Comment and uncomment values as needed.
+# Every value has a default within the application.
+# This file serves to be a drop in for config.yml
+
+# Unless specified in the comments, DO NOT put these options in quotes!
+# You can use https://www.yamllint.com/ if you want to check your YAML formatting.
+
+# Options for networking
+network:
+  # The IP to host on (default: 127.0.0.1).
+  # Use 0.0.0.0 to expose on all network adapters.
+  host: 0.0.0.0
+
+  # The port to host on (default: 5000).
+  port: 5000
+
+  # Disable HTTP token authentication with requests.
+  # WARNING: This will make your instance vulnerable!
+  # Turn on this option if you are ONLY connecting from localhost.
+  disable_auth: false
+
+  # Send tracebacks over the API (default: False).
+  # NOTE: Only enable this for debug purposes.
+  send_tracebacks: false
+
+  # Select API servers to enable (default: ["OAI"]).
+  # Possible values: OAI, Kobold.
+  api_servers: ["oai"]
+
+# Options for logging
+logging:
+  # Enable prompt logging (default: False).
+  log_prompt: false
+
+  # Enable generation parameter logging (default: False).
+  log_generation_params: false
+
+  # Enable request logging (default: False).
+  # NOTE: Only use this for debugging!
+  log_requests: false
+
+# Options for model overrides and loading
+# Please read the comments to understand how arguments are handled
+# between initial and API loads
+model:
+  # Directory to look for models (default: models).
+  # Windows users, do NOT put this path in quotes!
+  model_dir: models
+
+  # Allow direct loading of models from a completion or chat completion request (default: False).
+  inline_model_loading: false
+
+  # Sends dummy model names when the models endpoint is queried.
+  # Enable this if the client is looking for specific OAI models.
+  use_dummy_models: false
+
+  # An initial model to load.
+  # Make sure the model is located in the model directory!
+  # REQUIRED: This must be filled out to load a model on startup.
+  model_name: WizardLM-2-8x22B_exl2_2.6bpw
+
+  # Names of args to use as a fallback for API load requests (default: []).
+  # For example, if you always want cache_mode to be Q4 instead of on the inital model load, add "cache_mode" to this array.
+  # Example: ['max_seq_len', 'cache_mode'].
+  use_as_default: []
+
+  # Max sequence length (default: Empty).
+  # Fetched from the model's base sequence length in config.json by default.
+  max_seq_len: 32768
+
+  # Overrides base model context length (default: Empty).
+  # WARNING: Don't set this unless you know what you're doing!
+  # Again, do NOT use this for configuring context length, use max_seq_len above ^
+  override_base_seq_len:
+
+  # Load model with tensor parallelism.
+  # Falls back to autosplit if GPU split isn't provided.
+  # This ignores the gpu_split_auto value.
+  tensor_parallel: false
+
+  # Automatically allocate resources to GPUs (default: True).
+  # Not parsed for single GPU users.
+  gpu_split_auto: true
+
+  # Reserve VRAM used for autosplit loading (default: 96 MB on GPU 0).
+  # Represented as an array of MB per GPU.
+  autosplit_reserve: [0]
+
+  # An integer array of GBs of VRAM to split between GPUs (default: []).
+  # Used with tensor parallelism.
+  gpu_split: []
+
+  # Rope scale (default: 1.0).
+  # Same as compress_pos_emb.
+  # Use if the model was trained on long context with rope.
+  # Leave blank to pull the value from the model.
+  rope_scale: 1.0
+
+  # Rope alpha (default: None).
+  # Same as alpha_value. Set to "auto" to auto-calculate.
+  # Leaving this value blank will either pull from the model or auto-calculate.
+  rope_alpha:
+
+  # Enable different cache modes for VRAM savings (default: FP16).
+  # Possible values: 'FP16', 'Q8', 'Q6', 'Q4'.
+  cache_mode: Q4
+
+  # Size of the prompt cache to allocate (default: max_seq_len).
+  # Must be a multiple of 256 and can't be less than max_seq_len.
+  # For CFG, set this to 2 * max_seq_len.
+  cache_size:
+
+  # Chunk size for prompt ingestion (default: 2048).
+  # A lower value reduces VRAM usage but decreases ingestion speed.
+  # NOTE: Effects vary depending on the model.
+  # An ideal value is between 512 and 4096.
+  chunk_size: 1024
+
+  # Set the maximum number of prompts to process at one time (default: None/Automatic).
+  # Automatically calculated if left blank.
+  # NOTE: Only available for Nvidia ampere (30 series) and above GPUs.
+  max_batch_size:
+
+  # Set the prompt template for this model. (default: None)
+  # If empty, attempts to look for the model's chat template.
+  # If a model contains multiple templates in its tokenizer_config.json,
+  # set prompt_template to the name of the template you want to use.
+  # NOTE: Only works with chat completion message lists!
+  prompt_template:
+
+  # Number of experts to use per token.
+  # Fetched from the model's config.json if empty.
+  # NOTE: For MoE models only.
+  # WARNING: Don't set this unless you know what you're doing!
+  num_experts_per_token:
+
+  # Enables fasttensors to possibly increase model loading speeds (default: False).
+  fasttensors: true
+
+# Options for draft models (speculative decoding)
+# This will use more VRAM!
+draft_model:
+  # Directory to look for draft models (default: models)
+  draft_model_dir: models
+
+  # An initial draft model to load.
+  # Ensure the model is in the model directory.
+  draft_model_name:
+
+  # Rope scale for draft models (default: 1.0).
+  # Same as compress_pos_emb.
+  # Use if the draft model was trained on long context with rope.
+  draft_rope_scale: 1.0
+
+  # Rope alpha for draft models (default: None).
+  # Same as alpha_value. Set to "auto" to auto-calculate.
+  # Leaving this value blank will either pull from the model or auto-calculate.
+  draft_rope_alpha:
+
+  # Cache mode for draft models to save VRAM (default: FP16).
+  # Possible values: 'FP16', 'Q8', 'Q6', 'Q4'.
+  draft_cache_mode: FP16
+
+# Options for Loras
+lora:
+  # Directory to look for LoRAs (default: loras).
+  lora_dir: loras
+
+  # List of LoRAs to load and associated scaling factors (default scale: 1.0).
+  # For the YAML file, add each entry as a YAML list:
+  # - name: lora1
+  #   scaling: 1.0
+  loras:
+
+# Options for embedding models and loading.
+# NOTE: Embeddings requires the "extras" feature to be installed
+# Install it via "pip install .[extras]"
+embeddings:
+  # Directory to look for embedding models (default: models).
+  embedding_model_dir: models
+
+  # Device to load embedding models on (default: cpu).
+  # Possible values: cpu, auto, cuda.
+  # NOTE: It's recommended to load embedding models on the CPU.
+  # If using an AMD GPU, set this value to 'cuda'.
+  embeddings_device: cpu
+
+  # An initial embedding model to load on the infinity backend.
+  embedding_model_name:
+sampling:
+
+# Options for development and experimentation
+developer:
+  # Skip Exllamav2 version check (default: False).
+  # WARNING: It's highly recommended to update your dependencies rather than enabling this flag.
+  unsafe_launch: false
+
+  # Disable API request streaming (default: False).
+  disable_request_streaming: false
+
+  # Enable the torch CUDA malloc backend (default: False).
+  cuda_malloc_backend: true
+
+  # Run asyncio using Uvloop or Winloop which can improve performance.
+  # NOTE: It's recommended to enable this, but if something breaks turn this off.
+  uvloop: true
+
+  # Set process to use a higher priority.
+  # For realtime process priority, run as administrator or sudo.
+  # Otherwise, the priority will be set to high.
+  realtime_process_priority: true

generation_config.json

@@ -0,0 +1,6 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "transformers_version": "4.36.2"
+}

huggingface-metadata.txt

@@ -0,0 +1,64 @@
+url: https://huggingface.co/microsoft/WizardLM-2-8x22B
+branch: main
+download date: 2024-04-15 16:48:15
+sha256sum:
+    9e5cc2f91f7830364eff657b53df4f855091996cb2db7de2b69b8f11fa06be0a model-00001-of-00059.safetensors
+    24dbc249686530b5bcf630c6edae25fff212900d340946fb868ebd3dd8c62851 model-00002-of-00059.safetensors
+    d8ebc25463564c00d8e5d3f8a08ca5ce60c0efd53dd23aecfa7a2e3b0112b941 model-00003-of-00059.safetensors
+    81abed9dadae5877c49a57990fe5d8a449ed507987517728b1da7c195eacdd54 model-00004-of-00059.safetensors
+    45a5914445b0d8a95e02881d342a37b7d7c69943980b373b78331cf5c406191f model-00005-of-00059.safetensors
+    ea96b16084f0b6fddf232343e2ea55734ec324aeaa2eb60759fca724b80e292d model-00006-of-00059.safetensors
+    a78d60bb853a77110a815f6a20c5819cef21bcc0b4d50e700b560be54d6f53b5 model-00007-of-00059.safetensors
+    ba757c9109937a28e912c10bdb18042d347c55e768ec2dc89859c0e25823fa35 model-00008-of-00059.safetensors
+    ab90762b6b58313c37b2d1e09a8f9ae956bc4b5d0cc52c379b0937ce752dca98 model-00009-of-00059.safetensors
+    0b167436f7049226dbe7153e1ec602bf06d4cc384e5c16289e7207408c66068a model-00010-of-00059.safetensors
+    c4cc3a3f71b37f107b20f96d04067abbaee78f16e74196cdc68469d551bbe38c model-00011-of-00059.safetensors
+    b045740b8c2377430a5f42e06d9eec2606b8db45c3e5ac87a316fbc715569cc0 model-00012-of-00059.safetensors
+    175977b687acd05686a22c0f2b89f0ab0afb79c03db3480fa7fd989aec786287 model-00013-of-00059.safetensors
+    2703b8b32154f524baf105cb15f54ffbf134d13ccd923a30315e698fb85e54ff model-00014-of-00059.safetensors
+    bbe6004093d3a5e29f26c5e3f75e4e0b8a40c16498102dac31b0180746880671 model-00015-of-00059.safetensors
+    54c8f475521181cb4b93031b17c2a5c08ad0de1a81b035c2864c2bce19234c21 model-00016-of-00059.safetensors
+    0b4e356a83fc7e42a056d790213fee7624abc0d12b8ca113dbe734a139a99491 model-00017-of-00059.safetensors
+    188a8a3611852f704dae7804b47cfabb5a2ccf6c7f19226ae4e3fce6a3aa0506 model-00018-of-00059.safetensors
+    36d1e452bb1e962a83fdd712f4b018195a0a6102f378e9297291acf2d82cb3c0 model-00019-of-00059.safetensors
+    f99ae9d9cf903af58255d0774edec5ec1382c0d56a609108a59ce111518d40f0 model-00020-of-00059.safetensors
+    1223e5b007d833bcb459b23f3d7f59f646b16acf2a1a685085389b8de2dc6128 model-00021-of-00059.safetensors
+    48a99d78aaea1949ad85dce7847fa7d033173a22a58add4b2cd732410e7e15cf model-00022-of-00059.safetensors
+    db3c4baffbc4fc5a4eb34926e342d91fe4bcae59e0f712d06db4c2c4deb985da model-00023-of-00059.safetensors
+    141a6b26fb0a959967c24629bbfcf0c970a475d7698d2125e622cb38fb385b9b model-00024-of-00059.safetensors
+    433db989f798b99a87a64829f0a9d067e89daf1352cda3014e15c5414cbd33d2 model-00025-of-00059.safetensors
+    6cc8f60655049f3c1cc04c74cc3f02f72a3f820dcac263b483fa8bcba026ac6c model-00026-of-00059.safetensors
+    20b6cc117252f16fcb3d41f2372aca13a8f7426c13c86ad623c43cf0b96b5e72 model-00027-of-00059.safetensors
+    8bbbb1329d0fbe43933a49d37dada0518609bc18cc762549c67f677051ac46e8 model-00028-of-00059.safetensors
+    0734dbd916a3e5789d4338fdb053da6882fb7a6d636fd1495710557c032b5a24 model-00029-of-00059.safetensors
+    01f5ace1d4086c96950f57604626a30821ff3ca6f60dddea74678d914d93dd72 model-00030-of-00059.safetensors
+    8c362f13e9e0889f09bb552e57e5e7ac45e2f28510a9ad9fab5b50147b9c6dab model-00031-of-00059.safetensors
+    e93b983001b7b890da1a96405c630efb3eee7617482af1491822f213a6640b38 model-00032-of-00059.safetensors
+    61bcaf0a63535ba8f756509e7cca4b6ce1c4b741aca01f2afba40f5c3a727cfd model-00033-of-00059.safetensors
+    1c97dd5624ad3332e21c86836fc063059caab9262125a986e6f331c4ecf5aeff model-00034-of-00059.safetensors
+    8e9e4be3a6296da50cc1b4af53115818aa75d266c430a6baa2f91709885d18df model-00035-of-00059.safetensors
+    a699b76e6a8633bd6ba0527270bb81a4c91604cd3a7118de09c9361fda3b144e model-00036-of-00059.safetensors
+    ff7bbbeb2f68415f368f2e99e91b905e145c87beadcf352d0311f85ec6ddaf6c model-00037-of-00059.safetensors
+    2fe8533aac43a8c1c7b0885efef6a3080fdc6a61c880746c47520c25a10bf17d model-00038-of-00059.safetensors
+    4846e39f398d04411859f18b2697f4e70e70ad33deeefa93135911d10c5a23e8 model-00039-of-00059.safetensors
+    4b7de8e7af1d27c665f9e8dee5c904f9ee88ccd48c831361fc5999b979c16897 model-00040-of-00059.safetensors
+    02c8320c607f11bbd377930a056873d84f0192d950176fc22e3c59d1dfb5995d model-00041-of-00059.safetensors
+    68a8a3b2780336a8012b795aa6295fd9248c322f939e8131315a520fe4e43480 model-00042-of-00059.safetensors
+    d859d466ef5da5d0cf036bc6ed57fa0033d15c3921eb042d1a31617c6f0eef34 model-00043-of-00059.safetensors
+    06d32c1ed652616e42fffdcce1735fad67b50a4f13397cfd030a4304ce035a08 model-00044-of-00059.safetensors
+    beb9181e4b827f0136d213c11a05c569e0bcb85b7e68ade06b38c5d3da2d1ae0 model-00045-of-00059.safetensors
+    44a573a390174b0f0a41f8c0b198761384c46ae328a089f97d3e622468507224 model-00046-of-00059.safetensors
+    32a3bc20ff1805ae3e14e0bcf7ac3a1f9c21a31fb04922d3d31e440e658c32b6 model-00047-of-00059.safetensors
+    69a198153c9725aa0b6de50831b55fb0f04fc7a0f402731257836110358b7bde model-00048-of-00059.safetensors
+    a9d4adc81dbb3f12ec1300cb9c53bae070b7dbe41a66e30b5160841504fb5ba3 model-00049-of-00059.safetensors
+    9d7cf6acb12c6907a102f9b836e3db345d2aa885493b3904084b95d0ac55c739 model-00050-of-00059.safetensors
+    582322e446962d374f4b6fd5e469040d331150f7ef3d4a69fa460deded7c3ebc model-00051-of-00059.safetensors
+    d6fe7d0d82aa12dd5f40c083b4046ac6ef15ed0627433d63795bed9e50c55fb4 model-00052-of-00059.safetensors
+    af0c588e8ffe2e87ce5dee116d70ec083a3bfd2f4c4a09618d8a2791de5cd55f model-00053-of-00059.safetensors
+    ecda36174ea4e376919360f6e08c17d5b3d0a870469e9fd52cac0b7c8e12351e model-00054-of-00059.safetensors
+    5d7c98c60576932850e8ef8945e38f24ca0f37cca895bf04c4a23d251081859c model-00055-of-00059.safetensors
+    7410c9689fd0d2d384f4b2886707fb377327457937c91da914c725b7c35c2396 model-00056-of-00059.safetensors
+    5c684ae59038495c44d01507e8047511db57fa720cc9997f7719fba76dbc29ee model-00057-of-00059.safetensors
+    b1234554df96e1f1e2a7adcef5f680e7b57e9b6ae4f4a16ceff8225b20921c5d model-00058-of-00059.safetensors
+    e36fa9e3f053db7a27e06f5b2804a2aa650a026a66e3f7db9efd35c5103cd9ae model-00059-of-00059.safetensors
+    dadfd56d766715c61d2ef780a525ab43b8e6da4de6865bda3d95fdef5e134055 tokenizer.model

output-00001-of-00006.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ca7fe6b12c0ccbef691e7ace8489af1f155f6d28fc9b340fdeac15895e348eb7
+size 8575532528

output-00002-of-00006.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:be96ba2fc6abe281dbbc156cb7bc42e34351fe8b29197de27652c301618c9a6e
+size 8578903808

output-00003-of-00006.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7b5196a15d7a0fc2c91e062916913a6fc5cddce2ff635adb730aefd34b1d614a
+size 8583086392

output-00004-of-00006.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b9d3f7755e8fc81c169f11f97daf4ac5acdf55e621282df7b1b543edae93db4f
+size 8571095736

output-00005-of-00006.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:362f7f846f579d888583435fc0aca41cadc0ac1fd924e0cc124a8ca04ba15627
+size 8589761752

output-00006-of-00006.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e99fd9a0b5362add62cf0c89fe08453e7136eead6fb67f095c3e6e9ee397b326
+size 3231878312

special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "<unk>",
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dadfd56d766715c61d2ef780a525ab43b8e6da4de6865bda3d95fdef5e134055
+size 493443

tokenizer_config.json

@@ -0,0 +1,44 @@
+{
+  "add_bos_token": true,
+  "add_eos_token": false,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [],
+  "bos_token": "<s>",
+  "chat_template": "{% if messages[0]['role'] == 'system' %}{% set loop_messages = messages[1:] %}{{ messages[0]['content'].strip() }}{% else %}{% set loop_messages = messages %}{{ 'A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user\\'s questions.' }}{% endif %}{% for message in loop_messages %}{% if loop.index0 == 0 %}{% if message['role'] == 'system' or message['role'] == 'user' %}{{ ' USER: ' + message['content'].strip() }}{% else %}{{ ' ASSISTANT: ' + message['content'].strip() + eos_token }}{% endif %}{% else %}{% if message['role'] == 'system' or message['role'] == 'user' %}{{ '\nUSER: ' + message['content'].strip() }}{% else %}{{ ' ASSISTANT: ' + message['content'].strip() + eos_token }}{% endif %}{% endif %}{% endfor %}{% if add_generation_prompt %}{{ ' ASSISTANT:' }}{% endif %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "</s>",
+  "legacy": true,
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "<unk>",
+  "padding_side": "right",
+  "sp_model_kwargs": {},
+  "spaces_between_special_tokens": false,
+  "tokenizer_class": "LlamaTokenizer",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": true
+}

upload.py

@@ -0,0 +1,45 @@
+from huggingface_hub import HfApi
+from pathlib import Path
+
+# Define the parameters for uploading
+repo_id = "DBMe/WizardLM-2-8x22B-2.6bpw-h6-exl2"  # Replace with your actual repo ID
+folder_path = "/home/asusws-x570-ace/programs/tabbyAPI/models/WizardLM-2-8x22B_exl2_2.6bpw/"  # Replace with your folder path
+repo_type = "model"  # Change to "model" or "space" if applicable
+revision = "main"  # Optional: specify the branch or use "main"
+private = False  # Set to True if the repository should be private
+allow_patterns = None  # Optional: specify patterns of files to include
+ignore_patterns = None  # Optional: specify patterns of files to exclude
+num_workers = 1  # Set based on your system; lower if your internet is unstable
+print_report = True  # Enable progress reporting
+print_report_every = 60  # Report frequency in seconds
+
+# Initialize the Hugging Face API client
+api = HfApi()
+
+# Function to upload the folder in a resumable manner
+def upload_resumable():
+    try:
+        print("Starting upload process...")
+
+        # Perform the upload with the provided parameters
+        api.upload_large_folder(
+            repo_id=repo_id,
+            folder_path=Path(folder_path),
+            repo_type=repo_type,
+            revision=revision,
+            private=private,
+            allow_patterns=allow_patterns,
+            ignore_patterns=ignore_patterns,
+            num_workers=num_workers,
+            print_report=print_report,
+            print_report_every=print_report_every,
+        )
+
+        print("Upload completed successfully!")
+
+    except Exception as e:
+        print(f"Upload interrupted due to error: {e}")
+        print("You can resume the upload by running the script again.")
+
+# Call the function to start the upload
+upload_resumable()