TheBloke

/

vicuna-13b-v1.3.0-GPTQ

like 21

Sorry I don't understand, what do you mean by previous version? I've not changed the GPTQ model in the main branch since it was first uploaded 27 days ago.

hii i am looking for this
model_name_or_path = "TheBloke/vicuna-13b-v1.3-GPTQ"
model_basename = "vicuna-13b-v1.3-GPTQ-4bit-128g.no-act.order"
but its not available am i right?

It's available and it works fine. Here's what I get when I run the code provided in the README:

Downloading (…)okenizer_config.json: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 727/727 [00:00<00:00, 1.81MB/s]Downloading tokenizer.model: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 500k/500k [00:00<00:00, 2.00MB/s]Downloading (…)/main/tokenizer.json: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1.84M/1.84M [00:00<00:00, 9.74MB/s]Downloading (…)cial_tokens_map.json: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 435/435 [00:00<00:00, 1.26MB/s]Downloading (…)lve/main/config.json: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 627/627 [00:00<00:00, 1.88MB/s]Downloading (…)quantize_config.json: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 135/135 [00:00<00:00, 393kB/s]Downloading (…)ct.order.safetensors: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 7.45G/7.45G [01:11<00:00, 104MB/s]
The safetensors archive passed at /home/ubuntu/.cache/huggingface/hub/models--TheBloke--vicuna-13b-v1.3.0-GPTQ/snapshots/0348d3dcc0b518c371aaaee8aea360078b3ea1b7/vicuna-13b-v1.3.0-GPTQ-4bit-128g.no-act.order.safetensors does not contain metadata. Make sure to save your model with the `save_pretrained` method. Defaulting to 'pt' metadata.
skip module injection for FusedLlamaMLPForQuantizedModel not support integrate without triton yet.


*** Generate:
<s> A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions.

USER: Tell me about AI
ASSISTANT:

Artificial Intelligence (AI) is a field of computer science that aims to create intelligent machines that can perform tasks that typically require human intelligence, such as visual perception, speech recognition, decision-making, and language translation. AI can be divided into two main categories: Narrow AI, which is designed to perform a specific task, and General AI, which can perform a wide range of tasks.

There are several types of AI, including:

1. Reactive Machines: These machines can only react to a given input and do not have the ability to form memories or plan for the future.
2. Limited Memory: These machines can store and use past experiences to make better decisions.
3. Theory of Mind: These machines can understand the beliefs, intentions, and emotions of others.
4. Self-Aware: These machines have a sense of self-awareness and the ability to reflect on their own thoughts and actions.

AI is used in a wide range of applications, including:

1. Natural Language Processing: AI can be used to analyze and understand human language, including speech and text.
2. Computer Vision: AI can be used to analyze and understand visual data, such as images and videos.
3. Robotics: AI can be used to control robots and automate tasks.
4. Predictive Analytics: AI can be used to analyze data and make predictions about future events.
5. Autonomous Systems: AI can be used to create self-driving cars, drones, and other autonomous systems.

As AI continues to advance, it has the potential to revolutionize many industries and improve our lives in countless ways. However, it also raises important ethical and societal questions, such as the impact on employment and the potential for misuse.</s>
*** Pipeline:
A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions.

USER: Tell me about AI
ASSISTANT:

Artificial Intelligence (AI) is a field of computer science that focuses on creating intelligent machines capable of performing tasks that typically require human intelligence, such as visual perception, speech recognition,
decision-making, and language translation. There are several subfields within AI, including machine learning, natural language processing, robotics, and computer vision.

Machine Learning (ML) is a subset of AI that involves training algorithms to make predictions or decisions based on data, without being explicitly programmed to do so. ML algorithms can learn from patterns in data and improve their performance over time as they are exposed to more examples. Some common applications of ML include image classification, speech recognition, and recommendation systems.

Natural Language Processing (NLP) is another important area of AI that deals with enabling computers to understand and generate human language. NLP techniques are used in applications such as sentiment analysis, text summarization, and question answering.

Robotics is a field that combines AI with mechanical engineering to create robots that can perform physical tasks autonomously. Robotics has many practical applications, such as manufacturing, healthcare, and space exploration.

Computer Vision is an area of AI that focuses on developing algorithms that enable computers to interpret and analyze visual information from the world around them. Computer vision techniques are used in applications such as facial recognition, object detection, and self-driving cars.

FYI there was a typo in the README, a comma was missing after model_basename. I have fixed this.

TheBloke/vicuna-13b-v1.3-GPTQ this model path name give me 404 on huggingface

Oh, it's just because you are using the wrong name. Look at the name of the repo. It's TheBloke/vicuna-13b-v1.3.0-GPTQ

The code in the README is correct, just use that!

i know that .. but every time i used TheBloke/vicuna-13b-v1.3-GPTQ and its load perfectly . so i asked are the model changed or updated . because i saw difference in results in our use case
check screenshot

The base vicuna 13B 1.3 model was updated by LmSys nearly a month ago, and I re-did the quantisations:

The original version of this model was in a repo called vicuna-13b-v1.3-GPTQ, and I did the new quants to vicuna-13b-v1.3.0-GPTQ. I then deleted vicuna-13b-v1.3-GPTQ as I was told by LmSys that 1.3.0 fixed problems in 1.3 and that there was no reason to keep 1.3 any more.

So no, sorry I can't put the other one back. It's gone.

hey bloke i found the safetensors of v1.3 on my cache . but its says the model not found . when I load again .