license: mit
datasets:
- glaiveai/glaive-code-assistant-v2
- TokenBender/code_instructions_122k_alpaca_style
language:
- en
metrics:
- code_eval
pipeline_tag: text-generation
tags:
- code
- text-generation-inference
1. Introduction of CodeNinja
CodeNinja is a fine tuned version of the excellent model openchat/openchat-3.5-1210. It is a 7B model that was fine tuned using Supervised Fine Tuning in 2 instructions datasets containing in total more than 400 000 code instructions.
The model is quite good for coding tasks and it's goal is to be a coding assistant that you can use daily.
The quantized versions can be found here: beowolx/CodeNinja-1.0-OpenChat-7B-GGUF.
The model performs very good in a serie of different tasks.
Massive Training Data: Fine-tuned using the datasets glaiveai/glaive-code-assistant-v2 and TokenBender/code_instructions_122k_alpaca_style it contains around 400 000 code instructions in different programming languages such as Python, C, C++, Rust, Java, JavaScript and etc.
Highly Flexible & Scalable: Offered in model sizes of 7B, enabling users to run it locally.
Superior Model Performance: State-of-the-art performance among publicly available code models on HumanEval.
Advanced Code Completion Capabilities: A context window size of 8192 supporting project-level code completion.
2. Prompt Format
CodeNinja uses the same prompt format than OpenChat 3.5 so you will need to use it to get good results.
The prompt format looks like this:
GPT4 Correct User: Hello<|end_of_turn|>GPT4 Correct Assistant: Hi<|end_of_turn|>GPT4 Correct User: How are you today?<|end_of_turn|>GPT4 Correct Assistant:
🚨 Notice: Remember to set <|end_of_turn|> as end of generation token.
You must use this prompt format to get good results
3. How to Use
Using LM Studio
The easiest way to start using the model is to download one of the quantized versions using LM Studio.
You then need to make sure that you are using the "OpenChat" preset that contain the prompt format mentioned above already set.
If you want, you can get the preset from this gist.
Using the transformers library
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
# Initialize the model
model_path = "beowolx/CodeNinja-1.0-OpenChat-7B"
model = AutoModelForCausalLM.from_pretrained(model_path, device_map="auto")
# Import openchat tokenizer
tokenizer = AutoTokenizer.from_pretrained("openchat/openchat-3.5-1210", use_fast=True)
def generate_one_completion(prompt: str):
messages = [
{"role": "user", "content": prompt},
{"role": "assistant", "content": ""} # Placeholder for the model's response
]
# Apply the chat template to get the list of token IDs
input_ids = tokenizer.apply_chat_template(messages, add_generation_prompt=True)
# Generate completion
generate_ids = model.generate(
torch.tensor([input_ids]).to("cuda"), # Convert list to tensor and send to GPU
max_length=256,
pad_token_id=tokenizer.pad_token_id,
eos_token_id=tokenizer.eos_token_id
)
# Decode and clean up the completion
completion = tokenizer.decode(generate_ids[0], skip_special_tokens=True)
completion = completion.split("\n\n\n")[0].strip()
return completion
4. License
This code repository is licensed under the MIT License. The use of CodeNinja model is subject to the Model License.
5. Contact
If you have any questions, please raise an issue here.