metadata
language:
- en
pipeline_tag: text-generation
tags:
- unsloth
- axolotl
DreamGen Opus V1
Models for (steerable) story-writing and role-playing.
All Opus V1 models, including quants.
Resources
- Opus V1 prompting guide with many (interactive) examples and prompts that you can copy.
- Google Colab for interactive role-play using
opus-v1.2-7b. - Python code to format the prompt correctly.
- Join the community on Discord to get early access to new models.
Prompting
The models use an extended version of ChatML.
<|im_start|>system
(Story description in the right format here)
(Typically consists of plot description, style description and characters)<|im_end|>
<|im_start|>user
(Your instruction on how the story should continue)<|im_end|>
<|im_start|>text names= Alice
(Continuation of the story from the Alice character)<|im_end|>
<|im_start|>text
(Continuation of the story from no character in particular (pure narration))<|im_end|>
<|im_start|>user
(Your instruction on how the story should continue)<|im_end|>
<|im_start|>text names= Bob
(Continuation of the story from the Bob character)<|im_end|>
The Opus V1 extension is the addition of the text role, and the addition / modification of role names.
Pay attention to the following:
- The
textmessages can (but don't have to have)names, names are used to indicate the "active" character during role-play. - There can be multiple subsequent message with a
textrole, especially if names are involved. - There can be multiple names attached to a message.
- The format for names is
names= {{name[0]}}; {{name[1]}}, beware of the spaces afternames=and after the;. This spacing leads to most natural tokenization for the names.
While the main goal for the models is great story-writing and role-playing performance, the models are also capable of several writing related tasks as well as general assistance.
Here's how you can prompt the model for the following tasks
- Steerable Story-writing and Role-playing:
- Input:
- System prompt: You provide story / role-play description, which consists of:
- Plot description
- Style description
- Characters and their descriptions
- Conversation turns:
- Text / message turn: This represents part of the story or role play
- Instruction: This tells the model what should happen next
- System prompt: You provide story / role-play description, which consists of:
- Output: Continuation of the story / role-play.
- Input:
- Story plot summarization
- Input: A story, or a few chapters of a story.
- Output: A description of the story or chapters.
- Examples:
Examples
Summary of [CNN News article](https://edition.cnn.com/2024/02/29/health/pets-diseases-viruses-bacteria-wellness/index.html) in 200 words
**Output:**
The article discusses the recent human deaths from diseases transmitted by animals, specifically Alaskapox and bubonic plague. Dr. Leana Wen, an emergency physician and former Baltimore health commissioner, explains the various ways household pets can transmit diseases to humans. These include fecal-oral transmission, scratches and bites, and vector-borne diseases like ticks, fleas, and mosquitoes. Wen emphasizes the importance of vaccinating pets, maintaining cleanliness, and taking commonsense precautions to reduce disease transmission. She also advises that individuals who are immunocompromised, babies, and pregnant women should take additional care when interacting with animals.
<details>
<summary>Input:</summary>
<|im_start|>system
You are an intelligent, knowledgeable, helpful, general-purpose assistant.
You are given a story and will be asked to perform various tasks with it. Respond without any additional fluff.
The story:
"""
Health officials in Alaska recently reported the first known human death from a virus called Alaskapox.
The man, who died in January, lived in a wooded area and cared for a stray cat that hunted small animals, according to health officials. He had a weakened immune system and is thought to have contracted Alaskapox through contact with animals.
At around the same time, an individual living in Oregon was diagnosed with bubonic plague. Health officials suspect this person was infected by a cat. The individual is reported to be in early stages of the illness, and the patient and close contacts are being treated with antibiotics.
These infections are reminders that diseases from nonhuman animals can sometimes be transmitted to humans. To get a better sense of what such diseases are and what precautions pet owners and animal lovers should take to keep themselves and their animals healthy, I spoke with CNN wellness medical expert Dr. Leana Wen.
Wen is an emergency physician and adjunct associate professor at George Washington University. She previously was Baltimore’s health commissioner, where as part of her responsibilities, she also oversaw animal control and protection.
CNN: What types of diseases can household pets like cats and dogs transmit to humans?
Dr. Leana Wen: Just as humans can transmit diseases to other humans, animals can transmit diseases to others in their own species. Sometimes, animals can transmit diseases to other species, and that includes humans with whom pets have had close contact.
One way to classify the diseases that can be spread by household pets to humans is by method of transmission, such as the fecal-oral route. Someone who is in contact with the feces of infected animals could get the disease themselves if they don’t wash their hands well or if the fecal contents end up contaminating water or food that is then ingested by the human.
These diseases include bacterial infections like campylobacter and salmonella and parasitic infections like cryptosporidium, giardia and tapeworm.
Symptoms can include abdominal cramping, diarrhea, fever and vomiting. Depending on the illness, people may need targeted treatment, such as antibiotics or antiparasitic medications. The infections are generally mild, although they can be serious for people with underlying medical conditions and those who are very young or elderly.
There is a specific parasitic infection called toxoplasmosis that is associated with cleaning cat litter boxes or eating food or drinking water that has been contaminated with cat feces. Most people with this infection have little or no symptoms, but there is a unique risk to pregnant women because toxoplasmosis can be transmitted through the placenta. Their infants could become infected before birth and then develop serious neurological problems such as seizures, vision loss and mental disability.
A second route of infection is through scratches and bites. Cat scratch disease, for instance, is a bacterial infection spread when a cat scratches or bites a person enough to break their skin. It also can be spread if an infected cat licks a person’s open wound. People who are infected can develop swollen and painful lymph nodes, fever and headache.
Many people have heard of rabies, which is a deadly viral infection spread to humans through the bite of infected animals. Rabies is nearly universally deadly. Most domestic animals in the United States are vaccinated against rabies, but this is not the case in some other countries. Thankfully, there are rabies shots that can be taken after suspected exposure.
A third route of infection is through another vector such as ticks, fleas and mosquitoes. Ticks, for instance, can transmit infections that cause Lyme disease, babesiosis, ehrlichiosis and tularemia. These diseases spread when a person is bitten by an infected tick that has fed off an infected animal. Dogs that spend time in wooded habitats could have ticks that infect them, and the ticks could also spread the disease to humans.
CNN: What about Alaskapox and the bubonic plague? How could someone contract these diseases from animals?
Wen: Alaskapox is in the category of viruses called orthopoxvirus. There are several viruses in this category that can be transmitted from animals to humans, including mpox (formerly known as monkeypox) and cowpox, smallpox and vaccinia viruses. It’s not known how the man who died from Alaskapox contracted it, but a possible route is an infected animal biting the individual or exposure of an infected animal to an existing break in the man’s skin.
Bubonic plague is caused by the Yersinia pestis bacterium and can be transmitted through a break in the skin, too, if someone has an open wound and who comes into contact with an animal’s infectious bodily fluids. It also could be transmitted through the bite of an infected flea.
CNN: What are steps people can take to reduce the risk of contracting diseases from their pets?
Wen: There are several important steps pet owners and animal lovers can take to protect themselves, their family members and their pets.
To begin with, they should ensure that their dogs and cats are fully vaccinated. Vaccines prevent not only many illnesses that can spread from animals to humans, but also diseases limited to the animal but that could be harmful and even fatal. Pets should also be cared for with regularly scheduled checkups.
It’s also important to take your pet to the veterinarian if it becomes ill. Try to reduce contact with sick pets in the meantime. Identifying the cause of the illness can prompt quicker treatment for the animal and possible testing and treatment for the human.
There are also commonsense precautions people should take to reduce disease transmission. This includes washing your hands well after handling feces and other waste. Make sure the pet’s living area is clean. Keep away from wild animals and animals that are behaving strangely.
Children who are helping out with cleaning litter boxes or picking up after their dogs should be reminded to wash their hands well. Kids should also be reminded to avoid kissing their pet’s mouth and reduce activities that could result in accidental bites. They should be sure to wash their hands after going to friends’ houses with animals and visiting farms with petting zoos. And it’s important to clean waste outdoors, including in your own yard, and not to let kids play in areas with animal feces.
To lower the risk of fleas, it’s important to maintain good hygiene and reduce rodents in and around the animal’s living space. To decrease tick-borne illness, look carefully for ticks after going to wooded areas and remove ticks right away after spotting them.
CNN: Does it mean no one should have animals?
Wen: Certainly not. People seek animal companionship for many reasons. For many people, having a dog or cat is essential to their own well-being and mental health. There are simple steps everyone can take to reduce the risk of animal to human disease transmission. Individuals who should take additional care include those who are immunocompromised, babies and pregnant women.
"""<|im_end|>
<|im_start|>user
Summarize the overall story in roughly 150 words.<|im_end|>
</details>
</details>
</details>
Sampling params
For story-writing and role-play, I recommend "Min P" based sampling with min_p in the range [0.01, 0.1] and with temperature in the range [0.5, 1.5], depending on your preferences. A good starting point would be min_p=0.1; temperature=0.8.
You may also benefit from setting presence, frequency and repetition penalties, especially at lower temperatures.
Dataset
The fine-tuning dataset consisted of ~100M tokens of steerable story-writing, role-playing, writing-assistant and general-assistant examples. Each example was up to 31000 tokens long.
All story-writing and role-playing examples were based on human-written text.
Running the model
The model is should be compatible with any software that supports the base model, but beware of prompting and tokenization.
I recommend using these model versions:
Running on DreamGen.com (free)
You can run the models on dreamgen.com for free — you can use the built-in UI for story-writing & role-playing, or use the API.
Running Locally
- Make sure your prompt is as close as possible to the Opus V1
- Regardless of which backend you use, it's important that you format your prompt well and that the tokenization works correctly.
- Read the prompt guide
- Read the prompt formatting code
- Make sure
<|im_start|>and<|im_end|>are tokenized correctly
- vLLM
- Google Colab: This is a simple interactive Google Colab to do role-play with the 7B model, it should fit on the T4 GPU.
- Code: This is simple script for interactive chat for one hard-coded scenario.
- SillyTavern
- Settings, v2 kindly provided by @MarinaraSpaghetti
- Settings screenshot
- This is just an attempt at approximating the Opus V1 prompt, it won't be perfect
- LM Studio
- Config
- Just like ChatML, just changed "assistant" to "text" role.
- There's a bug in LM Studio if you delete a message or click "Continue", see here for details.
- HuggingFace
- Chat template
- Just like ChatML, just changed "assistant" to "text" role.
Known Issues
- 34B repetition:
- The 34B sometimes gets stuck repeating the same word, or synonyms. This seems to be a common problem across various Yi 34B fine-tunes.
- GGUF:
- The tokenization might be messed up. Some users reported that
<|im_start|>and<|im_end|>are tokenized as multiple tokens. Also llama.cpp may not tokenize correctly (the Yi tokenizer is subtly different from the Llama 2 tokenizer).
- The tokenization might be messed up. Some users reported that
License
- This model is intended for personal use only, other use is not permitted.