dreamgen/opus-v1.4-70b-llama3-exl2-3.5bpw-h6

Llama 3 DreamGen Opus V1

Models for (steerable) story-writing and role-playing.
Other quants
Other models

Resources

• Opus V1 prompting guide with many prompts and interactive examples you can copy.
• Story-writing examples (model output in red)
• Role-playing examples (model output in red, including the user turns)
• Join the community on Discord to discuss the models.

Prompting

This model uses an extended version of the Llama 3 template (click for details).

Compared to the official Llama 3 template, a new writer role was added.

<|start_header_id|>system<|end_header_id|>

Lorem ipsum<|eot_id|>
<|start_header_id|>user<|end_header_id|>

Dolor sit amet<|eot_id|>
<|start_header_id|>writer character: Alice<|end_header_id|>

Ut enim ad minim veniam<|eot_id|>
<|start_header_id|>writer character: Bob<|end_header_id|>

Duis aute irure dolor in reprehenderit<|eot_id|>
<|start_header_id|>writer character: Alice + Bob<|end_header_id|>

Excepteur sint occaecat cupidatat non proident<|eot_id|>

Concretely, this what the full prompt could look like:

<|start_header_id|>system<|end_header_id|>

You are an intelligent, skilled, versatile writer.

Your task is to write a story based on the information below.

## Overall plot description:

In a world where magic is fading, a young mage named Elara embarks on a perilous journey to the Enchanted Forest. There, she hopes to uncover ancient secrets and restore magic to the land, battling dangerous creatures and facing personal trials along the way.

As Elara and Caelan traverse treacherous terrains, face mythical creatures, and unravel the mysteries of the Enchanted Forest, their relationship grows stronger, and they discover the true extent of their own powers. Along the way, they encounter allies and enemies, unearthing the dark forces responsible for the decline of magic. Their ultimate goal is to unlock the ancient magic that lies within the heart of the forest, revitalizing the world and fulfilling their destiny as true protectors of magic.

## Characters:

### Elara

Elara is the protagonist and young mage. Elara is 23 years old. She is a determined and compassionate young

### Caelan

Caelan is Elara's loyal companion and protector. He is a seasoned warrior with a heart of gold. He is known for his exceptional combat skills and unyielding bravery. Caelan has a tragic past intertwined with the loss of his family to a malevolent darkness, fueling his desire to protect others from a similar fate. Despite his initial skepticism towards magic, he develops a deep bond with Elara and becomes her trusted confidante and guardian on their quest. Caelan's unwavering loyalty and formidable swordsmanship prove invaluable throughout their journey.<|eot_id|>
<|start_header_id|>user<|end_header_id|>

In a serene clearing, Elara, a young mage, performs a powerful ritual, awakening the dormant magic in the Enchanted Forest. Vibrant bursts of energy transform the vegetation, filling the air with wonder. Unbeknownst to Elara, a mysterious figure watches from the shadows, setting the stage for an extraordinary adventure that will unravel secrets and determine the fate of the realms.<|eot_id|>
<|start_header_id|>writer<|end_header_id|>

The Enchanted Forest was once a place of magic and wonder, filled with mythical creatures and the most powerful spells known to man. But now, it was a shadow of its former self. The trees were barren, the air was still, and the creatures that once thrived here had all but vanished.

But there was hope. A young mage named Elara had discovered a hidden chamber deep within the forest, filled with ancient artifacts and forgotten knowledge. She had spent months studying these relics, learning their secrets and unlocking their power. And now, on this fateful day, she was ready to perform a powerful ritual that would awaken the dormant magic of the Enchanted Forest.<|eot_id|>
<|start_header_id|>writer character: Elara<|end_header_id|>

In the waning light of dusk, Elara stood in the quiet heart of Aylesford, her focus intent upon the ancient runes etched into the earth before her. The air, usually thrumming with the unspoken words of magic, lay heavy and still around her. Each gesture and incantation, practiced countless times, faltered, as if the very essence of magic was slipping through her fingers like grains of sand.<|eot_id|>
<|start_header_id|>writer character: Caelan<|end_header_id|>

Caelan watched from a distance, his brow furrowed in concern. As the shadows grew longer, he approached, his steps soft upon the grass. "Elara," he began, his voice low, "There are whispers on the wind, tales from far-off lands. They speak of magic waning, of spells unspun and charms undone."<|eot_id|>
<|start_header_id|>writer character: Elara<|end_header_id|>

Elara turned, her eyes wide with fear. “What do you mean?” Her voice trembled with uncertainty, a stark contrast to the confidence that usually filled her being when she cast spells.<|eot_id|>

• The writer messages can (but don't have to have) character:, these are used to indicate the "active" character during role-play.
• There can be multiple subsequent message with a writer role, especially if names are involved.

While the main goal for the models is great story-writing and role-playing performance, the models are also capable of several writing related tasks as well as general assistance.

Read the official guide to learn how to prompt the model for various tasks beyond story-writing and role-playing, such as summarization, plot idea generation and more.

Sampling params

For story-writing and role-play, I recommend "Min P" based sampling with min_p in the range [0.01, 0.1] and with temperature in the range [0.5, 1.5], depending on your preferences. A good starting point would be:

• min_p=0.05
• temperature=1.0
• presence_penalty=0.1
• frequency_penalty=0.1

Dataset

The fine-tuning dataset consisted of >100M tokens of steerable story-writing, role-playing, writing-assistant and general-assistant examples. Each example was up to 8192 tokens long.

All story-writing and role-playing examples were based on human-written text.

Running the model on DreamGen.com (free)

You can run the models on dreamgen.com for free — you can use the built-in UI for story-writing & role-playing, or use the OpenAI compatible API.

Running the model locally

The model is should be compatible with any software that supports the base model, but beware of the chat template differences.

• Make sure your prompt is as close as possible to the Opus V1
  • Regardless of which method you use, it's important that you format your prompt well and that the tokenization works correctly.
  • Read the prompt guide
• SillyTavern
  • Use instruct mode with the built in DreamGen Llama 3 presets (context json, instruct json)
  • As a backend I suggest using Aphrodite with the largest exl2 quant you can fit, but llama.cpp with one of the gguf quants should also work.
  • Official SillyTavern documentation for DreamGen -- applies to both the API an local models.
  • Character cards specifically optimized for the DreamGen preset:
    • Seraphina (based on the default Seraphina card)
    • Lara Lightland (based on the card by Deffcolony)
• LM Studio
  • Story-Writing Config
    • Plot: Battle between Gandalf and Dumbledore
    • Here the user is steering the writing of the AI.
  • Role-Playing Config
    • Plot: Battle between Gandalf and Dumbledore
    • Here the user is playing the role of Gandalf and the AI the role of Dumbledore.

License

• This model is intended for personal use only, other use is not permitted.

Writing benchmarks

Here are results from internal gpt-4o judged writing benchmark. In this limited setting, the exl2 4.25 h6 seems on-par with the fp16 version. Of course the benchmark is limited and might not capture all the issues that arise from quantization, these are often more subtle or only apparent on repeated use.

Eval stats (total 566)
Model: human
  Rank 0 (42.23%): 239
  Rank 1 (33.04%): 187
  Rank 2 (24.73%): 140
  Quality great (37.81%): 214
  Quality good (46.47%): 263
  Quality ok (15.72%): 89
Model: dreamgen/opus-v1.4-70b-fp16
  Rank 0 (28.45%): 161
  Rank 1 (34.81%): 197
  Rank 2 (36.75%): 208
  Quality great (24.91%): 141
  Quality good (49.29%): 279
  Quality ok (25.27%): 143
  Quality bad (0.53%): 3
Model: dreamgen/opus-v1.4-70b-exl2-4.25bpw-h6
  Rank 0 (29.33%): 166
  Rank 1 (32.16%): 182
  Rank 2 (38.52%): 218
  Quality great (25.80%): 146
  Quality good (46.82%): 265
  Quality ok (26.50%): 150
  Quality bad (0.88%): 5
Model pair: dreamgen/opus-v1.4-70b-fp16 vs human: 566
  dreamgen/opus-v1.4-70b-fp16: 233 (41.17%)
  human: 333 (58.83%)
  (significant)
    dreamgen/opus-v1.4-70b-fp16: 119 (38.02%)
    human: 194 (61.98%)
Model pair: dreamgen/opus-v1.4-70b-exl2-4.25bpw-h6 vs human: 566
  dreamgen/opus-v1.4-70b-exl2-4.25bpw-h6: 234 (41.34%)
  human: 332 (58.66%)
  (significant)
    dreamgen/opus-v1.4-70b-exl2-4.25bpw-h6: 124 (38.75%)
    human: 196 (61.25%)
Model pair: dreamgen/opus-v1.4-70b-exl2-4.25bpw-h6 vs dreamgen/opus-v1.4-70b-fp16: 566
  dreamgen/opus-v1.4-70b-exl2-4.25bpw-h6: 280 (49.47%)
  dreamgen/opus-v1.4-70b-fp16: 286 (50.53%)
  (significant)
    dreamgen/opus-v1.4-70b-exl2-4.25bpw-h6: 138 (50.74%)
    dreamgen/opus-v1.4-70b-fp16: 134 (49.26%)