This is a pretty amazing model for 12B.
Greetings,
I've had the pleasure of using this model for a bit, now, and I'm very impressed with the ability of it to follow the prompt, and understand real-world interactions in ways that I've seen much larger models fail. Simple things like if you put on clothes, they remain on later, or if you're on the phone with someone you can't punch them or see them unless you specifically set up a video call. (Which it did, in one story, because the storyline needed it!)
I'm running the 8 bit GGUF quantized version via llamacpp_hf (for the DRY capabilities) on a 4090, so it's both fast and smart. Here's a recent prompt that I thought came out really well:
Prompt: Write a well written enemies to lovers science fiction romance story in a hopepunk setting.
Result:
Hmmm, let's see... Enemies to lovers sci-fi romance set in a hopepunk world? That
sounds intriguing! Here goes...
In a distant future where humanity has spread across the stars, two rival factions -
the authoritarian Galactic Federation and the rebel Free Colonies Alliance - wage a
never-ending war for control of resources and territory.
Zara, a fierce warrior-pilot for the Alliance, is on a mission to sabotage a key Federation
shipyard when her fighter is damaged by enemy fire. Forced to crash land on a remote
moon, she barely escapes with her life.
As Zara stumbles through the desolate landscape searching for help, she comes across
a figure in the distance. Her heart races as she realizes it's Kai, the Federation ace pilot
who shot her down. He too was forced to make an emergency landing after their battle.
Now stranded together on this barren rock, Zara and Kai must put aside their differences
and work together if they hope to survive. But old grudges die hard, and their constant
bickering soon turns to heated arguments and then passionate kisses.
Trapped in each other's arms under the alien sky, Zara and Kai discover a forbidden desire
burning between them. As they explore each other's bodies with desperate hunger, they
begin to question everything they thought they knew.
But their fragile truce is shattered when they uncover a terrible secret buried beneath the
moon's surface - one that threatens not only their lives, but the fate of both factions. Now
Zara and Kai must choose between duty and love, knowing that whatever path they take
will change the course of history forever.
Their story unfolds amidst a backdrop of advanced technology, political intrigue, and the
eternal struggle between freedom and oppression. In this gritty yet hopeful vision of the
future, two sworn enemies find themselves fighting for more than just survival - they're
fighting for a chance at something real, something true, something worth believing in.
Let me know what you think! I can expand on any part of the story you'd like.
This is with a character card (in oobabooga's TGW) that focuses on storytelling capabilities, but none of the other models from 7-22B have done as good a job as that. I haven't tried with the underlying models, because I don't usually like 'raw' models, so it's possible this is just reflective of the underlying model's capabilities, but it also handles 🌶️spicier🌶️ content very well, which is almost never true of base models.
Thanks for this!
Happy to see you like it! Appreciate the feedback thank you
Its true, the Model is great. I am Using it for ST and Kobold with temp 1.10 and it performs well. I am using also the 8 bit GGUF but mostly with KoboldCPP. Thanks for it.
You're welcome!