|
--- |
|
license: llama2 |
|
library_name: peft |
|
tags: |
|
- generated_from_trainer |
|
base_model: unsloth/llama-3-8b-bnb-4bit |
|
metrics: |
|
- accuracy |
|
model-index: |
|
- name: llama3-qwantz-coherent |
|
results: [] |
|
pipeline_tag: text-classification |
|
--- |
|
|
|
<!-- This model card has been generated automatically according to the information the Trainer had access to. You |
|
should probably proofread and complete it, then remove this comment. --> |
|
|
|
# llama3-qwantz-coherent |
|
|
|
This model is a fine-tuned version of [unsloth/llama-3-8b-bnb-4bit](https://huggingface.co/unsloth/llama-3-8b-bnb-4bit) on an unknown dataset. |
|
It achieves the following results on the evaluation set: |
|
- Loss: 0.3295 |
|
- Accuracy: 0.8758 |
|
|
|
## Model description |
|
|
|
More information needed |
|
|
|
## Intended uses & limitations |
|
|
|
More information needed |
|
|
|
## Training and evaluation data |
|
|
|
More information needed |
|
|
|
## Training procedure |
|
|
|
### Training hyperparameters |
|
|
|
The following hyperparameters were used during training: |
|
- learning_rate: 0.0001 |
|
- train_batch_size: 8 |
|
- eval_batch_size: 8 |
|
- seed: 42 |
|
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 |
|
- lr_scheduler_type: linear |
|
- num_epochs: 1 |
|
|
|
### Training results |
|
|
|
| Training Loss | Epoch | Step | Validation Loss | Accuracy | |
|
|:-------------:|:-----:|:----:|:---------------:|:--------:| |
|
| 0.4482 | 1.0 | 1428 | 0.3295 | 0.8758 | |
|
|
|
``` |
|
2-4 fake words at the end (like training set): |
|
Can save 90% of coherent strings by discarding 82% of dp strings (cutoff is -67.26065874099731) |
|
Can save 95% of coherent strings by discarding 72% of dp strings (cutoff is -88.40824365615845) |
|
Can save 98% of coherent strings by discarding 62% of dp strings (cutoff is -95.06730437278748) |
|
Can save 99% of coherent strings by discarding 54% of dp strings (cutoff is -97.79982566833496) |
|
|
|
1 fake word at the end: |
|
Can save 90% of coherent strings by discarding 48% of dp strings (cutoff is -77.83336043357849) |
|
Can save 95% of coherent strings by discarding 30% of dp strings (cutoff is -92.52431392669678) |
|
Can save 98% of coherent strings by discarding 26% of dp strings (cutoff is -95.45100927352905) |
|
Can save 99% of coherent strings by discarding 21% of dp strings (cutoff is -97.32990860939026) |
|
|
|
Examples (2-4 fake words): |
|
My only problem (s) have to do with ==> coherent: 99.12% |
|
My only problem (s) to cheer them personally ==> dp: 99.69% |
|
(in small text) crazy utahraptor ==> coherent: 88.54% |
|
(in small text) ". ==> coherent: 54.82% |
|
Well, I've made up my own joke to get him today. All I need to do is " ==> coherent: 77.98% |
|
Well, I've made up my own joke to get him today. All I need a father and gentlemen ==> dp: 99.79% |
|
I will be immortalized by kicking an evil ==> dp: 72.79% |
|
I will be immortalized by kicking other punches ==> dp: 99.49% |
|
Aw shoot, I was supposed to ==> coherent: 99.80% |
|
Aw shoot, I was APOCALYPSE PORN ==> dp: 94.10% |
|
Get it? Because CRIME DOESN'T PAY!! Listen, my story has ==> coherent: 66.25% |
|
Get it? Because CRIME DOESN'T PAY!! Listen, transcriptions of it ==> dp: 99.75% |
|
Utahraptor!! DON'T LISTEN TO ==> coherent: 99.96% |
|
Utahraptor! This is sort of ==> coherent: 95.96% |
|
Doesn't exist in my mouth, that is!! Because it's too big ==> coherent: 95.38% |
|
Doesn't exist in my mouth, that is!! Because if Superman. ==> dp: 99.66% |
|
Now, HERE'S how ==> coherent: 98.67% |
|
Now, guys would ==> dp: 95.30% |
|
But I am a rock star ==> coherent: 92.34% |
|
But I am a guy come ==> dp: 99.40% |
|
But I have a solution to make them interesting again: all you need is stories where not ==> coherent: 94.51% |
|
But I have a solution to make them interesting again: all you need is gonna! Diseases ==> dp: 99.94% |
|
At that point, there's a sequence of six nines in a row, and his joke was that he'd like to memorize pi up to that point, so that when reciting he could end with "9,9,9,9,9,9... and so on. " Others ==> coherent: 70.68% |
|
At that point, there's a sequence of six nines in a row, and his joke was that he'd like to memorize pi up to that point, so that when reciting he could end with "9,9,9,9,9,9... and so it's great he looks ==> dp: 99.54% |
|
This is definitely called " T -Rex's Hilarious e joke ", okay ==> coherent: 79.86% |
|
This is definitely called " T -Rex's Hilarious e joke AND IN THE ==> dp: 98.83% |
|
" Your mouth is full of cockroaches: ==> coherent: 93.64% |
|
" Your mouth is full of smooches. ==> coherent: 99.10% |
|
Excuse me, sexual congress? Everyone else on the planet is dead, and ==> coherent: 89.66% |
|
Excuse me, sexual congress? Everyone else on the planet without syntactic ambiguity! ==> dp: 97.02% |
|
Sony is going to write swears on my bathroom ==> coherent: 99.75% |
|
Sony is going to write their babies need to ==> dp: 99.68% |
|
Beginning with the most modest: why am I ==> coherent: 99.62% |
|
Beginning with the most modest: why T - ==> dp: 92.38% |
|
Is there any greater meaning -to anything ==> coherent: 95.96% |
|
Is there any greater meaning? When you ==> dp: 73.11% |
|
I've also got steaks AND ==> coherent: 92.46% |
|
I've also cold -deterministic ==> dp: 99.48% |
|
I had a friend (female) who dated her roommate (also female) ==> coherent: 98.78% |
|
I had a friend (female) who dated her roommate, je suis grand ==> dp: 97.52% |
|
Yes... TOO BAD INDEED ==> dp: 93.13% |
|
Yes... TOO MANY YEARS ==> coherent: 65.94% |
|
|
|
Examples (1 fake word): |
|
My only problem (s) have to do with you ==> dp: 83.54% |
|
My only problem (s) have to do with no ==> dp: 53.68% |
|
(in small text ==> coherent: 93.38% |
|
(in small changes ==> dp: 97.64% |
|
Well, I've made up my own joke to get him today. All I need to do is " ==> coherent: 77.98% |
|
Well, I've made up my own joke to get him today. All I need to do is already ==> dp: 97.05% |
|
I will be immortalized by kicking an evil kangaroo ==> coherent: 92.81% |
|
I will be immortalized by kicking an evil! ==> dp: 99.50% |
|
Aw shoot, I was supposed ==> coherent: 98.61% |
|
Aw shoot, I was how ==> dp: 99.55% |
|
Get it? Because CRIME DOESN'T PAY!! Listen, my story has both a hilarious twist ending and also ==> coherent: 99.64% |
|
Get it? Because CRIME DOESN'T PAY!! Listen, my story has both a hilarious twist ending and genders ==> dp: 88.29% |
|
Utahraptor!! DON'T LISTEN TO MY ==> coherent: 94.44% |
|
Utahraptor!! DON'T LISTEN TO THE ==> coherent: 97.86% |
|
Doesn't exist in my mouth, that is!! Because it's too ==> coherent: 96.91% |
|
Doesn't exist in my mouth, that is!! Because it's Well ==> dp: 97.87% |
|
Now, HERE'S how putting the things ==> dp: 97.33% |
|
Now, HERE'S how putting the wall ==> dp: 87.15% |
|
But I am a rock ==> coherent: 52.13% |
|
But I am a time ==> dp: 93.86% |
|
But I have a solution to make them interesting again: all you need is ==> coherent: 99.83% |
|
But I have a solution to make them interesting again: all you need to ==> coherent: 99.81% |
|
At that point, there's a sequence of six nines in a row, and his joke was that he'd like to memorize pi up to that point, so that when reciting he could end with "9,9,9 ==> coherent: 99.95% |
|
At that point, there's a sequence of six nines in a row, and his joke was that he'd like to memorize pi up to that point, so that when reciting he could end with "9,9, and ==> coherent: 95.76% |
|
This is definitely ==> coherent: 89.56% |
|
This is especially ==> coherent: 98.87% |
|
" Your mouth is full of cockroaches: many of them are dead, but those that ==> coherent: 99.96% |
|
" Your mouth is full of cockroaches: many of them are dead, but those taste ==> dp: 78.53% |
|
Excuse me, sexual congress? Everyone ==> dp: 94.45% |
|
Excuse me, sexual congress? " ==> dp: 91.74% |
|
Sony is going to write swears ==> coherent: 74.56% |
|
Sony is going to write that ==> dp: 98.15% |
|
Beginning with the ==> coherent: 84.62% |
|
Beginning with! ==> dp: 98.17% |
|
Is there any greater meaning -to ==> dp: 61.69% |
|
Is there any greater meaning -Rex ==> dp: 99.31% |
|
I've also got steaks AND pork chops ==> coherent: 99.84% |
|
I've also got steaks AND pork meat ==> coherent: 95.80% |
|
I had a friend (female ==> coherent: 85.77% |
|
I had a friend (on ==> coherent: 57.30% |
|
Yes... TOO ==> dp: 77.83% |
|
Yes... All ==> dp: 67.88% |
|
``` |
|
|
|
### Framework versions |
|
|
|
- PEFT 0.10.0 |
|
- Transformers 4.40.1 |
|
- Pytorch 2.2.1+cu121 |
|
- Datasets 2.19.0 |
|
- Tokenizers 0.19.1 |
