File size: 4,321 Bytes
27c2cad 7e65c15 27c2cad 7e65c15 27c2cad 7e65c15 27c2cad 7e65c15 27c2cad 7e65c15 27c2cad 2551a24 27c2cad 7e65c15 27c2cad 8c10c53 55baf49 8c10c53 8d6df28 8c10c53 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 |
---
license: mit
model-index:
- name: Rubra-Phi-3-mini-128k-instruct
results:
- task:
type: text-generation
dataset:
type: MMLU
name: MMLU
metrics:
- type: 5-shot
value: 67.87
verified: false
- task:
type: text-generation
dataset:
type: GPQA
name: GPQA
metrics:
- type: 0-shot
value: 29.69
verified: false
- task:
type: text-generation
dataset:
type: GSM-8K
name: GSM-8K
metrics:
- type: 8-shot, CoT
value: 79.45
verified: false
- task:
type: text-generation
dataset:
type: MATH
name: MATH
metrics:
- type: 4-shot, CoT
value: 30.80
verified: false
- task:
type: text-generation
dataset:
type: MT-bench
name: MT-bench
metrics:
- type: GPT-4 as Judge
value: 8.21
verified: false
tags:
- function-calling
- tool-calling
- agentic
- rubra
- conversational
language:
- en
---
# Rubra Phi-3 Mini 128k Instruct GGUF
Original model: [rubra-ai/Phi-3-mini-128k-instruct](https://huggingface.co/rubra-ai/Phi-3-mini-128k-instruct)
## Model description
The model is the result of further post-training [microsoft/Phi-3-mini-128k-instruct](https://huggingface.co/microsoft/Phi-3-mini-128k-instruct). This model is designed for high performance in various instruction-following tasks and complex interactions, including multi-turn function calling and detailed conversations.
| Model | Function Calling | MMLU | GPQA | GSM-8K | MATH | MT-bench | Win | Loss | Tie | Win Rate | Loss Rate | Adjusted Win Rate |
|----------------------------------------------|------------------|-------|-------|--------|-------|----------|-----|------|-----|----------|-----------|-------------------|
| Phi-3 Mini 128k Instruct (June) | - | 69.36 | 27.01 | 83.7 | 32.92 | 8.02 | 21 | 72 | 67 | 0.13125 | 0.45000 | 0.340625 |
| Rubra Enhanced Phi-3 Mini 128k Instruct (June)| 70.00% | 67.87 | 29.69 | 79.45 | 30.80 | 8.21 | 72 | 21 | 67 | 0.45000 | 0.13125 | **0.659375** |
| Phi-3 Mini 128k Instruct (April) | - | 68.17 | 25.90 | 80.44 | 28.12 | 7.92 | 51 | 45 | 64 | 0.31875 | 0.28125 | 0.51875 |
| Rubra Enhanced Phi-3 Mini 128k Instruct (April)| 65.71% | 66.66 | 29.24 | 74.09 | 26.84 | 7.45 | 45 | 51 | 64 | 0.28125 | 0.31875 | 0.48125 |
* Commit `e2ecb24bd9dae689bb30dafcf13cbbc9dbddead5` is the last commit to have the April-based Phi-3 model. The latest in main is built off the June model
## Training Data
The model underwent additional training on a proprietary dataset encompassing diverse instruction-following, chat, and function calling data. This post-training process enhances the model's ability to integrate tools and manage complex interaction scenarios effectively.
## How to use
Refer to https://docs.rubra.ai/inference/llamacpp for usage. Feel free to ask/open issues up in our Github repo: https://github.com/rubra-ai/rubra
## Limitations and Bias
While the model performs well on a wide range of tasks, it may still produce biased or incorrect outputs. Users should exercise caution and critical judgment when using the model in sensitive or high-stakes applications. The model's outputs are influenced by the data it was trained on, which may contain inherent biases.
## Ethical Considerations
Users should ensure that the deployment of this model adheres to ethical guidelines and consider the potential societal impact of the generated text. Misuse of the model for generating harmful or misleading content is strongly discouraged.
## Acknowledgements
We would like to thank Microsoft for the model.
## Contact Information
For questions or comments about the model, please reach out to [the rubra team](mailto:[email protected]).
## Citation
If you use this work, please cite it as:
```
@misc {rubra_ai_2024,
author = { Sanjay Nadhavajhala and Yingbei Tong },
title = { Phi-3-mini-128k-instruct },
year = 2024,
url = { https://huggingface.co/rubra-ai/Phi-3-mini-128k-instruct },
doi = { 10.57967/hf/2682 },
publisher = { Hugging Face }
}
``` |