tensorplex-labs
commited on
Commit
•
3650c29
1
Parent(s):
b78b2e4
Update README.md
Browse files
README.md
CHANGED
@@ -16,17 +16,17 @@ datasets:
|
|
16 |
---
|
17 |
|
18 |
|
19 |
-
# Sumo-
|
20 |
|
21 |
![image/png](https://cdn-uploads.huggingface.co/production/uploads/65a8a4c5539e211436ef5485/RXiIpU1BwTpvUdhzv-XK9.png)
|
22 |
|
23 |
-
### Tensorplex Labs Unveils Sumo-
|
24 |
|
25 |
-
[Tensorplex Labs]((https://tensorplex.ai)) is proud to announce that its latest top-performing model on Bittensor Subnet 9, Sumo-
|
26 |
has outperformed notable models such as TII Falcon 7B and Meta's Llama-2-7b-hf. This achievement highlights the potential of decentralized networks
|
27 |
like Bittensor and underscores Tensorplex Labs' commitment to advancing open-source AI technologies.
|
28 |
|
29 |
-
"Sumo" represents the family of models developed by Tensorplex, and "
|
30 |
|
31 |
Bittensor Subnet 9 serves a unique role within the Bittensor ecosystem by rewarding miners who produce pretrained foundational models on the Falcon Refined Web dataset. This subnet functions as a continuous benchmark, where miners are incentivized to achieve the best performance metrics using a model under the parameter limit. The competitive nature of Subnet 9 drives rapid advancements and refinements in large language model training.
|
32 |
|
@@ -45,7 +45,7 @@ Since the parameter limit was upgraded to 7 billion on April 19, 2024, Tensorple
|
|
45 |
- **Training Objective**: Causal Language Modeling (next token prediction)
|
46 |
- **Original Model Repo**: [tensorplex-labs/pretraining-sn9-7B-1](https://huggingface.co/tensorplex-labs/pretraining-sn9-7B-1)
|
47 |
|
48 |
-
Sumo-
|
49 |
|
50 |
⛔ **This is a pretrained base model, which hasn't been aligned yet. Use with caution or finetune further on downstream tasks before deployment.**
|
51 |
|
@@ -63,7 +63,7 @@ from transformers import AutoTokenizer, AutoModelForCausalLM
|
|
63 |
import transformers
|
64 |
import torch
|
65 |
|
66 |
-
model = "tensorplex-labs/Sumo-
|
67 |
|
68 |
tokenizer = AutoTokenizer.from_pretrained(model)
|
69 |
pipeline = transformers.pipeline(
|
@@ -95,7 +95,7 @@ This model has been trained with [tiiuae/falcon-refinedweb](https://huggingface.
|
|
95 |
|
96 |
## Evaluation
|
97 |
|
98 |
-
Sumo-
|
99 |
establishing itself as the leading model in aggregate across various evaluation tasks.
|
100 |
Such benchmarks include ARC Challenge, GSM8K, HellaSwag, MMLU, TruthfulQA, and Winogrande.
|
101 |
|
|
|
16 |
---
|
17 |
|
18 |
|
19 |
+
# Sumo-T9-7B-v0.1
|
20 |
|
21 |
![image/png](https://cdn-uploads.huggingface.co/production/uploads/65a8a4c5539e211436ef5485/RXiIpU1BwTpvUdhzv-XK9.png)
|
22 |
|
23 |
+
### Tensorplex Labs Unveils Sumo-T9-7B: Beating Notable 7b Pretrained Models
|
24 |
|
25 |
+
[Tensorplex Labs]((https://tensorplex.ai)) is proud to announce that its latest top-performing model on Bittensor Subnet 9, Sumo-T9-7B,
|
26 |
has outperformed notable models such as TII Falcon 7B and Meta's Llama-2-7b-hf. This achievement highlights the potential of decentralized networks
|
27 |
like Bittensor and underscores Tensorplex Labs' commitment to advancing open-source AI technologies.
|
28 |
|
29 |
+
"Sumo" represents the family of models developed by Tensorplex, and "T9" designates the top-performing model specifically trained for Bittensor Subnet 9.
|
30 |
|
31 |
Bittensor Subnet 9 serves a unique role within the Bittensor ecosystem by rewarding miners who produce pretrained foundational models on the Falcon Refined Web dataset. This subnet functions as a continuous benchmark, where miners are incentivized to achieve the best performance metrics using a model under the parameter limit. The competitive nature of Subnet 9 drives rapid advancements and refinements in large language model training.
|
32 |
|
|
|
45 |
- **Training Objective**: Causal Language Modeling (next token prediction)
|
46 |
- **Original Model Repo**: [tensorplex-labs/pretraining-sn9-7B-1](https://huggingface.co/tensorplex-labs/pretraining-sn9-7B-1)
|
47 |
|
48 |
+
Sumo-T9-7B-v0.1 features a larger vocabulary size (100k), compatible with the GPT-4 tokenizer, ensuring its versatility across various natural language processing tasks.
|
49 |
|
50 |
⛔ **This is a pretrained base model, which hasn't been aligned yet. Use with caution or finetune further on downstream tasks before deployment.**
|
51 |
|
|
|
63 |
import transformers
|
64 |
import torch
|
65 |
|
66 |
+
model = "tensorplex-labs/Sumo-T9-7B-v0.1"
|
67 |
|
68 |
tokenizer = AutoTokenizer.from_pretrained(model)
|
69 |
pipeline = transformers.pipeline(
|
|
|
95 |
|
96 |
## Evaluation
|
97 |
|
98 |
+
Sumo-T9-7B-v0.1 has outperformed notable models such as TII Falcon 7B, Meta's Llama-2-7b and Llama-1-7b in zero-shot performance,
|
99 |
establishing itself as the leading model in aggregate across various evaluation tasks.
|
100 |
Such benchmarks include ARC Challenge, GSM8K, HellaSwag, MMLU, TruthfulQA, and Winogrande.
|
101 |
|