---
license: apache-2.0
datasets:
- gair-prox/RedPajama-pro
language:
- en
tags:
- math
- reasoning
---

# ProX-RedPJ-0.3B

<p align="center">
  <img src="prox.png" width="200">
</p>

[ArXiv](http://arxiv.org/abs/xxxx) | [Models](https://huggingface.co/gair-prox/ProX-RedPJ-0.3B) | [Data](https://huggingface.co/datasets/gair-prox/RedPajama-pro) | [Code](https://github.com/GAIR-NLP/program-every-example)

**ProX-RedPJ-0.3B** is a tiny language model. It was and trained on the [RedPajama-V2-pro](https://huggingface.co/datasets/gair-prox/RedPajama-pro) for 25B tokens. 

## Evaluations

ProX models are evaluated over 10 language model benchmarks in zero-shot setting.

| ARC-C  | ARC-E  | CSQA   | HellaSwag | MMLU   | OBQA   | PiQA   | SIQA   | WinoG  | SciQ   | AVG  |
|--------|--------|--------|-----------|--------|--------|--------|--------|--------|--------|------|
| 25.90 | 47.50 | 29.20 | 36.70    | 28.11 | 30.20 | 64.60 | 38.00 | 51.70 | 71.40 | 42.3 |

### Citation
```
@misc{TBD
}
```