Hugging Face
Models
Datasets
Spaces
Posts
Docs
Solutions
Pricing
Log In
Sign Up
grimjim
/
wizard-elem-to-32k-7B
like
0
Text Generation
Transformers
Safetensors
mistral
mergekit
Merge
conversational
text-generation-inference
Inference Endpoints
arxiv:
2212.04089
License:
apache-2.0
Model card
Files
Files and versions
Community
Train
Deploy
Use this model
7fb0bbb
wizard-elem-to-32k-7B
/
params.json
grimjim
Initial release
7fb0bbb
4 months ago
raw
Copy download link
history
blame
No virus
203 Bytes
{
"dim"
:
4096
,
"n_layers"
:
32
,
"head_dim"
:
128
,
"hidden_dim"
:
14336
,
"n_heads"
:
32
,
"n_kv_heads"
:
8
,
"norm_eps"
:
1e-05
,
"vocab_size"
:
32000
,
"rope_theta"
:
1000000.0
}