ilu000 commited on
Commit
251a6e5
1 Parent(s): f6819ae

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -33,7 +33,7 @@ This model was trained using [H2O LLM Studio](https://github.com/h2oai/h2o-llmst
33
 
34
  ## Model Architecture
35
 
36
- We adjust the Llama 2 architecture for a total of around 1.8b parameters. We use the original Llama 2 tokenizer with a vocabulary size of 32,000 and train our model up to a context length of 16,384. We incorporate the sliding window attention from mistral with a size of 4,096.
37
 
38
  The details of the model architecture are:
39
 
 
33
 
34
  ## Model Architecture
35
 
36
+ We adjust the Llama 2 architecture for a total of around 1.8b parameters. For details, please refer to our [Technical Report](https://arxiv.org/abs/2401.16818). We use the original Llama 2 tokenizer with a vocabulary size of 32,000 and train our model up to a context length of 16,384. We incorporate the sliding window attention from mistral with a size of 4,096.
37
 
38
  The details of the model architecture are:
39