ibm-granite
/

granite-3.0-1b-a400m-base

Text Generation

Model card Files Files and versions Community

amezasor commited on 16 days ago

Commit

cf1a1df

•

1 Parent(s): 6793f30

typo fix

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -272,7 +272,7 @@ Granite-3.0-1B-A400M-Base is based on a decoder-only sparse Mixture of Experts (
 | Initialization std           | 0.1      | 0.1      | **0.1**      | 0.1      |
 | Sequence Length              | 4096     | 4096     | **4096**     | 4096     |
 | Position Embedding           | RoPE     | RoPE     | **RoPE**     | RoPE     |
-| # Paremeters                 | 2.5B     | 8.1B     | **1.3B**     | 3.3B     |
 | # Active Parameters          | 2.5B     | 8.1B     | **400M**     | 800M     |
 | # Training tokens            | 12T      | 12T      | **10T**      | 10T      |

 | Initialization std           | 0.1      | 0.1      | **0.1**      | 0.1      |
 | Sequence Length              | 4096     | 4096     | **4096**     | 4096     |
 | Position Embedding           | RoPE     | RoPE     | **RoPE**     | RoPE     |
+| # Parameters                 | 2.5B     | 8.1B     | **1.3B**     | 3.3B     |
 | # Active Parameters          | 2.5B     | 8.1B     | **400M**     | 800M     |
 | # Training tokens            | 12T      | 12T      | **10T**      | 10T      |