Hugging Face's logo

alasdairforsythe
/
tokenmonster

Model card Files Community
alasdairforsythe commited on
Commit
5e0a1e4
•
1 Parent(s): ea199cd

Upload 9 files

Browse files
Files changed (10) hide show
  1. .gitattributes +3 -0
  2. vocabs/code-40000-strict-v1.vocab +3 -0
  3. vocabs/code-65536-consistent-v1.vocab +3 -0
  4. vocabs/englishcode-16000-unfiltered-v1.vocab +0 -0
  5. vocabs/englishcode-4096-clean-v1.vocab +0 -0
  6. vocabs/englishcode-8000-clean-v1.vocab +0 -0
  7. vocabs/englishcode-8000-consistent-v1.vocab +0 -0
  8. vocabs/fiction-16000-consistent-v1.vocab +0 -0
  9. vocabs/fiction-2048-unfiltered-v1.vocab +0 -0
  10. vocabs/fiction-40000-clean-v1.vocab +3 -0
.gitattributes CHANGED
@@ -91,3 +91,6 @@ vocabs/fiction-50256-consistent-v1.vocab filter=lfs diff=lfs merge=lfs -text
91
  vocabs/code-100256-consistent-v1.vocab filter=lfs diff=lfs merge=lfs -text
92
  vocabs/englishcode-65536-balanced-v1.vocab filter=lfs diff=lfs merge=lfs -text
93
  vocabs/fiction-40000-unfiltered-v1.vocab filter=lfs diff=lfs merge=lfs -text
 
 
 
 
91
  vocabs/code-100256-consistent-v1.vocab filter=lfs diff=lfs merge=lfs -text
92
  vocabs/englishcode-65536-balanced-v1.vocab filter=lfs diff=lfs merge=lfs -text
93
  vocabs/fiction-40000-unfiltered-v1.vocab filter=lfs diff=lfs merge=lfs -text
94
+ vocabs/code-40000-strict-v1.vocab filter=lfs diff=lfs merge=lfs -text
95
+ vocabs/code-65536-consistent-v1.vocab filter=lfs diff=lfs merge=lfs -text
96
+ vocabs/fiction-40000-clean-v1.vocab filter=lfs diff=lfs merge=lfs -text
vocabs/code-40000-strict-v1.vocab ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fe67e47d30c54a7c077714aa68e9636f79a005797d5547ef8fdb23bcfebc5288
3
+ size 1093224
vocabs/code-65536-consistent-v1.vocab ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:070dc3152f18a080af4a5b078237af483f984de35584026128e559894c31574c
3
+ size 1878772
vocabs/englishcode-16000-unfiltered-v1.vocab ADDED
Binary file (418 kB). View file
 
vocabs/englishcode-4096-clean-v1.vocab ADDED
Binary file (102 kB). View file
 
vocabs/englishcode-8000-clean-v1.vocab ADDED
Binary file (200 kB). View file
 
vocabs/englishcode-8000-consistent-v1.vocab ADDED
Binary file (201 kB). View file
 
vocabs/fiction-16000-consistent-v1.vocab ADDED
Binary file (408 kB). View file
 
vocabs/fiction-2048-unfiltered-v1.vocab ADDED
Binary file (51 kB). View file
 
vocabs/fiction-40000-clean-v1.vocab ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d223111c67b05b7761552daa2ce2a45e9f0ddebc1300fcab53206cea9d5e0c75
3
+ size 1061894