Hugging Face's logo

alasdairforsythe
/
tokenmonster

Model card Files Community
alasdairforsythe commited on
Commit
d0485c7
•
1 Parent(s): 3470e92

Upload 30 files

Browse files
Files changed (18) hide show
  1. .gitattributes +5 -0
  2. vocabs/code-50256-consistent-v1.vocab +3 -0
  3. vocabs/english-1024-consistent-v1.vocab +0 -0
  4. vocabs/english-16000-balanced-v1.vocab +0 -0
  5. vocabs/english-2048-consistent-v1.vocab +0 -0
  6. vocabs/english-24000-consistent-v1.vocab +0 -0
  7. vocabs/english-50256-balanced-v1.vocab +3 -0
  8. vocabs/english-50256-strict-v1.vocab +3 -0
  9. vocabs/englishcode-16000-balanced-v1.vocab +0 -0
  10. vocabs/englishcode-16000-consistent-v1.vocab +0 -0
  11. vocabs/englishcode-16000-strict-v1.vocab +0 -0
  12. vocabs/englishcode-2048-clean-v1.vocab +0 -0
  13. vocabs/englishcode-4096-balanced-v1.vocab +0 -0
  14. vocabs/englishcode-65536-strict-v1.vocab +3 -0
  15. vocabs/fiction-1024-balanced-v1.vocab +0 -0
  16. vocabs/fiction-1024-clean-v1.vocab +0 -0
  17. vocabs/fiction-24000-consistent-v1.vocab +0 -0
  18. vocabs/fiction-65536-balanced-v1.vocab +3 -0
.gitattributes CHANGED
@@ -103,3 +103,8 @@ vocabs/code-40000-balanced-v1.vocab filter=lfs diff=lfs merge=lfs -text
103
  vocabs/english-100256-balanced-v1.vocab filter=lfs diff=lfs merge=lfs -text
104
  vocabs/english-65536-clean-v1.vocab filter=lfs diff=lfs merge=lfs -text
105
  vocabs/english-65536-unfiltered-v1.vocab filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
 
103
  vocabs/english-100256-balanced-v1.vocab filter=lfs diff=lfs merge=lfs -text
104
  vocabs/english-65536-clean-v1.vocab filter=lfs diff=lfs merge=lfs -text
105
  vocabs/english-65536-unfiltered-v1.vocab filter=lfs diff=lfs merge=lfs -text
106
+ vocabs/code-50256-consistent-v1.vocab filter=lfs diff=lfs merge=lfs -text
107
+ vocabs/english-50256-balanced-v1.vocab filter=lfs diff=lfs merge=lfs -text
108
+ vocabs/english-50256-strict-v1.vocab filter=lfs diff=lfs merge=lfs -text
109
+ vocabs/englishcode-65536-strict-v1.vocab filter=lfs diff=lfs merge=lfs -text
110
+ vocabs/fiction-65536-balanced-v1.vocab filter=lfs diff=lfs merge=lfs -text
vocabs/code-50256-consistent-v1.vocab ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7a8d454a0742d60afdac8287a157d1cd50b3e62e3fb440834f0c26d48e2b239f
3
+ size 1431097
vocabs/english-1024-consistent-v1.vocab ADDED
Binary file (25.7 kB). View file
 
vocabs/english-16000-balanced-v1.vocab ADDED
Binary file (409 kB). View file
 
vocabs/english-2048-consistent-v1.vocab ADDED
Binary file (50.9 kB). View file
 
vocabs/english-24000-consistent-v1.vocab ADDED
Binary file (620 kB). View file
 
vocabs/english-50256-balanced-v1.vocab ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cdb4bd389de76f9bcba4ae2a3ae582c1e28ab492555e51bf9ca38b6d58edb308
3
+ size 1331709
vocabs/english-50256-strict-v1.vocab ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f6ac708a37531416faf58076094d6272df13da9e1a8d77433044cd6ac60d5f56
3
+ size 1337179
vocabs/englishcode-16000-balanced-v1.vocab ADDED
Binary file (410 kB). View file
 
vocabs/englishcode-16000-consistent-v1.vocab ADDED
Binary file (410 kB). View file
 
vocabs/englishcode-16000-strict-v1.vocab ADDED
Binary file (407 kB). View file
 
vocabs/englishcode-2048-clean-v1.vocab ADDED
Binary file (51.2 kB). View file
 
vocabs/englishcode-4096-balanced-v1.vocab ADDED
Binary file (102 kB). View file
 
vocabs/englishcode-65536-strict-v1.vocab ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d65915cc6986f8b5a2a79385f743c3bad4128cbf11b3884221a2bcc20a95c404
3
+ size 1763890
vocabs/fiction-1024-balanced-v1.vocab ADDED
Binary file (25.6 kB). View file
 
vocabs/fiction-1024-clean-v1.vocab ADDED
Binary file (25.6 kB). View file
 
vocabs/fiction-24000-consistent-v1.vocab ADDED
Binary file (621 kB). View file
 
vocabs/fiction-65536-balanced-v1.vocab ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:86cdf29a50d79a6105c49025c25e29e0d476708c5e4637c641debb73ab7ff4f3
3
+ size 1770143