Hugging Face's logo

alasdairforsythe
/
tokenmonster

Model card Files Community
alasdairforsythe commited on
Commit
ea199cd
•
1 Parent(s): 1d1827f

Upload 13 files

Browse files
Files changed (14) hide show
  1. .gitattributes +3 -0
  2. vocabs/code-100256-consistent-v1.vocab +3 -0
  3. vocabs/code-16000-strict-v1.vocab +0 -0
  4. vocabs/code-32000-consistent-v1.vocab +0 -0
  5. vocabs/englishcode-2048-strict-v1.vocab +0 -0
  6. vocabs/englishcode-24000-balanced-v1.vocab +0 -0
  7. vocabs/englishcode-24000-clean-v1.vocab +0 -0
  8. vocabs/englishcode-32000-balanced-v1.vocab +0 -0
  9. vocabs/englishcode-4096-unfiltered-v1.vocab +0 -0
  10. vocabs/englishcode-65536-balanced-v1.vocab +3 -0
  11. vocabs/englishcode-8000-unfiltered-v1.vocab +0 -0
  12. vocabs/fiction-16000-strict-v1.vocab +0 -0
  13. vocabs/fiction-32000-clean-v1.vocab +0 -0
  14. vocabs/fiction-40000-unfiltered-v1.vocab +3 -0
.gitattributes CHANGED
@@ -88,3 +88,6 @@ vocabs/fiction-100256-consistent-v1.vocab filter=lfs diff=lfs merge=lfs -text
88
  vocabs/fiction-40000-strict-v1.vocab filter=lfs diff=lfs merge=lfs -text
89
  vocabs/fiction-50256-clean-v1.vocab filter=lfs diff=lfs merge=lfs -text
90
  vocabs/fiction-50256-consistent-v1.vocab filter=lfs diff=lfs merge=lfs -text
 
 
 
 
88
  vocabs/fiction-40000-strict-v1.vocab filter=lfs diff=lfs merge=lfs -text
89
  vocabs/fiction-50256-clean-v1.vocab filter=lfs diff=lfs merge=lfs -text
90
  vocabs/fiction-50256-consistent-v1.vocab filter=lfs diff=lfs merge=lfs -text
91
+ vocabs/code-100256-consistent-v1.vocab filter=lfs diff=lfs merge=lfs -text
92
+ vocabs/englishcode-65536-balanced-v1.vocab filter=lfs diff=lfs merge=lfs -text
93
+ vocabs/fiction-40000-unfiltered-v1.vocab filter=lfs diff=lfs merge=lfs -text
vocabs/code-100256-consistent-v1.vocab ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:65d03b15dd3b27e800e6715d76daf16e42c36a4c4350a97f7a079c11c4131c1b
3
+ size 2897899
vocabs/code-16000-strict-v1.vocab ADDED
Binary file (423 kB). View file
 
vocabs/code-32000-consistent-v1.vocab ADDED
Binary file (895 kB). View file
 
vocabs/englishcode-2048-strict-v1.vocab ADDED
Binary file (50 kB). View file
 
vocabs/englishcode-24000-balanced-v1.vocab ADDED
Binary file (624 kB). View file
 
vocabs/englishcode-24000-clean-v1.vocab ADDED
Binary file (620 kB). View file
 
vocabs/englishcode-32000-balanced-v1.vocab ADDED
Binary file (838 kB). View file
 
vocabs/englishcode-4096-unfiltered-v1.vocab ADDED
Binary file (104 kB). View file
 
vocabs/englishcode-65536-balanced-v1.vocab ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2d74cfefd22cc6a412bb0fd1e4afe1d052b6f994560b80649b565543c1db7e56
3
+ size 1761619
vocabs/englishcode-8000-unfiltered-v1.vocab ADDED
Binary file (205 kB). View file
 
vocabs/fiction-16000-strict-v1.vocab ADDED
Binary file (409 kB). View file
 
vocabs/fiction-32000-clean-v1.vocab ADDED
Binary file (842 kB). View file
 
vocabs/fiction-40000-unfiltered-v1.vocab ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8a034d135e0dbe08958240a0a28549ce9b36ff228882b6d8261b7fe2da15d1fe
3
+ size 1064286