Hugging Face's logo

alasdairforsythe
/
tokenmonster

Model card Files Community
alasdairforsythe commited on
Commit
f5c7ec3
•
1 Parent(s): 1be91c1

Upload 13 files

Browse files
Files changed (14) hide show
  1. .gitattributes +5 -0
  2. vocabs/code-65536-clean-v1.vocab +3 -0
  3. vocabs/code-65536-unfiltered-v1.vocab +3 -0
  4. vocabs/english-100256-strict-v1.vocab +3 -0
  5. vocabs/englishcode-1024-balanced-v1.vocab +0 -0
  6. vocabs/englishcode-1024-consistent-v1.vocab +0 -0
  7. vocabs/englishcode-2048-consistent-v1.vocab +0 -0
  8. vocabs/englishcode-24000-strict-v1.vocab +0 -0
  9. vocabs/englishcode-32000-clean-v1.vocab +0 -0
  10. vocabs/englishcode-32000-consistent-v1.vocab +0 -0
  11. vocabs/englishcode-65536-clean-v1.vocab +3 -0
  12. vocabs/fiction-100256-strict-v1.vocab +3 -0
  13. vocabs/fiction-4096-clean-v1.vocab +0 -0
  14. vocabs/fiction-4096-unfiltered-v1.vocab +0 -0
.gitattributes CHANGED
@@ -113,3 +113,8 @@ vocabs/code-40000-clean-v1.vocab filter=lfs diff=lfs merge=lfs -text
113
  vocabs/code-40000-unfiltered-v1.vocab filter=lfs diff=lfs merge=lfs -text
114
  vocabs/englishcode-40000-balanced-v1.vocab filter=lfs diff=lfs merge=lfs -text
115
  vocabs/fiction-40000-balanced-v1.vocab filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
 
113
  vocabs/code-40000-unfiltered-v1.vocab filter=lfs diff=lfs merge=lfs -text
114
  vocabs/englishcode-40000-balanced-v1.vocab filter=lfs diff=lfs merge=lfs -text
115
  vocabs/fiction-40000-balanced-v1.vocab filter=lfs diff=lfs merge=lfs -text
116
+ vocabs/code-65536-clean-v1.vocab filter=lfs diff=lfs merge=lfs -text
117
+ vocabs/code-65536-unfiltered-v1.vocab filter=lfs diff=lfs merge=lfs -text
118
+ vocabs/english-100256-strict-v1.vocab filter=lfs diff=lfs merge=lfs -text
119
+ vocabs/englishcode-65536-clean-v1.vocab filter=lfs diff=lfs merge=lfs -text
120
+ vocabs/fiction-100256-strict-v1.vocab filter=lfs diff=lfs merge=lfs -text
vocabs/code-65536-clean-v1.vocab ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:adb153c8bcf51f77996ff945d56d106299a285355451c913fe326dc3a4476f91
3
+ size 1819682
vocabs/code-65536-unfiltered-v1.vocab ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c2706cfb52bda4aacf37a9cf639cb816d06deef4c4d85e7eee140eb0923b80dc
3
+ size 1905892
vocabs/english-100256-strict-v1.vocab ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e0eddc93d7baf827e1abd3f47c18e1afb4461912ed26f2d312a903f6fa2b5db4
3
+ size 2746592
vocabs/englishcode-1024-balanced-v1.vocab ADDED
Binary file (25.7 kB). View file
 
vocabs/englishcode-1024-consistent-v1.vocab ADDED
Binary file (25.5 kB). View file
 
vocabs/englishcode-2048-consistent-v1.vocab ADDED
Binary file (50.9 kB). View file
 
vocabs/englishcode-24000-strict-v1.vocab ADDED
Binary file (620 kB). View file
 
vocabs/englishcode-32000-clean-v1.vocab ADDED
Binary file (833 kB). View file
 
vocabs/englishcode-32000-consistent-v1.vocab ADDED
Binary file (840 kB). View file
 
vocabs/englishcode-65536-clean-v1.vocab ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:896529eaee076b14e0ce48b38a1732fc4b137ff6b687fe6c318df641e9d5c35d
3
+ size 1745161
vocabs/fiction-100256-strict-v1.vocab ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:77f0684a29632a51b41df7bf2d1519f6d3cf8c2cdee0a7ad4bd0b50a4a35ab3e
3
+ size 2776502
vocabs/fiction-4096-clean-v1.vocab ADDED
Binary file (103 kB). View file
 
vocabs/fiction-4096-unfiltered-v1.vocab ADDED
Binary file (103 kB). View file