Francesco-A
commited on
Commit
•
ae47b0b
1
Parent(s):
669ef59
Update README.md
Browse files
README.md
CHANGED
@@ -1,5 +1,9 @@
|
|
1 |
---
|
2 |
-
|
|
|
|
|
|
|
|
|
3 |
---
|
4 |
**Model Card: (TEST) code-search-net-tokenizer**
|
5 |
|
@@ -25,7 +29,7 @@ The tokenizer was trained on a corpus of Python code snippets from the CodeSearc
|
|
25 |
|
26 |
**Usage:**
|
27 |
|
28 |
-
You can use the `code-search-net-tokenizer` to preprocess code snippets and convert them into numerical representations suitable for feeding into language models
|
29 |
|
30 |
**Limitations:**
|
31 |
|
|
|
1 |
---
|
2 |
+
license: apache-2.0
|
3 |
+
datasets:
|
4 |
+
- code_search_net
|
5 |
+
language:
|
6 |
+
- code
|
7 |
---
|
8 |
**Model Card: (TEST) code-search-net-tokenizer**
|
9 |
|
|
|
29 |
|
30 |
**Usage:**
|
31 |
|
32 |
+
You can use the `code-search-net-tokenizer` to preprocess code snippets and convert them into numerical representations suitable for feeding into language models.
|
33 |
|
34 |
**Limitations:**
|
35 |
|