Francesco-A commited on
Commit
ae47b0b
1 Parent(s): 669ef59

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +6 -2
README.md CHANGED
@@ -1,5 +1,9 @@
1
  ---
2
- {}
 
 
 
 
3
  ---
4
  **Model Card: (TEST) code-search-net-tokenizer**
5
 
@@ -25,7 +29,7 @@ The tokenizer was trained on a corpus of Python code snippets from the CodeSearc
25
 
26
  **Usage:**
27
 
28
- You can use the `code-search-net-tokenizer` to preprocess code snippets and convert them into numerical representations suitable for feeding into language models like GPT-2, BERT, or RoBERTa.
29
 
30
  **Limitations:**
31
 
 
1
  ---
2
+ license: apache-2.0
3
+ datasets:
4
+ - code_search_net
5
+ language:
6
+ - code
7
  ---
8
  **Model Card: (TEST) code-search-net-tokenizer**
9
 
 
29
 
30
  **Usage:**
31
 
32
+ You can use the `code-search-net-tokenizer` to preprocess code snippets and convert them into numerical representations suitable for feeding into language models.
33
 
34
  **Limitations:**
35