mhhmm
/

typescript-instruct-20k-v2

Text Generation

instruction-tuning

code-generation

Model card Files Files and versions Metrics Training metrics Community

mhhmm commited on Dec 21, 2023

Commit

b54267f

•

1 Parent(s): dc7be24

Update README.md

Files changed (1) hide show

README.md +4 -3

README.md CHANGED Viewed

@@ -86,9 +86,10 @@ The following hyperparameters were used during training:
 I'm using MultiPL-E benchmark, the same as Code Llmama using in their paper
-| Dataset                                          | Pass@k | Estimate             | NumProblems | MinCompletions | MaxCompletions |
-|--------------------------------------------------|--------|----------------------|-------------|----------------|----------------|
-| mhhmm/typescript-instruct-20k | 1      | 0.4241   | 159         | 13             | 20             |
 How to reproduce my evaluation? Just run like the offical document of MultiPL-E: https://nuprl.github.io/MultiPL-E/tutorial.html, change the modal name by my model here: `mhhmm/typescript-instruct-20k`

 I'm using MultiPL-E benchmark, the same as Code Llmama using in their paper
+| Modal                                   | Pass@k | Estimate | Num problems  |
+|-----------------------------------------|--------|----------|---------------|
+| Code LLama - Instruct 13B               | 1      | 0.390   | 159           |
+| Our                                     | 1      | 0.424   | 159           |
 How to reproduce my evaluation? Just run like the offical document of MultiPL-E: https://nuprl.github.io/MultiPL-E/tutorial.html, change the modal name by my model here: `mhhmm/typescript-instruct-20k`