webpolis
/

zenos-gpt-j-6B-instruct-4bit

Text Generation

Inference Endpoints

4-bit precision

Model card Files Files and versions Community

Nicolas Iglesias commited on Oct 15, 2023

Commit

c72a93e

•

1 Parent(s): 4ab4475

Update README.md

Files changed (1) hide show

README.md +3 -3

README.md CHANGED Viewed

@@ -46,11 +46,11 @@ input_ids = inputs["input_ids"].to(model.device)
 attention_mask = inputs["attention_mask"].to(model.device)
 generation_config = GenerationConfig(
-    temperature=0.1,
-    top_p=0.75,
     top_k=40,
     num_beams=1,
-    repetition_penalty=1.5,
     do_sample=True
 )

 attention_mask = inputs["attention_mask"].to(model.device)
 generation_config = GenerationConfig(
+    temperature=0.2,
+    top_p=0.8,
     top_k=40,
     num_beams=1,
+    repetition_penalty=1.3,
     do_sample=True
 )