MLP-KTLim
/

llama-3-Korean-Bllossom-8B-gguf-Q4_K_M

Inference Endpoints

Model card Files Files and versions Community

limhyeonseok commited on May 10

Commit

897ebfe

•

1 Parent(s): eb39122

Update README.md

Files changed (1) hide show

README.md +2 -1

README.md CHANGED Viewed

@@ -119,8 +119,9 @@ prompt = tokenizer.apply_chat_template(
 generation_kwargs = {
     "max_tokens":512,
     "stop":["<|eot_id|>"],
     "echo":True, # Echo the prompt in the output
-    "top_k":1 # This is essentially greedy decoding, since the model will always return the highest-probability token. Set this value > 1 for sampling decoding
 }
 resonse_msg = model(prompt, **generation_kwargs)

 generation_kwargs = {
     "max_tokens":512,
     "stop":["<|eot_id|>"],
+    "top-p":0.9,
+    "temperature":0.6,
     "echo":True, # Echo the prompt in the output
 }
 resonse_msg = model(prompt, **generation_kwargs)