derek-thomas HF staff commited on
Commit
efbddb3
1 Parent(s): 4cd60cc

Documenting the new prompting

Browse files
Files changed (1) hide show
  1. README.md +16 -0
README.md CHANGED
@@ -13,6 +13,22 @@ license: apache-2.0
13
  pipeline_tag: conversational
14
  ---
15
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
16
  # Jais-13b-chat
17
 
18
  <!-- Provide a quick summary of what the model is/does. -->
 
13
  pipeline_tag: conversational
14
  ---
15
 
16
+ # jais-13b-chat-hf
17
+ I made a couple changes, I use LLM.int8() to load this in 8 bits rather than full precision which lowers the GPU VRAM requirements by 3x.
18
+ Further I allow you to set the whole prompt like this:
19
+ ```python
20
+ import requests
21
+ API_URL='your API url' # You get this from your deployed Inference Endpoint
22
+ BEARER='your bearer token' # You get this from your deployed Inference Endpoint
23
+ headers = {
24
+ "Authorization": f"Bearer {BEARER}",
25
+ "Content-Type": "application/json"
26
+ }
27
+ prompt = "Your clever prompt to drive value here..."
28
+ payload = {'inputs': '', 'prompt': prompt} # 'inputs' is a required key...
29
+ response = requests.post(API_URL, headers=headers, json=payload)
30
+ ```
31
+
32
  # Jais-13b-chat
33
 
34
  <!-- Provide a quick summary of what the model is/does. -->