Spaces:
Sleeping
Sleeping
Commit History
Add Mistral API support due to my RunPod serverless system reliability issues
8093276
Add TODO for Runpod timeout handling
3c6c618
Assert RunPod env vars are setup before trying to use them
00af17e
Revert expected serverless output metadata stripper
c013599
Luke Stanley
commited on
Avoid unneeded imports, make serverless output more sensible, removing some debugging and comments
469f650
Luke Stanley
commited on
Fix RUNPOD_ENDPOINT_ID environment variable
ce5ad5f
Luke Stanley
commited on
Introduces worker mode env var
56e785c
Luke Stanley
commited on
Ensure N_GPU_LAYERS is int
9475016
Luke Stanley
commited on
Update default GPU layer, temperature values
e327a9e
Add env vars to set GPU layer count and context size, make verbose
e01e28e
Add n_gpu_layers parameter to Llama initialization
88e6118
Fix: Move n_ctx parameter to model setup!
358cd20
Fix check for LLM_MODEL_PATH to avoid load error
ff938c3
Auto-downloads model if env var is not set
74d6e52
Luke Stanley
commited on
Make llm_stream_sans_network actually stream to stdout
a0f49a0
Luke Stanley
commited on
Default to in-memory LLM interface
ddb0d91
Luke Stanley
commited on