README.md

metadata

license: llama2

Usage:

1. Install OpenChat

2. python -m ochat.serving.openai_api_server --model-type openchat_llama2 --model Open-Orca/Llama2_GPT4_1M --engine-use-ray --worker-use-ray --max-num-batched-tokens 5120

To use features such as tensor parallelism on consumer GPUs, API keys and logging, follow the OpenChat documentation.