ICTNLP

/

Llama-3.1-8B-Omni

like 367

Follow

Natural Language Processing Group, Institute of Computing Technology, Chinese Academy of Science 25

It needs ~20GB of VRAM. We tried inference on the RTX 3090 and it worked well.

RTX 3070 TI laptop with 8gigs is not working while running the python -m omni_speech.serve.model_worker --host 0.0.0.0 --controller http://localhost:10000 --port 40000 --worker http://localhost:40000 --model-path Llama-3.1-8B-Omni --model-name Llama-3.1-8B-Omni --s2s