This is the code and data repo of paper HDFlow: Enhancing LLM Complex Problem-Solving with Hybrid Thinking and Dynamic Workflows. If you think this repo is useful, please cite us!
@article{yao2024hdflow,
title={HDFlow: Enhancing LLM Complex Problem-Solving with Hybrid Thinking and Dynamic Workflows},
author={Yao, Wenlin and Mi, Haitao and Yu, Dong},
journal={arXiv preprint arXiv:2409.17433},
year={2024}
}
Download our model into the dir /xxxx/checkpoints/wenlinyao/HDFlow-Llama-3-8B-Instruct/. Next, use vllm to host the model service. Here provides an example to host two models at the same time (using different ports).
CUDA_VISIBLE_DEVICES=0 python -m vllm.entrypoints.openai.api_server --model /xxxx/checkpoints/wenlinyao/HDFlow-Llama-3-8B-Instruct/ --tensor-parallel-size 1 --max-num-batched-tokens 8192 --dtype bfloat16 --port 4231
Refer to https://github.com/wenlinyao/HDFlow for more details.
- Downloads last month
- 6