Update README.md
Browse files
README.md
CHANGED
@@ -68,7 +68,13 @@ pip install -e ".[train]"
|
|
68 |
### B. Inference
|
69 |
|
70 |
```bash
|
71 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
72 |
|
73 |
# example:
|
74 |
# >> Enter 'exit' to end the conversation, 'reset' to clear the chat history.
|
@@ -77,6 +83,9 @@ CUDA_VISIBLE_DEVICES=0 python infer.py --model_dir /path/to/your/model
|
|
77 |
# >> Assistant: The image features a stylized representation of a cat, characterized by its vibrant and abstract depiction.
|
78 |
# >> User: What color is this cat?
|
79 |
# >> Assistant: The cat in the image is primarily white with blue, orange and pink accents, creating a visually appealing and unique appearance.
|
|
|
|
|
|
|
80 |
```
|
81 |
|
82 |
### C. Evaluation for Embodied Ability
|
@@ -132,7 +141,17 @@ bash scripts/eval/eval_robo.sh /path/to/your/model
|
|
132 |
Install the evaluation tool and execute the evaluation script:
|
133 |
```bash
|
134 |
pip install lmms-eval==0.2.0
|
135 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
136 |
```
|
137 |
|
138 |
We would like to express our gratitude to [Huajie Tan](https://huggingface.co/tanhuajie2001), [Yumeng Wang](https://huggingface.co/devymex), [Yin Xie](https://huggingface.co/Yin-Xie) for his significant contributions to the experimental validation in MLLMs.
|
|
|
68 |
### B. Inference
|
69 |
|
70 |
```bash
|
71 |
+
git clone https://github.com/deepglint/unicom
|
72 |
+
cd unicom
|
73 |
+
pip install --upgrade pip
|
74 |
+
pip install -e ".[train]"
|
75 |
+
pip install flash-attn --no-build-isolation
|
76 |
+
|
77 |
+
CUDA_VISIBLE_DEVICES=0 python infer.py --model_dir DeepGlint-AI/MLCD-Embodied-7B
|
78 |
|
79 |
# example:
|
80 |
# >> Enter 'exit' to end the conversation, 'reset' to clear the chat history.
|
|
|
83 |
# >> Assistant: The image features a stylized representation of a cat, characterized by its vibrant and abstract depiction.
|
84 |
# >> User: What color is this cat?
|
85 |
# >> Assistant: The cat in the image is primarily white with blue, orange and pink accents, creating a visually appealing and unique appearance.
|
86 |
+
# >> User: <image>请你介绍一下这个图片
|
87 |
+
# >> Assistant: 这是一幅充满创意的猫头艺术作品。它采用了多色渐变和抽象风格,将猫的头部描绘成一个充满活力和色彩的视觉冲击。猫的眼睛用金色渲染,显得非常有神采,
|
88 |
+
# 而粉色的鼻子则增添了一丝可爱感。整体设计融合了现代艺术与传统猫头图案,创造出一种既独特又引人入胜的视觉效果。。
|
89 |
```
|
90 |
|
91 |
### C. Evaluation for Embodied Ability
|
|
|
141 |
Install the evaluation tool and execute the evaluation script:
|
142 |
```bash
|
143 |
pip install lmms-eval==0.2.0
|
144 |
+
PYTHONPATH=./ CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 python -m accelerate.commands.launch \
|
145 |
+
--main_process_port=12444 \
|
146 |
+
--num_processes=8 \
|
147 |
+
-m lmms_eval \
|
148 |
+
--model llava \
|
149 |
+
--model_args pretrained=DeepGlint-AI/MLCD-Embodied-7B,conv_template=qwen_1_5 \
|
150 |
+
--tasks mme \
|
151 |
+
--batch_size 1 \
|
152 |
+
--log_samples \
|
153 |
+
--log_samples_suffix mlcd \
|
154 |
+
--output_path ./eval_log/
|
155 |
```
|
156 |
|
157 |
We would like to express our gratitude to [Huajie Tan](https://huggingface.co/tanhuajie2001), [Yumeng Wang](https://huggingface.co/devymex), [Yin Xie](https://huggingface.co/Yin-Xie) for his significant contributions to the experimental validation in MLLMs.
|