qwenvl/README.md

59 lines
1.4 KiB
Markdown
Raw Normal View History

2024-11-12 15:32:14 +08:00
# Qwen2-VL deployment instances
2024-11-12 16:39:19 +08:00
## dependents
2024-11-12 15:32:14 +08:00
git+https://git.kaiyuancloud.cn/yumoqing/apppublic
git+https://git.kaiyuancloud.cn/yumoqing/ahserver
2024-11-12 16:39:19 +08:00
## preinstallation
2024-11-12 15:32:14 +08:00
first, create a new python virtual env
```
python3 -m venv ~/vl
```
create two shell scripts named vlpy and vlpip:
```vlpy
#!/usr/bin/bash
~/vl/bin/python $*
```
and
```vlpip
#!/usr/bin/bash
~/vl/bin/pip $*
```
and copy them to the bin under you $HOME folder, and chmod +x to them
```
mv vlpip vlpy ~/bin
chmod +x ~/bin/vl*
```
## isntallation
2024-11-13 11:22:53 +08:00
follow instuctions from [Qwen2-VL](https://github.com/QwenLM/Qwen2-VL), remember to change pip to vlpip
2024-11-12 15:32:14 +08:00
do the following
```
2024-11-13 11:22:53 +08:00
vlpip install git+https://github.com/huggingface/transformers@21fac7abba2a37fae86106f87fcf9974fd1e3830 accelerate
pip install qwen-vl-utils[decord]
2024-11-12 15:32:14 +08:00
git clone https://git.kaiyauncloud.cn/yumoqing/qwenvl
cd qwenvl/script
sudo isntall.sh
```
## Change model or http port
there is a config.json file under qwenvl folder, change the "modelname" and "port" value to suite your requirements
2024-11-12 16:39:19 +08:00
## model to use
* Qwen/Qwen2-VL-7B-Instruct-AWQ
* Qwen/Qwen2-VL-7B-Instruct
* Qwen/Qwen2-VL-7B-Instruct-GPTQ-Int4
* Qwen/Qwen2-VL-7B-Instruct-GPTQ-Int8
* Qwen/Qwen2-VL-72B-Instruct
* Qwen/Qwen2-VL-72B-Instruct-AWQ
* Qwen/Qwen2-VL-72B-Instruct-GPTQ-Int4
* Qwen/Qwen2-VL-72B-Instruct-GPTQ-Int8
* Qwen/Qwen2-VL-2B-Instruct
* Qwen/Qwen2-VL-2B-Instruct-AWQ
* Qwen/Qwen2-VL-2B-Instruct-GPTQ-Int4
* Qwen/Qwen2-VL-2B-Instruct-GPTQ-Int8