59 lines
1.4 KiB
Markdown
59 lines
1.4 KiB
Markdown
# Qwen2-VL deployment instances
|
|
|
|
## dependents
|
|
git+https://git.kaiyuancloud.cn/yumoqing/apppublic
|
|
git+https://git.kaiyuancloud.cn/yumoqing/ahserver
|
|
|
|
## preinstallation
|
|
first, create a new python virtual env
|
|
```
|
|
python3 -m venv ~/vl
|
|
```
|
|
create two shell scripts named vlpy and vlpip:
|
|
```vlpy
|
|
#!/usr/bin/bash
|
|
~/vl/bin/python $*
|
|
```
|
|
and
|
|
```vlpip
|
|
#!/usr/bin/bash
|
|
~/vl/bin/pip $*
|
|
```
|
|
and copy them to the bin under you $HOME folder, and chmod +x to them
|
|
```
|
|
mv vlpip vlpy ~/bin
|
|
chmod +x ~/bin/vl*
|
|
```
|
|
|
|
## isntallation
|
|
|
|
follow instuctions from [Qwen2-VL](https://github.com/QwenLM/Qwen2-VL), remember to change pip to vlpip
|
|
|
|
do the following
|
|
```
|
|
vlpip install git+https://github.com/huggingface/transformers@21fac7abba2a37fae86106f87fcf9974fd1e3830 accelerate
|
|
pip install qwen-vl-utils[decord]
|
|
|
|
git clone https://git.kaiyauncloud.cn/yumoqing/qwenvl
|
|
cd qwenvl/script
|
|
sudo isntall.sh
|
|
```
|
|
|
|
## Change model or http port
|
|
there is a config.json file under qwenvl folder, change the "modelname" and "port" value to suite your requirements
|
|
|
|
## model to use
|
|
|
|
* Qwen/Qwen2-VL-7B-Instruct-AWQ
|
|
* Qwen/Qwen2-VL-7B-Instruct
|
|
* Qwen/Qwen2-VL-7B-Instruct-GPTQ-Int4
|
|
* Qwen/Qwen2-VL-7B-Instruct-GPTQ-Int8
|
|
* Qwen/Qwen2-VL-72B-Instruct
|
|
* Qwen/Qwen2-VL-72B-Instruct-AWQ
|
|
* Qwen/Qwen2-VL-72B-Instruct-GPTQ-Int4
|
|
* Qwen/Qwen2-VL-72B-Instruct-GPTQ-Int8
|
|
* Qwen/Qwen2-VL-2B-Instruct
|
|
* Qwen/Qwen2-VL-2B-Instruct-AWQ
|
|
* Qwen/Qwen2-VL-2B-Instruct-GPTQ-Int4
|
|
* Qwen/Qwen2-VL-2B-Instruct-GPTQ-Int8
|