# Qwen2-VL deployment instances

## dependents
git+https://git.kaiyuancloud.cn/yumoqing/apppublic
git+https://git.kaiyuancloud.cn/yumoqing/ahserver

## preinstallation
first, create a new python virtual env
```
python3 -m venv ~/vl
```
create two shell scripts named vlpy and vlpip:
```vlpy
#!/usr/bin/bash
~/vl/bin/python $*
```
and 
```vlpip
#!/usr/bin/bash
~/vl/bin/pip $*
```
and copy them to the bin under you $HOME folder, and chmod +x to them
```
mv vlpip vlpy ~/bin
chmod +x ~/bin/vl*
```

## isntallation

follow instuctions from [Qwen2-VL](https://github.com/QwenLM/Qwen2-VL), remember to change pip to vlpip

do the following
```
vlpip install git+https://github.com/huggingface/transformers@21fac7abba2a37fae86106f87fcf9974fd1e3830 accelerate
pip install qwen-vl-utils[decord]

git clone https://git.kaiyauncloud.cn/yumoqing/qwenvl
cd qwenvl/script
sudo isntall.sh
```

## Change model or http port
there is a config.json file under qwenvl folder, change the "modelname" and "port" value to suite your requirements

## model to use

* Qwen/Qwen2-VL-7B-Instruct-AWQ
* Qwen/Qwen2-VL-7B-Instruct
* Qwen/Qwen2-VL-7B-Instruct-GPTQ-Int4
* Qwen/Qwen2-VL-7B-Instruct-GPTQ-Int8
* Qwen/Qwen2-VL-72B-Instruct
* Qwen/Qwen2-VL-72B-Instruct-AWQ
* Qwen/Qwen2-VL-72B-Instruct-GPTQ-Int4
* Qwen/Qwen2-VL-72B-Instruct-GPTQ-Int8
* Qwen/Qwen2-VL-2B-Instruct
* Qwen/Qwen2-VL-2B-Instruct-AWQ
* Qwen/Qwen2-VL-2B-Instruct-GPTQ-Int4
* Qwen/Qwen2-VL-2B-Instruct-GPTQ-Int8