Skip to content

Commit 3aa6248

Browse files
Update README.md
1 parent 3301ba7 commit 3aa6248

1 file changed

Lines changed: 41 additions & 4 deletions

File tree

sonic-o1/05_evaluation_inference/README.md

Lines changed: 41 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -10,14 +10,24 @@ This directory handles model evaluation and inference for video question-answeri
1010
05_evaluation_inference/
1111
├── run_evaluation.py # Main evaluation pipeline orchestrator
1212
├── README.md # This file
13-
13+
├── external_repos/ # External model repositories
14+
│ ├── Baichuan-Omni-1.5/ # Baichuan-Omni 1.5 repository
15+
│ ├── Ola/ # OLA omni model repository
16+
│ ├── omnivinci/ # OmniVinci repository
17+
│ ├── Uni-MoE/ # Uni-MoE repository
18+
│ ├── VideoLLaMA2/ # VideoLLaMA2 repository
19+
│ ├── VITA/ # VITA repository
20+
│ └── README.md # Details on included repos
1421
├── inference/ # Inference execution
1522
│ └── run_inference.py # Standalone inference script
1623
1724
├── models/ # Model implementations
1825
│ ├── base_model.py # Base class for all models
26+
│ ├── baichuan.py # Baichuan-Omni 1.5 model
1927
│ ├── gemini.py # Google Gemini API
2028
│ ├── gpt4o.py # OpenAI GPT-4o API
29+
│ ├── ola.py # OLA omni model
30+
│ ├── omnivinci.py # OmniVinci model
2131
│ ├── qwen3.py # Qwen3 VL model
2232
│ ├── minicpm.py # MiniCPM-V model
2333
│ ├── phi4.py # Phi-4 Vision model
@@ -49,6 +59,9 @@ This directory handles model evaluation and inference for video question-answeri
4959
├── models_config.yaml # Model and evaluation configuration
5060
5161
├── models_requirements/ # Model-specific requirements
62+
│ ├── requirements_baichuan.txt
63+
│ ├── requirements_ola.txt
64+
│ ├── requirements_omnivinci.txt
5265
│ ├── requirements_venv_llama.txt
5366
│ ├── requirements_venv_minicpm.txt
5467
│ ├── requirements_venv_phi4.txt
@@ -108,13 +121,16 @@ The pipeline supports multiple video understanding models:
108121
- **GPT-4o** - OpenAI GPT-4o Vision API
109122

110123
**Open-Source Models:**
111-
- **Qwen3** - Qwen3 VL model
124+
- **Baichuan-Omni 1.5** - Baichuan omnimodal model
112125
- **MiniCPM-V** - MiniCPM Vision model
126+
- **OLA** - OLA omni-modal language model
127+
- **OmniVinci** - NVIDIA OmniVinci model
113128
- **Phi-4** - Phi-4 Vision model
129+
- **Qwen3** - Qwen3 VL model
114130
- **Uni-MoE** - Uni-MoE multimodal model
115131
- **Video-LLaMA** - Video-LLaMA2 model
116132
- **VITA** - VITA video understanding model
117-
133+
-
118134
### Features
119135

120136
- **Multiple Model Support**: Evaluate various open-source and commercial models
@@ -221,6 +237,27 @@ source venv_unimoe/bin/activate
221237
pip install -r models_requirements/requirements_venv_unimoe.txt
222238
```
223239

240+
**Baichuan-Omni 1.5:**
241+
```bash
242+
python -m venv venv_baichuan
243+
source venv_baichuan/bin/activate
244+
pip install -r models_requirements/requirements_baichuan.txt
245+
```
246+
247+
**OLA:**
248+
```bash
249+
python -m venv venv_ola
250+
source venv_ola/bin/activate
251+
pip install -r models_requirements/requirements_ola.txt
252+
```
253+
254+
**OmniVinci:**
255+
```bash
256+
python -m venv venv_omnivinci
257+
source venv_omnivinci/bin/activate
258+
pip install -r models_requirements/requirements_omnivinci.txt
259+
```
260+
224261
**VITA:**
225262
```bash
226263
python -m venv venv_vita
@@ -651,7 +688,7 @@ Or check LLM judge model configuration in `models_config.yaml`.
651688

652689
## Notes
653690

654-
- **Environment Management**: Most models work with the general environment. Only use model-specific environments (in `models_requirements/`) when running models with special requirements (Uni-MoE, VITA, Video-LLaMA, MiniCPM-V, Phi-4).
691+
- **Environment Management**: Most models work with the general environment. Only use model-specific environments (in `models_requirements/`) when running models with special requirements (Baichuan-Omni 1.5, OLA, OmniVinci, Uni-MoE, VITA, Video-LLaMA, MiniCPM-V, Phi-4).
655692

656693
- **Path Resolution**: All scripts expect to be run from the `sonic-o1` parent directory to properly resolve relative imports and data paths.
657694

0 commit comments

Comments
 (0)