Skip to content

Commit 6aaa75a

Browse files
Updated NPI files gemma3-27B (#567)
Updated NPI files to get good output in AIC. Signed-off-by: Ann Kuruvilla <[email protected]>
1 parent 4c68412 commit 6aaa75a

File tree

6 files changed

+480
-1343
lines changed

6 files changed

+480
-1343
lines changed

examples/gemma3_example/fp32_nodes_gemma3_27b_image.yaml renamed to examples/gemma3_example/fp32_nodes_gemma3_27b.yaml

Lines changed: 475 additions & 289 deletions
Large diffs are not rendered by default.

examples/gemma3_example/fp32_nodes_gemma3_27b_text.yaml

Lines changed: 0 additions & 500 deletions
This file was deleted.
File renamed without changes.

examples/gemma3_example/fp32_nodes_gemma3_4b_image.yaml

Lines changed: 0 additions & 275 deletions
This file was deleted.

examples/gemma3_example/fp32_nodes_gemma3_4b_text.yaml

Lines changed: 0 additions & 275 deletions
This file was deleted.

examples/gemma3_example/gemma3_mm.py

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@
1111

1212
from QEfficient import QEFFAutoModelForImageTextToText
1313

14+
# Change model_id to "google/gemma-3-27b-it" for 27B model
1415
model_id = "google/gemma-3-4b-it"
1516
config = AutoConfig.from_pretrained(model_id)
1617
# For Testing Purpose Only
@@ -25,8 +26,8 @@
2526
model_id, config=config, attn_implementation="eager", kv_offload=True
2627
)
2728

28-
### use skip_vision=Ture, if want to run only text, or false ###
29-
skip_vision = True
29+
### use skip_vision=True, if want to run only text, or false ###
30+
skip_vision = False
3031

3132
if skip_vision:
3233
## Only Text ##
@@ -41,7 +42,7 @@
4142
aic_enable_depth_first=True,
4243
skip_vision=True,
4344
mos=1,
44-
node_precision_info="examples/gemma3_example/fp32_mm.yaml",
45+
node_precision_info="examples/gemma3_example/fp32_nodes_gemma3_4b.yaml", # Change to fp32_nodes_gemma3_27b.yaml for 27B model
4546
)
4647

4748
messages = [
@@ -77,7 +78,7 @@
7778
mxint8_kv_cache=False,
7879
aic_enable_depth_first=True,
7980
mos=1,
80-
node_precision_info="examples/gemma3_example/fp32_mm.yaml",
81+
node_precision_info="examples/gemma3_example/fp32_nodes_gemma3_4b.yaml", # Change to fp32_nodes_gemma3_27b.yaml for 27B model
8182
)
8283

8384
### IMAGE + TEXT ###

0 commit comments

Comments
 (0)