Commit fa5fa07
feat(model): support FunctionGemma-270m-it for local inference (#1279)
FunctionGemma is a specialized variant of Gemma 3 270M optimized for
function calling.
However, it introduces a unique formatting convention for tool
definitions and calls that is incompatible with the existing
`GemmaHandler`.
This commit adds local support for FunctionGemma-270m-it by:
- Added `FunctionGemmaHandler` for local inference.
- Implemented specific prompt formatting that matches the official
documentation.
References:
-
https://ai.google.dev/gemma/docs/functiongemma/formatting-and-best-practices#base-prompt-structure
-
https://ai.google.dev/gemma/docs/functiongemma/model_card#benchmark-results
Co-authored-by: Huanzhi Mao <huanzhimao@gmail.com>1 parent 11fc254 commit fa5fa07
File tree
4 files changed
+541
-0
lines changed- berkeley-function-call-leaderboard
- bfcl_eval
- constants
- model_handler/local_inference
4 files changed
+541
-0
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
50 | 50 | | |
51 | 51 | | |
52 | 52 | | |
| 53 | + | |
53 | 54 | | |
54 | 55 | | |
55 | 56 | | |
| |||
Lines changed: 13 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
39 | 39 | | |
40 | 40 | | |
41 | 41 | | |
| 42 | + | |
42 | 43 | | |
43 | 44 | | |
44 | 45 | | |
| |||
1247 | 1248 | | |
1248 | 1249 | | |
1249 | 1250 | | |
| 1251 | + | |
| 1252 | + | |
| 1253 | + | |
| 1254 | + | |
| 1255 | + | |
| 1256 | + | |
| 1257 | + | |
| 1258 | + | |
| 1259 | + | |
| 1260 | + | |
| 1261 | + | |
| 1262 | + | |
1250 | 1263 | | |
1251 | 1264 | | |
1252 | 1265 | | |
| |||
Lines changed: 1 addition & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
107 | 107 | | |
108 | 108 | | |
109 | 109 | | |
| 110 | + | |
110 | 111 | | |
111 | 112 | | |
112 | 113 | | |
| |||
0 commit comments