Skip to content

Add custom Vertex AI Model Garden example #2868

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Sep 30, 2025
Merged

Add custom Vertex AI Model Garden example #2868

Changes from 5 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
e6221ae
Add custom Vertex AI Model Garden example
stefannae Sep 11, 2025
b0f17a2
Add Model Garden link
stefannae Sep 24, 2025
5841e76
Remove custom client
stefannae Sep 24, 2025
67c1b97
Clarify model access for generateContent API
stefannae Sep 24, 2025
acdcf14
Use the simplest model name pattern
stefannae Sep 30, 2025
fa8cf90
Add & use the publisher/model name pattern
stefannae Sep 30, 2025
b0977bf
Update docs/models/google.md
DouweM Sep 30, 2025
f2cfb6e
Update docs/models/google.md
DouweM Sep 30, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 25 additions & 0 deletions docs/models/google.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,31 @@ agent = Agent(model)
...
```

#### Customizing Model

You can access models from the [Model Garden](https://cloud.google.com/model-garden?hl=en) that support the generateContent API and are available under your GCP project, including but not limited to Gemini, using one of the following `model_name` patterns: `{model_id}`, `publishers/{publisher}/models/{model_id}`, or `projects/{project}/locations/{location}/publishers/{publisher}/models/{model_id}`.

```python
from google.oauth2 import service_account

from pydantic_ai import Agent
from pydantic_ai.models.google import GoogleModel
from pydantic_ai.providers.google import GoogleProvider

credentials = service_account.Credentials.from_service_account_file(
'path/to/service-account.json',
scopes=['https://www.googleapis.com/auth/cloud-platform'],
)
provider = GoogleProvider(
credentials=credentials,
project='your-gcp-project-id',
location='us-central1', # the region where the model is available
)
model = GoogleModel('llama-3.3-70b-instruct-maas', provider=provider)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From googleapis.github.io/python-genai/genai.html#genai.models.AsyncModels.generate_content_stream, it seems like this should be meta/llama-3.3-70b-instruct-maas, so we should fix it here and specify {publisher}/{model_id} in the docs above.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I missed this 4th way of specifying the model. Still, the simplest way, according to the same docs (and tested - I've tested them all), is llama-3.3-70b-instruct-maas.

Should we keep llama-3.3-70b-instruct-maas in the example and add {publisher}/{model_id} to the list of patterns?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@stefannae Hmm, the docs make it sound like excluding the publisher only works with Gemini models, so even though you've verified it works with llama as well, I'd prefer to document and use only {publisher}/{model_id}, or say "{model_id} for Gemini models" like the docs do.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@DouweM, they have a formatting issue with the docs, which visually reads better from the docstring https://github.com/googleapis/python-genai/blob/538c755e84777e6b76d5152aac18268e4d0c99c6/google/genai/models.py#L6377-L6401

This way, seems Gemini is only used as an example:

For the `model` parameter, supported formats for Vertex AI API include:
- The Gemini model ID, for example: 'gemini-2.0-flash'
- The full resource name starts with 'projects/', for example:
  'projects/my-project-id/locations/us-central1/publishers/google/models/gemini-2.0-flash'
- The partial resource name with 'publishers/', for example:
  'publishers/google/models/gemini-2.0-flash' or
- `/` separated publisher and model name, for example:
  'google/gemini-2.0-flash'

For the `model` parameter, supported formats for Gemini API include:
- The Gemini model ID, for example: 'gemini-2.0-flash'
- The model name starts with 'models/', for example:
  'models/gemini-2.0-flash'
- For tuned models, the model name starts with 'tunedModels/',
  for example:
  'tunedModels/1234567890123456789'

Though given how the endpoints are defined in Vertex AI, it is probably better to switch to {publisher}/{model_id} in the example. Let me know your preference, and I'll make the change.

Screenshot from 2025-09-30 18-42-07

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@stefannae Yeah let's do that, I'm interpreting "Gemini model ID" as "this only works for Gemini models"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@DouweM ok, pushed the update!

agent = Agent(model)
...
```

## Provider Argument

You can supply a custom `GoogleProvider` instance using the `provider` argument to configure advanced client options, such as setting a custom `base_url`.
Expand Down
Loading