Skip to content

Add support for safety_settings in google LLM constructor #3733

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed

Add support for safety_settings in google LLM constructor #3733

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions livekit-plugins/livekit-plugins-google/livekit/plugins/google/llm.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,7 @@ class _LLMOptions:
gemini_tools: NotGivenOr[list[_LLMTool]]
http_options: NotGivenOr[types.HttpOptions]
seed: NotGivenOr[int]
safety_settings: NotGivenOr[list[types.SafetySettingOrDict] | dict[str, str]]


class LLM(llm.LLM):
Expand All @@ -88,6 +89,7 @@ def __init__(
gemini_tools: NotGivenOr[list[_LLMTool]] = NOT_GIVEN,
http_options: NotGivenOr[types.HttpOptions] = NOT_GIVEN,
seed: NotGivenOr[int] = NOT_GIVEN,
safety_settings: NotGivenOr[list[types.SafetySettingOrDict] | dict[str, str]] = NOT_GIVEN,
Copy link
Contributor

@tinalenguyen tinalenguyen Oct 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be best to only accept SafetySettingOrDict types for this parameter. If you want to support accepting dict[str, str] then you would need to cast it to SafetySetting anyway to pass to GenerateContentConfig

Suggested change
safety_settings: NotGivenOr[list[types.SafetySettingOrDict] | dict[str, str]] = NOT_GIVEN,
safety_settings: NotGivenOr[list[types.SafetySettingOrDict]] = NOT_GIVEN,

) -> None:
"""
Create a new instance of Google GenAI LLM.
Expand Down Expand Up @@ -116,6 +118,14 @@ def __init__(
automatic_function_calling_config (AutomaticFunctionCallingConfigOrDict, optional): The automatic function calling configuration for response generation. Defaults to None.
gemini_tools (list[LLMTool], optional): The Gemini-specific tools to use for the session.
http_options (HttpOptions, optional): The HTTP options to use for the session.
seed (int, optional): Random seed for reproducible generation. Defaults to None.
safety_settings (list[SafetySettingOrDict] | dict[str, str], optional): Safety settings for content filtering. Defaults to None.
Can be specified as:
- Dict mapping harm categories to thresholds (e.g., {"HARM_CATEGORY_HATE_SPEECH": "BLOCK_LOW_AND_ABOVE"})
- List of SafetySetting objects from google.genai.types
Available categories: HARM_CATEGORY_HARASSMENT, HARM_CATEGORY_HATE_SPEECH, HARM_CATEGORY_SEXUALLY_EXPLICIT, HARM_CATEGORY_DANGEROUS_CONTENT
Available thresholds: BLOCK_NONE, BLOCK_ONLY_HIGH, BLOCK_MEDIUM_AND_ABOVE, BLOCK_LOW_AND_ABOVE

""" # noqa: E501
super().__init__()
gcp_project = project if is_given(project) else os.environ.get("GOOGLE_CLOUD_PROJECT")
Expand Down Expand Up @@ -179,6 +189,7 @@ def __init__(
gemini_tools=gemini_tools,
http_options=http_options,
seed=seed,
safety_settings=safety_settings,
)
self._client = Client(
api_key=gemini_api_key,
Expand Down Expand Up @@ -286,6 +297,9 @@ def chat(
if is_given(self._opts.automatic_function_calling_config):
extra["automatic_function_calling"] = self._opts.automatic_function_calling_config

if is_given(self._opts.safety_settings):
extra["safety_settings"] = self._opts.safety_settings

gemini_tools = gemini_tools if is_given(gemini_tools) else self._opts.gemini_tools

return LLMStream(
Expand Down