[Question]: Adding safety settings to vertex ai

[Prem-Nitin]

Question Validation

• I have searched both the documentation and discord for an answer.

Question

I am trying to add safety settings to vertex ai models.

safety_settings={
                    generative_models.HarmCategory.HARM_CATEGORY_HATE_SPEECH: generative_models.HarmBlockThreshold.BLOCK_ONLY_HIGH,
                    generative_models.HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT: generative_models.HarmBlockThreshold.BLOCK_ONLY_HIGH,
                    generative_models.HarmCategory.HARM_CATEGORY_SEXUALLY_EXPLICIT: generative_models.HarmBlockThreshold.BLOCK_ONLY_HIGH,
                    generative_models.HarmCategory.HARM_CATEGORY_HARASSMENT: generative_models.HarmBlockThreshold.BLOCK_ONLY_HIGH,
                }

when I pass the above settings I get the following error.

pydantic.v1.errors.ConfigError: field "safety_settings_List[ForwardRef('SafetySetting')]" not yet prepared so type is still a ForwardRef, you might need to call Vertex.update_forward_refs().

Then whats the correct format to pass the safety settings

[dosubot]

The correct format to pass the safety settings to Vertex AI models is to use the SafetySettingsType from the vertexai.generative_models._generative_models module. You should pass it as an optional parameter named safety_settings when creating an instance of the Vertex class.

Here is an example of how to do it:

from vertexai.generative_models._generative_models import SafetySettingsType
from llama_index.llms.vertex import Vertex

# Define your safety settings
safety_settings = SafetySettingsType(
    # Add your specific safety settings here
)

# Set up necessary variables
credentials = {
    "project_id": "INSERT_PROJECT_ID",
    "api_key": "INSERT_API_KEY",
}

# Create an instance of the Vertex class with safety settings
llm = Vertex(
    model="text-bison",
    project=credentials["project_id"],
    credentials=credentials,
    safety_settings=safety_settings
)

# Access the complete method from the instance
response = llm.complete("Hello world!")
print(str(response))

This ensures that the safety_settings parameter is correctly passed and avoids the pydantic.v1.errors.ConfigError ^[1].

Details

About Dosu
This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

To continue the conversation, mention @dosu.

[logan-markewich]

As the error says, update forward refs needs to be called, so the class has to be updated

The workaround would be calling it yourself before using it

Vertex.update_forward_refs()

llm = Vertex(....)

[Prem-Nitin]

@dosu-bot, @logan-markewich

safety_settings={
                HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT: HarmBlockThreshold.BLOCK_ONLY_HIGH,
                HarmCategory.HARM_CATEGORY_HATE_SPEECH: HarmBlockThreshold.BLOCK_MEDIUM_AND_ABOVE,
                HarmCategory.HARM_CATEGORY_HARASSMENT: HarmBlockThreshold.BLOCK_LOW_AND_ABOVE,
                HarmCategory.HARM_CATEGORY_SEXUALLY_EXPLICIT: HarmBlockThreshold.BLOCK_NONE,
            }

Vertex.update_forward_refs()
llm = Vertex(
    model="gemini-1.5-pro-preview-0514", 
    safety_settings=safety_settings, 
    project=credentials.project_id, 
    credentials=credentials,
    temperature = 0.3,
    max_tokens = 8192,
)

NameError: name 'SafetySetting' is not defined. Did you mean: 'SafetySettingsType'?

[dosubot]

To resolve the NameError: name 'SafetySetting' is not defined error, you need to ensure that SafetySetting is properly defined or imported. However, in your case, it seems you should be using SafetySettingsType instead.

Here's how you can correctly define and pass the safety settings:

1. Import the necessary classes:

from vertexai.preview.language_models import HarmCategory, HarmBlockThreshold
from vertexai.generative_models._generative_models import SafetySettingsType
from llama_index.llms.vertex import Vertex

2. Define your safety settings using SafetySettingsType:

safety_settings = SafetySettingsType(
    {
        HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT: HarmBlockThreshold.BLOCK_ONLY_HIGH,
        HarmCategory.HARM_CATEGORY_HATE_SPEECH: HarmBlockThreshold.BLOCK_MEDIUM_AND_ABOVE,
        HarmCategory.HARM_CATEGORY_HARASSMENT: HarmBlockThreshold.BLOCK_LOW_AND_ABOVE,
        HarmCategory.HARM_CATEGORY_SEXUALLY_EXPLICIT: HarmBlockThreshold.BLOCK_NONE,
    }
)

3. Call update_forward_refs and create the Vertex instance:

Vertex.update_forward_refs()

llm = Vertex(
    model="gemini-1.5-pro-preview-0514",
    safety_settings=safety_settings,
    project=credentials.project_id,
    credentials=credentials,
    temperature=0.3,
    max_tokens=8192,
)

This should resolve the NameError and correctly pass the safety settings to your Vertex AI model ^[1].

To continue talking to Dosu, mention @dosu.