-
-
Notifications
You must be signed in to change notification settings - Fork 264
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Gemini client #1953
Gemini client #1953
Conversation
Thanks for the contribution 👍 I'll take a look at the PR! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The client is working properly however it doesn't integrate with the Giskard (scan, raget, ...).
The ChatMessage
that are sent as input to the LLM client can have 3 roles (system
, assistant
and user
).
In our case assistant
should be mapped to model
. For system we will need to map it to user
with a custom format the that the model understand the importance of the system prompt: https://www.googlecloudcommunity.com/gc/AI-ML/Implementing-System-Prompts-in-Gemini-Pro-for-Chatbot-Creation/m-p/715501
let me know if you need help on this. You can take a look on our Bedrock client implementation.
PS: ideally this code snippet should be working:
client = GeminiClient()
response = client.complete(messages=[
ChatMessage(role='system', content='You are a "ping" service that always reply with "IT WORKS!"'),
ChatMessage(role='user', content='Hello, does it works?'),
ChatMessage(role='assistant', content='IT WORKS!'),
ChatMessage(role='user', content='What is your goal?'),
])
assert response.role == 'assistant' # model should be mapped to assistant in the response too
assert response.role == 'IT WORKS!'
# Conflicts: # pdm.lock # tests/llm/test_llm_client.py
Description
Implement a Gemini LLMClient and test for it.
Related Issue
Implement a Gemini LLMClient #1901
Type of Change
Checklist
CODE_OF_CONDUCT.md
document.CONTRIBUTING.md
guide.pdm.lock
runningpdm update-lock
(only applicable whenpyproject.toml
has beenmodified)