You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
Currently several other frameworks support tool calling for ollama via a "wrapper", which I think has led this to become an expected feature. As of today, we rely on model api support for whether or not we can enable tool calling. We already retrofit functionality such as "context" and "system prompts" via middleware.
Describe the solution you'd like
I have not fully thought this out, but we could add a middleware that would roughly do the following:
Append instructions to the prompt that include tool definitions
Append instructions to the prompt that specify the expected json output for a tool call request
Translate json tool call requests into genkit tool call requests
Tool calling should work as usual?
Translate tool call responses back into "user" messages
MichaelDoyle
changed the title
[FR] Support tool calling for any arbitrary model
[FR] Support tool calling for any arbitrary (ollama?) model
Jul 22, 2024
Is your feature request related to a problem? Please describe.
Currently several other frameworks support tool calling for ollama via a "wrapper", which I think has led this to become an expected feature. As of today, we rely on model api support for whether or not we can enable tool calling. We already retrofit functionality such as "context" and "system prompts" via middleware.
Describe the solution you'd like
I have not fully thought this out, but we could add a middleware that would roughly do the following:
Describe alternatives you've considered
Additional context
See also: #661
Notes: Such a feature is largely dependent on models that can reliably produce json output.
The text was updated successfully, but these errors were encountered: