-
Notifications
You must be signed in to change notification settings - Fork 4.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Ollama Client (with tool calling) #3056
Open
marklysze
wants to merge
19
commits into
main
Choose a base branch
from
ollamaclient
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
+1,579
−1
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Basic program:
Add |
Tool calling:
and result:
|
Parallel tool calling (LLM recommends multiple tool calls at a time):
and result:
|
This was referenced Aug 1, 2024
Open
Okay - I think if we can review and try and get this out and available for developers to use and provide feedback that would be great. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
An Ollama client! Run your local models with AutoGen using a dedicated client class.
One of the key features of this library (and which is still very much experimental) is the support for tool calling. This is done "manually" by injecting tools into the prompt and translating between AutoGen's tool call objects and text messages (updated to also support Ollama's native tool calling). The use of tool calling will be described in further detail below but essentially you should be able to get up and running with it as it stands without customising the text injected to support it.
This manual tool calling approach and the actual text injected is an initial attempt to handle tool calling so if you can help improve it, please do!
I'll use the client with some notebooks and local models and summarise the results in another comment.
To run the code, you'll need to install the
ollama
andfix-busted-json
packages (these will be automatically installed when this is merged and you install throughpip install pyautogen[ollama]
):pip install ollama
pip install fix-busted-json
2024-07-27: Updated to include Ollama's native tool calling (just released, v0.3.0 Ollama library)
Related issue numbers
#2893
Checks