-
Notifications
You must be signed in to change notification settings - Fork 4.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Bug]: After registering the tool for the agent, no matter what information I input, it will always call the tool. #2893
Comments
I'm experiencing the same issue. Any update? |
Please see issue #2953. |
Besides other fix or improvement in autogen side, may try with adding "...Do not use function unless it is asked" or ""...Do not use tool unless it is asked" in assistant sys message. |
I tried it, but it doesn't seem to work. Even if I say “do not use tools”, it still tries to call it. |
Only using the GPT API seems fine. |
I wanted to add some background on LiteLLM when using 'ollama_chat' for the model. LiteLLM will add instruction onto the end of your agent's system message instructing it to only return JSON in a function call format. So this is a big reason why it's difficult to get it to not return a function call. The text it adds in is: Here's output from litellm (using the
I'll keep experimenting with prompts and perhaps trying |
Nice find! Have you figured out any possible way we get around this? |
Hey @Noir97 - well, yes, somewhat. I'm working on new PRs, which will be for a LiteLLM client and an Ollama client. I'm not sure if I'll submit the LiteLLM one as I've built it to only work with Ollama anyway, so may just go with the Ollama native one. In these branches I've created a "manual" tool calling approach, whereby the format for the model is no longer set to "JSON" and instead guidance is injected into the prompts and at various stages of tool calling to guide the LLM to return JSON compatible text or just text. Then, I parse the text to see if it is a function call compatible JSON and convert that into a tool call. It is working well, in that I can get it to send through the JSON for the tool call, read the tool call execution and comment on it with normal text. I feel the current LiteLLM function calling implementation is impossible to get around because it injects the very strict JSON format and also passes a parameter for inference to get a JSON response (though the LLM may still return a non-JSON response). My working LiteLLM branch is here. I think it's close enough to being able to be used. If you wanted to test it you could try it out. You could try this program (change the IP address and model):
As mentioned, I'll probably move forward with the Ollama one as I'm not sure the LiteLLM provides any benefit as it's built specifically to work with Ollama (rather than be more generalised, which may be the way to go with it). My Ollama one works in exactly the same way except it will be hardcoded to the manual function parsing because Ollama doesn't support tool calling. |
Hey @Noir97 and anyone who's able to run Ollama, please check my draft PR #3056, I've created an Ollama Client class that has tool calling support (preliminary). It will require a lot of testing so if you get a chance to try it out that would be great. I'll add comments to the PR with results of my tests across different models and test programs. |
I'll test it out in my scenario. Much appreciated for the work and effort. |
Describe the bug
I used LLaMA 3 with ollama and LiteLLM.
After registering the tool for the agent, no matter what information I input, it will always call the tool.
Steps to reproduce
Model Used
llama3
Expected Behavior
The agent should only call the tool when I want it to.
Screenshots and logs
Additional Information
I tried various models of Ollama, and this issue occurred with all of them.
The text was updated successfully, but these errors were encountered: