Programmable Tool Confirmation

[Stebalien]

It would be nice if tool confirmation were a bit more programmable.

In my specific case:

1. I'd like the tool to be able to provide a description of what the tool is doing rather than displaying the raw call description. This especially matters when some of the call arguments are large.
2. I'd like to be able to ask for confirmation for risky operations and/or the first time some operation is performed on a buffer. E.g., when the LLM tries to edit a buffer in its context the first time, I want to confirm that tool call to give the LLM permission to edit the buffer, but I don't want to confirm every single call.

The simplest solution, IMO, is to allow the tool's :confirm parameter to be a synchronous function that:

1. Takes the same parameters as the tool.
2. Returns one of allow, deny (maybe with an optional reason?), ask, or a string (to be displayed in the buffer as the tool confirmation).

A more flexible alternative would be to have the tool call some kind of gptel-ask-confirmation function, but that starts turning into callback hell pretty quickly.

[karthink]

See #968. This is on the docket, but I need to set some time aside to figure out the best way to do it.

[Stebalien]

Ah, awesome! I searched the open issues but not the open PRs.

[karthink]

An update: gptel now provides the hooks gptel-pre-tool-call-functions and gptel-post-tool-call-functions for more fine-grained control. You can also edit tool calls interactively when using gptel-send.

Between the three options, you can inspect and act on tool calls and tool results, modify arguments, short-circuit the call and provide results yourself, modify the result of the tool call, block tool calls and supply the LLM with a reason, or stop the request entirely. See the documentation of the hooks. (If you want more information see commit messages from a080e68 to af0d39f.)

There are two missing features:

• You can't erase tool calls on your turn once the LLM makes them, although see commit 7bd4bc9
• and you cannot edit tool call results interactively from the gptel buffer, you have to use one of the two hooks.