I’m integrating models from OpenAI, Anthropic’s Claude, Grok, and DeepSeek, and I’m stuck on one specific part: tool calling. The models can all work with tools in some form, but their request and response formats aren’t identical. What’s a sensible way to keep my app’s behavior consistent without pretending every provider supports the same things?
Separate the tool’s meaning from its API format
The useful boundary is between what a tool does and how a particular provider describes it. Your application might expose a tool called search_orders with a few arguments. That definition is part of your product; the JSON structure used to register it with a model provider is an adapter detail.
Keep your own canonical tool-call format inside the application. It could represent a call with a tool name and validated arguments, without depending on any provider’s field names or message structure. Then translate that internal representation to and from each provider’s API format at the edge of your system.
Keep execution out of the provider adapter
A provider adapter should translate requests and responses. It should not decide whether the model is allowed to run a particular action. Pass any returned call to a shared tool execution layer that checks the tool name, validates the arguments, enforces permissions, and invokes the application function. Send the result back through the appropriate provider adapter as a tool result.
This distinction matters for safety. A model’s request to run a tool is not authorization to do it. For example, an assistant suggesting a refund should not cause a refund unless your application checks the user’s permissions and any required approval rules.
Don’t assume feature parity
Provider APIs evolve, and similar-looking features may have different constraints. One provider may support parallel calls; another integration may only handle one call at a time. Tool choice controls, streaming events, argument validation, and built-in tools can also differ. Keep a provider capability matrix that records what your integration actually supports, and check the current API documentation before relying on a feature.
Perplexity is a useful reminder that “tools” can mean different things. A provider may offer its own search or browsing behavior, while your application may be registering custom functions. Those are not automatically interchangeable. Decide whether you need an application-owned tool, a provider-managed capability, or both, and represent that distinction clearly in your code.
Make failures ordinary, not mysterious
- Validate tool names and arguments before execution. Treat malformed or unknown calls as recoverable errors rather than passing them blindly to a function.
- Give every call a stable internal identifier so the result can be matched to the request, even if provider response formats differ.
- Handle timeouts, tool errors, and model refusals separately. A failed tool is not the same as a failed model request.
- Consider parallel tool calls only when the tools are independent and your application can safely manage concurrent execution.
- Store a provider-neutral record of each call and result. This makes debugging easier without requiring your logs to mirror one vendor’s message format.
When sending results back, preserve the conversation sequence the provider expects. The model generally needs to see which call produced which result, so don’t discard call identifiers or turn a tool response into an ordinary assistant message. Your adapter can translate the internal result into the right tool result messages for that API.
Where does prompt design fit?
Prompts should describe when a tool is useful and what information the model should provide. They shouldn’t be the only place you enforce permissions, required fields, or business rules. Those belong in application logic and tool-call validation. If a call is rejected, return a short, useful error to the model when the provider supports that flow; otherwise, handle the failure in the user-facing response.
So I’d avoid trying to make every provider’s tool API look identical. I’d normalize the parts my application owns, translate at the boundaries, and make unsupported capabilities explicit. For anyone who has shipped this across multiple providers: which differences caused the most trouble in practice—streaming, parallel calls, or matching tool results back to their calls?
