I’m building a chat feature that may send one turn to OpenAI and a later turn to Claude, Grok, or DeepSeek—for example, if a provider is unavailable or a user changes models. The request itself is easy to redirect. I’m less sure what to do with the conversation that came before it.
Can I pass the same transcript to the new provider and expect a coherent continuation, or does switching models require some kind of translation? I’m especially curious about tool calls, system instructions, and context limits. What have people found works reliably?
The transcript can travel; the exact behavior cannot
A chat history is not just a list of sentences. It also carries roles, instructions, tool activity, and sometimes provider-specific metadata. Even if two APIs accept messages that look similar, their models may interpret the history differently. A new provider has not seen the earlier turns or any hidden state from the previous request; it only gets what your application sends.
That means you can preserve the visible conversation, but you should not promise that the new model will continue exactly as the old one would have. It may answer in a different style, revisit a settled point, or interpret an instruction differently. That is a model handoff, not a perfectly transparent continuation.
Keep an application-owned conversation record
Rather than treating any provider’s request format as your permanent storage format, keep a canonical message format in your application. Store the role and content for each message, along with structured records for events such as tool requests and results. When calling a provider, convert that record into the format its API expects.
This makes it easier to preserve conversation history between providers without assuming their message formats or features are identical. Keep the original user and assistant text where possible. If a provider response contains data your canonical record cannot represent, decide deliberately whether to retain it as metadata, convert it to a readable note, or omit it. Silently dropping information can make a later answer seem inexplicably confused.
Tool calls need special care
A tool interaction is not merely an assistant message that says “I searched.” It commonly includes a structured request, the tool’s response, and a follow-up assistant message. Preserve that tool-call history as a sequence in your application record.
When changing providers, check whether the destination supports the same tool and whether it expects a compatible call format. If it does not, you may need to summarize the result in ordinary conversation text—for example, “A weather lookup returned 18°C and light rain”—rather than sending an unsupported tool event. Don’t replay a tool call automatically just because the new model cannot see it: an earlier call may have created a ticket, sent a message, or made another change outside the chat.
Reapply instructions and account for limits
Do not assume that a system message from one API is automatically included when you build a request for another. Store your application’s durable instructions separately, then render them into the destination provider’s supported instruction fields. These provider-specific system instructions may need small adjustments, but they should express the same product rules and safety expectations.
Also check context window limits before forwarding a long transcript. Providers can have different limits, and token counts are not interchangeable across models. If the history will not fit, summarize older turns while retaining the active question, important decisions, user preferences, and any facts the current answer depends on. Keep recent turns intact when you can; an over-compressed summary often loses the detail that made the conversation useful.
Don’t discard information the user can see
Some providers return useful material beyond plain text. If the conversation includes source citations, a refusal marker, or another visible annotation, consider how it should appear after a handoff. A citation should remain attached to the claim it supports, rather than becoming an unattributed URL in a later prompt. If your app cannot preserve an item’s meaning, make that limitation clear instead of presenting an altered transcript as though nothing changed.
In the interface, a small notice such as “Continuing with a different model” can set expectations without interrupting the chat. It is also worth recording which provider handled each turn for debugging and support, while avoiding the storage of secrets or unnecessary sensitive data.
My working rule would be: keep the conversation in a provider-neutral record, translate it for each request, preserve tool activity explicitly, and treat a switch as a change in model—not a guarantee of identical memory or behavior. If others have built this, I’d be interested to hear which parts of a transcript you preserve verbatim and which you summarize during a handoff.
