I am using the Python openai SDK to stream chat responses from gpt-4o for a web application. Randomly, during longer output generations, the response stream breaks and raises an httpx.ReadTimeout error.
What causes read timeouts during streaming?
By default, the openai client uses HTTPX under the hood with default timeout settings (typically around 60 seconds). When streaming complex responses, if the model pauses briefly between tokens or if network latency spikes, HTTPX detects no incoming data and closes the socket.
How to override default timeouts in the client
You can customize the httpx.Timeout parameters directly when instantiating the client to handle long streaming sessions without unexpected disconnects.
import httpx
from openai import OpenAI
# Configure custom timeout thresholds using httpx.Timeout
custom_timeout = httpx.Timeout(
connect=10.0, # Time allowed to establish server connection
read=180.0, # Max allowed gap between streamed chunks (in seconds)
write=10.0, # Time allowed to transmit payload
pool=5.0 # Time allowed to acquire connection from pool
)
# Pass the configured timeout to the OpenAI client instance
client = OpenAI(
timeout=custom_timeout
)
# Stream completions using the custom client configuration
response = client.chat.completions.create(
model="gpt-4o",
messages=[{"role": "user", "content": "Write a detailed architectural design document."}],
stream=True
)Is adjusting the read timeout the best way to handle this, or should client applications also implement chunk-level reconnect logic?
