---
title: "Why Does an LLM Return Invalid JSON, and How Should My API Handle It?"  
description: "Why can an LLM produce invalid or unusable JSON even when prompted to follow a format? Explore structured outputs, validation, and practical recovery choices for API integrations."  
author: "Jithin Rahman"  
published: 2026-10-15  
updated: 2026-10-15  
canonical: https://answers.mindstick.com/blog/696/why-does-an-llm-return-invalid-json-and-how-should-my-api-handle-it  
category: "Mix Category"  
tags: ["artificial intelligence", "LLM prompts", "API Integration", "JSON", "Python"]  
reading_time: 4 minutes  

---

# Why Does an LLM Return Invalid JSON, and How Should My API Handle It?

I’m adding an LLM call to an API that expects a predictable JSON object. Asking the model to “return valid JSON” helps, but every so often I still get a missing field, the wrong value type, or a response wrapped in extra text. What’s a sensible way to make this reliable without treating every odd response as something to blindly retry?

The tricky part is that “valid JSON” and “valid for my application” are different standards. A response like `{"quantity":"three"}` is valid JSON, but it may be useless if your code requires `quantity` to be an integer. A response can also parse cleanly while omitting a required field or inventing a value your system cannot accept.

## Ask for structure, then enforce it

When the chosen model and endpoint support structured output or a schema-constrained response mode, use that instead of relying on prompt wording alone. These features can reduce formatting errors, but their availability and behavior vary across providers, models, and API endpoints. Check the current documentation for the exact model you call; don’t assume that a feature available in one provider’s chat endpoint works identically in another.

Even with constrained output, validate the result in your own service. Treat the model response as untrusted input, much like a request from a client or a message from another system. Validation should check both shape and meaning: [required fields](https://www.mindstick.com/forum/158276/how-do-you-use-the-required-attribute-in-html5-to-make-form-fields-mandatory), allowed values, sensible ranges, and any relationships between fields.

## Validate at the application boundary

For example, suppose an application asks a model to classify a support message. It expects a category from a fixed list and a confidence score between zero and one. A small validation layer makes those expectations explicit before the result reaches [business logic](https://answers.mindstick.com/qa/30543/what-is-business-logic):

```python
from pydantic import BaseModel, Field, ValidationError
from typing import Literal
import json

class Classification(BaseModel):
    # Restrict categories to values the rest of the application understands.
    category: Literal["billing", "technical", "account", "other"]
    # Reject scores outside the range used by downstream decisions.
    confidence: float = Field(ge=0.0, le=1.0)

def parse_classification(response_text: str) -> Classification:
    # Parse the text as JSON and validate its fields and value constraints.
    try:
        data = json.loads(response_text)
        return Classification.model_validate(data)
    except (json.JSONDecodeError, ValidationError) as exc:
        # Keep the original response available for logs or a controlled recovery path.
        raise ValueError("The model returned an invalid classification") from exc
```

This example deliberately doesn’t try to repair malformed output by stripping arbitrary text or guessing what the model meant. If your endpoint promises a particular data shape, quietly coercing unexpected values can hide a prompt or integration problem. Log enough information to investigate failures, but avoid recording sensitive user content unless your privacy and retention rules allow it.

## Make recovery deliberate

When validation fails, decide what the application should do before the failure happens. Depending on the task, reasonable options include returning a clear error, asking the model once to correct a specific validation issue, falling back to a safe default, or routing the request for human review. A repair attempt should include the validation feedback and still pass through the same validator. Put a firm limit on retries; repeated calls can add latency and cost without making an ambiguous answer trustworthy.

Also distinguish a malformed response from an incomplete one. A provider may return a refusal, hit a token limit, or report a tool handoff rather than supply the JSON your code expects. Those are separate outcomes, and a production integration should handle them according to the provider’s response format instead of treating every response as ordinary text.

Finally, test with more than a happy-path example. Include missing fields, unexpected categories, boundary values, empty output, and provider errors. The goal isn’t to force an LLM to behave like a database. It’s to keep unpredictable model output from crossing into your application as if it were already trusted data.

---

Original Source: https://answers.mindstick.com/blog/696/why-does-an-llm-return-invalid-json-and-how-should-my-api-handle-it

Copyright © MindStick Software Pvt. Ltd. This Markdown version is provided for developers, AI systems, and offline reading.