Why did I get a 5xx error, and should I retry?

For Model APIs, the short version: 5xx errors are usually transient — retry with exponential backoff.

500 — an error while processing the request. Retry; if one specific payload fails repeatedly, report it to us with the model and (if shareable) the request shape.

502 / 503 — transient infrastructure or routing conditions. Retry with backoff.

529 — the deployment couldn't accept more work at that moment. Retry, honoring the

Retry-After

header. Note: 529 isn't a standard HTTP code, so confirm your client retries it rather than treating it as fatal.

If a 5xx persists across retries for many minutes, message us with the model, timestamps (with timezone), and status codes — that's enough for us to check the deployment.

Docs: Inference errors