Skip to main content
504Retryable

OpenAI 504 Gateway Timeout

The edge gave up waiting for the origin — typical for long non-streaming generations exceeding proxy timeouts.

Most likely causes

  1. 1.Non-streaming request generating for minutes
  2. 2.Very large context + large max output in one call

Fix checklist

  • Stream responses for anything long
  • Use background/Batch processing for slow jobs
  • Reduce max output per call and continue across calls

Retry guidance

Retry once with streaming enabled; repeated identical non-streaming calls will time out again.

// Retry 504 with exponential backoff + full jitter.
async function callWithBackoff(payload: unknown, maxAttempts = 5) {
  for (let attempt = 0; attempt < maxAttempts; attempt++) {
    const res = await fetch("https://api.openai.com/v1/chat/completions", {
      method: "POST",
      headers: {
        "content-type": "application/json",
        Authorization: `Bearer ${process.env.OPENAI_API_KEY}`,
      },
      body: JSON.stringify(payload),
    });
    if (res.status !== 504) return res;
    // Honor Retry-After when present; otherwise exponential backoff, capped at 32s.
    const retryAfter = Number(res.headers.get("retry-after"));
    const base = Number.isFinite(retryAfter) && retryAfter > 0
      ? retryAfter * 1000
      : Math.min(1000 * 2 ** attempt, 32_000);
    await new Promise((r) => setTimeout(r, base * (0.5 + Math.random() * 0.5)));
  }
  throw new Error("OpenAI 504: still failing after backoff — check https://status.openai.com");
}

Provider status page: status.openai.com