Prompt Size Impact on Time to First Token
latency
Time to First Token (TTFT) is strongly affected by uncached input prompt size and reasoning complexity. Large prompts or complex reasoning tasks can cause significant TTFT spikes even when token velocity remains stable.
OpenAI insight details requires a free account. Sign in with Google or GitHub to access the full knowledge base.
Sign in to access