Fundamentals
Latency vs Throughput
Latency vs Throughput
Fast single requests and high volume are related—but optimizing one can hurt the other.
{#each concepts as c} {@const Icon = getIcon(c.icon)}
{c.title}
{c.desc}
{/each}
Practical Tips
Track p50/p95/p99; p99 drives user experience and incident pain.
Bound queues and add timeouts; unbounded retries create latency collapse.
Use load shedding when overloaded to keep the system responsive for some users.
Prefer idempotent operations so retries don’t amplify failures.
References
{#each references as ref}
{ref.label}
{/each}
Scaling
Compute