KV-cache pressure
Diagnostic confidence
86%
SERVING DIAGNOSTICS
Find the real bottleneck
before buying more GPUs.
Feed in workload shape and serving telemetry. The doctor separates queueing, prefill, decode, KV-cache and memory pressure — then recommends the next fix.
OBSERVE→CLASSIFY→DIAGNOSE→OPTIMIZE→VERIFY
68
Recoverable inefficiency
Multiple tuning opportunities
04 · PRESSURE PROFILE
Synthetic pressure index
Where serving time and capacity are going
05 · OPTIMIZATION STACK
Priority ordered
What to change next
06 · SYNTHETIC IMPACT
Directional estimate
Before vs. after the next fix
Queue time
420 ms→315 ms
Decode speed
42 tok/s→54 tok/s
Memory pressure
84%→71%
Effective batch
8→12
07 · DIAGNOSTIC RECORD
Machine-readable JSON
How Inference Bottleneck Doctor works Deterministic serving heuristics, not live profiling ＋
The doctor combines request shape with queue latency, prefill latency, decode speed, GPU utilization, memory usage, KV-cache pressure, batch size, prefix reuse and retries. It produces relative pressure scores for common serving bottlenecks.
Recommendations follow a staged optimization philosophy: fix the actual bottleneck first, then re-measure before stacking additional optimizations.