FluidPD: In-Place Elasticity for SLO-Aware Prefill-Decode Disaggregated LLM Serving

FluidPD is a P/D-disaggregated LLM serving system that provides SLO-aware in-place elasticity through two complementary mechanisms: FluidToken and FluidRole.

Traditional autoscaling reacts slowly, requires spare GPUs, and does not directly address short-timescale phase imbalance. FluidPD handles both transient and sustained imbalance in place without adding workers.

FluidToken handles transient imbalance by offloading a bounded portion of prefill computation to decode workers when decode-side slack is available. FluidRole handles sustained imbalance by reassigning running workers between prefill and decode roles in place, avoiding model reload and engine restart.

The published results are positive on production Azure traces, but this remains arXiv research. Infrastructure teams can evaluate the concept in test environments while awaiting independent verification before production adoption.