support to set an SLO threshold level and then calculate the goodput, rather than measuring the total throughput. #169
Description
What would you like to be added:
support to set an SLO threshold level and then calculate the goodput, rather than measuring the total throughput.
Why is this needed:
Normally, the vllm-benchmark only provides average/median/P99 metrics for TTFT/TPOT, making it impossible to calculate statistics such as what percentage of 200 requests fall below the SLO threshold.
like below:
