Proposal: LLM Billing & Benchmarking Standard (LBBS) — Call for Public Review

[huouer]

Hi MLPerf Training team,

I'm sharing a public draft of a new proposed standard designed to complement the MLPerf benchmarking ecosystem by introducing a vendor-neutral billing and efficiency metric for LLM evaluation:

📄 Standard draft (v0.1):
https://github.com/huouer/LBBS-Standard/tree/main/docs/LBBS_v0_1_Standard_Draft.pdf

🧩 Motivation
MLPerf focuses on system-level performance (throughput, time-to-train, accuracy).
LBBS focuses on cost-per-task × latency, enabling comparable $·s/task reporting across LLM providers (OpenAI, Anthropic, Google, etc.).

🔍 Why this may be relevant
• MLPerf benchmarks runtime and scaling, LBBS benchmarks billing and deployment cost
• Used together, they offer full economics of inference + training
• Standard is open for 90-day community comment

🗣️ Feedback Thread
huouer/LBBS-Standard#2

Would the MLPerf Training WG be open to reviewing this for potential alignment or cross-reference?

Thanks for your time — happy to answer questions here or in a WG call.

[ShriyaRishab]

@huouer - can you please attend the Dec 4th, 2025 WG meeting and present this?