[Feature Request ]Add macOS Benchmark Suite for Local Agent / Model Execution

[joaobenedetmachado]

Motivation

macOS (especially Apple Silicon M1/M2/M3) is widely used by developers, but currently there is no standardized way to benchmark Meridian performance on macOS.

This makes it hard to:

• Compare performance across Intel vs Apple Silicon
• Detect regressions specific to macOS
• Evaluate local agent / model execution latency and memory usage

A lightweight benchmark suite would greatly improve developer experience and reproducibility.

---

Proposal

Introduce a macOS-focused benchmark suite that can be run locally and in CI.

The benchmark suite could include:

• Startup time
• First-token latency
• Tokens/sec
• Peak memory usage
• CPU utilization

Benchmarks should run on:

• macOS Intel (x86_64)
• macOS Apple Silicon (arm64)

---

Possible Implementation

• Add a benchmarks/macos/ directory
• Provide a simple CLI:

  meridian benchmark --platform macos   

[cpulavarthi]

Hello @joaobenedetmachado,

Thank you for sharing your requirements and proposing the addition of macOS compatible benchmarking of Meridian model performance. Our team is looking into your requests and this shall be considered for future Meridian upgrades.

You can track the previous changes to Meridian and unreleased feature updates in our Change Log. We appreciate you taking the time to help us improve Meridian for all users.

Do reach out if you have any suggestions or questions regarding Meridian!

Google Meridian Support Team