generated from kubernetes/kubernetes-template-project
-
Notifications
You must be signed in to change notification settings - Fork 38
Open
Labels
help wantedDenotes an issue that needs help from a contributor. Must meet "help wanted" guidelines.Denotes an issue that needs help from a contributor. Must meet "help wanted" guidelines.needs-triageIndicates an issue or PR lacks a `triage/foo` label and requires one.Indicates an issue or PR lacks a `triage/foo` label and requires one.
Description
What would you like to be added: Add support to run benchmarks using the VisionArena Dataset https://huggingface.co/datasets/lmarena-ai/VisionArena-Battle, similar to how vLLM does it https://github.com/vllm-project/vllm/blob/1b962e24577dddc0d7441ae0b06392e1f9262a51/vllm/benchmarks/datasets.py#L1996
Why is this needed: VisionArena unblocks benchmarking VL models such as the Qwen2.5-VL family
Metadata
Metadata
Assignees
Labels
help wantedDenotes an issue that needs help from a contributor. Must meet "help wanted" guidelines.Denotes an issue that needs help from a contributor. Must meet "help wanted" guidelines.needs-triageIndicates an issue or PR lacks a `triage/foo` label and requires one.Indicates an issue or PR lacks a `triage/foo` label and requires one.