Skip to content

Add support for VisionArena Dataset #225

@Edwinhr716

Description

@Edwinhr716

What would you like to be added: Add support to run benchmarks using the VisionArena Dataset https://huggingface.co/datasets/lmarena-ai/VisionArena-Battle, similar to how vLLM does it https://github.com/vllm-project/vllm/blob/1b962e24577dddc0d7441ae0b06392e1f9262a51/vllm/benchmarks/datasets.py#L1996

Why is this needed: VisionArena unblocks benchmarking VL models such as the Qwen2.5-VL family

Metadata

Metadata

Assignees

No one assigned

    Labels

    help wantedDenotes an issue that needs help from a contributor. Must meet "help wanted" guidelines.needs-triageIndicates an issue or PR lacks a `triage/foo` label and requires one.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions