Skip to content

About

Evaluate embedded vector indexes performance in retrieval

Resources

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

rag-evaluator

Gradio UI to build and search TileDB vector indexes, run RAG-style evaluations on golden datasets, and compare indexes. Uses LangGraph and Anthropic (Claude) for the agent path.

RAG evaluator Gradio UI

Requirements

  • Python 3.11+
  • ANTHROPIC_API_KEY — required for the RAG agent and optional LLM judge features (standard Anthropic env var).

Install

From the repo root (uses uv):

uv sync

Optional dev tools: uv sync --extra dev.

Dev server

uv run rag-evaluator-ui

Opens the app at http://127.0.0.1:7860 by default. Override bind address and port:

uv run rag-evaluator-ui --host 0.0.0.0 --port 7860

Agent skills

Agent instructions for this repo live under .agents/skills/:

Skill Path Use when
Weights & Biases (W&B / Weave) .agents/skills/wandb-primary/SKILL.md Runs, metrics, artifacts, Weave traces, evals, scorers
TileDB Vector Search CLI .agents/skills/tiledb-vector-search-cli/SKILL.md tiledb vs — build indexes from docs, semantic search, CLI workflows

Attach or register these with your agent tooling if you want assistants to load them for W&B or tiledb vs work.

About

Evaluate embedded vector indexes performance in retrieval

Resources

Stars

0 stars

Watchers

0 watching

Forks

Contributors

Languages