🌐 Multi-DB Translation Agent

An agentic RAG translation assistant that routes each request across three specialized vector/data stores — instead of dumping everything into one index.

---

🧭 Architecture

User Input (text, src_lang, tgt_lang, domain)
        │
        ▼
   🤖 Router Agent  ── decides which DBs are worth querying
        │
   ┌────┼──────────────┬───────────────┐
   ▼                   ▼               ▼
🗂️ Translation Memory  📖 Glossary DB   🎨 Style DB
  (FAISS IVF-PQ)        (FAISS flat)    (ChromaDB)
   │                   │               │
   └───────────┬────────┴───────────────┘
               ▼
        🔤 Translator Agent (Groq / Llama 3.3 70B)
               ▼
        🧐 Critic Agent ── checks glossary terms used, tone matches
               │
        retry ─┴─ approve → ✅ Final translation + retrieval trace

🎯 Why three separate DBs

DB	Problem it solves	Storage	Why this index
🗂️ Translation Memory	"Has this been translated before?"	FAISS IVF-PQ (falls back to flat under ~2000 vectors)	Grows unbounded over time — PQ compression keeps it small (~190x smaller than flat)
📖 Glossary	"How should this specific term be translated?"	FAISS flat + keyword prefilter	Small, static, needs exact recall — compression would only hurt here
🎨 Style Examples	"What tone/register does this domain expect?"	ChromaDB (embedded, no server)	Small and simple — avoids running a DB server for a tiny collection

Mixing all of this into one index would blend translation-memory hits with random style examples on every query, diluting relevance. The router agent decides per-request which stores are actually relevant, which also keeps latency and API cost down — most short/simple inputs skip retrieval entirely.

---

⚙️ Setup (all free tools — no paid services)

git clone https://github.com/10div10/translation-agent.git
cd translation-agent

python3.11 -m venv .venv
source .venv/bin/activate      # Windows: .venv\Scripts\activate

pip install -r requirements.txt
cp .env.example .env
# edit .env and add a free Groq API key from https://console.groq.com

python -m dbs.seed_all         # populates all 3 DBs with sample data
streamlit run app.py

Open in VS Code: code . — the repo includes no local model weights, so it's light to clone; embeddings (BGE-small) download once via sentence-transformers and cache locally.

---

🧪 Running tests

pytest eval/test_dbs.py -v

These tests cover the DB layer only (no Groq API key required), which is why CI can run them on every push for free. Agent/graph-level tests that call the LLM are intentionally kept separate.

---

⚠️ Known limitations

• 📉 IVF-PQ recall drops noticeably below ~2000 translation-memory vectors, which is why the store falls back to a flat index until it has enough data to train stable clusters. If you seed a large real corpus, recall should be evaluated (see eval/) before trusting IVF-PQ results.
• 🔍 Glossary term matching on the critic side is currently a simple substring check, so paraphrased translations may cause false-positive "missing term" retries.
• 🔁 No true multi-hop reasoning — the graph is a fixed pipeline (router → retrieve → translate → critique) with a bounded retry loop, not a fully autonomous agent that replans.
• 🌍 Currently scoped to en/fr/es for the seed data; adding language pairs just means adding rows to dbs/seed_all.py, no code changes needed.

---

🧱 Stack