Trip Idea Agent

A single-turn "trip idea" agent: give it one natural-language travel request, get back a typed, structured suggestion (destinations, flights, accommodation, rough budget, caveats).

No UI, no persistence, no real travel APIs — see Design notes for why.

Provider & model

• Provider: OpenRouter (OpenAI-compatible chat completions API)
• Default model: openai/gpt-oss-120b — free, reliable enough at returning JSON on request.
• Override the model via OPENROUTER_MODEL (any OpenRouter model slug) without touching code.

Why OpenRouter first: one API key gives access to many underlying models (Claude, Gemini, GPT, open models), so swapping models is an env var change, not a code change. If usage outgrows what's economical there, the same completeJSON interface in src/lib/llm/openrouter.ts can be duplicated for a direct Anthropic/Gemini SDK client behind the same shape.

Setup (under 5 minutes)

npm install
cp .env.example .env.local
# edit .env.local and paste your OpenRouter key (https://openrouter.ai/keys)

npm run trip -- "Relaxing beach week in February under \$2000 for two, leaving from JFK. Want a nice hotel."

That prints a JSON TripSuggestion to stdout.

Required env vars

Var	Required	Default	Notes
OPENROUTER_API_KEY	yes	—	From https://openrouter.ai/keys
OPENROUTER_MODEL	no	openai/gpt-4o-mini	Used for the agent's extraction + synthesis passes (the customer-facing path)
OPENROUTER_JUDGE_MODEL	no	falls back to OPENROUTER_MODEL	Used only by the eval harness's LLM-as-judge step (npm run eval) — pick a stronger model here since judge reliability matters more than judge latency/cost

What happens when credentials are missing or invalid

• Missing OPENROUTER_API_KEY: fails fast, before any network call, with Configuration error: OPENROUTER_API_KEY is not set... and CLI exit code 2. The API route returns HTTP 500 with the same message in the body.
• Invalid/rejected key (401 from OpenRouter): Configuration error: OpenRouter rejected the API key (401 Unauthorized)..., exit code 2 / HTTP 500.
• Other request failures (network error, rate limit, non-2xx, malformed JSON from the model after one retry): Request error: ..., exit code 3 / HTTP 502.

Try it: run npm run trip -- "test" without setting up .env.local to see the missing-key message live.

Other ways to run it

HTTP (Next.js API route):

npm run dev
curl -X POST http://localhost:3000/api/trip \
  -H "Content-Type: application/json" \
  -d '{"request": "Long weekend in Europe, flying from SFO. Walkable city, great food, boutique hotel under $300/night."}'

Eval harness:

npm run eval

Runs 7 cases through the agent and prints a per-case + aggregate report (see below).

Design notes

Tool slicing: one tool per concern, plus a deterministic budget calculator

Four tools in src/lib/tools/: searchDestinations, searchFlights, searchHotels, and computeBudget. Flights and hotels take different inputs (origin/dates vs. location/nights) and have different mock shapes, so merging them would just push the branching inside one function instead of removing it. computeBudget is plain arithmetic — it's deliberately not an LLM-callable tool, since totals are exactly the kind of thing that shouldn't be left to a model to "remember" correctly.

Orchestration: two structured LLM passes, no ReAct loop

Single-turn requests over a small, fixed tool universe don't benefit from a multi-step reason/act/observe loop — there's nothing for the model to discover by looping. Instead (src/lib/agent.ts):

1. Extract — one LLM call turns free text into a typed TripRequest (origin, budget, dates, vibe, exclusions, etc.), leaving anything unstated as null/false/[].
2. Tools — code deterministically calls searchDestinations → searchFlights → searchHotels → computeBudget for each destination candidate. No model involved.
3. Synthesize — one LLM call picks the best candidate and writes the reasoning/caveats, given the tool data as context. It is explicitly instructed not to invent numbers.
4. Assemble — the final response is built in code from step 2's data plus step 3's reasoning/caveats/confidence, then validated against the response schema. Prices and hotel names always come from the mock tools, never from model text, so a flaky model can't hallucinate a price into a structurally valid response.

Missing/ambiguous info: infer with defaults, flag with caveats, never fabricate

Three-tier handling, enforced in code (not left to model judgment):

• Missing, non-blocking (e.g. no traveler count) → default applied (2 travelers, 5 days)
  • a caveat stating the assumption.
• Missing, blocking for one tool (e.g. no origin city) → that field degrades to null (no flight price) + a caveat asking for the missing info — never a guessed price.
• Nearly no information (e.g. "Cheap.") → still returns a valid structured response, but confidence: "low" and a caveat saying the suggestion is a generic placeholder.

This is a single-turn tool (no follow-up question loop), so "ask" takes the form of a caveat in the response rather than a clarifying question back to the user.

Eval dimensions

eval/run-eval.ts runs 7 cases (the prompt's examples) through the agent and scores 3 dimensions:

1. Schema validity (deterministic) — does the response parse against the Zod schema.
2. Constraint adherence (deterministic, per-case) — case-specific checks, e.g. "flight price present when an origin was given," "destination excludes Santorini," "no flight cost when the user already has one booked."
3. Groundedness (LLM-as-judge, 1 extra call per case) — does the response avoid inventing facts beyond the mock data, and does it caveat appropriately what wasn't provided.

Two deterministic + one judged dimension keeps the harness cheap and the deterministic checks give a hard signal; the judge call only covers the part deterministic checks can't reach (whether the prose is grounded).

The judge step can use a different (typically stronger) model than the agent's own extraction/synthesis passes via OPENROUTER_JUDGE_MODEL — judge reliability matters more than judge cost/latency, since it only runs in CI/local evals, not on every customer request.

Dimensions considered but intentionally not scored for this prototype: latency/call-count (useful to observe, not a quality signal), output determinism across reruns (worth a manual spot-check, not worth automating at this scale), and caveat precision beyond "are there any caveats" (folded into the groundedness judge rather than given its own dimension).

What's out of scope (by design)

No real travel APIs (mocked data only), no UI, no persistence/multi-turn state, no auth, no production concerns (deploy/cache/observability/retries). See the task brief for the full list — this prototype optimizes for a clear, typed, testable single-turn flow over completeness.