Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,14 @@ jobs:
env:
TRASK_SKIP_BUILD: "1"

- name: Install Playwright Chromium (Discord embed harness)
run: pnpm exec playwright install chromium --with-deps

- name: Discord /ask Playwright harness (offline)
run: pnpm trask:e2e:discord:playwright
env:
TRASK_SKIP_BUILD: "1"

- name: Lint (pazaak-world)
run: pnpm --filter pazaak-world lint

Expand Down
4 changes: 3 additions & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -145,6 +145,8 @@ pnpm verify:trask-cli

`pnpm verify:trask-cli` runs `scripts/verify_trask_cli_qa.mjs` (golden queries). `pnpm verify:trask-cli:ci` / `pnpm trask:verify-import-smoke:ci` use `--import-smoke` (no LLM). `pnpm verify:trask-discord` runs live Discord-format checks via the research wizard.

**Playwright (offline Discord + live Holocron):** `pnpm trask:e2e:discord:playwright` — static harness on **:4012** (`scripts/discord-ask-e2e-webserver.mjs`, `e2e/trask-discord-ask.spec.mjs`); mirrors import-smoke embed contract in a real browser (no discord.com, no LLM). `pnpm trask:e2e:playwright` runs Discord harness then `pnpm holocron:e2e:playwright`. CI Build job runs Discord Playwright after `trask:gate:ci`.

#### Offline faithfulness gate (citation alignment)

After code changes to answer formatting, citation alignment, or `grounded-evidence.ts`, run:
Expand All @@ -164,7 +166,7 @@ pnpm trask:faithfulness-eval # faithfulness fixtures only

### Trask Discord `/ask` — mandatory verification (agents)

**Do not claim Discord `/ask` is fixed until live checks pass.** Run `pnpm verify:trask-discord` (full expert set). Holocron requires `pnpm holocron:e2e` and, when browser MCP works, all five UI queries on `:4010`. Discord web UI is not Playwright-gated in CI; live verify exercises the same compose/format path as the bot.
**Do not claim Discord `/ask` is fixed until live checks pass.** Run `pnpm verify:trask-discord` (full expert set). Offline Playwright gate: `pnpm trask:e2e:discord:playwright` (five golden import-smoke embeds on **:4012**). Holocron requires `pnpm holocron:e2e` and, when browser MCP works, all five UI queries on `:4010`. CI runs offline Discord Playwright in Build; live `verify:trask-discord` still needs a bot token locally.

1. **Worker retrieve** at `TRASK_INDEXER_BASE_URL=http://127.0.0.1:8787` — `pnpm verify:trask-discord` auto-bootstraps indexer+Worker when unhealthy (same as CLI/Holocron e2e); manual path: `bash scripts/trask_live_stack.sh` with QA seed `bash scripts/bootstrap_trask_indexer.sh`, `bash scripts/trask_index_seed_for_qa.sh`.
2. **Restart** Trask bot after `@openkotor/trask` changes: kill old `trask-bot/dist/main.js`, rebuild (`pnpm build`), start with `TRASK_INDEXER_BASE_URL` + `TRASK_WEB_RESEARCH_PYTHON` + `.env` token.
Expand Down
9 changes: 9 additions & 0 deletions apps/holocron-web/e2e/holocron-research.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -205,6 +205,15 @@ for (const [index, querySpec] of RESEARCH_QUERIES.entries()) {

assertSubstantiveAnswer(bodyText, sourcesText, querySpec)

expect(bodyText, 'answer should not leak spaced markdown link syntax').not.toMatch(/\]\s*\(https:\/\//)
if (hasSourcesPanel) {
const sourceLabels = await sourcesRegion.locator('[role="listitem"]').allInnerTexts()
for (const label of sourceLabels) {
expect(label, 'source card label').not.toMatch(/githubusercontent\.com/i)
expect(label, 'source card label').not.toMatch(/^[A-Za-z0-9_.-]+\/[A-Za-z0-9_.-]+#L\d+$/i)
}
}

const httpsCount = Math.max(
await httpsInSources.count(),
countHttpsUrls(sourcesText),
Expand Down
1 change: 1 addition & 0 deletions apps/holocron-web/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,7 @@
"zod": "^3.25.76"
},
"devDependencies": {
"@openkotor/trask": "workspace:*",
"@openkotor/trask-config": "workspace:*",
"@eslint/js": "^9.21.0",
"@tailwindcss/postcss": "^4.1.8",
Expand Down
52 changes: 10 additions & 42 deletions apps/holocron-web/src/lib/answer-presentation.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@
* Kept separate from Message.tsx for unit testing without React.
*/

import { webCitationDisplayLabel } from '@openkotor/trask/github-citation-url'

export interface SourceLike {
name: string
url: string
Expand Down Expand Up @@ -205,44 +207,8 @@ function sourceHostname(url: string): string {

function formatSourceDisplayName(name: string, url: string): string {
if (!url) return name
try {
const parsed = new URL(url)
const host = parsed.hostname.replace(/^www\./, '')
const genericName =
!name.trim()
|| name.trim().toLowerCase() === host.toLowerCase()
|| name.trim().toLowerCase() === 'github.com'

if (host === 'github.com') {
const path = decodeURIComponent(parsed.pathname)
const hash = parsed.hash && /^#L/i.test(parsed.hash) ? parsed.hash : ''
const blobMatch = path.match(/\/blob\/[^/]+\/(.+)$/i)
if (blobMatch?.[1]) {
const shortPath = blobMatch[1].replace(/\/+$/, '').split('/').slice(-2).join('/') || blobMatch[1]
return `${shortPath}${hash}`
}
const wikiMatch = path.match(/\/wiki\/(.+)$/i)
if (wikiMatch?.[1]) {
const page = wikiMatch[1].replace(/\/+$/, '')
return `wiki: ${page.split('/').pop() ?? page}${hash}`
}
const repoMatch = path.match(/^\/([^/]+)\/([^/]+)\/?$/i)
if (repoMatch?.[2]) {
return `${repoMatch[2]}${hash}`
}
}

if (genericName) {
const pathSegments = parsed.pathname.replace(/\/+$/, '').split('/').filter(Boolean)
if (pathSegments.length > 1) {
return `${pathSegments.slice(-2).join('/')}`.replace(/[-_]+/g, ' ')
}
return host
}
} catch {
/* keep name */
}
return name
const label = webCitationDisplayLabel(url, name)
return label.trim() || name
}

export function sourceKey(source: Pick<SourceLike, 'name' | 'url'>): string {
Expand All @@ -263,7 +229,7 @@ function stripSourceNoise(text: string): string {
function isNumberedBibliographyLine(line: string): boolean {
const trimmed = line.trim()
if (!parseNumberedSourceLine(trimmed)) return false
return extractHttpUrls(trimmed).length > 0 || trimmed.length > 24
return extractHttpUrls(trimmed).length > 0
}

/**
Expand Down Expand Up @@ -430,11 +396,13 @@ export function buildAnswerPresentation(content: string, explicitSources: Source
if (existingIndex !== undefined) {
const existing = merged[existingIndex]
if (!existing) return
const url = cleanUrl(candidate.url || existing.url)
merged[existingIndex] = {
...existing,
name: existing.name || candidate.name,
url: existing.url || candidate.url,
hostname: existing.hostname || candidate.hostname,
...candidate,
url,
name: formatSourceDisplayName(existing.name || candidate.name, url),
hostname: url ? sourceHostname(url) : existing.hostname || candidate.hostname,
}
return
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,8 @@ pr_refs: [33, 34, 35, 36, 38]
| Sources in UI | Visible panel / API fields | **Hidden** — stripped from embed |
| Body presentation | Multi-paragraph / bullets allowed | ≤5 non-empty lines, char cap per line |
| Citations | `[n]` + URLs in Sources | Inline `[n](https://…)` only |
| Display entry | HTTP record + Holocron UI | `buildResearchEmbed` + `formatDiscordAskDisplay` |
| Display entry | HTTP record + Holocron UI (`answer-presentation.ts` + `webCitationDisplayLabel`) | `buildResearchEmbed` + `formatDiscordAskDisplay` |
| GitHub source permalinks | `github-citation-url.ts` at compose; Holocron re-labels via `@openkotor/trask/github-citation-url` | Same module for embed URL map |

[SYNTH] Faithfulness fixtures and Holocron e2e validate full answers; **discord-reply-format.test.js** stress tests validate the display transform only.

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
---
status: completed
branch: feat/holocron-topnav-ci-followup
origin: docs/plans/2026-06-04-007-feat-holocron-answer-citation-render-plan.md
---

# Plan: Holocron citation + GitHub permalink LFG closeout

## Problem (pre-fix)

Holocron grounded answers and the Sources panel misled users when:

1. Answer body leaked raw markdown (`[label] (https://…)`) or numbered bibliography without a `Sources` heading.
2. GitHub shallow-repo citations inferred file paths from `raw.githubusercontent.com/...` URLs in passage text, producing blob paths like `githubusercontent.com/owner/repo` and labels like `KobaltBlu/KotOR.js#L1` instead of `README.md#L1`.

**Remaining (this plan):** ship commits `ab117de` and `5c568d6` on `feat/holocron-topnav-ci-followup` after PR #97 merged to `main`.

## Requirements

| ID | Requirement |
|----|-------------|
| R1 | Visible answer paragraphs strip spaced markdown links/images; preserve `[n]` citation badges. |
| R2 | Trailing `1.` bibliography lines that include `https://` URLs, without a `Sources` heading, peel into the Sources panel (or source-only body). |
| R3 | `inferGitHubFilePath` ignores http(s) URL segments and rejects domain-like path segments. |
| R4 | Holocron Sources labels use `webCitationDisplayLabel` via `@openkotor/trask/github-citation-url` (no full `@openkotor/trask` browser bundle). |
| R5 | Unit tests cover markdown sanitization, permalink inference, and malformed-blob labels. |
| R6 | Open PR, CI green, local stack smoke on `:4010` for reone query. |

## Implementation units

### U1 — Answer presentation (landed `ab117de`)

- **Files:** `apps/holocron-web/src/lib/answer-presentation.ts`, `apps/holocron-web/src/components/Message.tsx`, `scripts/answer_presentation.test.mjs`
- **Verification:** `node --import tsx/esm --test scripts/answer_presentation.test.mjs`

### U1b — Compose markdown strip (landed `ab117de`)

- **Files:** `packages/trask/src/grounded-evidence.ts`
- **Verification:** `node --test packages/trask/dist/grounded-evidence.test.js` (after `pnpm build`)

### U2 — GitHub permalink labels (landed `5c568d6`)

- **Files:** `packages/trask/src/github-citation-url.ts`, `packages/trask/package.json` (subpath export), `packages/trask/src/github-citation-url.test.ts`, `apps/holocron-web/package.json`, `apps/holocron-web/src/lib/answer-presentation.ts`
- **Verification:** `pnpm --filter @openkotor/trask build && node --test packages/trask/dist/github-citation-url.test.js`

### U3 — Docs alignment (landed `5c568d6`)

- **Files:** `docs/solutions/tooling-decisions/trask-citation-*.md`, `docs/knowledgebase/10-architecture-runtime/trask-citation-display-contract.md`
- **Verification:** paths in docs match repo; no contradictory "0 Playwright tests" claims.

## Test scenarios

- Passage with `raw.githubusercontent.com/.../icon.png` → blob URL ends with `/README.md`, label `README.md#Ln`.
- Answer with `3. [broken](https://raw...)` line → sanitized prose, no `](https://` in UI.
- Numbered-only body → Sources grid populated; no duplicate numbered paragraphs in answer.

## Verification ladder

```bash
pnpm build
node --import tsx/esm --test scripts/answer_presentation.test.mjs
node --test packages/trask/dist/github-citation-url.test.js
pnpm trask:gate:ci
bash scripts/trask_live_stack.sh
# Holocron :4010 — reone canonical query; Sources card shows README.md#Ln not owner/repo#Ln
```

## Scope boundaries

- Out of scope: HF Space recovery, production Worker deploy (see `docs/plans/2026-06-04-006-fix-holocron-public-api-connection-plan.md`).
- Out of scope: Full five-query browser MCP gate in this PR; R6 closes with one reone smoke on `:4010`. Five-query Playwright e2e is optional post-merge (`pnpm holocron:e2e`).

### U4 — Ship / verify (R6)

- Open PR from `feat/holocron-topnav-ci-followup`, CI green, `:4010` reone canonical smoke.

## Deferred to implementation

- None — feature code landed; U4 tracks PR/CI/smoke only.
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
---
status: completed
branch: feat/holocron-topnav-ci-followup
origin: docs/plans/2026-06-03-001-feat-trask-playwright-holocron-discord-e2e-plan.md
date: 2026-06-04
---

# Plan: Trask Discord Playwright harness + Holocron citation e2e

## Problem (pre-fix)

Holocron has six Playwright tests (`holocron-research.spec.ts`), but Discord `/ask` contract validation is **Node-only** (`verify_trask_discord_live.mjs`). Agents and CI lack a **browser-level** Discord embed gate that uses Playwright while avoiding discord.com UI fragility and guild tokens.

Holocron e2e does not assert **source card permalink labels** (`README.md#Ln`) after the citation presentation slice (PR #98).

## Inferred Intent

- **Direct ask:** Continue Trask Q&A quality with Playwright for Holocron and Discord surfaces; browser-verify both.
- **Adjacent impact:** CI ladder parity, import-smoke ↔ Playwright alignment, citation label regressions caught in e2e.
- **Cohesive scope:** Offline Discord Playwright harness (golden compose, no LLM); Holocron Playwright assertions for citation hygiene; root scripts + CI step; local full ladder run.
- **Risks if partial:** Discord regressions only caught by slow live verify; Holocron UI label bugs slip past Playwright.

## Requirements

| ID | Requirement |
|----|-------------|
| R1 | Playwright spec exercises all five `DISCORD_IMPORT_SMOKE_SPECS` embed descriptions via static harness (no discord.com, no LLM) |
| R2 | Harness assertions mirror `verify_trask_discord_live.mjs` import-smoke contract (lines, inline links, no Sources block, expectPattern) |
| R3 | Holocron Playwright asserts Sources panel labels avoid `githubusercontent.com` path noise |
| R4 | Root `package.json` exposes `trask:e2e:discord:playwright` and `trask:e2e:playwright` (Holocron + Discord) |
| R5 | CI runs Discord Playwright after `trask:gate:ci` (fast, offline) |
| R6 | Local: stack up → `pnpm trask:e2e:playwright` or Holocron-only with `HOLOCRON_REUSE_SERVER=1` |

## Scope boundaries

- **In:** Playwright harness server, specs, Holocron assertion tweak, package scripts, CI step.
- **Out:** Playwright against discord.com in CI; live Discord token in GitHub Actions UI tests.

## Implementation units

### U1 — Discord ask display audit module

- **Files:** `scripts/lib/discord_ask_display_audit.mjs` (extract from verify script)
- **Tests:** existing import-smoke via `pnpm verify:trask-discord:ci`

### U2 — Discord Playwright harness

- **Files:** `scripts/discord-ask-e2e-webserver.mjs`, `e2e/trask-discord-ask.spec.mjs`, `playwright.trask-discord.config.mjs`
- **Verification:** `pnpm trask:e2e:discord:playwright` — 5 tests pass in <30s

### U3 — Holocron citation Playwright assertions

- **Files:** `apps/holocron-web/e2e/holocron-research.spec.ts`
- **Verification:** sources panel + answer body hygiene checks per query

### U4 — Scripts + CI

- **Files:** `package.json`, `.github/workflows/ci.yml`
- **Verification:** CI job step green

### U5 — Local verification

```bash
pnpm build && pnpm trask:gate:ci
pnpm trask:e2e:discord:playwright
bash scripts/trask_live_stack.sh
HOLOCRON_REUSE_SERVER=1 pnpm holocron:e2e:playwright
```

## Deferred

- Optional headed Discord web UI proof (`discord_fetch_trask_token.mjs` profile) — operator-only.
53 changes: 53 additions & 0 deletions docs/plans/2026-06-04-010-feat-trask-e2e-ladder-closeout-plan.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
---
status: completed
branch: feat/holocron-topnav-ci-followup
origin: docs/plans/2026-06-04-009-feat-trask-discord-playwright-holocron-e2e-plan.md
date: 2026-06-04
---

# Plan: Trask e2e ladder closeout (Playwright + docs + CI)

## Problem (remaining)

Plan 009 landed Discord Playwright harness and Holocron citation assertions, but:

1. `AGENTS.md` does not document `trask:e2e:discord:playwright` / `trask:e2e:playwright`.
2. Full Holocron Playwright matrix (6 tests) not re-run after citation assertion changes on PR #98.
3. PR #98 CI must go green with new Build job Discord Playwright step.

## Inferred Intent

- **Direct ask:** Complete Trask Q&A e2e story — Playwright for Holocron + Discord, browser validation.
- **Adjacent impact:** Agent runbooks, PR merge readiness, validation-ladder accuracy.
- **Cohesive scope:** Doc drift fix, full local Holocron Playwright pass, Discord offline Playwright + import-smoke, CI watch.
- **Risks if partial:** Agents miss new scripts; merge with unverified Holocron 6-test matrix.

## Requirements

| ID | Requirement |
|----|-------------|
| R1 | `AGENTS.md` documents `trask:e2e:discord:playwright`, `trask:e2e:playwright`, harness port 4012 |
| R2 | `pnpm trask:e2e:discord:playwright` passes locally |
| R3 | `HOLOCRON_REUSE_SERVER=1 pnpm holocron:e2e:playwright` — 6/6 pass on live stack |
| R4 | PR #98 Build & Test + Holocron Playwright jobs green |
| R5 | Browser smoke: one Holocron expert query on `:4010` (agent-browser) |

## Implementation units

### U1 — Agent docs

- **Files:** `AGENTS.md` (Trask Discord / Holocron e2e sections)

### U2 — Local Playwright verification

- Discord: `pnpm trask:e2e:discord:playwright`
- Holocron: stack on `:4010` → `HOLOCRON_REUSE_SERVER=1 pnpm holocron:e2e:playwright`

### U3 — CI + PR

- Watch `gh pr checks 98`; autofix if Build job fails on Discord Playwright install/runtime

## Deferred

- Live `pnpm verify:trask-discord` (needs bot token in env)
- discord.com headed Playwright (operator-only)
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,10 @@ Authoritative display contract: [trask-citation-display-contract.md](../../knowl
citation-markers.ts ← regex + parseCitationIndex (**internal** — not exported from `@openkotor/trask` index)
query-anchor.ts ← BRIEF_DISCORD_MIN_CITATIONS, distinctiveAnchorTokens, claimMatchesQueryAnchor (**exported** from `@openkotor/trask` index)
research-answer-split.ts ← splitResearchAnswer, syncSourcesSectionToApproved (**exported** from `@openkotor/trask` index)
grounded-evidence.ts ← compose, claims, sufficiency (imports split + anchor; re-exports anchor)
github-citation-url.ts ← shallow repo → blob permalinks, webCitationDisplayLabel, passage path inference (**exported**)
grounded-evidence.ts ← compose, claims, sufficiency (imports split + anchor + github-citation-url; re-exports anchor)
discord-reply-format.ts ← line filters, embedInlineCitationLinks (imports markers, anchor, split; NOT grounded-evidence)
apps/holocron-web/src/lib/answer-presentation.ts ← Holocron Sources panel labels (`webCitationDisplayLabel` via `@openkotor/trask/github-citation-url` — browser-safe subpath)
```

[SYNTH] **No cycle** on answer parsing (split) or display anchors (query-anchor). Compose still owns claim selection; display imports anchor helpers only.
Expand Down
Loading
Loading