feat(images): opt-in VLM transcription of image-dominant pages#94
Merged
Conversation
added 2 commits
July 7, 2026 16:33
Stage 2 of image-dominant page support. When a page is image-dominant and transcription is enabled, content images are downloaded (streamed, per-image byte cap), tall marketing images are selected (MIN_IMAGE_HEIGHT_PX=800, MIN_ASPECT_RATIO=1.5 -- tall banners carry text, square gallery photos do not; max 5 per fetch), transcribed via the vision LLM at TRAWL_VLM_URL, and spliced into the markdown as '## Image text (...)' sections BEFORE chunking, so transcripts embed, retrieve, rerank, and pass the injection scan like normal page text. New module src/trawl/image_transcribe.py: dimension probe (PIL if present, else fitz.Pixmap -- pymupdf is already a hard dep; Pillow is not installed in the trawl env), transcript cache keyed on image sha256 + model + prompt version (TTL 7 days), enable_thinking:False + reasoning_content fallback per the Gemma quirk. Pipeline: the image scan moved from post-rerank to a pre-chunking helper (_scan_and_transcribe_images) shared by both paths, which also resolves the duplicated scan+warn block from PR #93. When transcription is requested the C8 fetch-cache read is bypassed (cache stores no raw HTML; write still happens), so transcribe_images=True works deterministically on recently-fetched URLs. Profile path chunk_ms excludes transcription time. MCP fetch_page gains a transcribe_images param (JSON null defers to env). Default OFF: TRAWL_IMAGE_TRANSCRIBE=1, fetch_relevant(transcribe_images=True), or the MCP param. PipelineResult.images_transcribed reports the count. Pre-registered gates, all PASS on whalesleep WS-1 (reference page): G1 rank-1 chunk is the density transcript for query '메모리폼 밀도가 왜 중요한가' (top-6 has 4 transcript chunks incl. spec dimensions); G2 parity 15/15 with flag OFF and ON; G3 OFF path +0 ms, ON cold 47 s (5 transcriptions); G4 warm transcript cache 6.5 s (~OFF + 1 s). MCP smoke test PASS, pytest 48 pass offline. Known limits (follow-ups, not in this change): transcript chunks inherit the page's extractor/source_selector (no provenance marker); leading gallery images are fully downloaded before the tall filter rejects them; no failed-host cooldown on the VLM (feature is per-call opt-in); telemetry does not record images_transcribed.
Follow-up to d228f1b -- this test file was part of the review-fix batch but missed the commit.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Stage 2 of image-dominant page support (follows #93). When a page is image-dominant and transcription is enabled, content images are downloaded (streamed, 4 MB/image cap), tall marketing images selected (height >= 800 px, aspect >= 1.5, max 5/fetch), transcribed via the vision LLM, and spliced into the markdown as
## Image text (...)sections before chunking — so transcripts embed, retrieve, rerank, and pass the injection scan like normal page text.src/trawl/image_transcribe.py: dimension probe (PIL if present, elsefitz.Pixmap— pymupdf is already a hard dep), transcript cache (image sha256 + model + prompt version, 7-day TTL), Gemmaenable_thinking:false+reasoning_contentfallback.chunk_msexcludes transcription time.fetch_pagegainstranscribe_images(JSON null defers to env).PipelineResult.images_transcribedreports the count.TRAWL_IMAGE_TRANSCRIBE=1/fetch_relevant(transcribe_images=True)/ MCP param.Pre-registered gates (all PASS, whalesleep WS-1 reference page)
Known limits / follow-ups
images_transcribed