Add web research context adapter#30
Conversation
Co-Authored-By: Codex <noreply@openai.com>
|
Bugbot is not enabled for your account, so this pull request was not reviewed. Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs. |
🤖 Codex PR ReviewPlease ensure a human reviewer checks this PR before merging. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0c86e20b2c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| with urlopen(request, timeout=self.timeout_seconds) as response: # noqa: S310 - operator-controlled research fetch. | ||
| body = response.read() |
There was a problem hiding this comment.
When a whitelisted source returns a 30x redirect, urlopen follows it and this code reads/parses the final response without validating response.geturl(). In that scenario, a configured source like https://allowed.example/feed can redirect to https://blocked.example/... or an internal metadata URL and bypass the whitelist, embedding non-approved content in the context bundle; validate the final URL after opening the response or disable redirects before reading the body.
Useful? React with 👍 / 👎.
| Path(args.web_research_sources), | ||
| timeout_seconds=args.web_research_timeout, | ||
| max_entries=args.web_research_max_entries, | ||
| ).build_context(pit_timestamp=generated_at) |
There was a problem hiding this comment.
When the CLI is run with both --end-date and --web-research-sources, this stamps/fetches web research at the current generated_at while the price context is limited to the requested historical end date. That produces a bundle whose as_of is historical but whose web_research contains current articles, which contaminates any point-in-time replay/evidence built from --end-date; either disable web research for historical builds or pass/filter by the requested as-of date.
Useful? React with 👍 / 👎.
| summary = child_text("summary", "description", "encoded") | ||
| title = child_text("title") or source_url | ||
| return { | ||
| "title": title, | ||
| "summary": summary, |
There was a problem hiding this comment.
For full-content RSS feeds, description/content:encoded can be the entire article, and this copies it into the bundle without any per-entry length limit; post_shadow_signal_request.py later embeds the whole bundle JSON in a GitHub issue body, so one whitelisted full-text feed can make the monthly issue creation fail or produce an unusably large operator notification despite max_entries being capped. Please truncate or size-limit the extracted title/summary fields before storing them.
Useful? React with 👍 / 👎.
| summary = child_text("summary", "description", "encoded") | ||
| title = child_text("title") or source_url |
There was a problem hiding this comment.
RSS titles and summaries are copied from the feed without escaping Markdown fence delimiters, and post_shadow_signal_request.context_markdown() embeds the resulting bundle inside a json fenced block. When a whitelisted or compromised feed item contains in its title or description, it can close the fence and inject arbitrary Markdown or mentions into the operator issue instead of remaining inert JSON; escape fence sequences or attach/render the JSON in a way that cannot be terminated by fetched text.
Useful? React with 👍 / 👎.
1 participant
Adds optional whitelist-based web/RSS research context collection for long-horizon context bundles and keeps default behavior unchanged.\n\nValidation: .venv/bin/pytest -q (38 passed), compileall passed.