Skip to content

Sync: crawl past Meetup events, improve parsing and debug, and expand workflow triggers#17

Merged
carloshvp merged 2 commits intomainfrom
codex/find-workflow-for-updating-events-75t6q6
Apr 3, 2026
Merged

Sync: crawl past Meetup events, improve parsing and debug, and expand workflow triggers#17
carloshvp merged 2 commits intomainfrom
codex/find-workflow-for-updating-events-75t6q6

Conversation

@carloshvp
Copy link
Copy Markdown
Member

Motivation

  • Improve event coverage by harvesting recent historical events from Meetup in addition to the iCal feed so the site can show recent past meetups.
  • Make the sync behavior more observable and debuggable in CI by emitting diagnostics and exposing a debug flag.
  • Allow validation on pull requests and avoid committing from non-push/schedule/manual contexts to reduce unexpected pushes.

Description

  • Expanded GitHub Actions workflow to trigger on pull_request, set MEETUP_SYNC_DEBUG by default in the job, and only run the commit step for push, schedule, or workflow_dispatch events, and enhance the run-time stats output to show the most recent past event and URL.
  • Updated README.md to document MEETUP_PAST_EVENTS_URL and MEETUP_SYNC_DEBUG and to reflect the new workflow triggers and behavior.
  • Enhanced scripts/sync_meetup_events.py with debug logging, unescaping of iCal text, better JSON-LD parsing (including @graph nodes), utilities to extract event URLs from meetup HTML, a fetch_url wrapper, merging of iCal and past-event lists, and a crawl of the events/past page (and individual event pages) to supplement parsed past events.
  • Added unit tests in tests/test_sync_meetup_events.py covering extract_event_urls_from_html and JSON-LD @graph parsing.

Testing

  • Ran the new unit tests with python -m unittest tests.test_sync_meetup_events and the two included tests passed.
  • Manual linting and basic smoke-run of scripts/sync_meetup_events.py were exercised during development to verify parsing branches (no automated failures reported).

Codex Task

@carloshvp carloshvp merged commit d0e18bf into main Apr 3, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant