fix(data-sync): fix scraper not running in Bun ESM + add manual trigger#813
Merged
Merged
Conversation
- Replace `require.main === module` with `import.meta.main` in both sync-courses.ts and update-courses.ts — the old check doesn't work in Bun ESM (module: esnext), causing the Docker container to exit silently without ever running the scraper - Fix run-docker.sh to use `bun run` instead of `tsx` (tsx is not installed in the container) - Add workflow_dispatch trigger to build.yaml with a run-scraper job so the scraper can be manually triggered from the GitHub Actions UI with a configurable semester parameter https://claude.ai/code/session_01RbmtUdLaaURBWLK2BrXfok
Deploying with
|
| Status | Name | Latest Commit | Preview URL | Updated (UTC) |
|---|---|---|---|---|
| ✅ Deployment successful! View logs |
courseweb-web | 3b4c2cc | Commit Preview URL Branch Preview URL |
Jun 05 2026, 01:46 AM |
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Contributor
There was a problem hiding this comment.
Pull request overview
This PR updates the data-sync scraper entrypoints and tooling to run correctly under Bun ESM (preventing silent exits), and adds a manual GitHub Actions trigger to run the scraper on demand.
Changes:
- Switch “run directly” guards in
sync-courses.ts/update-courses.tsto Bun-friendly semantics. - Update
run-docker.shto invoke the scraper withbun runinstead oftsx. - Add
workflow_dispatch+ arun-scraperjob to allow manual scraper runs with a semester input.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
| tools/data-sync/src/update-courses.ts | Updates the scheduled sync CLI entrypoint guard for Bun ESM execution. |
| tools/data-sync/src/sync-courses.ts | Updates the one-shot sync CLI entrypoint guard for Bun ESM execution. |
| tools/data-sync/run-docker.sh | Uses Bun to run the sync scripts inside the Docker container. |
| .github/workflows/build.yaml | Adds a manually-triggerable Actions job to run the scraper with secrets. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
|
||
| // If this file is run directly, execute the sync | ||
| if (require.main === module) { | ||
| if (import.meta.main) { |
|
|
||
| // If this file is run directly, start the scheduled sync | ||
| if (require.main === module) { | ||
| if (import.meta.main) { |
Comment on lines
159
to
164
| # Set command based on mode | ||
| if [[ "$MODE" == "once" ]]; then | ||
| CMD="tsx src/sync-courses.ts $SEMESTER" | ||
| CMD="bun run src/sync-courses.ts $SEMESTER" | ||
| else | ||
| CMD="tsx src/update-courses.ts \"$CRON_PATTERN\" $SEMESTER" | ||
| CMD="bun run src/update-courses.ts \"$CRON_PATTERN\" $SEMESTER" | ||
| fi |
Comment on lines
+166
to
+172
| env: | ||
| SUPABASE_URL: ${{ secrets.SUPABASE_URL }} | ||
| SUPABASE_SERVICE_ROLE_KEY: ${{ secrets.SUPABASE_SERVICE_ROLE_KEY }} | ||
| ALGOLIA_APP_ID: ${{ secrets.ALGOLIA_APP_ID }} | ||
| ALGOLIA_API_KEY: ${{ secrets.ALGOLIA_API_KEY }} | ||
| SEMESTER: ${{ github.event.inputs.semester || '11510' }} | ||
| run: bun run src/sync-courses.ts ${{ github.event.inputs.semester || '11510' }} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
require.main === modulewithimport.meta.mainin bothsync-courses.ts and update-courses.ts — the old check doesn't work in
Bun ESM (module: esnext), causing the Docker container to exit silently
without ever running the scraper
bun runinstead oftsx(tsx is notinstalled in the container)
the scraper can be manually triggered from the GitHub Actions UI
with a configurable semester parameter
https://claude.ai/code/session_01RbmtUdLaaURBWLK2BrXfok