Skip to content

Conversation

Xernisfy
Copy link

@Xernisfy Xernisfy commented Nov 5, 2024

I scraped Bulbapedia for the new cards from TCG Pocket.
Some props like rarity and artist are missing (because it's tricky to parse), I'll add them as soon as possible.

@Xernisfy
Copy link
Author

Xernisfy commented Nov 5, 2024

Related to #553

@Xernisfy
Copy link
Author

Xernisfy commented Nov 6, 2024

I've added all missing fields now.
Please let me know if this is alright or if anything can be improved.
If card images are needed, I can fetch them as well.

Comment on lines 2851 to 2858
{
"id": "tcgp1",
"name": "Genetic Apex",
"series": "TCG Pocket",
"printedTotal": 226,
"total": 286,
"ptcgoCode": "A1"
},
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey man, just asking, is there a reason why you didn't include the releaseDate and images properties, that are present in other sets, for the Pocket sets? I'm planning on doing a small personal project related to TCG Pocket using this API, so I'm just wondering if these fields will be available.

@Xernisfy
Copy link
Author

Thanks @Joaos98 for pointing that out. Fields are now up to date.
I updated my scraping tool to generate the sets/en.json file automatically now.

@adback03
Copy link
Member

Thanks for this - going to be reviewing and potentially getting this added in the next week or so.

@adback03
Copy link
Member

@Xernisfy, would you happen to have the newest expansion as well - Mythical Island?

@Xernisfy
Copy link
Author

Sorry for answering so late.
The new commit should include all sets and cards up to now.
Maybe I could turn my scraping tool into a GitHub action to run on demand?

Arno Klein added 3 commits April 13, 2025 03:14
- add regulation marks
- correct rarities (Bulbapedia made them more detailed)
- added new evolutions for older sets
- remove evolutions for ex (namely Pikachu ex)
- fix missing attacks (namely Meowth's Pay Day) (still incorrectly on Bulbapedia)
- slot sets in chronologically by release date (global)
- add regulation marks
- correct rarities (Bulbapedia made them more detailed)
- added new evolutions for older sets
- remove evolutions for ex (namely Pikachu ex)
- fix missing attacks (namely Meowth's Pay Day) (still incorrectly on Bulbapedia)
- slot sets in chronologically by release date (global)
@Xernisfy
Copy link
Author

I dockerized my scraping tool and made it available as an on-demand GitHub Action.
Feel free to use or modify it!
It will do the following:

  • Use my tool to scrape Bulbapedia for all TCG Pocket cards
  • Create a new branch
  • Commit the changes to the new branch
  • Create a merge request so the changes can be manually reviewed and approved
name: import TCG Pocket cards

on:
  # make this action "on-demand"
  workflow_dispatch:

jobs:
  import:
    runs-on: ubuntu-latest
    permissions:
      # needed to commit changes
      contents: write
      # needed to download the importer package
      packages: read
      # needed to create a pull-request
      # make sure "Allow GitHub Actions to create and approve pull requests" is enabled in Repository > Settings > Actions > General
      pull-requests: write
    container: ghcr.io/xernisfy/pokemon-tcg-importer
    steps:

      - name: install Git and GitHub CLI in container
        run: |
          apk --no-cache add git github-cli
          git config --global --add safe.directory "$GITHUB_WORKSPACE"

      - uses: actions/checkout@v4

      - name: import cards
        run : |
          cd /app
          deno run -A src/main.ts bulbapedia

      - name: copy cards to repo
        run: cp -r /app/out/* .

      - name: commit cards to new branch
        run: |
          git config --global user.name 'Xernisfy'
          git config --global user.email '[email protected]'
          git checkout -b tcgp-import-${{ github.run_id }}
          git add .
          git commit -m 'import TCG Pocket cards'
          git push origin tcgp-import-${{ github.run_id }}

      - name: create pull request
        env:
          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        run: |
          gh pr create \
            --title "Import TCG Pocket cards - ${{ github.run_id }}" \
            --body "This PR contains newly imported cards for TCG Pocket" \
            --base main \
            --head tcgp-import-${{ github.run_id }}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants