DO NOT MERGE feat: add script to merge all tokens into a single JSON file

[srambach]

AI-generated code via Cursor to output the deep merge of all the exported token files as JSON token format. First commit puts all into one file, but that would mix dark/default themes, so in the second commit I separated them into two files.

@garrett perhaps this helps?

[garrett]

I tried this with node packages/module/merge-tokens.js and the resulting JSON files are not properly loading in Penpot. Only colors load, and most of the colors are broken references. The basic color palette seems to work. Nothing other than colors work. Additionally, there's no Default vs Dark.

[garrett]

Thanks again for your start on this!

I kind of went a bit overboard and did some type sniffing based on name and content, to map the tokens into W3C types for standards support and Penpot compliance.

I also split off sets of tokens, so that Definitions are a set (so they can be avoided, generally, unless a certain hardcoded version is needed), Semantic ones are set (which is the main one to use), and Charts are separate (to avoid flooding all the colors with charts). Additionally, there are Dark and Chart-Dark overrides for dark support, selectable in the themes dropdown.

Anyway, it's a bit hackish, especially in some places, but it works for me and produces a W3C token compatible file that works in Penpot!

Penpot is supporting new token types soon, so some might just work (I added in some future support for them), and there are passthroughs from this too. There are more tokens in the resulting file than Penpot can work with currently, but it already handles all the main ones! (Currently, there aren't shadows or font/type related tokens. But all the rest of the most useful ones are there already.)