Source code for tokenization

[Guest400123064]

Hi Team, thanks for sharing the training code. However, I have a few follow-up questions regarding the training code and a few details in the original paper:

• Is the open-sourced training data pre-tokenized? If so, does the team plan to open-source the tokenizer code?
• The Algo 1 in the original paper does not seem to contain the vertex swap operation. For instance, suppose we have v1, v2, v3 and v1, v3, v4, it seems that, according to Algo 1, v1, v3, v4 will be filtered out since the face is not sharing the edge v2-v3.
• Could the team delineate in more detail how the tokenizer processes v1, v2, v3 and v3, v2, v4?

Thanks again.