What is the purpose of downloading "1 Billion Word Language Model Benchmark" in the code? Is it directly generated using one hot vector?

[ykcheng9966]

Thank you for publishing the model. Recently, after reading your article, I saw in the code that you downloaded the dataset "1 Bill Word Language Model Benchmark". This dataset seems to be unrelated to the Escherichia coli promoter. What is the purpose of this dataset?

I also want to ask you a question. Do you directly process the startup Subsequence into one hot vector in the code and generate it with GAN? Is there a dimensionality increase operation on the one-hot matrix?

Looking forward to your reply!

[Iris7788]

你的邮件我已经收到啦，我会尽快查收哒~

[HaochenW]

Thanks for your questions.

1. Actually it is no need to download the dataset, and I think it is a bug when I wrote the code.
2. We do not have the dimension reduction policy and directly use one-hot vector. If you have good encoding methods, I am very glad to learn from them.