what is the difference between hf unilm and microsoft unilm

[huangrenjie0]

does the optimization also apply to https://github.com/microsoft/unilm? do you also implement optimization similar to faster transformer?

[JiushengChen]

@fuliucansheng, could you help answer this question?

[fuliucansheng]

Sure, so sorry for the late reply. I have missed the email.

[fuliucansheng]

1. The official unilm repo has more tasks like unilm-v1/unilm-v2/infoxlm pretrain & finetune on different benchmarks and more options for model training experiments. The hf unilm in the fastseq repo is mainly used for faster generation inference. Of course, the inference results are the same.
2. Fastseq is quite different from the repo NVIDIA/FasterTransformer. FasterTransformers is a C++/CUDA repo, and it has GPU kernels/memory optimization and something like that. Fastseq is a Python repo that is easy to use, and the optimization including cache/beam search/io/attention is mainly by python with C/CUDA extensions.