Tutorial: Deploy RWKV-v4 World with ncnn

[MollySophia]

Download weights from HuggingFace

https://huggingface.co/BlinkDL/rwkv-4-world/tree/main

Clone the repo rwkv-ncnn

git clone https://github.com/MollySophia/rwkv-ncnn
cd rwkv-ncnn

Convert the weights

python convert_model.py ~/Downloads/RWKV-4-World-CHNtuned-3B-v1-20230625-ctx4096.pth ./output fp32
# replace the path with your downloaded file's path

Build chat_rwkv_ncnn binary

See rwkv-ncnn#build-how-to

Run!

cd build
./chat_rwkv_ncnn ../output/model.ncnn.bin ../output/model.ncnn.param ../output/emb_weight.bin ../rwkv_vocab_v20230424.bin ../output/parameters.txt