Repository navigation
[GPU-MPC] Sigma Loading Model Weights #240
Description
Activity
I'm also trying secure inference with BERT-tiny. I found that there is a load function under module.h that can be used to load model weights. As I understand it, the weights will be part of the input for key generation and online inference. The weights I'm using are from a PyTorch pre-trained model. I tried to use the load function to read in the binary file of the weights for secure inference, but I didn't succeed. I hope this information is helpful to you, and I hope that if you successfully implement secure inference with Llama, you could let me know how you did it.
I am trying to use sigma to make inference with llama2 7b model weights with GPU-MPC/experiments/sigma/sigma.cu or sigma_offline_online.cu.
However, I found that the created GPULlama were only initiated both in key generation phase and online inference phase. Could you show me where the server's model weights are loaded and used?