loss function returns nan 

I had the problem of running the code and got nan from the loss function after 2-3 iterations. While testing the problem I saw that some parameters such as gamma and beta in Layernorm are derived (requires_grad is true )and so are namd_hz_ and low_hz_in SincConv_fast.  

I changed Layernorm to nn.Layernorm and it resolves the problem