I cannot update the patch

[SuoHongfei]

Thank you for your contribution and sharing. When I reproduced the code, I found that the patch could not be optimized properly. Although all the loss values ​​were output normally, the gradient was always 0, and the gradient of the attacked network could not be accessed. How can I solve this problem?

[mayfly227]

Hi, sorry for the late reply. Do you mean the gradient of the patch is zero? Could you clarify under what circumstances the gradient cannot be updated?

[SuoHongfei]

Hi, sorry for the late reply. Do you mean the gradient of the patch is zero? Could you clarify under what circumstances the gradient cannot be updated?

Thank you for your reply!
To be more specific, when I reproduced TAOB's code, I found that the success rate of the attack was consistently low and showed no upward trend. During the debugging process, I found that the total loss of the code could be reasonably calculated, but a problem occurred during gradient optimization:

loss = (
total_conf
+ args.beta * ctx_loss
+ args.targetLoss * target_loss
+ args.tvloss * loss_tv
)

When I set the coefficient of tvloss to 0, the trained patch is almost indistinguishable from the initial patch.
Therefore, I output the specific gradient information during the training process:

grad = torch.autograd.grad(loss, patch, retain_graph=False, create_graph=False)[0]
print(grad)

The output "grad" is the all-zero tensor with dimensions [1, 3, 100, 200].
Since the gradient of the patch is 0, it cannot update specific pixels. Therefore, the patch we finally trained is the same as the initial patch. Do you know what caused this outcome?