Potential inconsistencies between the code and paper.

[Luizerko]

Hey guys. First things first, thank you very much not only for your work, but also for making it publicly available for everyone to try it :)

I'm writing you because I found some potential inconsistencies between your available implementation and your paper after carefully reading it. I wanted to clarify these points, both to make sure these were not simply misunderstandings on my end and to be more certain about the next steps I should take to potentially fix the issues and have a smoother paper reproduction:

1. In Algorithm 1 of the appendix, you say you initialize the perturbation with N(0, I), but in the code, the initial perturbation is given by torch.rand(*X.shape)*2*eps-eps, which corresponds to U(-eps, eps). What initialization should we actually use?

2. The perturbation computation introduced in Equation 5 seems to not correspond to what we implement in the code:

   a. The disparity between feature embeddings (f_{FE} which should correspond to loss_lpips in the code) has no lambda associated with it. Instead the lambda multiplies a regularization term (loss_encoder) never mentioned in the paper. Should we keep this regularization term? Should we multiply loss_lpips by lambda?

   b. The perturbation loss seems to be only given by the regularization term for the first 25% of iterations (loss = -loss_cvl * (1 if i >= iters * 0.35 else 0.0) + loss_encoder * 0.2 + loss_lpips * (1 if i > iters * 0.25 else 0.0)). This was also never mentioned in the paper and it's a bit concerning because Equation 5 was the central idea of your method. Should we use the core loss terms from the beginning?

   c. This one is not exactly a question. The disparity between feature embeddings is computed using LPIPS model in the code, which is only introduced for the evaluation pipeline in the paper. I think it's a bit concerning because your method seems to achieve the best LPIPS score among your experiments and it might lead readers to think that this was a byproduct, that your method inherently got there. This is not the case, however, since you actively optimize for it. There's nothing wrong with getting the best score for a target you optimized for, of course, but I think it should be mentioned that it was an optimization target.

3. What exactly is the l_infinity norm you mention on Table A1 of the appendix and where can I find it in the code?

4. You mention using 2000 images for your experiments. I assume these are the 2000 images from celeba-hq validation set, right?

5. I commented about this point on another issue, but I think it's worth saying it again. You only ever mention working with 512x512 images on table A2 of the appendix. Even the dataset your paper links us to contains 1024x1024 images, and the code has no resizing operation. I think it's nice to include that in the paper because using 1024x1024 images directly on your code leads to both a gigantic memory consumption and, even if you have the GPU to handle that, overflow during gradient computation.

Sorry to bother you with so many points, and thank you for your time and attention in advance, as well as the paper and the open-source work again!

EDIT:
Found the l_infinity norm in the code (point 3). I didn't realize it before, but it's given by:

X_adv = torch.minimum(torch.maximum(X_adv, X - eps), X + eps)
X_adv.data = torch.clamp(X_adv, min=clamp_min, max=clamp_max)