Hard to follow the equation (7), (8) in the OptNet paper

[AtsushiHashimoto]

Thank you for your excellent work.
I am trying to understand the formulae of the OptNet paper, but some parts are hard to follow for me.

Especially in Eq. (7), the paper says

Thus, if we multiply the backward pass vector by the transpose of the differential matrix ...

But I cannot get the heart of the idea to multiply the transpose of the differential matrix (also, I wonder if the Q in the transpose in Eq. (7) is not Q^T?).

I could not understand how the equations in Eq. (8) are obtained from Eqs. (6) and (7).
It would help me so much if you could give me any hints.

[bamos]

Hi, thanks for taking a closer look! For better intuition on all of that, I recommend considering a quadratic function (e.g. y=x^T Q x) instead of the optimization problem (y=argmin ...) and working out the necessary derivatives for differentiating L(y) w.r.t. Q for some loss L. This will help see where the transpose comes up (from the chain rule) and a more manageable step from the derivatives from the differentials. Here are also a few alternative derivations:

• https://arxiv.org/pdf/1607.05447.pdf
• https://arxiv.org/pdf/1804.05098.pdf
• https://arxiv.org/pdf/1904.09043.pdf
• https://web.stanford.edu/~boyd/papers/pdf/diff_cvxpy.pdf

[AtsushiHashimoto]

I appreciate for the quick response and informative comments with references!