[QUESTION]Where does the attention_mask come from when the gpt_model is not the first or last pipeline stage?

[janelu9]

I know the hidden_states are the output of previous stage, but I don't understand the how the attention_mask is passed to the next transformer block.

[necrophagists]

Hi,do you how the attention_mask is passed to the middle pp stage now? I am facing this problem now.