Take a look at https://www.openssh.com/agent-restrict.html
This should enable more straightforward support for when multiple deploy keys are in use with docker.
All we have to do it seems is, really, just pass the host as a -h argument when invoking ssh-add.