To set up SSH agent forwarding, first add your admin SSH key to your local SSH agent using the ssh-add command:
> ssh-add -K ~/.ssh/admin.pem
Identity added: /Users/jmenga/.ssh/admin.pem (/Users/jmenga/.ssh/admin.pem)
> ssh-add -L
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCkF7aAzIRayGHiiR81wcz/k9b+ZdmAEkdIBU0pOvAaFYjrDPf4JL4I0rJjdpFBjFZIqKXM9dLWg0skENYSUl9pfLT+CzValQat/XpBw/HfwzbzMy8wqcKehN0pB4V1bpzfOYe7lTLmTYIQ/21wW63QVlZnNyV1VZiVgN5DcLqgiG5CHHAooMIbiExAYvRrgo8XEXoqFRODLwIn4HZ7OAtojWzxElBx+EC4lmDekykgxnfGd30QgATIEF8/+UzM17j91JJohfxU7tA3GhXkScMBXnxBhdOftVvtB8/bGc+DHjJlkYSxL20792eBEv/ZsooMhNFxGLGhidrznmSeC8qL /Users/jmenga/.ssh/admin.pem
The -K flag is specific to macOS and adds the passphrase for your SSH key to your OS X keychain, meaning that this configuration will persist across reboots. If you are not using macOS, you can just omit the -K flag.
You can now access your Swarm manager using the -A flag, which configures the SSH client to use your SSH agent identities. Using the SSH agent also enables SSH agent forwarding, which means that the SSH key used to establish your SSH session with the Swarm manager can be automatically used or forwarded for other SSH connections you might establish from within your SSH session:
> ssh -A docker@54.145.175.148
Welcome to Docker!
~ $ ssh docker@ip-172-31-27-91.ec2.internal
Welcome to Docker!
As you can see, using SSH agent forwarding solves the issue of being able to access your worker nodes.