Download File From AWS Private ec2 To Local

In one of my project there was a scenario where I had to download a file from AWS ec2 instance of a private subnet vpc to my local machine. I could do it with the help of AWS s3 and I could upload this file on s3 (configure aws on machine) then download this file from s3 to my local machine.

But it is not the solution as it –

1. Required an extra overhead of uploading file on s3.
2. Will take some space on s3 which we need to pay money for that.
3. Is not automation friendly approach.

First of all let me give some understanding of aws architecture implemented in our project.

1. There were multiple ec2 instances were running on multiple private subnets VPC.
2. In order to get connected with these ec2 instances there were an instance running in a public subnet VPC. 
3. Whenever we need to connect(ssh operations) with private subnet instances, we take help of public subnet instance because all the instances are in same VPC.

One more thing, whenever I had to connect with aws instance I was not using pem file because I had already put my public key on my user’s authorized_keys file. See how to connect with ec2 without giving pem file here

See how to create private and public keys here

So this is the one line command to download file from AWS private ec2 instance to local machine directly throw bastion(public subnet ec2 machine).

ssh-add ~/.ssh/id_rsa ; ssh -oStrictHostKeyChecking=no -Att -l username B ssh -oStrictHostKeyChecking=no -Att -l username C scp PATH_TO_FILE_ON_EC2 username PUBLIC_DNS:~/. ; scp  username@PUBLIC_DNS:FILE_NAME . ; ssh -oStrictHostKeyChecking=no -Att -l username B rm FILE_NAME

Let’s understand scp command to download file from AWS private instance- 

We can divide this one line command into multiple lines.

#1. ssh-add ~/.ssh/id_rsa ;

#2. ssh -oStrictHostKeyChecking=no -Att -l username B ssh -oStrictHostKeyChecking=no -Att -l rajatgupta C scp PATH_TO_FILE_ON_C/FILE_NAME username B_PUBLIC_DNS:~/. ;

• C = Private IP of main instance where the file exists.

• • PATH_TO_FILE_ON_C = Give the full path of file on main ec2 instance(C) which is in private subnet.
  • FILE_NAME = File name with extention.
  •  Username = There is an user created on ec2 instance(C). On /home/username/.ssh there is a file authorized_keys where my public key is stored.