Setup SSH access on EC2

1. When creating an EC2 instance, you have a chance to download key pair (.pem file) for the instance

2. The pem file is the private key for accessing the EC2 instance. Change the extension .pem to .ppk

3. Install and open putty. 

   Host Name: the public DNS of the EC2 instance. e.g  ubuntu@ec2-54-148-40-159.us-west-2.compute.amazonaws.com

                       The 'ubuntu' here is the default user name. Different linux distribution has different default username

   Connections->SSH->Auth:

                      Choose the .ppk file as the private key

4. Click Ok to connect to the server

5. Add a new user group ('hadoop') and user ('hduser') for running hadoop

    sudo addgroup hadoop

    sudo adduser --ingroup hadoop hduser

6. Add sudo previlege to hduser

   sudo adduser hduser sudo

7. Switch to user hduser

   su hduser

   here we switch to hduser so the following step will generate keys for this user

8. On the EC2 instance, the SSH software has been installed, so just use it to generate private - public keys

   ssh-keygen -t rsa

   The command will generate id_rsa (private key) and id_rsa.pub (public key) under /home/hduser/.ssh directory

   Leave the password empty if you don't need protection to the keys

9. To be able to SSH to another EC2 instance, you need to copy the public key (i.e. id_rsa.pub) to the target instance. 

    9.1  set up hduser on the target EC2 instance

    9.2  enable password access to the target EC2 instance (this is not safe to will disable later)

           putty to the target instance

           sudo vi /etc/ssh/sshd_config

           change "PasswordAuthentication no" to "PasswordAuthentication yes"

           reboot the instance, sudo reboot

    9.3 go back to the original EC2 instance

          ssh-copy-id -i /home/hduser/.ssh/id_rsa.pub hduser@target_instance_address

          it will ask for the password.

          this command will copy the public key to the target instance under /home/hduser/.ssh/authorized_keys

    9.4 from the original EC2 instance, test connection to target instance

          ssh hduser@target_instace_address

          it should just connect without asking for password

    9.5 disable password access on the target instance

          simply change 'yes' to 'no'

10. repeat the process 9 for all the slave instances, so from the master instance it can ssh to any slave without password

      also there are other ways (e.g. pscp) to copy public key to target instance, ssh-copy-id is not the only way