Amazon Machine Images (AMIs) are used to build and launch instances. They store snapshots of EBS volumes, permissions and a block device mapping which configures how the instance's OS sees the attached volumes. AMIs can be shared, free or paid and can be copied to other AWS regions.

Step 1: Configure and Launch Instance from Source (AWS) AMI. Attach EBS volumes are configured with any required software and configurations.

Step 2: Stop the launched instance and create a snapshot. Snapshot creates a custom AMI of our launched instance preserving all data and configurations stored on EBS volumes attached to our Instance. It also includes instance permissions and block device (EBS) mappings. This type of AMI is also called EBS backed AMI as it preserves all block device (EBS) mappings and its widely used in production environments globally.

There are two types of AMI permissions. Public and Private.

Step 3: Launch instance from snapshot AMI taken in step 2. Note, we can select any type of instance type when launching an Instance from our AMI snapshot since the snapshots is essentially only for the EBS volumes attached to our original instance.

By creating an AMI of a launched instance, all configurations are baked into the AMI for faster future launch of new instances. Hence saving valuable time. AMI also helps with a mutable architecture where the configuration never changes once an application is launched.

AMIs are also used to launch new instances based on CPU load or instance health using Auto Scaling Groups.

Limitations of AMI include configurations, subnetting, IP addresses are built-in and cannot be changed. To overcome this limitation, we use Boot Strapping where we use User Data to build and customize instances before launching.

Bootstrapping is a process where instructions are executed on an instance during its launch process. Bootstrapping is used to configure the instance, perform software installation and add application configuration.

In EC2, user data can be used to run shell scripts (Bash or PowerShell) or run cloud-init directives.

There are two types of Instances:

• Private Instance - Elastic Network Interface only contains a private IP address and a private DNS address. This instance is not accessible from Public Networks and only accessible within the VPC this Instance resides in. Private IP is unchanged during the stop/start cycles. However, IP is released when instance is terminated. Multiple private IP addresses are assigned to ENI of private instance.

• Public Instance: Elastic Network Interface is allocated a public IP address (dynamic IP). Public IP address is NOT allocated to the OS of the instance. Infact, Public IP is allocated via Internet Gateway (IGW) residing in VPC. IGW is responsible for Network Address Translation (NAT) between Public IP and Private IP allocated to an Instance. Public IP allocated to an instance does not change or gets removed when an Instance is restarted. However, if an instance is Stopped, Public IP is removed and allocated a brand new Pubic IP when restarted. To overcome this issue, Elastic IPs are used which are static public IPs.

Instance Roles

EC2 instance roles are IAM roles that can be assumed by EC2 using an intermediary called an instance profile. An instance profile is either created automatically when using the console UI or manually when using the CLI. It's a container for the role that is associated with an EC2 instance.

The instance profile allows applications on the EC2 instance to access the credentials from the role using the instance metadata. Creating an instance role using console UI also creates an Instance profile. However, using CLI, we need to create both instance role and instance profile using two separate commands. Once an instance role is created, we can attach that instance profile to a resource (e.g. attach to EC2 to access S3 bucket). In EC2 control panel, right click on our running EC2 instance-->Instance settings-->Attach/Replace IAM Role-->IAM role-->select the role we have just created in IAM console UI.

Hands-on Lab

1. Log in to test lab AWS account

2. Navigate to EC2 and create a public instance. This instance will be used as a Bastion Host.

3. Go to IAM and download access keys for the user.

4. Create IAM Assume Role Policy document for an IAM Role (DEV_ROLE)

Open Command Line Interface (make sure that AWS command line is installed). SSH to the Bastion Host Instance.

ssh user_id@public_ip_of_instance

aws configure (input access key id and secret key, default region: us-east-1, default output format: json)

sudo nano trust_policy_ec2.json

copy below in nano

{

"Version": "2012-10-17",

"Statement": [

{ "Effect": "Allow",

"Principal": {"Service": "ec2.amazonaws.com"},

"Action": "sts:AssumeRole"

}

]

}

5. Create Dev IAM Role (DEV_ROLE) using Assume Role Policy Document

aws iam create-role --role-name DEV_ROLE --assume-role-policy-document file://trust_policy_ec2.json

6. Create S3 Dev Bucket Read Access file dev_s3_read_access.json

sudo nano dev_s3_read_access.json

Copy below text and paste in nano text editor window. Save and exit. Replace <DEV_S3_BUCKET_NAME> with actual S3 bucket name.

{

"Version": "2012-10-17",

"Statement": [

{

"Sid": "AllowUserToSeeBucketListInTheConsole",

"Action": ["s3:ListAllMyBuckets", "s3:GetBucketLocation"],

"Effect": "Allow",

"Resource": ["arn:aws:s3:::*"]

},

{

"Effect": "Allow",

"Action": [ "s3:Get*", "s3:List*" ],

"Resource": [ "arn:aws:s3:::<DEV_S3_BUCKET_NAME>/*", "arn:aws:s3:::<DEV_S3_BUCKET_NAME>" ]

}

]

}

7. Create the IAM Policy (DevS3ReadAccess) using dev_s3_read_access.json

aws iam create-policy --policy-name DevS3ReadAccess --policy-document file://dev_s3_read_access.json

8. Attach IAM Policy (arn of DevS3ReadAccess) to IAM Role (DEV_ROLE)

aws iam attach-role-policy --role-name DEV_ROLE --policy-arn "<POLICY_ARN_FROM_LAST_STEP>"

9. Verifying Configurations

aws iam list-attached-role-policies --role-name DEV_ROLE (This verifies the attached policy to role)

aws iam get-policy --policy-arn "<POLICY_ARN_FROM_LAST_STEP>" (This shows the policy details)

aws iam get-policy-version --policy-arn "<POLICY_ARN>" --version-id "<DEFAULT_VERSION_ID>" (This shows the permissions associated with the policy)

10. Create the Instance Profile (DEV_PROFILE) and add IAM Role (DEV_ROLE) to the Instance Profile

aws iam create-instance-profile --instance-profile-name DEV_PROFILE

aws iam add-role-to-instance-profile --instance-profile-name DEV_PROFILE --role-name DEV_ROLE

aws iam get-instance-profile --instance-profile-name DEV_PROFILE (This verifies the configuration)

11. Attach Instance Profile (DEV_PROFILE) to an Instance (e.g. Web Server)

aws ec2 associate-iam-instance-profile --instance-id <WEB_SERVER_INSTANCE_ID> --iam-instance-profile Name="DEV_PROFILE"

aws ec2 describe-instances --instance-ids <WEB_SERVER_INSTANCE_ID> (This verifies the configuration)

12. Test IAM Role (DEV_ROLE) permissions. SSH into Web Server Instance.

aws sts get-caller-identity

aws s3 ls

aws s3 ls s3://<DEV_S3_BUCKET_NAME>

How AWS Command Line Interface (CLI) credentials priority works?

In above, our webserver is accessing S3 buckets using Instance profile credentials (level-5) defined in VPC under IAM using Bastion Host. Now we will explore order of AWS CLI credentials order. Since Level-4 is container credential so we will skip this level here. Next is

AWS CLI credentials file (Level-3)

aws configure

This command creates a credentials file stored at ~/.aws/credentials on Linux, macOS, or Unix or at C:\Users\USERNAME\.aws\credentials on Windows. This file can contain the credentials for the default profile and any named profiles. This approach uses longer term credentials for the default profile and any named profiles. This approach uses longer term credentials stored locally on the instance and is NOT RECOMMENDED for production environments.

aws configure

input IAM user access, secret access key, default region: us-east-1, default output format: json

aws sts get-caller-identity (we should be able to see the ARN for our IAM user and this user will also be able to access S3 buckets.

Hand-on Lab: Using AWS Tags and Resource Groups

AWS Tagging Categories:

Technical Tags: Name, Application ID, Application Role, Cluster, Environment, Version

Tags for Automation: Date/Time, Opt in/Opt out, Security

Business Tags: Owner, Cost Center/Business Unit, Customer, Project

Security Tags: Confidentiality

Tags for Cost Allocation:

Tags for Access Control:

We have two Subnets; A and B, each having two webservers and one test webserver. Two S3 buckets; Developer and Production.

In AWS console, search Config and click Get Started. Check All resource; Record all resources supported in this region. Amazon S3 bucket-->Check Create Bucket. Amazon SNS topic, uncheck Stream configuration changes and notifications to an Amazon SNS topic. AWS Config role-->Check Use an existing AWS Config service-linked role. Click Next; Next; and Confirm.

In AWS console, Click EC2-->Instances. Click Mod.1 Web Server A. Select Actions-->Image-->Create Image. Image Name: Base AMI-20200626-->Create Image. Goto AMI-->Select AMI (being created). Bottom window, Select Tags-->Add Tag/Edits-->Input Key and Value-->Save. Once AMI is available, Select AMI-->Actions-->Launch. Accept default settings; on Add Tags page-->Input Key and Value and Click Next. Select an existing Security Group. Click Launch.

Select Resource Groups-->Tag Editor. Filter Resource Type-->EC2 Instances and S3 Bucket. Click Search Resources. Search for the desired resources and click Manage tags of selected resources. Click Add Tag--Tag Key: Module; Tag Value: Developer-->Review and Apply tag changes. We can now search Resources based on Tag key (Module) we have just created and assigned to our resources in step above.

Click Resource Groups-->Create a Group-->Create Query-Based Group. Check Tag based, Tags: Module and click Add. Click Preview Group Resources. In Group Details, Input Group Name-->Create Group. Now on left menu-->Saved Resource Groups.

Now back to EC2-->AMI-->copy the AMI ID. Now go to Config-->Rules-->Add Rule-->Approved AMI by Id-->Trigger; Scope of changes; Tags; input Resources by tag; Rule parameters; input Value (AMI ID copy earlier). Click Save.

Check Rule Name we have just created and it will mention that two non-compliant resources exist in this rule. Why? Because we have mentioned the AMI ID of our Image while our Resource Group contains other EC2 instances that are different to Image Instance AMI ID.

Next: Compute Advanced