AWS Storage Gateway is a hybrid cloud storage service that gives you on-premises access to virtually unlimited cloud storage. Customers use Storage Gateway to simplify storage management and reduce costs for key hybrid cloud storage use cases.

Storage Gateway allows storage of data in the AWS cloud for scalable and cost-effective storage while maintaining data security.

Exposes compatible iSCSI interface on the front end to easily integrate with existing backup applications and represents an other disk drive.

AWS Storage Gateway backs up the data in Amazon Storage as incremental EBS snapshots.

AWS Storage Gateway can run either on-premises, as a virtual machine (VM) appliance, or in AWS, as an EC2 instance. So if the on-premises data center goes offline and there is no available host, the gateway can be deployed on an EC2 instance.

Gateways hosted on EC2 instances can be used for disaster recovery, data mirroring, and providing storage for applications hosted on EC2.

AWS Storage Gateway, by default, uploads data using SSL and provides data encryption at rest when stored in S3 or Glacier using AES-256

AWS Storage Gateway performs compression of data in-transit and at-rest.

Storage Gateway Types: AWS Storage Gateway offers file-based, volume-based, and tape-based storage solutions

You can create EBS volume from Storage Gateway,

Supports a file interface into S3 and combines a service and a virtual software appliance.

Allows storing and retrieving of objects in S3 using industry-standard file protocols such as Network File System (NFS) and Server Message Block (SMB).

Software appliance, or gateway, is deployed into on-premises environment as a virtual machine (VM) running on VMware ESXi or Microsoft Hyper-V hypervisor.

Provides access to objects in S3 as files or file share mount points. It can be considered as a file system mount on S3.

Provides a cost-effective alternative to on-premises storage.

Provides low-latency access to data through transparent local caching.

Manages data transfer to and from AWS, buffers applications from network congestion, optimizes and streams data in parallel, and manages bandwidth consumption.

Easily integrates with AWS services like IAM, KMS, CloudWatch etc.

File Gateway allows you to

• store and retrieve files directly using the NFS version 3 or 4.1 protocol.

• store and retrieve files directly using the SMB file system version, 2 and 3 protocol.

• access the data directly in S3 from any AWS Cloud application or service.

• manage S3 data using lifecycle policies, cross-region replication, and versioning.

Volume gateways provide cloud-backed storage volumes that you can mount as Internet Small Computer System Interface (iSCSI) devices from your on-premises application servers. For Volume gateways all data is securely stored in AWS, the approach differs with how much data is stored on-premises.

Volume gateway provides an iSCSI target, which enables you to create volumes and mount them as iSCSI devices from your on-premises or EC2 application servers.

Data is stored in S3 and acts as a Primary data storage

In the cached mode, your primary data is written to S3, while gateway retains a copy of recently read data locally for low latency access to the frequently accessed data,

Gateway-cached volumes offer a substantial cost savings on primary storage and minimize the need to scale your storage on-premises.

Each gateway configured for gateway-cached volumes can support up to 32 volumes, with each volume ranging from 1GiB to 32TiB, for a total maximum storage volume of 1,024 TiB (1 PiB).

Gateway-cached volumes can be attached as iSCSI devices from on-premises application servers.

Gateway-cached volumes can be backed up incrementally by taking snapshots which are stored as EBS snapshots in Amazon S3. These snapshots can be restored as gateway storage volume or used to create EBS volumes (if < 16TiB) to attached to an EC2 instance.

All gateway-cached volume data and snapshot data is stored in Amazon S3 encrypted at rest using server-side encryption (SSE) and it cannot be accessed with S3 API or any other tools.

Gateway VM can be allocate disks

• Cache storage

  • Cache storage, acts as the on-premises durable storage, stores the data before uploading it to Amazon S3

  • Cache storage also stores recently read data for low-latency access

• Upload buffer

  • Upload buffer acts as a staging area, before the data is uploaded to S3

  • Gateway uploads data over (SSL) connection to AWS, where it is stored encrypted in Amazon S3.

You can take point-in-time snapshots of gateway volumes made available in the form of Amazon EBS snapshots. A new EBS volume can be created from the snapshot which can be mounted to an existing EC2 instance.

Gateway-stored volumes maintain the entire data set locally to provide low latency access, while asynchronously backed up to AWS.

If you need low-latency access to your entire dataset, first configure your on-premises gateway to store all your data locally. Then asynchronously back up point-in-time snapshots of this data to Amazon S3.

Gateway asynchronously backs up point-in-time snapshots (in the form of EBS snapshots) of the data to S3 which provides durable off-site backups.

Gateway-stored volumes can be attached as iSCSI devices from on-premises application servers.

Each gateway configured for gateway-stored volumes can support up to 32 volumes, ranging from 1GiB to 16TiB, and a total volume storage of 192 TiB.

Actually Gateway stored volumes can only store only 512TB worth of data.

Gateway-stored volume configuration provides durable and inexpensive off-site backups that you can recover to your local data center or Amazon EC2. For example, if you need replacement capacity for disaster recovery, you can recover the backups to Amazon EC2.

Gateway-stored volumes can be backed up incrementally by taking snapshots which are stored as EBS snapshots in Amazon S3. These snapshots can be restored as gateway storage volume or used to create EBS volumes (if < 16TiB) to attached to an EC2 instance

• Gateway VM can be allocate disks

  • Volume Storage

    • For storing the actual data

    • Can be mapped to on-premises direct-attached storage (DAS) or storage area network (SAN) disks

  • Upload buffer

    • Upload buffer acts as a staging area, before the data is uploaded to S3

    • Gateway uploads data over (SSL) connection to AWS, where it is stored encrypted in Amazon S3

Gateway-virtual tape library (VTL) provides cost-effective and durable archival of backup data in Amazon Glacier.

Gateway-VTL provides a virtual tape infrastructure.

VTL interface lets you leverage your existing tape-based backup application infrastructure to store data on virtual tape cartridges that you create on your gateway-VTL.

Each gateway-VTL is preconfigured with a media changer and tape drives, which are available to the existing client backup applications as iSCSI devices. Tape cartridges can be added as needed to archive your data.

Gateway VTL has the following components :-

• Virtual tape

  • Virtual tape are similar to the physical tape cartridge, except that the data is stored in the AWS storage solution

  • Each gateway can contain 1500 tapes or up to 150 TiB of total tape data, with each tape ranging from 100 GiB to 2.5 TiB.

• Virtual tape library

  • Virtual tape library is similar to the physical tape library with tape drives (replaced with VTL tape drive) and robotic arms (replaced with Media changer)

  • Tapes in the Virtual tape library are backup in Amazon S3

  • Backup software writes data to the gateway, the gateway stores data locally and then asynchronously uploads it to virtual tapes in Amazon S3.

• Virtual tape shelf

  • Virtual tape shelf is similar to the offsite tape holding facility

  • Tapes in the Virtual tape library are backup in Amazon Glacier providing an extremely low-cost storage service for data archiving and backup

  • VTS is located in the same region where the gateway was created and every region would have a single VTS irrespective of the number of gateways.

  • Archiving tapes

    • When the backup software ejects a tape, the gateway moves the tape to the VTS for long term storage

  • Retrieving tapes

    • Tape can be retrieved from VTS only by first retrieving the tapes first to VTL and would be available in the VTL in about 24 hours

Storage Gateway Essentials