EFS


Basics

Elastic - growing & shrinking automatically

Managed file infrastructure - no patching, no system configuration

Supports tags

The resource in EFS is File System


Access

Accessible to

  • Multiple EC2 at same time
  • Within same region
  • Covering all (multiple) AZ in the region
  • EC2 owned by another account in same VPC
    • through Working with Shared VPCs in the Amazon VPC Peering Guide
  • From another VPC
    • use a VPC peering connection or transit gateway to connect VPCs

Protocol: NFS v4 (4.1 & 4.0) - 4.1 recommended (support depending on AMI)

Mount

  • To EC2
  • In VPC (only one VPC at a time)
    • Create 0..1 mount target in each AZ in one VPC, a mount target provides an IP address for an NFSv4 endpoint
      • Recommended: one for each AZ, multiple subnets in same AZ share one mount point in a subnet
    • Mount file system using the mount target's DNS
    • Create one mount target in each AZ, in one of the subnets, and all EC2 in the AZ in all subnets share the mount target
    • Mount targets are highly available.
    • IP addresses and DNS for mount targets in each AZ are static.
    • Available to on-premises hosts through Direct Connect (cable connect) to VPC (use IP not DNS for simplicity)
  • Recommended mounting option: https://docs.aws.amazon.com/efs/latest/ug/mounting-fs-mount-cmd-general.html
  • Linux mount tool available - see 'tools'
$ sudo mount -t nfs -o nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2,noresvport file-system-DNS-name:/ ~/efs-mount-point

File system access semantics, such as strong data consistency and file locking

Mount Target

Has:

  • mount target ID
  • subnet ID (in which)
  • file system ID (for which)
  • an IP address (at which to mount) / DNS name
  • mount target state
  • associated Security Group (firewall): max 5 each mount target

Access Control

Operate on the File System

IAM control

POSIX permissions

  • by default only one '/' dir & only root (UID 0 ) has read-write-execute permission
  • no root-squashing:
    • if UID=0, ignore permission check (full access whatever)
  • permission caching (enabled, so might be small window using expired permission)
  • chown_restricted enforced:
    • only root user can change the owner
    • root or the owner user can change the owner group
      • unless the user is root, the group can only be changed to one that the owner user is a member of
  • user & group - numeric ID used (EFS don't care name) - best keep ID consistent
  • read
  • write
  • exec

IP Mapper (not recommended)

The NFS utilities in the operating system include a daemon called an ID Mapper that manages mapping between user names and IDs. In Amazon Linux, the daemon is called rpc.idmapd and on Ubuntu is called idmapd.

File System

Properties:

  • Meta data
    • Name
    • Size - dynamic
    • Creation time
    • FilsSystemId
    • Creation Token (to ensure idempotent, i.e. repeated/redundant operation as single, operation)
  • Performance Mode: General Purpose and Max I/O
  • Throughput Mode: Bursting or Provisioned
  • Encryption
  • LifeCycle Policy: select from a set of available
  • Owner
  • Mount Targets:
    • select one VPC
    • multi-select AZ
    • select one subnet for each AZ, then manual assign IP or have IP automatically assigned, assign security group
  • Tags

Encryption

At rest

Enable at creation

In transit

Enable at mounting

Payment

Pay for usage

Storage classes & Life cycle Management

Classes:

  • Standard and
  • Infrequent Access (lower cost, no sacrificing the high availability, high durability, elasticity)

Using - through lifecycle management:

When enabled, lifecycle management migrates files that haven't been accessed for a set period of time (7,14,....90 days) to the Infrequent Access (IA) storage class.

Backup

AWS Backup service

The EFS-to-EFS backup solution

It includes an AWS CloudFormation template that launches, configures, and runs the AWS services required to deploy this solution.

Backing Up Amazon EFS File Systems Using AWS Data Pipeline (legacy, not recommended)

Transferring Data Into EFS

AWS DataSync service recommended

Data Consistency

NFS close-to-open consistency:

  • An application performs a synchronous write operation (for example, using the open Linux command with the O_DIRECT flag, or the fsync Linux command).
  • An application closes a file

Applications that perform synchronous data access and perform nonappending writes have read-after-write consistency for data access.

Performance & Throughput

Performance Modes:

  • General
  • Max I/O: can scale to higher levels of aggregate throughput and operations per second with a tradeoff of slightly higher latencies for file operations

Throughput Modes:

  • (default) Bursting Throughput mode: scales as your file system grows
  • Provisioned Throughput mode: can specify the throughput

Store

Data and metadata across multiple Availability Zones in an AWS Region (EBS only in one AZ)

Tools

amazon-efs-utils

Opensource EFS tools for linux:

  • mount helper - recommended for mounting EFS