Deep Dive on Elastic Block Store

  • If your data and workflow require a file system to store files or database data then Amazon EBS is the best storage option. Amazon EBS provides many features such as high durability and reliability, encryption, provisioned IOPS, and point-in-time snapshots amongst others. The built-in volume snapshot feature is a good option for backing up data.
  • EBS drives are network attached drives, behave like a normal block drive. They are block storage to EC2 instances. What is more important to you in terms of your application performance? Disk IOPS or Throughput? This is the critical question to be asked when thinking of different types of EBS options. EBS is directly attached to the instance, so they are in the same AZ always. Tied to AZ because of the latency issue
  • You cannot resize an existing EBS, you can only snapshot with a larger volume. Use Lambda scheduled events for snapshotting EBS volumes
  • Root volumes cannot be encrypted by default, you need a third party tool (such as bit locker etc) to encrypt the volume
  • Volumes occupy on Availability Zones. You can utilise Snapshots to provide availability.
  • Performance Measurement:
    • Throughput vs IOPS. IOPS is the number of input output operations per second that can occur to a given storage device.
    • IO size: The size of a particular IO operation. Let’s say you have 1 Mb IO size and with a given number of IOPS, you will workout the maximum throughput of the device. 1 Mb and 1750 IOPS and then you have a throughput of 1750 MiB/s
  • Max you can ever get from EBS is 80.000 IOPS and 1750 MiB/s throughput.

SSD General purpose gp2

  • Great for boot volumes, low-latency apps, and bursty database workloads. Default for almost allow
  • 3 IOPS per GB up to 16.000 IOPS - burstable to 3000 IOPS for volumes up to 1 TB
  • Credit Bucket: 5.4 million I/O credits stored if you want additional performance and consume credits. Max performance is 16,000 IOPS p/vol
  • Throughput: 250 MB/s and Latency: Single-digit milliseconds
  • Capacity: 1GB to 16 TB

SSD Provisioned IOPS io1/io2

  • This EBS type is ideal for critical applications and databases with sustained IOPS - extreme IOPS needs
  • 100 to 64.000 IOPS per volume
  • Throughput: 1,000 MB/s and Latency: Single-digit milliseconds
  • Capacity: 4 GB to 16 TB
  • There are also instance limitations as well.
  • io1 supports multi-attach where you can attach the volume to multiple instances.
  • io2 supports up to 500 IOPS per GB.

HDD Throughput optimized st1

  • Ideal for Big Data Analytics solutions, large-block, throughput sequential workloads and cost savings compared to io1. Media based data where it’s sequentially stored on the disk, it’s not good for random IOPS type of workloads.
  • Baseline: 40 MB/s per TB up to 500 MB/s
  • Burst: 250 MB/s per TB up to 500 MB/s
  • Capacity: 500 Gb to 16 TB
  • IOPS: 500 max

Cold HDD sc1

  • This is compliment to st1 and used for backup, archival, and logging. Substantial cost savings compared to st1
  • Baseline: 12MB/s per TB up to 250 MB/s
  • IOPS: 250 max

EBS Elastic Volumes

  • You can increase the size of the volume on the go but you cannot decrease
  • You can also change the volume, only applicable to gp2, io1, st1, sc1
  • Ability to decrease or increase provisioned IOPS

Where can I use EBS Volumes?

  • In all regions you can use EBS Elastic Volume
  • M3 Medium: You can do live volume modifications. The rest, you need to either start/stop or detach/attach the modify.
  • Each volume can be modified once every 6 hours!
  • The volumes of EBS are only available in the region they are created but you can take snapshots of them to another region.
  • Data that is stored on an Amazon EBS volume will persist independently of the life of the instance.
  • If the root device for your instances is an EBS volume, you can change the size of the instance simply by changing its instance type, which is known as resizing it. If it is an instance store volume, you must migrate your app to a new instance with the instance type you want.
  • On an EBS backed instance, the default action is for the root EBS volume to be deleted when the instance is terminated You don’t need to stop or disassociate the EBS instance if you need to upgrade your instance, just change the instance type and it’s ok. Stop/start is fine to perform this.

Case Scenario

  • Take a snapshot, the snapshot excludes data held in the cache by applications and the OS. This tends not to matter on a single volume, however using multiple volumes in a RAID array, this can be a problem due to interdependencies of the array. How would you achieve this? Solution Take an application consistent snapshot Stop the application from writing to the disk
  • Flush all the caches to the disk
  • Freeze the file system
  • Unmount the RAID array
  • Shutting down the associated EC2 instance

Automated backup retention period for your RDS DB is 35 days

  • You can only reimport and export data from an existing DB instance to another instance in a different VPC
  • You can only reimport and export data from an existing DB instance to another instance in a different VPC

Storage Array Considerations

  • RAID 0 allows you to distribute I/O across all volumes in a stripe, allowing straight gains with each addition.
  • RAID 1 can be used for durability to mirror volumes, but in this case, it requires more Amazon EC2 to Amazon EBS bandwidth because the data is written to multiple volumes simultaneously. If you’re doing this, definitely make sure you’re using Amazon EBS–optimization.

Instance Profile

  • Credentials are get from an IAM role to the application. IAM roles provide short term or temporary access credentials when used with sts:AssumeRole API call. Identities within AWS can be granted access to assume roles using that roles ‘Trust’ policy.
  • Instance profiles allow a role to be assigned to a single EC2 instance, and for applications running on that instance to assume a role while being abstracted away from any AWS identity. Applications running on EC2 instances are not AWS identities and so it’s the instance profile that bridges the gap.

Instance Store Volume

  • These are disks physically in an EC2 host.
  • A selection allocated to EC2 instances, different types.
  • Data lost if hardware fails or instance moves between hosts (stop/start action as well).
  • They provide the highest performance (IOPS and MB/s) because they are locally connected.

EC2 + EBS Instance IOPS Limits

  • Raid 0 Stripe: Let’s say you have io1 and io2 attached to an instance. You can create a Raid 0 Striped configuration to achieve 128K IOPS at a time with one logical volume.
    • However, EC2 instances have their own performance limits. Maximum for any instance is 160K IOPS. You need to use instance store for more than 160K IOPS.
  • Raid 1 Mirror: Data is written onto a number of EBS volumes, writing twice. Can tolerate 1 volume failure. Not much used as there are backups and snapshots occuring in AWS anyway.