Amazon’s Elastic Compute Cloud (EC2) provides an environment for running virtual servers on demand. You can manage each virtual server like a physical machine, installing the software you need and configuring it to work the way you want, or you can use preprepared servers created by third parties. The service allows you to create a resizable pool of servers for handling computing tasks. You can start as many virtual servers as necessary to perform a task, increase or decrease the number of servers as demand rises and falls, and stop them all when the task is finished. You pay only for the computing resources you use. In addition to scaling out by increasing the number of servers that will work on a task, you can scale your computing power up or down by using more or less powerful virtual server types.
To help you run your own instances in the EC2 service, we will need to discuss two things: how to use the service’s API interface to launch and manage your servers and how to create your own customized servers.
We will discuss the EC2 Application Programming Interface (API) in the current chapter. We will start by describing how to use the service’s API operations to start, stop, configure, and monitor EC2 server instances, and then we will look at the API operations for managing the Amazon Machine Images (AMIs) that will store your customized servers. This chapter will give you the information you need to interact with the EC2 service and use pre-prepared server images, but we will not discuss how to configure your own servers.
In the next chapter, we will describe how you can create your own customized servers and store them permanently as Amazon Machine Images, ready and waiting to spring to life at your command. This chapter delves into the nitty-gritty aspects of running a Linux server in EC2. We demonstrate how to configure the server to take advantage of the features the environment offers, and we highlight some of the issues you will have to address to keep your servers and data safe.
Amazon offers command-line tools for interacting with the EC2 API and managing your Amazon Machine Images. You may wish to use these tools instead of implementing your own API client. Refer to the section on EC2 client tools in Appendix A. AWS Resources for further details.
The EC2 service comprises three key components:
EC2 instances are the virtual machines that run in the EC2 environment and perform computing tasks that would typically be done by physical servers.
Instances run in the EC2 environment, which provides configurable access control, contextual data, and other information that instances need to do their work.
Amazon Machine Images (AMIs) are files that capture a complete snapshot of an EC2 instance at a point in time, including its software, configuration, and potentially even its data. These images serve as the boot disk for the instances you launch.
An EC2 virtual machine instance is a Linux computer system to which you have full root (administrator) access. The instances run in a Xen virtual environment. This means that the underlying computer hardware is virtualized and can be tailored to give performance within defined specifications, regardless of what real physical hardware Amazon is using in their data centers. Despite the fact your instances do not run on hardware in the usual sense, they are configured to provide a well-defined amount of computing power.
To provide a baseline guide to the computing capacity you can expect from an EC2 instance, Amazon defines their own measure of processing power, called an EC2 Compute Unit. This measure is based on several benchmarks set by Amazon to ensure that the performance of EC2 instances remains consistent and predictable over time. As a reference point, you can expect an instance with a rating of 1 EC2 compute unit to provide the same CPU capacity as a physical machine with a 1.0 to 1.2 GHz AMD Opteron processor, circa 2007.
EC2 offers a choice of three virtual machine instance types that offer different levels of performance and resourcing, and based on different platforms, with corresponding differences in pricing. The three instance types are referred to as small, large, and extra-large. Table 5-1 lists the specifications provided by the three instance types. The processing power you can expect from each instance type is defined in terms of EC2 compute units (ECUs). The measure of I/O performance is more nebulous, and is merely divided into two broad categories: “moderate” and “high.”
Table 5-1. EC2 instance types
| Resource | Small | Large | Extra Large |
|---|---|---|---|
| Platform | 32-bit x86 | 64-bit x86 | 64-bit x86 |
| CPU rating | 1 ECU (1 virtual core) | 4 ECUs (2 virtual cores of 2 ECUs each) | 8 ECUs (4 virtual cores of 2 ECUs each) |
| Memory (RAM) | 1.7 GB | 7.5 GB | 15 GB |
| Storage (ephemeral) | 150 GB | 840 GB (two 420 GB partitions) | 1680 GB (four 420 GB partitions) |
| Storage (root partition) | 10 GB | 10 GB | 10 GB |
| I/O Performance | Moderate | High | High |
| Instance Type Name | m1.small | m1.large | m1.xlarge |
| Price per hour | 10¢ | 40¢ | 80¢ |
Unless you explicitly choose otherwise when you launch your instances, the service will launch the small instance type as a default.
An EC2 instance is based on an AMI that captures the root filesystem of an instance in a series of files. When you launch an instance, it boots from the image’s filesystem and runs with the software, configuration settings, and data that were stored in the AMI.
You can log in to a running instance as the root user and customize it for your own purposes by installing and configuring any additional software that is compatible with the instance’s Linux kernel. Any changes you make to the instance can then be bundled up into a new AMI to be used as the starting point the next time you start an instance. You can even create your own AMI from scratch using your preferred Linux distribution.
AMIs are stored in Amazon’s S3 service and must be registered to be recognized by EC2. The images may be registered for private use only, in which case only you may run the AMIs you have created; or they may be shared with other EC2 users. Images can be shared with specific users or made completely public and available to all EC2 account holders.
In addition to sharing your AMIs, you can also rent them out to other users for a fee. If you create an image that is of value to others, you can register the image as a Paid AMI through Amazon’s DevPay system and arrange to have a product code associated with it. EC2 uses the product code to recognize when your AMI is run and by whom, at which point the service will charge the AMI user the additional usage fee premium you specify on top of Amazon’s hourly fees. Users who wish to run your AMI must first go through a sign-up process to provide Amazon with their credit card details, so they can be billed for using the image. Amazon collects the hourly premium on your behalf and passes the money on to you. Product codes can also be used for other purposes, such as to identify images you have sold to a user with a support agreement.
We do not discuss the DevPay service in detail in this book. For more information on DevPay refer to the home page of the service: http://www.amazon.com/devpay
For an in-depth guide for creating, configuring, and using AMIs, refer to Chapter 6. In the current chapter we will discuss the API operations for managing AMIs under Managing and Sharing AMIs.”
The EC2 environment provides instances with a range of services, such as access control mechanisms, a network firewall, network address allocation, and ephemeral storage volumes.
EC2 provides support for public and private key-based login access to instances, using the widely-used Secure Shell program (SSH). This mechanism is based on keypairs, a topic we discuss in detail in this chapter. Keypairs comprise a private key that you receive from Amazon and store on your client computers and a corresponding public key component that is stored within the EC2 environment. When you start an EC2 instance, you can choose to have the environment make one or more of your public keys available to the instance. If the instance is suitably configured, it will use this public key as half of the access credentials necessary to log in, so that only someone with the corresponding private key can access the instance.
The EC2 environment also provides a basic firewall mechanism that allows you to limit the network traffic that will reach your system. Firewall settings are configured using security groups that define rules specifying which incoming network connections will be allowed. These connection rules can impose limits based on the transport protocol, the port number, and the source IP address of the network traffic. Security groups, which we will discuss shortly, can be combined together to produce a cumulative set of rules, and these firewall rules can be modified while your instances are running.
When you launch an instance, the EC2 environment assigns it dynamic internal and external network addresses. These addresses will refer to your instance during its lifetime, and when the instance is terminated, the addresses will be reclaimed by the service and reused. There is no mechanism in EC2 to maintain a single, static network address for any of your instances. For more information about network addressing in EC2, including potential issues caused by the lack of static addressing, refer to Network Addressing” in Chapter 6.
Each instance is given storage space in the EC2 environment. The amount of space allocated to the instance depends on the instance type. This storage space is ephemeral, meaning that it is only allocated to your instance for its lifetime. Any data written to an instance’s data storage resources will be lost when the instance is terminated. It is the responsibility of you, the instance owner, to implement a persistence strategy if you need your data to outlive the instance on which it is stored. For more information about data storage in EC2 instances and data persistence strategies, refer to Data Management in EC2” in Chapter 6.
EC2 account holders are billed monthly for their usage of the service. The service charges are based on two aspects of your EC2 usage: the number of instance hours used and the volume of data transferred to or from your EC2 instances.
Amazon charges a fee for every hour, or portion of an hour, that an instance is running. Table 5-2 lists the hourly usage fee for each instance type:
Data received by EC2 instances costs 10¢ per GB (10243 bytes). Data sent from EC2 instances is charged on a sliding scale, depending on the volume of data transferred during the month: 18¢/GB from 0 to 10 TB, 16¢/GB from 10 to 50 TB, and 13¢/GB for any amount over 50 TB.
Data transfers between EC2 instances incur no transfer fees. Data transfers between EC2 instances and S3 buckets located in the United States are also free, but data transfers between EC2 instances and S3 buckets located in Europe incur the standard transfer fees.