Chapter 2. EC2 Recipes

If you are launching an instance, most likely you will want to log in to that instance once it is up and running. In the interest of security, AWS uses passwordless SSH for instance access. This requires the use of a public/private keypair that is used to control access to your instance. The public key is installed on the newly launched instance by EC2 and registered as an authorized key with the SSH software on the instance. Then, when you log into the instance, you provide the private key and the SSH software is able to cryptographically compare the public and private keys and determine if the login attempt should be allowed or not. So, prior to running our first instance, we need to create an SSH keypair.

In addition to the keypair, we also need to create a security group. Security groups are a distributed firewall used to control access to your instances. By default, all ports on your instance will be disabled so no access would be possible. If we want to access the instance via SSH, we need to create a security group that contains a specific rule that will enable access to the instance on the specific port we want to use for SSH (default is 22).

Example 2-1 shows a convenience function that does all of the hard work related to launching an instance.

Example 2-1. Launching an Instance

import os
import time
import boto
import boto.manage.cmdshell

def launch_instance(ami='ami-7341831a',
                    instance_type='t1.micro',
                    key_name='paws',
                    key_extension='.pem',
                    key_dir='~/.ssh',
                    group_name='paws',
                    ssh_port=22,
                    cidr='0.0.0.0/0',
                    tag='paws',
                    user_data=None,
                    cmd_shell=True,
                    login_user='ec2-user',
                    ssh_passwd=None):
    """
    Launch an instance and wait for it to start running.
    Returns a tuple consisting of the Instance object and the CmdShell
    object, if request, or None.

    ami        The ID of the Amazon Machine Image that this instance will
               be based on.  Default is a 64-bit Amazon Linux EBS image.

    instance_type The type of the instance.

    key_name   The name of the SSH Key used for logging into the instance.
               It will be created if it does not exist.

    key_extension The file extension for SSH private key files.
    
    key_dir    The path to the directory containing SSH private keys.
               This is usually ~/.ssh.

    group_name The name of the security group used to control access
               to the instance.  It will be created if it does not exist.

    ssh_port   The port number you want to use for SSH access (default 22).

    cidr       The CIDR block used to limit access to your instance.

    tag        A name that will be used to tag the instance so we can
               easily find it later.

    user_data  Data that will be passed to the newly started
               instance at launch and will be accessible via
               the metadata service running at http://169.254.169.254.

    cmd_shell  If true, a boto CmdShell object will be created and returned.
               This allows programmatic SSH access to the new instance.

    login_user The user name used when SSH'ing into new instance.  The
               default is 'ec2-user'

    ssh_passwd The password for your SSH key if it is encrypted with a
               passphrase.
    """
    cmd = None
    
    # Create a connection to EC2 service.
    # You can pass credentials in to the connect_ec2 method explicitly
    # or you can use the default credentials in your ~/.boto config file
    # as we are doing here.
    ec2 = boto.connect_ec2()

    # Check to see if specified keypair already exists.
    # If we get an InvalidKeyPair.NotFound error back from EC2,
    # it means that it doesn't exist and we need to create it.
    try:
        key = ec2.get_all_key_pairs(keynames=[key_name])[0]
    except ec2.ResponseError, e:
        if e.code == 'InvalidKeyPair.NotFound':
            print 'Creating keypair: %s' % key_name
            # Create an SSH key to use when logging into instances.
            key = ec2.create_key_pair(key_name)
            
            # AWS will store the public key but the private key is
            # generated and returned and needs to be stored locally.
            # The save method will also chmod the file to protect
            # your private key.
            key.save(key_dir)
        else:
            raise

    # Check to see if specified security group already exists.
    # If we get an InvalidGroup.NotFound error back from EC2,
    # it means that it doesn't exist and we need to create it.
    try:
        group = ec2.get_all_security_groups(groupnames=[group_name])[0]
    except ec2.ResponseError, e:
        if e.code == 'InvalidGroup.NotFound':
            print 'Creating Security Group: %s' % group_name
            # Create a security group to control access to instance via SSH.
            group = ec2.create_security_group(group_name,
                                              'A group that allows SSH access')
        else:
            raise

    # Add a rule to the security group to authorize SSH traffic
    # on the specified port.
    try:
        group.authorize('tcp', ssh_port, ssh_port, cidr)
    except ec2.ResponseError, e:
        if e.code == 'InvalidPermission.Duplicate':
            print 'Security Group: %s already authorized' % group_name
        else:
            raise

    # Now start up the instance.  The run_instances method
    # has many, many parameters but these are all we need
    # for now.
    reservation = ec2.run_instances(ami,
                                    key_name=key_name,
                                    security_groups=[group_name],
                                    instance_type=instance_type,
                                    user_data=user_data)

    # Find the actual Instance object inside the Reservation object
    # returned by EC2.

    instance = reservation.instances[0]

    # The instance has been launched but it's not yet up and
    # running.  Let's wait for its state to change to 'running'.

    print 'waiting for instance'
    while instance.state != 'running':
        print '.'
        time.sleep(5)
        instance.update()
    print 'done'

    # Let's tag the instance with the specified label so we can
    # identify it later.
    instance.add_tag(tag)

    # The instance is now running, let's try to programmatically
    # SSH to the instance using Paramiko via boto CmdShell.
    
    if cmd_shell:
        key_path = os.path.join(os.path.expanduser(key_dir),
                                key_name+key_extension)
        cmd = boto.manage.cmdshell.sshclient_from_instance(instance,
                                                          key_path,
                                                          user_name=login_user)
                                                            
    return (instance, cmd)

Example 2-2 shows an example of an interactive Python session that uses the launch_instance function.

Example 2-2. Using the launch_instance Function

>>> from ec2_launch_instance import launch_instance
>>> launch_instance() 1
Creating keypair: paws
Security Group: paws already authorized
waiting for instance
.
.
done
SSH Connection refused, will retry in 5 seconds
(Instance:i-98847ef8, ≤boto.manage.cmdshell.SSHClient object at 0x10141fb90>)
>>> instance, cmdshell = _ 2
>>> cmdshell.shell() 3

       __|  __|_  )
       _|  (     /   Amazon Linux AMI
      ___|\___|___|

See /usr/share/doc/system-release/ for latest release notes.
No packages needed for security; 1 packages available
[ec2-user@domU-12-31-39-00-E4-53 ~]$ ls
[ec2-user@domU-12-31-39-00-E4-53 ~]$ pwd
/home/ec2-user
[ec2-user@domU-12-31-39-00-E4-53 ~]$ df
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/xvda1             8256952    918228   7254864  12% /
tmpfs                   305624         0    305624   0% /dev/shm
[ec2-user@domU-12-31-39-00-E4-53 ~]$ 4
[ec2-user@domU-12-31-39-00-E4-53 ~]$ logout

*** EOF
>>>
1

Launch an instance with all of the default choices.

2

The launch_instance function returns a tuple consisting of the Instance object and the CmdShell object. This line uses the special Python shell symbol _, which represents the last returned value to unpack the two values in the tuple into their own variables.

3

The CmdShell object represents an SSH connection to our new instance. Here, we use the shell method to start up an interactive command shell to our new instance.

4

Typing exit into the SSH session will close the interactive SSH session and return us to our Python prompt. You could now call instance.terminate() to get rid of the instance, or you can keep it around for more experimentation.