Chapter 9. Authentication

Twisted comes with a protocol-independent, pluggable, asynchronous authentication system called Cred that can be used to add any type of authentication support to your Twisted server. Twisted also ships with a variety of common authentication mechanisms that you can use off the shelf through this system.

Because it is a general and extensible system, there are a number of components to understand and use in even a basic example. Getting over the initial learning curve will pay off for using Cred in real-world systems, so stick with me through the terminology and these examples.

Let me state up front that this is not a chapter on cryptography or password management best practices. This chapter uses hashing examples that are short and convenient for describing the capabilities of Twisted Cred with minimal overhead; if you want more information on securely managing user data, please consult a resource dedicated to this topic like Secure Coding: Principles and Practices (O’Reilly).

The Components of Twisted Cred

Before we get into the usage examples, there are a few terms that you should familiarize yourself with:

Credentials

Information used to identify and authenticate a user. Common credentials are a username and password, but they can be any data or object used to prove a user’s identity, such as a certificate or challenge/response protocol. Objects that provide credentials implement twisted.cred.credentials.ICredentials.

Avatar

A business logic object in a server application that represents the actions and data available to a user. For example, an avatar for a mail server might be a mailbox object, an avatar for a web server might be a resource, and an avatar for an SSH server might be a remote shell.

Avatars implement an interface that inherits from zope.interface.Interface.

Avatar ID

A string returned by the credentials checker that identifies the avatar for a user. This is often a username, but it could be any unique identifier. Example avatar IDs are “Joe Smith”, “joe@localhost”, and “user926344”.

Credentials checker

An object that takes credentials and attempts to verify them. The credentials checker is the bridge between the many ways credentials can be stored—for example, in a database, in a file, or in memory—and the rest of Cred.

If the credentials correctly identify a user, the credentials checker will return an avatar ID. Credentials checkers can also support anonymous access by returning twisted.cred.checkers.ANONYMOUS.

Credentials checkers implement the twisted.cred.checker.ICredentialsChecker interface.

Realm

An object that provides access to all the possible avatars in an application. A realm will take an avatar ID identifying a specific user and return an avatar object that will work on behalf of that user. A realm can support multiple types of avatars, allowing different types of users to have access to different services on a server.

Realm objects implement the twisted.cred.portal.IRealm interface.

Portal

The portal mediates interactions between the many parts of Cred. At the protocol level, the only thing you need to use Cred is a reference to a portal. The portal’s login method will authenticate users to the system.

The portal is not subclassed. Customization instead happens in the realm, credentials checkers, and avatars.

Twisted Cred: An Example

Now that we’re primed with those definitions, let’s look at a basic example. Example 9-1 shows an authenticating echo server.

Example 9-1. echo_cred.py

from zope.interface import implements, Interface

from twisted.cred import checkers, credentials, portal
from twisted.internet import protocol, reactor
from twisted.protocols import basic

class IProtocolAvatar(Interface):
    def logout():
        """
        Clean up per-login resources allocated to this avatar.
        """

class EchoAvatar(object):
    implements(IProtocolAvatar)

    def logout(self):
        pass

class Echo(basic.LineReceiver):
    portal = None
    avatar = None
    logout = None

    def connectionLost(self, reason):
        if self.logout:
            self.logout()
            self.avatar = None
            self.logout = None

    def lineReceived(self, line):
        if not self.avatar:
            username, password = line.strip().split(" ")
            self.tryLogin(username, password)
        else:
            self.sendLine(line)

    def tryLogin(self, username, password):
        self.portal.login(credentials.UsernamePassword(username,
                                                       password),
                          None,
                          IProtocolAvatar).addCallbacks(self._cbLogin,
                                                        self._ebLogin)

    def _cbLogin(self, (interface, avatar, logout)):
        self.avatar = avatar
        self.logout = logout
        self.sendLine("Login successful, please proceed.")

    def _ebLogin(self, failure):
        self.sendLine("Login denied, goodbye.")
        self.transport.loseConnection()

class EchoFactory(protocol.Factory):
    def __init__(self, portal):
        self.portal = portal

    def buildProtocol(self, addr):
        proto = Echo()
        proto.portal = self.portal
        return proto

class Realm(object):
    implements(portal.IRealm)

    def requestAvatar(self, avatarId, mind, *interfaces):
        if IProtocolAvatar in interfaces:
            avatar = EchoAvatar()
            return IProtocolAvatar, avatar, avatar.logout
        raise NotImplementedError(
            "This realm only supports the IProtocolAvatar interface.")

realm = Realm()
myPortal = portal.Portal(realm)
checker = checkers.InMemoryUsernamePasswordDatabaseDontUse()
checker.addUser("user", "pass")
myPortal.registerChecker(checker)

reactor.listenTCP(8000, EchoFactory(myPortal))
reactor.run()

To test the echo server, start it with python echo_cred.py. Connect to the server over telnet with telnet localhost 8000. To log in successfully, provide as the first line of client input user pass. You will then get a login message, and future lines will be echoed. Logging in with invalid credentials causes the server to send an invalid login message and terminate the connection. Here is an example client transcript:

$ telnet localhost 8000
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
user pass
Login successful, please proceed.
Hi
Hi
Quit
Quit
^]
telnet> quit
Connection closed.
localhost:~ jesstess$ telnet localhost 8000
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
foo bar
Login denied, goodbye.
Connection closed by foreign host.

Figure 9-1 illustrates Cred’s authentication process diagrammatically.

Figure 9-1. The Twisted Cred authentication process

The steps are:

1. Our protocol factory, EchoFactory, produces instances of Echo in its buildProtocol method, just like in Chapter 2. Unlike in Chapter 2, these protocols have a reference to a Portal.

   When we receive our first line from a connected client in Echo.lineReceived, we call our Portal’s login method to initiate a login request. Portal.login’s function signature is login(credentials, mind, *interfaces). In detail, the three arguments it requires are:

   1. Credentials, in this case a credentials.UsernamePassword created from the username and password parsed out of the line received.

   2. A “mind” which is almost always None. We won’t ever care about the mind in this book; if you are curious, the Portal.login documentation explains it.

   3. A list of avatar interfaces for which we are requesting authentication. This is usually a single interface (in this example, IProtocolAvatar).

2. The Portal hands off the credentials to the appropriate credentials checker based on the avatar interface requested.

   Each credentials checker exposes a set of credentialInterfaces for which it is able to authenticate. This example has only one checker, a toy checkers.InMemoryUsernamePasswordDatabaseDontUse that Twisted provides for learning about cred. This checker happens to support two types of credentials, credentials.IUsernamePassword and credentials.IUsernameHashedPassword. Because the call to Portal.login specified credentials.UsernamePassword, which implements credentials.IUsernamePassword, this credentials checker is able to authenticate the provided credentials.

3. A credentials checker returns a Deferred to the Portal, containing either an avatar ID if the credentials were correct or a login failure that terminates the login process and fires the errback chain for Portal.login. In this example, a failure would invoke Echo._ebLogin.

4. At this point, the user has successfully logged in. The Portal invokes the Realm’s requestAvatar method, providing the avatar ID and the appropriate avatar interface.

5. requestAvatar returns a triple of avatar interface, avatar instance, and avatar logout method. If no per-login resources need to get cleaned up after a user logs out, the logout method can do nothing.

6. Portal.login returns a Deferred containing either the avatar interface, avatar instance, and avatar logout method triple or a login failure, as mentioned in Step 3. In this example, on success _cbLogin is called, sending a welcome message to the now-authenticated user.

Once authenticated, the echo client and server interact as in Chapter 2.

Credentials Checkers

With a minimal example under our belt, we can start to explore why cred’s flexibility makes it so powerful. First, what if instead of using the toy in-memory checker we wanted to check the username and password against a file-based username and password database?

Twisted comes with a FilePasswordDB checker, so all we have to do is create a credentials file containing some usernames and passwords and swap in this FilePasswordDB checker:

-checker = checkers.InMemoryUsernamePasswordDatabaseDontUse()
-checker.addUser("user", "pass")
+checker = checkers.FilePasswordDB("passwords.txt")

FilePasswordDB’s line format is customizable and defaults to username:password. Try running echo_cred.py with these changes and a test passwords.txt.

What if we wanted to use hashed passwords in our password file instead? FilePasswordDB takes an optional hash argument that it will apply to a password before comparing it to the hash stored on disk. To augment Example 9-1 to support hashed passwords, swap in:

+import hashlib 
+def hash(username, password, passwordHash):
+    return hashlib.md5(password).hexdigest()
+
 realm = Realm()
 myPortal = portal.Portal(realm)
-checker = checkers.InMemoryUsernamePasswordDatabaseDontUse()
-checker.addUser("user", "pass")
+checker = checkers.FilePasswordDB("passwords.txt", hash=hash)

and use the same hash logic to generate the passwords in passwords.txt.

What if we wanted to store our passwords in a database?

Twisted does not ship with a database-backed credentials checker, so we’ll need to write our own. It must implement the ICredentialsChecker interface, namely:

• Expose a class variable credentialInterfaces, which lists the credentials types the checker is able to check

• Implement the requestAvatarId method, which, given a set of credentials, must either authenticate the user and return its avatar ID or return a login failure

Example 9-2 implements a database-backed credentials checker.

Example 9-2. db_checker.py

from twisted.cred import error
from twisted.cred.checkers import ICredentialsChecker
from twisted.cred.credentials import IUsernameHashedPassword
from twisted.internet.defer import Deferred

from zope.interface import implements

class DBCredentialsChecker(object):
    implements(ICredentialsChecker)

    credentialInterfaces = (IUsernameHashedPassword,)

    def __init__(self, runQuery, query):
        self.runQuery = runQuery
        self.query = query

    def requestAvatarId(self, credentials):
        for interface in self.credentialInterfaces:
            if interface.providedBy(credentials):
                break
            else:
                raise error.UnhandledCredentials()

        dbDeferred = self.runQuery(self.query, (credentials.username,))
        deferred = Deferred()
        dbDeferred.addCallbacks(self._cbAuthenticate, self._ebAuthenticate,
                                callbackArgs=(credentials, deferred),
                                errbackArgs=(credentials, deferred))
        return deferred

    def _cbAuthenticate(self, result, credentials, deferred):
        if not result:
            deferred.errback(error.UnauthorizedLogin('User not in database'))
        else:
            username, password = result[0]
            if credentials.checkPassword(password):
                deferred.callback(credentials.username)
            else:
                deferred.errback(error.UnauthorizedLogin('Password mismatch'))

    def _ebAuthenticate(self, failure, credentials, deferred):
        deferred.errback(error.LoginFailed(failure))

To be database-agnostic, an instance of DBCredentialsChecker is initialized with an adbapi.ConnectionPool handle and the query to run to retrieve user credentials.

requestAvatarID returns a Deferred containing the avatar ID. The method takes a set of credentials, does a database lookup on the username from those credentials, and checks the password provided in the credentials against the one looked up in the database. On a password match, the Deferred’s callback chain is invoked with credentials.username, which will be the avatar ID for this user. If the passwords don’t match, the errback chain is invoked with cred.error.UnauthorizedLogin.

This checker expects credentials implementing IUsernameHashedPassword; the passwords are hashed before insertion into the database so the checker does not have access to the plain-text password, and credentials.checkPassword is invoked with the user-provided password to determine a match.

The only modifications needed to our original authenticating echo server are to swap in the DBCredentialsChecker and store hashed credentials in a database. Make these changes in echo_server.py:

First, at the top of the file define the hash used when inserting passwords into the database:

+import hashlib
+def hash(password):
+    return hashlib.md5(password).hexdigest()

Then, swap in the new type of credentials expected:

-        self.portal.login(credentials.UsernamePassword(
-                username, password),
+        self.portal.login(credentials.UsernameHashedPassword(
+                username, hash(password)),

Finally, swap in the new DBCredentialsChecker:

-checker = checkers.InMemoryUsernamePasswordDatabaseDontUse()
-checker.addUser("user", "pass")
+from twisted.enterprise import adbapi 
+from db_checker import DBCredentialsChecker 
+dbpool = adbapi.ConnectionPool("sqlite3", "users.db")
+checker = DBCredentialsChecker(
+    dbpool.runQuery,
+    query="SELECT username, password FROM users WHERE username = ?")

Where a simple hash implementation could be something similar to the function from our earlier modification to Example 9-1:

Tip

Let me again remind you that this chapter is intentionally sticking to simple, concise examples. Don’t use md5 to hash passwords. Don’t store passwords in plaintext, do salt your passwords, and do use a cryptographically secure hash. If you want more information on how to securely manage user data, consult a resource dedicated to the topic. Your users will thank you!

Authentication in Twisted Applications

So far these Twisted Cred examples have used servers outside the Twisted application infrastructure discussed in Chapter 6. Twisted makes it easy to integrate authentication into applications deployed through twistd using the AuthOptionMixin class, and this is in fact where Twisted Cred really shines for providing a standard interface for swapping in and out authentication mechanisms decoupled from the business logic of your application.

As a concrete example, let’s convert our authenticating echo server from Example 9-1 to a Twisted application. First, delete the realm, portal, and reactor code, which twistd and the plugin will handle instead, from that server file:

-realm = Realm()
-myPortal = portal.Portal(realm)
-checker = checkers.InMemoryUsernamePasswordDatabaseDontUse()
-checker.addUser("user", "pass")
-myPortal.registerChecker(checker)
-
-reactor.listenTCP(8000, EchoFactory(myPortal))
-reactor.run()

Then, create a plugin for this application using the same template from Example 6-4: in the directory containing the server application, create a twisted directory containing a plugins directory containing a file echo_cred_plugin.py. Example 9-3 has the code for this plugin.

Example 9-3. echo_cred_plugin.py

from twisted.application.service import IServiceMaker
from twisted.application import internet
from twisted.cred import credentials, portal, strcred
from twisted.plugin import IPlugin
from twisted.python import usage

from zope.interface import implements

from echo_cred import EchoFactory, Realm

class Options(usage.Options, strcred.AuthOptionMixin):
    supportedInterfaces = (credentials.IUsernamePassword,)
    optParameters = [["port", "p", 8000, "The port number to listen on."]]

class EchoServiceMaker(object):
    implements(IServiceMaker, IPlugin)
    tapname = "echo"
    description = "A TCP-based echo server."
    options = Options

    def makeService(self, options):
        """
        Construct a TCPServer from EchoFactory.
        """
        p = portal.Portal(Realm(), options["credCheckers"])
        return internet.TCPServer(int(options["port"]), EchoFactory(p))

serviceMaker = EchoServiceMaker()

This echo_cred_plugin.py looks exactly like the plugin from Example 6-4, with one difference: the authenticating EchoFactory needs to interface with a Portal, which in turn needs to interface with a Realm and register credentials checkers. We want to be able to configure the available credentials checkers from the command line, and to do this we make our command-line Options class inherit from strcred.AuthOptionMixin.

Using AuthOptionMixin, all we have to do is enumerate the supported credentials types in a supportedInterface class variable; and that gives us full access to command-line credentials configuration. For this example, let’s reuse a credentials type we’ve seen before, credentials.IUsernamePassword.

With this AuthOptionMixin-enabled plugin in place, twistd echo grows command-line authentication configuration and documentation:

$ twistd echo --help-auth
Usage: --auth AuthType[:ArgString]
For detailed help: --help-auth-type AuthType

AuthTypeArgString format
========================
memory  A colon-separated list (name:password:...)
file    Location of a FilePasswordDB-formatted file.
unix    No argstring required.

Let’s try out our authenticating echo server with the twistd command-line version of checkers.InMemoryUsernamePasswordDatabaseDontUse from Example 9-1:

$ twistd -n echo --auth memory:user:pass:foo:bar
2012-12-01 14:04:11-0500 [-] Log opened.
2012-12-01 14:04:11-0500 [-] twistd 12.1.0 (/usr/bin/python 2.7.1) ...
2012-12-01 14:04:11-0500 [-] reactor class: twisted.internet.select...
2012-12-22 14:07:26-0500 [-] EchoFactory starting on 8000
2012-12-01 14:04:11-0500 [-] Starting factory <echo.EchoFactory ...

As before, we can now run telnet localhost 8000 to play with this server.

With no application configuration, we can switch to authenticating against a password file like our passwords.txt by specifying the file auth type:

twistd -n echo --auth file:passwords.txt

On Unix, we can even use a built-in unix checker that “will attempt to use every resource available to authenticate against the list of users on the local UNIX system,” which currently includes checking against /etc/passwd and /etc/shadow:

sudo twistd -n echo  --auth unix 

You can then use your login username and password for this machine to authenticate to the echo server.

What if we wanted to add one of our custom checkers to this pool of available command-line checkers, alongside memory, file, and unix?

We do this with, as you might guess, a plugin. If you look in twisted/plugins/ in the Twisted source code, you’ll see a cred_* file for each of the checkers we’ve used so far, as well as some others. Each Cred plugin implements and exposes a credentials checker factory. The list of credentials checkers available in twistd --help-auth is the set of checkers that implement the credentials interfaces specified in AuthOptionMixin’s supportedInterfaces in your server’s plugin file. In this echo example we specified credentials.IUsernamePassword, so the checkers available are those in twisted/plugins/ that list IUsernamePassword in their credentialInterfaces.

So, to add our own checker for a particular credential interface to twistd, we would place the credentials checker and factory plugin in the twisted/plugins/ subdirectory of our top-level project. After that, the checker will show up as an option in twisted --help-auth!

More Practice and Next Steps

This chapter discussed Twisted’s Cred authentication system. In the Cred model, protocols authenticate users through a Portal, which mediates the validation of credentials against a credentials checker and returns an avatar which can act on behalf of the authenticated user. Cred uses the plugin system introduced in Chapter 6 to be a general and extensible framework.

Twisted’s Web in 60 Seconds series walks through adding basic or digest HTTP authentication to a web server using Twisted Cred. For more practice, try adding authentication to one of your web servers from Chapter 4.

Conch, Twisted’s SSH subproject, is discussed in Chapter 14 and makes extensive use of Twisted Cred.