NetCipher is a library from the Guardian Project to improve the privacy and security of HTTP network communications. In particular, it makes it easier for your app to integrate with Orbot, an Android proxy server that forwards HTTP requests via Tor.

Prerequisites

This chapter assumes that you have read the core chapters of the book, particularly the one on Internet access. Having read the chapter on SSL is also a very good idea.

Network Security’s Got Onions

Maintaining privacy and security on the Internet, in the face of so-called “advanced persistent threats”, is a continuous challenge facing many people, particularly those under threats from hostile forces, ranging from organized crime syndicates to your average rampaging warlord. Tor was created to help deal with this sort of problem; Orbot was created to extend Tor to Android.

A Quick Primer on Tor

Originally named The Onion Router, Tor was created by researchers in the US Naval Research Laboratory back in the mid-1990’s, with an eye towards protecting US intelligence communications. In 2006, the technology spun out into an independent non-profit organization, which has continued to improve upon the core Tor software and expand the reach of Tor. Through packages like the Tor Browser Bundle, it is fairly easy for at-risk people to start using Tor to help shroud their communications.

Without getting into the full technical details of Tor — which are well beyond the scope of this chapter — Tor basically works by routing a request through a series of relay servers, through a process known as onion routing. Requests are secured through layers of encryption, to keep any two connected relays from knowing the full details of the communications. Some relays serve as “exit nodes”, for requests being made of ordinary Web servers. Certain servers — Tor hidden services — are only reachable through Tor; requests made of these servers never leave the Tor network.

Of course, technology like Tor is agnostic in terms of its users and usages, and there have been plenty of examples of people using Tor for illicit purposes, such as the Silk Road. This has a tendency to obscure Tor’s benefits to people who need to remain somewhat hidden online, whether from stalkers or other harassers or from the security forces of dictatorships.

Introducing Orbot

The entry path into Tor is usually via some sort of proxy server, that a regular Internet client can connect to. Orbot is one such proxy server, that runs on Android. Apps can use Orbot’s HTTP or SOCKS proxies to route requests; those requests will then wind up traversing the Tor network to the end site, whether that site is on the public Internet (reached from a Tor exit node) or a Tor hidden service.

By default, Orbot is limited to localhost use, meaning that it does not have open ports that can be reached from other devices on the local WiFi LAN segment (or some subnet of the mobile carrier, if not on WiFi). For an Android app on the same device, this is not a problem, and it in fact simplifies things a fair bit, as there is no guesswork as to what the IP address should be for the proxy. As we will see, though, finding out exactly how to connect to Orbot is a bit tricky, though with some helper code it is not too bad.

What NetCipher Provides

While we know that Orbot will be listening on localhost, we do not necessarily know the port that it is using for its HTTP proxy. Partly, that is because the user might configure it manually. Partly, that is because there are occasional conflicts with Orbot’s default port.

The NetCipher HTTP Integration APIs

NetCipher offers two levels of API for integration. This chapter focuses on the newer of those, a suite that offers simple plug-and-play integration with popular HTTP client APIs: HttpURLConnection, OkHttp, Volley, and Apache’s HttpClient. This focus stems from two main reasons:

The Internet/HTTPStacks sample application demonstrates all four of the HTTP integration APIs. Each of the four is based on the Stack Overflow sample app from the chapter on Internet access, with NetCipher integration added in.

There are a few simple steps for adding in NetCipher integration: choosing your HTTP stack, adding the dependencies, setting up OrbotHelper, and then using a secure connection.

Choose an HTTP Stack

As noted above, NetCipher offers integration APIs for four major HTTP client implementations (a.k.a., “HTTP stacks”):

HttpURLConnection support is part of the core NetCipher library (info.guardianproject.netcipher:netcipher), as HttpURLConnection is part of standard Java and Android. The other three HTTP stacks have separate libraries:

Add the Dependencies

HTTP Stack	NetCipher Artifact
OkHttp3	`info.guardianproject.netcipher:netcipher-okhttp3`
HttpClient	`info.guardianproject.netcipher:netcipher-httpclient`
Volley	`info.guardianproject.netcipher:netcipher-volley`

Unfortunately, some packaging issues with the 2.0.0-alpha1 edition of NetCipher, adding the dependencies is more complicated than it needs to be. Your project needs to have dependencies on:

So, for example, the okhttp3 module in the HTTPStacks project is a sample app that uses OkHttp 3.x for its HTTP client API. It needs three artifacts in its dependencies closure to pull in OkHttp and NetCipher’s support for OkHttp:

    compile 'info.guardianproject.netcipher:netcipher:2.0.0-alpha1'
    compile 'info.guardianproject.netcipher:netcipher-okhttp3:2.0.0-alpha1'
    compile 'com.squareup.okhttp3:okhttp:3.8.0'

Volley integration has been tested with com.android.volley:volley:1.0.0, while the HttpClient integration work with the cz.msebera.android:httpclient:4.4.1.2 independent repackaging of Apache HttpClient for Android.

Set up OrbotHelper

OrbotHelper is a singleton that manages a lot of the asynchronous communication between your app and Orbot. It is designed to be initialized fairly early on in your app’s lifecycle. One likely candidate is to have a custom Application subclass, where you override onCreate() and set up OrbotHelper.

package com.commonsware.android.http;

import android.app.Application;
import com.squareup.leakcanary.LeakCanary;
import info.guardianproject.netcipher.proxy.OrbotHelper;

public class SampleApplication extends Application {
  @Override
  public void onCreate() {
    super.onCreate();

    LeakCanary.install(this);
    OrbotHelper.get(this).init();
  }
}

SampleApplication is then tied into the app via the android:name attribute on the <application> element in the manifest:

  <application
    android:name=".SampleApplication"
    android:allowBackup="true"
    android:icon="@drawable/ic_launcher"
    android:label="@string/app_name"
    android:theme="@style/Theme.Apptheme">

Choose and Create a Builder

Each module defines a corresponding builder class that can be used to configure NetCipher for use with that stack, with names based on the classes used with those HTTP stacks:

HTTP Stack	Builder Class
`HttpURLConnection`	`StrongConnectionBuilder`
OkHttp3	`StrongOkHttpClientBuilder`
HttpClient	`StrongHttpClientBuilder`
Volley	`StrongVolleyQueueBuilder`

You will need an instance of your chosen builder class. The simplest way to do that is to call the forMaxSecurity() static method on the builder class. forMaxSecurity() takes a Context as a parameter, though it only holds onto the Application singleton internally, so any Context is safe. forMaxSecurity() returns a builder configured for the best protection that NetCipher can offer.

Get a Connection

Then, call build() on the builder object. It will take a StrongBuilder.Callback object as a parameter, typed for whatever HTTP stack you chose. So, for example, if you went with StrongConnectionBuilder, your callback will be a StrongBuilder.Callback<HttpURLConnection>.

Seeing the Builder in Action

HTTP Stack	Builder Class	Connection Class
`HttpURLConnection`	`StrongConnectionBuilder`	`HttpURLConnection`
OkHttp3	`StrongOkHttpClientBuilder`	`OkHttpClient`
HttpClient	`StrongHttpClientBuilder`	`HttpClient`
Volley	`StrongVolleyQueueBuilder`	`RequestQueue`

Each of the four modules in the sample app (hurl, httpclient, okhttp3, and volley) have a similar MainActivity implementation, one that populates a ListView with the latest Stack Overflow Android questions. The difference in which HTTP stack the sample uses.

For example, the okhttp3 module, in onCreate() of its MainActivity, uses StrongOkHttpClientBuilder:

  @Override
  protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);

    try {
      StrongOkHttpClientBuilder
        .forMaxSecurity(this)
        .withTorValidation()
        .build(this);
    }
    catch (Exception e) {
      Toast
        .makeText(this, R.string.msg_crash, Toast.LENGTH_LONG)
        .show();
      Log.e(getClass().getSimpleName(),
        "Exception loading SO questions", e);
      finish();
    }
  }

Here, we use forMaxSecurity() to create the StrongOkHttpClientBuilder, then configure it further with withTorValidation(). This requests that we do a test HTTP request to a Tor status URL to confirm that our request has indeed gone over Tor.

Note that StrongConnectionBuilder — for use with HttpURLConnection – also requires that you call connectTo(), before build(), to indicate the specific URL for which you want an HttpURLConnection. This is unique among the builders. These sorts of per-builder differences are discussed later in this chapter.

build() is passed this, referencing MainActivity itself, which is implementing the StrongBuilder.Callback interface:

public class MainActivity extends ListActivity implements
  StrongBuilder.Callback<OkHttpClient> {

That Callback is tied to the particular type of connection we are creating. We are using OkHttp3 and StrongOkHttpClientBuilder, so we are creating an OkHttpClient connection.

Our onConnected() method for that Callback gets the OkHttpClient and makes an HTTP request using it:

  @Override
  public void onConnected(final OkHttpClient client) {
    new Thread() {
      @Override
      public void run() {
        try {
          Request request=new Request.Builder().url(SO_URL).build();
          Response response=client.newCall(request).execute();

          final SOQuestions result=
            new Gson().fromJson(response.body().charStream(), SOQuestions.class);

          runOnUiThread(new Runnable() {
            @Override
            public void run() {
              setListAdapter(new ItemsAdapter(result.items));
            }
          });
        }
        catch (IOException e) {
          onConnectionException(e);
        }
      }
    }.start();
  }

SO_URL, passed into url(), is a Web service request URL from the Stack Exchange API, looking for Stack Overflow questions tagged with the android tag:

  String SO_URL=
    "https://api.stackexchange.com/2.1/questions?"
      + "order=desc&sort=creation&site=stackoverflow&tagged=android";

Note that onConnected() will be called on the main application thread, so you will need to get your connection over to whatever background thread will be doing your work. In this case, we create a background thread right here to retrieve the JSON, parse it, and use runOnUiThread() to update the ListActivity with an ItemsAdapter to show the parsed Stack Overflow questions:

  class ItemsAdapter extends ArrayAdapter<Item> {
    ItemsAdapter(List<Item> items) {
      super(MainActivity.this,
        android.R.layout.simple_list_item_1, items);
    }

    @Override
    public View getView(int position, View convertView, ViewGroup parent) {
      View row=super.getView(position, convertView, parent);
      TextView title=(TextView)row.findViewById(android.R.id.text1);

      title.setText(Html.fromHtml(getItem(position).title));

      return(row);
    }
  }

The other three methods that we need to implement for our Callback are for error conditions: onConnectionException(), onTimeout(), and onInvalid():

  @Override
  public void onConnectionException(Exception e) {
    Log.e(getClass().getSimpleName(),
      "Exception loading SO questions", e);

    runOnUiThread(new Runnable() {
      @Override
      public void run() {
        Toast
          .makeText(MainActivity.this, R.string.msg_crash,
            Toast.LENGTH_LONG)
          .show();
        finish();
      }
    });
  }

  @Override
  public void onTimeout() {
    runOnUiThread(new Runnable() {
      @Override
      public void run() {
        Toast
          .makeText(MainActivity.this, R.string.msg_timeout,
            Toast.LENGTH_LONG)
          .show();
        finish();
      }
    });
  }

  @Override
  public void onInvalid() {
    runOnUiThread(new Runnable() {
      @Override
      public void run() {
        Toast
          .makeText(MainActivity.this, R.string.msg_invalid,
            Toast.LENGTH_LONG)
          .show();
        finish();
      }
    });
  }

Other than initializing OrbotHelper, setting up the builder, and implementing StrongBuilder.Callback somewhere to handle the results, the rest of the code is tied to application logic, not NetCipher itself.

The Rest of the Builder API

The API shown above for getting a NetCipher-secured connection via your favorite HTTP stack is designed for ease of use. However, as shown, it is not very flexible.

The rest of the builder API offers that flexibility, at the cost of some additional code.

Common Configuration Methods

The StrongBuilder interface defines the common public API for all four of the builder classes:


/*
 * Copyright (c) 2016 CommonsWare, LLC
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 * http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

package info.guardianproject.netcipher.client;

import android.content.Intent;
import java.io.IOException;
import java.security.KeyManagementException;
import java.security.KeyStore;
import java.security.KeyStoreException;
import java.security.NoSuchAlgorithmException;
import java.security.UnrecoverableKeyException;
import java.security.cert.CertificateException;
import javax.net.ssl.TrustManager;

public interface StrongBuilder<T extends StrongBuilder, C> {
  /**
   * Callback to get a connection handed to you for use,
   * already set up for NetCipher.
   *
   * @param <C> the type of connection created by this builder
   */
  interface Callback<C> {
    /**
     * Called when the NetCipher-enhanced connection is ready
     * for use.
     *
     * @param connection the connection
     */
    void onConnected(C connection);

    /**
     * Called if we tried to connect through to Orbot but failed
     * for some reason
     *
     * @param e the reason
     */
    void onConnectionException(Exception e);

    /**
     * Called if our attempt to get a status from Orbot failed
     * after a defined period of time. See statusTimeout() on
     * OrbotInitializer.
     */
    void onTimeout();

    /**
     * Called if you requested validation that we are connecting
     * through Tor, and while we were able to connect to Orbot, that
     * validation failed.
     */
    void onInvalid();
  }

  /**
   * Call this to configure the Tor proxy from the results
   * returned by Orbot, using the best available proxy
   * (SOCKS if possible, else HTTP)
   *
   * @return the builder
   */
  T withBestProxy();

  /**
   * @return true if this builder supports HTTP proxies, false
   * otherwise
   */
  boolean supportsHttpProxy();

  /**
   * Call this to configure the Tor proxy from the results
   * returned by Orbot, using the HTTP proxy.
   *
   * @return the builder
   */
   T withHttpProxy();

  /**
   * @return true if this builder supports SOCKS proxies, false
   * otherwise
   */
   boolean supportsSocksProxy();

  /**
   * Call this to configure the Tor proxy from the results
   * returned by Orbot, using the SOCKS proxy.
   *
   * @return the builder
   */
  T withSocksProxy();

  /**
   * Applies your own custom TrustManagers, such as for
   * replacing the stock keystore support with a custom
   * keystore.
   *
   * @param trustManagers the TrustManagers to use
   * @return the builder
   */
  T withTrustManagers(TrustManager[] trustManagers)
    throws NoSuchAlgorithmException, KeyManagementException;

  /**
   * Call this if you want a weaker set of supported ciphers,
   * because you are running into compatibility problems with
   * some server due to a cipher mismatch. The better solution
   * is to fix the server.
   *
   * @return the builder
   */
  T withWeakCiphers();

  /**
   * Call this if you want the builder to confirm that we are
   * communicating over Tor, by reaching out to a Tor test
   * server and confirming our connection status. By default,
   * this is skipped. Adding this check adds security, but it
   * has the chance of false negatives (e.g., we cannot reach
   * that Tor server for some reason).
   *
   * @return the builder
   */
  T withTorValidation();

  /**
   * Builds a connection, applying the configuration already
   * specified in the builder.
   *
   * @param status status Intent from OrbotInitializer
   * @return the connection
   * @throws IOException
   */
  C build(Intent status) throws Exception;

  /**
   * Asynchronous version of build(), one that uses OrbotInitializer
   * internally to get the status and checks the validity of the Tor
   * connection (if requested). Note that your callback methods may
   * be invoked on any thread; do not assume that they will be called
   * on any particular thread.
   *
   * @param callback Callback to get a connection handed to you
   *                 for use, already set up for NetCipher
   */
  void build(Callback<C> callback);
}

withTorValidation(), build(), and the Callback nested interface were covered earlier in this chapter, but the others offer finer-grained configuration options.

Five of the methods are tied into choosing what proxy protocol should be used with Orbot.

forMaxSecurity(), under the covers, uses withBestProxy(), which chooses the best proxy for the situation. Right now, the implementation chooses the SOCKS proxy where that is supported, falling back to the HTTP proxy where it is not.

The supportsHttpProxy() and supportsSocksProxy() methods indicate whether a given builder supports these proxy types.

The withHttpProxy() and withSocksProxy() methods tell the builder that you want to use that specific proxy. Use these with care, making sure that the proxy you want is supported. withBestProxy() is a far better choice overall.

withWeakCiphers() expands the roster of SSL ciphers that NetCipher allows the HTTPS connection to use. Normally, NetCipher tries to avoid ciphers with known security issues. However, that may cause problems with some servers, if NetCipher and the server cannot negotiate a common cipher. withWeakCiphers() allows NetCipher to use more ciphers, to perhaps overcome the negotiation problem, with the cost of possibly weaker security.

Differences Between the Stacks

While each of the builders supports the StrongBuilder API, there are some differences between the implementations.

StrongConnectionBuilder

Before calling build(), you need to call connectTo() to supply the URL (as a String or URL) that you want to connect to. The other builders give you objects that you can reuse across many requests (e.g., OkHttp3’s OkHttpClient), but that is not possible with HttpURLConnection.

  @Override
  protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);

    try {
      StrongConnectionBuilder
        .forMaxSecurity(this)
        .withTorValidation()
        .connectTo(SO_URL)
        .build(this);
    }
    catch (Exception e) {
      Toast
        .makeText(this, R.string.msg_crash, Toast.LENGTH_LONG)
        .show();
      Log.e(getClass().getSimpleName(),
        "Exception loading SO questions", e);
      finish();
    }
  }

The onConnected() method then just uses the fully-configured HttpURLConnection object:

  @Override
  public void onConnected(final HttpURLConnection conn) {
    new Thread() {
      @Override
      public void run() {
        try {
          InputStream in=conn.getInputStream();
          BufferedReader reader=
            new BufferedReader(new InputStreamReader(in));

          final SOQuestions result=
            new Gson().fromJson(reader, SOQuestions.class);

          runOnUiThread(new Runnable() {
            @Override
            public void run() {
              setListAdapter(new ItemsAdapter(result.items));
            }
          });

          reader.close();
        }
        catch (IOException e) {
          onConnectionException(e);
        }
        finally {
          conn.disconnect();
        }
      }
    }.start();
  }

To help make this a bit easier, StrongConnectionBuilder supports the copy constructor. You can create a master StrongConnectionBuilder with your base configuration, then make a copy, call connectTo() on the copy, then call build() on the copy, throwing away the copy when you are done.

StrongHttpClientBuilder

The builder for Apache’s independent packaging of HttpClient for Android extends Apache’s own HttpClientBuilder. As a result, you can call all the normal HttpClientBuilder methods in addition to calling the StrongBuilder methods. The noteworthy exception is that the standard zero-parameter build() offered by HttpClientBuilder is not supported.

The httpclient module’s MainActivity does not need any HttpClient-specific configuration:

  @Override
  protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);

    try {
      StrongHttpClientBuilder
        .forMaxSecurity(this)
        .withTorValidation()
        .build(this);
    }
    catch (Exception e) {
      Toast
        .makeText(this, R.string.msg_crash, Toast.LENGTH_LONG)
        .show();
      Log.e(getClass().getSimpleName(),
        "Exception loading SO questions", e);
      finish();
    }
  }

  @Override
  public void onConnected(final HttpClient client) {
    new Thread() {
      @Override
      public void run() {
        try {
          HttpGet get=new HttpGet(SO_URL);
          String json=client.execute(get, new BasicResponseHandler());

          final SOQuestions result=
            new Gson().fromJson(new StringReader(json),
              SOQuestions.class);

          runOnUiThread(new Runnable() {
            @Override
            public void run() {
              setListAdapter(new ItemsAdapter(result.items));
            }
          });
        }
        catch (IOException e) {
          onConnectionException(e);
        }
      }
    }.start();
  }

StrongVolleyQueueBuilder

This builder class adheres to the StrongBuilder API without any changes, making its use fairly straightforward:

  @Override
  protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);

    try {
      StrongVolleyQueueBuilder
        .forMaxSecurity(this)
        .withTorValidation()
        .build(this);
    }
    catch (Exception e) {
      Toast
        .makeText(this, R.string.msg_crash, Toast.LENGTH_LONG)
        .show();
      Log.e(getClass().getSimpleName(),
        "Exception loading SO questions", e);
      finish();
    }
  }

  @Override
  public void onConnected(final RequestQueue rq) {
    new Thread() {
      @Override
      public void run() {
        final StringRequest stringRequest=
          new StringRequest(StringRequest.Method.GET, SO_URL,
            new Response.Listener<String>() {
              @Override
              public void onResponse(String response) {
                final SOQuestions result=
                  new Gson().fromJson(new StringReader(response),
                    SOQuestions.class);

                runOnUiThread(new Runnable() {
                  @Override
                  public void run() {
                    setListAdapter(new ItemsAdapter(result.items));
                  }
                });
              }
            },
            new Response.ErrorListener() {
              @Override
              public void onErrorResponse(VolleyError error) {
                Log.e(getClass().getSimpleName(),
                  "Exception making Volley request", error);
              }
            });

        rq.add(stringRequest);
      }
    }.start();
  }

StrongOkHttpClientBuilder

Note that OkHttp3 does not support SOCKS proxies. Hence, supportsSocksProxy() returns false, causing withBestProxy() to fall back to the HTTP proxy. This is handled for you automatically.