Data Backup

Backing up your PC used to be essential. To some extent, it still is, but as more and more stuff moves to “the cloud”, local machine backups become less and less important.

Backing up mobile devices historically has been an afterthought, as a lot of what people use these devices for are gateways to Internet-hosted content and services. However, as more and more stuff becomes local to the device — for disconnected operation, for example — the greater the need for backing up that local data.

Android does not have a full-device backup as part of the OS. It does have some hooks that Google advertises as being “backup”, but IT professionals would not consider Google’s definition to match their own for “backup”. And, what hooks there are exist at the level of an app, not the device, providing opportunity — and requirements — for developers to tailor what gets backed up and, to a lesser extent, how it gets backed up.

This chapter will explore the steps to back up your app’s data, with and without Google’s assistance.

Prerequisites

Understanding this chapter requires that you have read the core chapters, particularly the ones on file access and Internet access.

Having read the chapters on SSL and SQLCipher for Android are not required but may prove to be useful background for some of the side topics in this chapter.

First, Some Terminology

One key concept when it comes to backups is what, exactly, we are backing up. The general rule is that you focus your backup regimen on the “system of record”. This is the one and only system that has the master copy of the data. While it may be one “system”, that “system” may be rather complex (e.g., cluster of database servers). However, anything else outside of that system — such as clients for those servers — are not part of the system of record. While they may have some data that is also held by the system of record, that data is considered to be a cached local copy; the system of record has the “real” copy of the data.

Differing Definitions of “Backup”

The problem is that we toss around the term “backup” as though there is a universal canonical definition for that term. Hence, what Google will tell users is “backup” will not necessarily line up with what an IT department will consider “backup” to mean.

What Google Thinks “Backup” Means

Google’s focus is on the cloud. Therefore, their focus is on apps using data resident in the cloud, with some servers forming the system of record. Google (presumably) does some sort of backup for their own systems of record, for their own Internet-based services, and Google assumes that other firms are doing the same.

The side-effect of this definition, though, is that Google does not view an app as having much in the way of local data that needs to be backed up. Cached data can always be reloaded from the system of record, after all. What Google expects needs to be backed up will be local preferences and perhaps authentication or authorization credentials for working with the system of record. This dataset is small and does not necessarily change all that often.

Because the dataset does not change that often, Google only really cares about restoring that data in case of a total device replacement. In other words, if your phone gets run over by a bakery truck, and you wind up replacing that phone with another Android phone, Google is interested in making sure that your old phone’s apps get restored along with the old phone’s last backup of the tiny dataset. After that, you are on your own. In particular, because the dataset does not change that often and does not have much in the way of critical data, Google is not concerned with allowing users to restore app data from backup for any reason other than replacing the device outright. In other words, Google is only concerned with disaster recovery.

Google does not offer any configurability for where backups themselves are stored. Whatever Google backs up, Google stores where Google wants. Terms of service and related agreements give Google — at least in Google’s eyes — the right to do pretty much anything they want with that data. While they will tout the fact that Android 6.0+ backups are stored in an encrypted fashion, they fail to note that Google — not the developer, not the user – holds the encryption keys. Thus, the security offered by this encryption is nominal, perhaps slowing down somebody who breaks into Google’s network, but otherwise not preventing anyone from accessing the data.

Also, there is a 25MB data cap on the size of the backup, so if your app might have data in excess of that, you need to handle backups yourself.

Finally, the author of this book cannot get Google’s backup system to work on production hardware, as will be explained a bit more later in this chapter.

What IT Thinks “Backup” Means

Apps may well be the system of record for the data that they work with. There is no requirement that all apps be front-ends for some server, any more than there is a requirement that all desktop OS apps be front-ends for some server. There may be plenty of business or technical reasons why an app will be the system of record for its data, either all of the time or in between specific sync operations with some central data store.

As a result, an IT department will recognize that apps need a much more robust backup and restoration service, one that takes into account conventional IT backup concepts.

Most IT-grade backup regimens have the notion of “backup aging”. Rather than Google’s approach of considering only one backup to be relevant, an IT department will maintain a series of backups (e.g., 14 days of nightly backups, plus 3 months of weekly backups, plus 5 years of monthly backups), to be able to handle data that might be lost, but where that loss is not detected for some time.

Most IT-grade backups regimens allow data to be restored, in part or completely, at any point, not just in case a device is stepped on by an elephant or otherwise destroyed. Disaster recovery is a scenario of a backup regimen, not the sole objective.

IT departments also tend to be very concerned about where their business data goes. The idea that the data should be available, unencrypted, to arbitrary third parties would be an anathema. Business data should be backed up on by IT-supplied technology on IT-supplied backup media, employing whatever security the IT department thinks is necessary.

Suffice it to say, Google’s approach to “backup” does not align well with what an IT department will want.

What Your Legal Counsel Thinks “Backup” Means

Legal counsel, at some point, should be brought into the discussion of backups, as, for better or worse, there are legal risks involved in backups.

Particularly with Google-style, send-the-data-to-a-third-party backups, you need to ensure that this will not get you in legal trouble. From European Union privacy laws to HIPAA in the US, there are plenty of laws that prohibit the careless distribution of data.

Beyond that, legal counsel will be worried about “the Ashley Madison scenario”. A firm’s IT department will be responsible for ensuring that their servers are not hacked into. However, once you start passing data to third parties, now you are at risk of those servers getting hacked into. Legal counsel can advise you on what your legal exposure is, in terms of potential lawsuits from people whose data might get leaked by these sorts of attacks.

Implementing IT-Style Backup

So, if we want to add backup and restore capability to our app, what is needed? To explore that, we will use the Backup/BackupClient sample project as an illustration. This is a clone of a sample that originally appeared in the chapter on files. We have a three-tab ViewPager, with a large EditText widget in each tab. The three tabs differ in where they persist their data:

This revised sample adds backup-and-restore functionality to this app.

The app also has one other change: it stores the most-recently-visited tab in SharedPreferences. To that end, MainActivity has a PrefsLoadThread static inner class that asynchronously loads the SharedPreferences, then delivers them via greenrobot’s EventBus:

  private static class PrefsLoadThread extends Thread {
    private final Context ctxt;

    PrefsLoadThread(Context ctxt) {
      this.ctxt=ctxt.getApplicationContext();
    }

    @Override
    public void run() {
      SharedPreferences prefs=
        PreferenceManager.getDefaultSharedPreferences(ctxt);
      PrefsLoadedEvent event=new PrefsLoadedEvent(prefs);

      EventBus.getDefault().post(event);
    }
  }
(from Backup/BackupClient/app/src/main/java/com/commonsware/android/backup/MainActivity.java)

MainActivity picks up this event in onPrefsLoaded(), an EventBus method that takes PrefsLoadedEvent as a parameter and updates the current page of the ViewPager (named pager):

  @Subscribe(threadMode =ThreadMode.MAIN)
  public void onPrefsLoaded(PrefsLoadedEvent event) {
    this.prefs=event.prefs;

    int lastVisited=prefs.getInt(PREF_LAST_VISITED, -1);

    if (lastVisited>-1) {
      pager.setCurrentItem(lastVisited);
    }
  }
(from Backup/BackupClient/app/src/main/java/com/commonsware/android/backup/MainActivity.java)

The PrefsLoadThread is kicked off in onStart(), and the PREF_LAST_VISITED value is saved in onStop(), along with the registration and unregistration from the event bus:

  @Override
  protected void onStart() {
    super.onStart();

    EventBus.getDefault().register(this);

    if (prefs==null) {
      new PrefsLoadThread(this).start();
    }
  }

  @Override
  protected void onStop() {
    EventBus.getDefault().unregister(this);

    if (prefs!=null) {
      prefs
        .edit()
        .putInt(PREF_LAST_VISITED, pager.getCurrentItem())
        .apply();
    }

    super.onStop();
  }
(from Backup/BackupClient/app/src/main/java/com/commonsware/android/backup/MainActivity.java)

The net effect is that we retain the last-visited tab across invocations of MainActivity. This forms part of the data that we would like to back up.

Choosing the Backup Scope

The first question is: what exactly are we backing up? Files? Databases? SharedPreferences? Stuff that is out in common areas, like top-level directories on external storage (e.g., DIRECTORY_DOCUMENTS) or the ContactsContract ContentProvider?

Typically, an individual app will focus on backing up only that app’s data, which would exclude the common areas from consideration. That does not mean that you can’t back up common data, but it makes restoration a bit more challenging, as you do not want to overwrite changes to that data that the user made from another app.

In BackupClient, we are backing up:

Notably, we are not backing up the file out on shared storage (the “Public” tab, set to store its data in Environment.getExternalStoragePublicDirectory(Environment.DIRECTORY_DOCUMENTS)). Hence, whatever is in that tab will be left alone when we restore the data from the backup.

Choosing a Backup Trigger

The next question is: when are we backing up the data?

There are any number of possibilities:

The automated options (push message, AlarmManager, JobScheduler) are great, so users do not forget to make a backup. On the other hand, there is the risk that the user is using the app at the time the automated backup is supposed to happen, which means you will need some additional logic to ensure that you postpone that backup until a quieter time. It is difficult to back up data that is actively in use.

The BackupClient sample will settle for a simple manual trigger, via a “Backup” action bar item in the main activity. We also have a “Restore” action bar item, to request to restore the data from a backup. So, MainActivity will load in a menu resource that contains these two options:

<?xml version="1.0" encoding="utf-8"?>
<menu xmlns:android="http://schemas.android.com/apk/res/android">
  <item
    android:id="@+id/backup"
    android:icon="@drawable/ic_backup_white_24dp"
    android:title="@string/menu_backup"/>
  <item
    android:id="@+id/restore"
    android:icon="@drawable/ic_restore_white_24dp"
    android:title="@string/menu_restore"/>
</menu>
(from Backup/BackupClient/app/src/main/res/menu/actions.xml)

It uses a pair of icons culled from Google’s material design icon set.

That resource is inflated in onCreateOptionsMenu(). If the user chooses the “Backup” option, we start a BackupService to do the work:

  @Override
  public boolean onCreateOptionsMenu(Menu menu) {
    getMenuInflater().inflate(R.menu.actions, menu);

    return(super.onCreateOptionsMenu(menu));
  }

  @Override
  public boolean onOptionsItemSelected(MenuItem item) {
    if (item.getItemId()==R.id.backup) {
      BackupService.enqueueWork(this);

      return(true);
    }
    else if (item.getItemId()==R.id.restore) {
      startActivity(new Intent(this, RestoreRosterActivity.class));

      return(true);
    }

    return(super.onOptionsItemSelected(item));
  }
(from Backup/BackupClient/app/src/main/java/com/commonsware/android/backup/MainActivity.java)

We will get into the restore scenario a bit later in this chapter.

Generating the Dataset

Next, we need to actually collect the data to be backed up and package it in some form to send to a server to serve as the backup dataset.

There are any number of ways to package this sort of data, but a ZIP file seems like a likely candidate:

It is the job of the BackupService to create a ZIP file of our desired data, then send that ZIP file to a backup server.

BackupService itself is a JobIntentService, as this sort of work is a nice “fire-and-forget” sort of request, where we no longer need the service once the work is done.

In onHandleWork(), we orchestrate the major steps in this process:

  @Override
  public void onHandleWork(@NonNull Intent i) {
    try {
      File backup=buildBackup();

      uploadBackup(backup);
      backup.delete();

      EventBus.getDefault().post(new BackupCompletedEvent());
    }
    catch (IOException e) {
      Log.e(getClass().getSimpleName(),
        "Exception creating ZIP file", e);
      EventBus.getDefault().post(new BackupFailedEvent());
    }
  }
(from Backup/BackupClient/app/src/main/java/com/commonsware/android/backup/BackupService.java)

We:

Those events can then trigger UI responses. In the case of this trivial sample app, they just result in Toast messages to the user:

  @Subscribe(threadMode =ThreadMode.MAIN)
  public void onCompleted(BackupService.BackupCompletedEvent event) {
    Toast
      .makeText(this, R.string.msg_backup_completed, Toast.LENGTH_LONG)
      .show();
  }

  @Subscribe(threadMode =ThreadMode.MAIN)
  public void onFailed(BackupService.BackupFailedEvent event) {
    Toast
      .makeText(this, R.string.msg_backup_failed, Toast.LENGTH_LONG)
      .show();
  }
(from Backup/BackupClient/app/src/main/java/com/commonsware/android/backup/MainActivity.java)

A production-grade app would do something more sophisticated, particularly for error messages, given that a Toast is ephemeral, and so the user might not see it.

buildBackup() is responsible for creating a file that contains our desired dataset and returning the File object pointing to that file:

  private File buildBackup() throws IOException {
    File zipFile=new File(getCacheDir(), BACKUP_FILENAME);

    if (zipFile.exists()) {
      zipFile.delete();
    }

    FileOutputStream fos=new FileOutputStream(zipFile);
    ZipOutputStream zos=new ZipOutputStream(fos);

    zipDir(ZIP_PREFIX_FILES, getFilesDir(), zos);
    zipDir(ZIP_PREFIX_PREFS, getSharedPrefsDir(this), zos);
    zipDir(ZIP_PREFIX_EXTERNAL, getExternalFilesDir(null), zos);
    zos.flush();
    fos.getFD().sync();
    zos.close();

    return(zipFile);
  }
(from Backup/BackupClient/app/src/main/java/com/commonsware/android/backup/BackupService.java)

We put the backup ZIP file in internal storage cache (getCacheDir()), as that is not something that we are backing up, and therefore we do not need to worry about somehow trying to back up the backup file itself.

We then call zipDir() three times, one for each directory of data to be backed up. Two of the three locations have SDK-supplied methods to get the File object pointing at those directories: getFilesDir() and getExternalFilesDir(). Unfortunately, the SDK does not provide any direct method that returns a File pointing at the directory for SharedPreferences. So, we have to hack one ourselves, in the form of getSharedPrefsDir():

  static File getSharedPrefsDir(Context ctxt) {
    return(new File(new File(ctxt.getApplicationInfo().dataDir),
      "shared_prefs"));
  }
(from Backup/BackupClient/app/src/main/java/com/commonsware/android/backup/BackupService.java)

getApplicationInfo() returns the ApplicationInfo object describing our app. That has a dataDir field that points to all of our internal storage (whereas getFilesDir() points to a subdirectory off of dataDir). The SharedPreferences are stored in XML files in a shared_prefs/ directory off of the location pointed to by the dataDir field. This is not an ideal solution, as in theory the SharedPreferences storage location could move. However, this code should work for all API levels from 1 through 23, and therefore it is reasonably likely that it will hold up over time.

zipDir() not only takes the File of data to be backed up and a ZipOutputStream representing where to package the data, but it also takes a path prefix. ZIP files do not really have a directory structure; that structure is faked based on path-style names associated with each entry. The prefix is added to each of those names, giving the effect of putting each directory’s contents into a separate “directory” within the ZIP archive. Those three prefixes are defined as simple String constants:

  static final String ZIP_PREFIX_FILES="files/";
  static final String ZIP_PREFIX_PREFS="shared_prefs/";
  static final String ZIP_PREFIX_EXTERNAL="external/";
(from Backup/BackupClient/app/src/main/java/com/commonsware/android/backup/BackupService.java)

zipDir() itself (mostly) is a typical recursive put-the-files-in-the-archive method:

  private void zipDir(String basePath, File dir,
                      ZipOutputStream zos) throws IOException {
    byte[] buf=new byte[16384];

    if (dir.listFiles()!=null) {
      for (File file : dir.listFiles()) {
        if (file.isDirectory()) {
          String path=basePath+file.getName()+"/";

          zos.putNextEntry(new ZipEntry(path));
          zipDir(path, file, zos);
          zos.closeEntry();
        }
        else if (!file.getName().equals(BACKUP_PREFS_FILENAME)) {
          FileInputStream fin=new FileInputStream(file);
          int length;

          zos.putNextEntry(
            new ZipEntry(basePath+file.getName()));

          while ((length=fin.read(buf))>0) {
            zos.write(buf, 0, length);
          }

          zos.closeEntry();
          fin.close();
        }
      }
    }
  }
(from Backup/BackupClient/app/src/main/java/com/commonsware/android/backup/BackupService.java)

The one wrinkle is that we filter out files with a particular name, denoted by the BACKUP_PREFS_FILENAME constant:

  private static final String BACKUP_PREFS_FILENAME=
    "com.commonsware.android.backup.BackupService.xml";
(from Backup/BackupClient/app/src/main/java/com/commonsware/android/backup/BackupService.java)

We will explore what this file is, and why we are not backing it up, later in this chapter.

This backup approach has its flaws, in the interests of keeping the example simple:

Transmitting the Dataset

Given the data to be backed up in a nice convenient package, we need to get that dataset off the device and someplace safe, where we can later download and restore it if needed. There are any number of possible solutions here, including many existing public Web services (Dropbox, Amazon’s AWS S3, Google Drive, etc.). If you are only worried about manual backups, you could even consider using ACTION_SEND to send the dataset as an email attachment, though size and content limitations on email attachments may make this impractical for many users.

BackupService works with some implementation of a particular REST-style API for backing up and restoring the data. This API is fairly lightweight, light enough that it can be implemented in ~70 lines of Ruby code, as will be seen later in this chapter. You could implement the same sort of API in any number of Web frameworks.

For backing up data, there are two REST operations that we need to perform:

To implement the client side, BackupService employs the OkHttp library profiled in the chapter on Internet access. Specifically, uploadBackup() does both of the HTTP requests necessary to back up the data, given the File pointing to the ZIP archive that is our dataset:


  private void uploadBackup(File backup) throws IOException {
    Request request=new Request.Builder()
      .url(URL_CREATE_BACKUP)
      .post(RequestBody.create(JSON, "{}"))
      .build();
    Response response=OKHTTP_CLIENT.newCall(request).execute();

    if (response.code()==201) {
      String backupURL=response.header("Location");

      request=new Request.Builder()
        .url(backupURL+RESOURCE_DATASET)
        .put(RequestBody.create(ZIP, backup))
        .build();
      response=OKHTTP_CLIENT.newCall(request).execute();

      if (response.code()==201) {
        String datasetURL=response.header("Location");
        SharedPreferences prefs=
          getSharedPreferences(getClass().getName(),
            Context.MODE_PRIVATE);

        prefs
          .edit()
          .putString(PREF_LAST_BACKUP_DATASET, datasetURL)
          .commit();
      }
      else {
        Log.e(getClass().getSimpleName(),
          "Unsuccessful request to upload backup");
      }
    }
    else {
      Log.e(getClass().getSimpleName(),
        "Unsuccessful request to create backup");
    }
  }
(from Backup/BackupClient/app/src/main/java/com/commonsware/android/backup/BackupService.java)

We create an OkHttp Request.Builder representing our POST request. The URL is defined as a constant, URL_CREATE_BACKUP:

  private static final String URL_CREATE_BACKUP=
    BuildConfig.URL_SERVER+"/api/backups";
(from Backup/BackupClient/app/src/main/java/com/commonsware/android/backup/BackupService.java)

This, in turn, is built up from the fixed REST endpoint path (/api/backups), with the rest of the URL coming from BuildConfig.URL_SERVER. This is defined out in our build.gradle file, allowing us to have different backup server locations based upon build types (or, in principle, product flavors):

  buildTypes {
    debug {
        buildConfigField "String", "URL_SERVER", '"http://10.0.2.2:4567"'
    }

    release {
      buildConfigField "String", "URL_SERVER", '"http://10.0.2.2:4567"'
    }
  }
(from Backup/BackupClient/app/build.gradle)

Here, they happen to both point to the same value at the moment, the IP address that, on an Android emulator, represents localhost of your development machine. However, you could easily change the release build type to point to some production instance of a backup server.

The body of the POST request is a JSON object containing whatever we want, in case we need to provide some sort of identifiers with the backup for server-side use or analysis. In this case, we are passing an empty JSON object ({}), using the JSON MediaType declared as another constant:

  private static final MediaType JSON=
    MediaType.parse("application/json; charset=utf-8");
(from Backup/BackupClient/app/src/main/java/com/commonsware/android/backup/BackupService.java)

We then use an instance of an OkHttpClient object to perform the request, getting the Response synchronously (since we are already on a background thread). If multiple components in your app will all be using OkHttp, the recommendation is to use a singleton instance of OkHttpClient, here defined on BackupService itself:

  static final OkHttpClient OKHTTP_CLIENT=new OkHttpClient();
(from Backup/BackupClient/app/src/main/java/com/commonsware/android/backup/BackupService.java)

The REST protocol to the backup server is that a 201 response code (“Created”) means that our backup metadata has been saved and an ID has been generated for our backup. The Location header in the response contains a REST URL pointing to the backup itself (/api/backups/... for some value of ...). We then use that to generate the URL for the dataset (/api/backups/.../dataset), and perform a PUT request for the dataset, using the ZIP MediaType defined as yet another constant:

  private static final MediaType ZIP=
    MediaType.parse("application/zip");
(from Backup/BackupClient/app/src/main/java/com/commonsware/android/backup/BackupService.java)

Once again, a 201 response indicates that our resource was created, and the Location header provides the URL for the backup dataset. We stuff that URL in a SharedPreferences object unique to BackupService, under a PREF_LAST_BACKUP_DATASET key. We will use that — at least, in theory – if we are restored from a Google disaster recovery process. We will explore that more later in the chapter.

If we get an unexpected response from the server, the sample app logs a message to Logcat and otherwise quietly fails. A production-grade app would handle these scenarios better, including informing the user about the problem.

Of course, a production-grade backup implementation might want more than what we have here, such as better security. For apps being publicly distributed through the Play Store or similar channels, you may want to offer multiple ways of saving off the backup, through some common API with multiple implementations. That way, users can choose whether to back up data via a private server or a public one (e.g., Amazon S3) or some other means that you offer.

Initiating a Restore

Unfortunately, on occasion, the user may have a need to restore the app’s data from a backup.

There are three primary possible triggers for this work to be done:

There is also the question of which backup to restore. Frequently, the user will want the most recent backup, but that is not always the case. The user might realize that the data has been wrong for days and needs to restore an earlier backup than the most recent one.

To that end, the BackupClient demo app will allow the user to manually request that data be restored, via a “Restore” action bar item. We will fetch a list of available backups from the backup server, so the user can choose what backup to restore from.

The “Restore” action bar item in MainActivity simply launches a RestoreRosterActivity, to allow the user to choose the backup to restore. That activity merely sets up a dynamic fragment, RestoreRosterFragment, in onCreate():

package com.commonsware.android.backup;

import android.os.Bundle;
import android.support.v4.app.FragmentActivity;

public class RestoreRosterActivity extends FragmentActivity {
  @Override
  protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);

    if (getSupportFragmentManager()
      .findFragmentById(android.R.id.content)==null) {
      getSupportFragmentManager().beginTransaction()
        .add(android.R.id.content,
          new RestoreRosterFragment()).commit();
    }
  }
}
(from Backup/BackupClient/app/src/main/java/com/commonsware/android/backup/RestoreRosterActivity.java)

RestoreRosterFragment has fairly basic implementations of the onCreate(), onStart(), and onStop() lifecycle methods, to mark the fragment as being a retained fragment, plus to register and unregister from the event bus:

  @Override
  public void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);

    setRetainInstance(true);
  }

  @Override
  public void onStart() {
    super.onStart();

    EventBus.getDefault().register(this);
  }

  @Override
  public void onStop() {
    EventBus.getDefault().unregister(this);

    super.onStop();
  }
(from Backup/BackupClient/app/src/main/java/com/commonsware/android/backup/RestoreRosterFragment.java)

RestoreRosterFragment is a ListFragment, so the ListView will be set up automatically in the inherited implementation of onCreateView(). In onViewCreated(), we can kick off a REST request to pull down the list of backups from the backup server. This client assumes that the REST server has an /api/backups endpoint that will return a JSON roster of the available backups, so we can use OkHttp to perform the GET request for that data:

  @Override
  public void onViewCreated(View view,
                            Bundle savedInstanceState) {
    super.onViewCreated(view, savedInstanceState);

    Request request=new Request.Builder()
      .url(URL_BACKUPS)
      .build();

    BackupService.OKHTTP_CLIENT.newCall(request).enqueue(this);
  }
(from Backup/BackupClient/app/src/main/java/com/commonsware/android/backup/RestoreRosterFragment.java)

Here, we use the same OkHttpClient instance as BackupService uses — since this is a static data member that is automatically initialized, it does not matter whether or not we have used BackupService already in this process. The endpoint URL is found in the URL_BACKUPS constant:

  private static final String URL_BACKUPS=
    BuildConfig.URL_SERVER+"/api/backups";
(from Backup/BackupClient/app/src/main/java/com/commonsware/android/backup/RestoreRosterFragment.java)

Since this is being driven by the UI, and we are calling OkHttp from the main application thread, we use enqueue() instead of execute(), to schedule the request to be performed on a background thread supplied and managed by OkHttp. RestoreRosterFragment implements the required Callback interface needed by enqueue(). That interface, in turn, requires two methods. One is onFailure(), to be called if there is a problem in executing the HTTP request. Here, we just inform the user about the problem in a Toast, though a production-grade app would do something more sophisticated:

  @Override
  public void onFailure(Request request, IOException e) {
    Toast.makeText(getActivity(), R.string.msg_roster_failure,
      Toast.LENGTH_LONG).show();
    Log.e(getClass().getSimpleName(),
      "Exception retrieving backup roster", e);
  }
(from Backup/BackupClient/app/src/main/java/com/commonsware/android/backup/RestoreRosterFragment.java)

The more important method is onResponse(), called when we get a valid-looking response from the server:

  @Override
  public void onResponse(Response response) throws IOException {
    Gson gson=new GsonBuilder()
      .setDateFormat("yyyy-MM-dd'T'HH:mm:ssZZZZZ")
      .create();

    Type listType=new TypeToken<List<BackupMetadata>>() {}.getType();

    EventBus
      .getDefault()
      .post(
        gson.fromJson(response.body().charStream(), listType));
  }
(from Backup/BackupClient/app/src/main/java/com/commonsware/android/backup/RestoreRosterFragment.java)

This sample could use Retrofit for performing this REST-style GET request, in which case Retrofit would work with OkHttp and Google’s Gson to parse our response. In this case, we are using OkHttp directly, and so we need to arrange to have Gson parse the response.

To that end, we:

And, since onResponse() is called on a background thread, we use the event bus to deliver that List of BackupMetadata objects to the fragment itself, so we can pick up that event on the main application thread.

The JSON we get back will be a JSON array containing a list of JSON objects, with each of those objects being mapped to a BackupMetadata instance by Gson:

package com.commonsware.android.backup;

import java.util.Date;

public class BackupMetadata {
  Date timestamp;
  String dataset;

  @Override
  public String toString() {
    return(timestamp.toString());
  }
}
(from Backup/BackupClient/app/src/main/java/com/commonsware/android/backup/BackupMetadata.java)

RestoreRosterFragment then has an onEventMainThread() method, to pick up the List of BackupMetadata, to wrap that in an ArrayAdapter and put those results in the fragment’s ListView:

  @Subscribe(threadMode =ThreadMode.MAIN)
  public void onEventMainThread(List<BackupMetadata> roster) {
    adapter=new ArrayAdapter<BackupMetadata>(getActivity(),
      android.R.layout.simple_list_item_1, roster);

    setListAdapter(adapter);
  }
(from Backup/BackupClient/app/src/main/java/com/commonsware/android/backup/RestoreRosterFragment.java)

RestoreRosterFragment, Showing Two Backups
Figure 776: RestoreRosterFragment, Showing Two Backups

Starting the Restore Activity

When the user clicks on an available backup in the ListView, onListItemClick() gets called:

  @Override
  public void onListItemClick(ListView l, View v, int position,
                              long id) {
    String url=
      BuildConfig.URL_SERVER+adapter.getItem(position).dataset;
    Intent i=
      new Intent(getActivity(), RestoreProgressActivity.class)
        .setData(Uri.parse(url))
        .addFlags(Intent.FLAG_ACTIVITY_NEW_TASK|
                  Intent.FLAG_ACTIVITY_CLEAR_TASK|
                  Intent.FLAG_ACTIVITY_EXCLUDE_FROM_RECENTS);

    startActivity(i);
  }
(from Backup/BackupClient/app/src/main/java/com/commonsware/android/backup/RestoreRosterFragment.java)

The BackupMetadata has a relative URL to the backup’s dataset, so we combine that with BuildConfig.URL_SERVER to get a fully-qualified URL. Then, we start up a RestoreProgressActivity, which will be responsible for kicking off the restore and showing some form of progress indicator along the way.

The tricky part with restoring your app’s data is that you cannot have any app components running that rely upon that data, as the data will be changing out from underneath those components. In our case, we need to get rid of our MainActivity.

To do that, we attach a few flags to the Intent used to start up the RestoreProgressActivity:

These will get rid of all of our previous activities (including the currently-active RestoreRosterActivity) and will prevent the RestoreProgressActivity from showing up in the overview screen.

RestoreProgressActivity has a simple layout with a large centered ProgressBar:

<?xml version="1.0" encoding="utf-8"?>
<FrameLayout
  xmlns:android="http://schemas.android.com/apk/res/android"
  android:layout_width="match_parent"
  android:layout_height="match_parent">

  <ProgressBar
    style="@android:style/Widget.ProgressBar.Large"
    android:layout_width="wrap_content"
    android:layout_height="wrap_content"
    android:layout_gravity="center"/>
</FrameLayout>
(from Backup/BackupClient/app/src/main/res/layout/progress.xml)

In onCreate() of RestoreProgressActivity, in addition to showing that ProgressBar, we kick off a RestoreService to actually download and restore the backup. We are passed the URL to the backup dataset in the Intent used to start RestoreProgressActivity, and we just pass that same URL along (as a Uri) to the service:

  @Override
  protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    setContentView(R.layout.progress);

    if (savedInstanceState==null) {
      RestoreService.enqueueWork(this);
    }
  }
(from Backup/BackupClient/app/src/main/java/com/commonsware/android/backup/RestoreProgressActivity.java)

However, we only do that if we are not being recreated after a configuration change, so this only happens on the first invocation of the activity.

RestoreProgressActivity also registers for events on the event bus, using the typical onStop()/onStart() pattern:

  @Override
  protected void onStart() {
    super.onStart();

    EventBus.getDefault().register(this);
  }

  @Override
  protected void onStop() {
    EventBus.getDefault().unregister(this);

    super.onStop();
  }
(from Backup/BackupClient/app/src/main/java/com/commonsware/android/backup/RestoreProgressActivity.java)

Downloading and Restoring the Dataset

Meanwhile, over in RestoreService, we download and unpack the dataset:

package com.commonsware.android.backup;

import android.content.Context;
import android.content.Intent;
import android.support.annotation.NonNull;
import android.support.v4.app.JobIntentService;
import android.util.Log;
import com.squareup.okhttp.Request;
import com.squareup.okhttp.Response;
import org.greenrobot.eventbus.EventBus;
import java.io.File;
import okio.BufferedSink;
import okio.Okio;

public class RestoreService extends JobIntentService {
  private static final int UNIQUE_JOB_ID=1337;

  static void enqueueWork(Context ctxt) {
    enqueueWork(ctxt, RestoreService.class, UNIQUE_JOB_ID,
      new Intent(ctxt, RestoreService.class));
  }

  @Override
  public void onHandleWork(@NonNull Intent i) {
    Request request=new Request.Builder()
      .url(i.getData().toString())
      .build();

    try {
      Response response=
        BackupService.OKHTTP_CLIENT.newCall(request).execute();
      File toRestore=new File(getCacheDir(), "backup.zip");

      if (toRestore.exists()) {
        toRestore.delete();
      }

      BufferedSink sink = Okio.buffer(Okio.sink(toRestore));

      sink.writeAll(response.body().source());
      sink.close();

      ZipUtils.unzip(toRestore, getFilesDir(),
        BackupService.ZIP_PREFIX_FILES);
      ZipUtils.unzip(toRestore,
        BackupService.getSharedPrefsDir(this),
        BackupService.ZIP_PREFIX_PREFS);
      ZipUtils.unzip(toRestore, getExternalFilesDir(null),
        BackupService.ZIP_PREFIX_EXTERNAL);

      EventBus.getDefault().post(new RestoreCompletedEvent());
    }
    catch (Exception e) {
      Log.e(getClass().getSimpleName(),
        "Exception restoring backup", e);
      EventBus.getDefault().post(new RestoreFailedEvent());
    }
  }

  static class RestoreCompletedEvent {

  }

  static class RestoreFailedEvent {

  }
}
(from Backup/BackupClient/app/src/main/java/com/commonsware/android/backup/RestoreService.java)

The URL for the dataset is coming in via the Intent passed into onHandleIntent(). We use that to build the OkHttp Request, then do a synchronous call via execute() to get the Response.

Previous uses of OkHttp in this chapter focused on REST responses, where we could either just use Location headers or pass the text of the response over to Gson. Here, we are expecting a ZIP file, and possibly a large one. The right way to get that written to disk (so we can unpack it) is to stream the data down and write that data out to disk, rather than attempting to read everything into memory first.

To that end, we take advantage of the fact that OkHttp itself is built atop Square’s Okio library, which offers a nice Java API for handling streams, based on sinks and sources. The recipe for streaming an HTTP response to disk involves:

At that point, we need to unpack the dataset into the places we got the data from in the first place when we backed it up:

To that end, we use a slightly modified version of the ZipUtils class first referenced in the tutorials. The one used in the tutorials comes from the CWAC-Security library. However, that ZipUtils class does not handle two things that we need here:

The BackupClient project has its own modified version of ZipUtils that handles those cases. Beyond that, the unzip() method is the same as before, taking:

When that is done, we post a RestoreCompletedEvent. If there is some problem, we post a RestoreFailedEvent, in addition to logging details to Logcat.

RestoreProgressActivity listens for both of those events:

  @Subscribe(threadMode =ThreadMode.MAIN)
  public void onCompleted(RestoreService.RestoreCompletedEvent event) {
    startActivity(new Intent(this, MainActivity.class));
    finish();
  }

  @Subscribe(threadMode =ThreadMode.MAIN)
  public void onFailed(RestoreService.RestoreFailedEvent event) {
    Toast.makeText(this, R.string.msg_restore_failed,
      Toast.LENGTH_LONG).show();
    finish();
  }
(from Backup/BackupClient/app/src/main/java/com/commonsware/android/backup/RestoreProgressActivity.java)

In the success case, we can now start up a fresh MainActivity (since the original was destroyed as part of launching RestoreProgressActivity), and it can read the restored data.

In the failure case… we are really screwed. We may have partially restored the data, but perhaps not all of it, and there is no telling what state the data is in. A production-grade app would handle this by:

This would reduce the odds of some catastrophic problem wiping out the app. In this sample, though, we just show a Toast, finish() the activity (thereby exiting the app, as we have no other active activities), and hoping the user uninstalls and reinstalls the app, or just uninstalls the app, or something.

Trying This Yourself… With a Little Help from Ol’ Blue Eyes

Everything discussed so far assumes the existence of some REST-style Web server that we can interact with for backups. As it so happens, the BackupClient project has a crude implementation of such a server, in the form of a Ruby script using the Sinatra gem:

require 'fileutils'
require 'time'
require 'sinatra'
require 'json'

BACKUP_ROOT='/tmp/backups'

get '/' do
  'Hello world!'
end

get '/api/backups' do
  result=[]

  if File.exist?(BACKUP_ROOT)
    Dir.foreach(BACKUP_ROOT) do |item|
      next if item == '.' or item == '..'

      subdir=File.join(BACKUP_ROOT, item)

      if File.directory?(subdir)
        f=File.join(subdir, "metadata.json")

        if File.exist?(f)
          metadata=JSON.load(open(f))
          metadata['dataset']="/api/backups/#{item}/dataset"

          result << metadata
        end
      end
    end
  end

  result.sort_by!{|metadata| metadata['timestamp']}
  result.reverse!

  JSON.pretty_generate(result)
end

post '/api/backups' do
  id=SecureRandom.uuid
  dir=File.join(BACKUP_ROOT, id)
  FileUtils.mkdir_p(dir)
  f=File.join(dir, "metadata.json")
  metadata={'timestamp'=>Time.new.xmlschema}
  File.open(f, 'w') {|io| io.write(JSON.generate(metadata))}

  redirect to('/api/backups/'+id), 201
end

put '/api/backups/:id/dataset' do
  dir=File.join(BACKUP_ROOT, params[:id])

  if File.exist?(dir)
    f=File.join(dir, "backup.zip")
    File.open(f, 'w') {|io| io.write(request.body.read)}

    redirect to("/api/backups/#{params[:id]}/dataset"), 201
  else
    status 404
  end
end

get '/api/backups/:id/dataset' do
  dir=File.join(BACKUP_ROOT, params[:id])
  f=File.join(dir, "backup.zip")

  if File.exist?(f)
    send_file f
  else
    status 404
  end
end
(from Backup/BackupClient/server.rb)

If you have familiarity with Ruby, you can:

That will give you a server, listening to localhost:4567… which happens to be what the BackupClient Android app is looking to talk to, if that app is running on an emulator. If you want to test with an actual Android device, the -o switch lets you specify the IP address to listen to, and -p lets you change up the port number if you wish.

The Google Backup Bootstrap

Once you get your real backup system going, then, if you wish, you can play around with Google’s disaster recovery bootstrap. By opting into what Google terms “backup”, you can have some of your data automatically backed up, then restored when the user replaces their device.

What to Bootstrap?

The biggest decision that you will need to make is what should be included in Google’s bootstrap backup and what should not.

The primary considerations are privacy and security. Any data included in the bootstrap is visible to other parties. If that data is not encrypted with a user-supplied passphrase, other parties will be able to do what they want with the data, without much recourse.

One option, therefore, is to opt out of these bootstrap backups entirely, and handle disaster recovery like any other restore process.

Another is to only include some identifying information in the bootstrap backup, to help expedite the restore process, but without really compromising security much. In the context of the BackupClient sample shown earlier in this chapter, if the backup server was adequately secured, including a dataset URL in the bootstrap backup would not be much of a problem. Having the URL itself is probably not that useful, and if only authorized users can download datasets from those URLs, attackers would not gain anything from peeking at the bootstrap. BackupClient itself has very little security, to keep the sample (reasonably) simple, but you can imagine requiring user accounts or similar means to try to lock down access to the backup server.

The far other end of the spectrum is to allow Android to backup “the whole shootin’ match” (i.e., everything), on the grounds that the data you have is not especially private.

You and your qualified legal counsel will need to make this decision before deciding what to do for implementing the bootstrap backup itself.

Bootstrap Backup on Android 6.0+

Android has had a backup API since Android 2.2. However, not only did developers have to opt into the backups, but they had to write special code to assist in those backups. As such, that API was not used that much.

Android 6.0 has gone the other direction, with opt-out backups of all likely data, if your targetSdkVersion is 23 or higher. Specifically:

Backups occur approximately once per day, if the device is idle, charging, and on WiFi.

Configuring the Backup

If what you want to back up is different than what Android 6.0+ will back up by default, you can add manifest entries to better control what is and is not backed up.

To opt out entirely, add android:allowBackup="false" to your <application> element in the manifest:


<application
  android:allowBackup="false"
  android:icon="@drawable/ic_launcher"
  android:label="@string/app_name"
  android:theme="@style/AppTheme"
  tools:replace="android:allowBackup">
  <!-- other cool stuff here -->
</application>

Here, the tools:replace ensures that no library attempts to override your allowBackup value.

Conversely, if you want to participate in the bootstrap backup, but you want to change the roster of what gets backed up, use the android:fullBackupContent attribute on the <application> element. This needs to point to an XML resource that describes what it is that you do and do not want backed up.

The BackupClient sample has this configured. The <application> element points to a res/xml/backup_rules.xml resource:

  <application
    android:fullBackupContent="@xml/backup_rules"
    android:icon="@drawable/ic_launcher"
    android:label="@string/app_name"
    android:theme="@style/Theme.Apptheme">
    <activity
      android:name=".MainActivity"
      android:label="@string/app_name">
      <intent-filter>
        <action android:name="android.intent.action.MAIN" />

        <category android:name="android.intent.category.LAUNCHER" />
      </intent-filter>
    </activity>
    <activity android:name=".RestoreRosterActivity" />
    <activity android:name=".RestoreProgressActivity" />

    <service
      android:name=".BackupService"
      android:permission="android.permission.BIND_JOB_SERVICE" />
    <service
      android:name=".RestoreService"
      android:permission="android.permission.BIND_JOB_SERVICE" />
  </application>
(from Backup/BackupClient/app/src/main/AndroidManifest.xml)

That XML resource can contain <include> and <exclude> elements, inside of a root <full-backup-content> element. The rules are:

The BackupClient sample has two <include> elements, in effect saying that only what is cited in these elements should be backed up:

<?xml version="1.0" encoding="utf-8"?>
<full-backup-content>
  <include
    domain="sharedpref"
    path="com.commonsware.android.backup_preferences.xml"/>
  <include
    domain="sharedpref"
    path="com.commonsware.android.backup.BackupService.xml"/>
</full-backup-content>
(from Backup/BackupClient/app/src/main/res/xml/backup_rules.xml)

The <include> and <exclude> elements must have a domain attribute and a path attribute. These combine to indicate what is being included or excluded.

The domain attribute indicates one of five locations relative to your app:

The path attribute then provides a relative path, from the base location indicated by domain, for the item to be included or excluded.

Hence, the BackupClient backup rules say to include two SharedPreferences files. One is written to by BackupService on every backup, holding a single value, keyed by lastBackupDataset, with the URL to the last backup dataset. The other is the default SharedPreferences, used for the last-visited tab by the UI. Because these SharedPreferences files are included in the bootstrap backup, they should be restored in case the user replaces the device. However, they are the only things that is supposed to be backed up — everything else in the app should be left alone.

Note that the documentation does not state clearly if the path attribute is required. It is possible that the path attribute is optional, where if it is missing, it means you want to include or exclude everything in the cited domain.

Testing the Backup and Restore Steps

In theory, to test your backup configuration, you can run three commands on the command line:


adb shell setprop log.tag.BackupXmlParserLogging VERBOSE
adb shell bmgr run
adb shell bmgr fullbackup ...

where ... is the application ID of the app to be backed up. For the sample app, that is com.commonsware.android.backup.

(the above assumes that you have adb in your PATH)

You can then manually initiate a restore operation via:


adb shell bmgr restore ...

for the same value of .... Presumably, you would do this after modifying or clearing the backed-up data, so you can confirm that the data was restored properly.

For the purposes of conducting lightweight experiments with the auto-backup facility, you do not need to mess around with the entire backup system outlined earlier in this chapter. That backs up the actual content; the auto-backup facility is backing up SharedPreferences, and that happens whether or not we are also backing up the content.

So, for example, you could do the following:


adb shell run-as com.commonsware.android.backup
"cat /data/data/com.commonsware.android.backup/
shared_prefs/com.commonsware.android.backup_preferences.xml"

(NOTE: the above should be all on one line; it is split here across three lines due to the length of the command)

You should see something like:


<?xml version='1.0' encoding='utf-8' standalone='yes' ?>
<map>
    <int name="lastVisited" value="2" />
</map>

The value will be the index of whatever tab you were on when you exited the activity.


adb shell setprop log.tag.BackupXmlParserLogging VERBOSE
adb shell bmgr run
adb shell bmgr fullbackup com.commonsware.android.backup

You should see output in Logcat indicating that the backup was taken:


14936-14936/? D/AndroidRuntime: Calling main entry com.android.commands.bmgr.Bmgr
800-2345/? D/BackupManagerService: fullTransportBackup()
800-14960/? I/PFTBT: Initiating full-data transport backup of ...
800-14961/? D/BackupManagerService: Binding to full backup agent : ...
800-14961/? D/BackupManagerService: awaiting agent for ApplicationInfo{...}
800-810/? D/BackupManagerService: agentConnected pkg=com.commonsware...
800-14961/? I/BackupManagerService: got agent android.app.IBackupAgent$Stub$Proxy@e17804c
800-14961/? I/BackupRestoreController: Getting widget state for user: 0
800-14962/? I/file_backup_helper:    Name: apps/com.commonsware.android...
800-14962/? D/BackupManagerService: Calling doFullBackup() on com.commonsware...
9380-9391/com.commonsware.android.backup I/file_backup_helper:    Name: ...
800-14960/? I/PFTBT: Transport suggested backoff=0
800-14960/? I/PFTBT: Full backup completed.
9380-9380/? I/Process: Sending signal. PID: 9380 SIG: 9
800-2345/? D/BackupManagerService: Done with full transport backup.

(NOTE: the lines have been truncated due to length)


16814-16814/? D/AndroidRuntime: Calling main entry com.android.commands.bmgr.Bmgr
800-7101/? V/BackupManagerService: beginRestoreSession: pkg=com.commonsware...
800-2345/? V/RestoreSession: restorePackage pkg=com.commonsware.android.backup ...
800-2345/? V/RestoreSession: restorePackage pkg=com.commonsware.android.backup ...
800-1111/? D/BackupManagerService: MSG_RUN_RESTORE observer=android.app.backup...
800-1111/? D/BackupManagerService: initiateOneRestore packageName=@pm@
800-1111/? E/SELinux: SELinux: Could not get canonical path /cache/@pm@.restore ...
800-1111/? I/BackupManagerService: Next restore package: RestoreDescription{...}
800-16839/? I/RestoreEngine: Sig + version match; taking data
800-16839/? D/RestoreEngine: Need to launch agent for com.commonsware.android.backup
800-16839/? D/RestoreEngine: Clearing app data preparatory to full restore
800-16839/? I/ActivityManager: Force stopping com.commonsware.android.backup ...
800-16839/? I/ActivityManager: Killing 15029:com.commonsware.android.backup/...
800-1195/? D/GraphicsStats: Buffer count: 5
800-1198/? W/ActivityManager: Spurious death for ProcessRecord...
5005-5986/? D/Documents: Update found 7 roots in 8ms
1888-16840/? D/PackageBroadcastService: Received broadcast ...
1888-16840/? D/AccountUtils: Clearing selected account for com.commonsware...
1888-16840/? I/LocationSettingsChecker: Removing dialog suppression flag...
1888-2082/? I/Icing: doRemovePackageData com.commonsware.android.backup
800-16839/? I/ActivityManager: Start proc 16848:com.commonsware.android...
800-16839/? D/BackupManagerService: awaiting agent for ApplicationInfo{...}
16848-16848/? I/art: Late-enabling -Xcheck:jni
16848-16848/? W/System: ClassLoader referenced unknown path: ...
800-1128/? D/BackupManagerService: agentConnected pkg=com.commonsware.android...
800-16839/? I/BackupManagerService: got agent android.app...
16848-16865/com.commonsware.android.backup V/BackupXmlParserLogging: ...
16848-16865/com.commonsware.android.backup V/BackupXmlParserLogging: ...
16848-16865/com.commonsware.android.backup V/BackupXmlParserLogging: ...
16848-16865/com.commonsware.android.backup V/BackupXmlParserLogging: ...
16848-16865/com.commonsware.android.backup V/BackupXmlParserLogging: ...
16848-16865/com.commonsware.android.backup V/BackupXmlParserLogging: ...
16848-16865/com.commonsware.android.backup V/BackupXmlParserLogging: Final tally.
16848-16865/com.commonsware.android.backup V/BackupXmlParserLogging: Includes:
16848-16865/com.commonsware.android.backup V/BackupXmlParserLogging:   domain=sp
16848-16865/com.commonsware.android.backup V/BackupXmlParserLogging: ...
16848-16865/com.commonsware.android.backup V/BackupXmlParserLogging: Excludes:
16848-16865/com.commonsware.android.backup V/BackupXmlParserLogging:   ...nothing to exclude.
16848-16865/com.commonsware.android.backup V/BackupXmlParserLogging:   
16848-16865/com.commonsware.android.backup V/BackupXmlParserLogging: ...
800-1111/? V/BackupManagerService: No more packages; finishing restore
800-2345/? D/RestoreSession: endRestoreSession
800-1111/? I/BackupRestoreController: restoreFinished for 0
800-1111/? I/BackupManagerService: Restore complete.
800-1111/? V/BackupManagerService: Clearing restore session and halting timeout

Bootstrap Backup on Android 2.2-5.1

Prior to Android 6.0, Android had a “backup service”, inaugurated in Android 2.2. As with the Android 6.0 approach, the original backup service was mostly for disaster recovery.

Unlike with Android 6.0’s approach, you needed to opt into having these backups. Partly, this opt-in was accomplished via code, as you had to extend a BackupAgentHelper and register it in your manifest as the android:backupAgent, via the <application> element. The BackupAgentHelper subclass would indicate what should be backed up, by instantiating one or more BackupHelper objects (e.g., FileBackupHelper), configuring them to back up certain items, then registering them with the BackupAgentHelper via an addHelper() method.

Partly, though, the opt-in was accomplished via registering for an API key. This process was never integrated into the rest of the Play Services architecture, which now has a standardized approach for registering for various API keys and agreeing to the terms of service for each.

Instead, you would need to visit an obscure Web page, agree to the terms of service, provide information about your app (notably the application ID), get the API key, and add it to your manifest via a <meta-data> element.

However, those terms of service contain some interesting clauses, ones that may give your legal counsel some concern, such as:

Beyond this, there are no statements about where the data is actually backed up, other than opening it up to just about anyone that Google wishes to characterize as “Subsidiaries and Affiliates”.

Please discuss these terms with legal counsel before registering for this service and integrating it within your app.

Additional documentation about this form of backup, should you choose to pursue it, can be found online.

Boosting Backup Security

Backups, in effect, are intentional data leaks. You want something other than the device to have access to your app’s data. Hence, it is important to take reasonable steps to ensure that those backups are secure, secure enough that nobody is going to be able to exploit them for uses that go against the user’s wishes. Rest assured that people will try to exploit backups and will succeed if your security is insufficient.

Securing Access to the Dataset

The backup dataset that you transfer off the device needs to be secure from attack. Unauthorized people should not be able to get at the dataset.

For a backup system like the one outlined in this chapter, the big thing to secure is access to the dataset via its URL. If anyone who gets the URL can download the dataset, now all an attacker needs to do is determine how to get that URL, such as by exploiting flaws in Google’s bootstrap backup. Or, for that matter, Google staff could get at the URL, at least in principle.

In this case, the URL alone must be insufficient. It would need to be combined with other information from the user, such as some sort of site authentication, where that other information is not retained.

If you are holding onto backup datasets yourself, on your own servers, you will also need to ensure that only authorized staff can get at those datasets and that such access is highly visible. Otherwise, you are at risk of an insider attack, whether through so-called “social engineering” or just good old-fashioned extortion.

Securing Transmission of the Dataset

Another way that an attacker could get at the dataset is to copy the data in motion, as it is sent from your app to the backup server. Make sure that you are using suitable security here:

Bear in mind that users may wind up making a backup from any sort of network, ranging from your office network to the free WiFi at a local coffee shop. In principle, you could detect this and refuse to back up the data when you do not recognize the network. However, this reduces the value of the backup system, as the user might not be able to make a manual backup at some point when they need it (e.g., on business travel).

Encrypting the Dataset

The ultimate in protection for the user is to have the data be encrypted by a user-supplied passphrase. Then, even you cannot access the data without the user’s assistance. There are ways of addressing this, perhaps involving brute force attacks or other sorts of brute force attacks. However, it certainly slows attackers down.

The simplest way to have encrypted backups — from the standpoint of the person writing the backup code — is to encrypt the data itself. For example, you do not necessarily need to re-encrypt a SQLCipher for Android database as part of a backup dataset, as it is already encrypted. Note, though, that having encrypted data at rest does not mean you can skip encrypting the data in motion, as it is sent to your backup server. While attackers would not be able to read the backed-up data readily, they could replace the backed-up data sent over the unencrypted communications channel and perhaps cause problems that way.

If, however, you are not in position to encrypt the data at rest within your app, you may wish to consider asking the user for a passphrase and using that to encrypt the backup dataset. Note that this passphrase requirement largely eliminates the ability for you to do unattended automated backups, as you either do not have the passphrase then (and so cannot encrypt the backups) or you are saving the passphrase (and so have just made it trivial for somebody to get it and decrypt the data).

Alternative Approaches

Backing up local data is essential where the device is the system of record, to be able to deal with catastrophe (e.g., the user accidentally uninstalls the app).

That being said, there are a few ways of dealing with backing up local data that might not necessarily seem to the user as though it is a backup process.

Data Versioning

Beyond the accidental wiping of data, such as through an erroneous install, a backup can also help recover from more fine-grained errors, like accidentally deleting a bit of data (e.g., a row or set of rows from database tables).

One way to address that is to use some sort of data versioning approach. Many software developers are familiar with this in the form of source code version control, such as git. Here, you never really “delete” anything forever. Instead, you delete (or change) things in your working copy of the data, with the versioning system tracking changes to the data, so you can roll back to some earlier version if the need arises.

This is not limited to source code or similar sorts of documents. One simple example of versioning that has been used for decades is to not actually delete database rows, but instead set some is_deleted column to a known value. Then, when you query the database, you filter out the “deleted” rows by excluding from the query those rows where is_deleted is set to that specific value. Recovering those deleted rows is then a matter of showing all the deleted ones to the user and clearing is_deleted for the ones to be restored.

Obviously, this gets much more complicated once you get into foreign key constraints (i.e., how can you restore X if it depends on Y that was also deleted?). And it is not a full replacement for a backup-and-restore system, since anything that damages or deletes the entire database cannot be recovered via this sort of versioning. But, if you are looking to implement a robust disk-based “undo” facility for users, just bear in mind that it also helps out for some sorts of cases where you might ordinarily think of restoring from a backup.

Import and Export

Another feature that you can add that has some relationship to data backups is data import and export. Whatever is exported can be backed up by the user by some other means; if the master copy of the app’s data gets damaged, you might be able to recover from that damage via importing a previous export.

Of course, import and export are also used for data exchange with foreign systems (e.g., exporting tabular data in a format that can be read in by a desktop spreadsheet program). Also, traditionally, import and export are tasks that are manually requested by users. However, you might consider giving the user an option of performing an automatic export as a replacement for, or adjunct to, some other form of regular backup.

Data Synchronization

The ultimate solution for not having to mess with a robust device-based backup system is to not have the device be the system of record. Instead, some server is the system of record, with the device holding what amounts to a persistent cache of some of that data: