SQLite databases, by default, are stored on internal storage, accessible only to the app that creates them.
At least, that is the theory.
In practice, it is conceivable that others could get at an app’s SQLite database, and that those “others” may not have the user’s best interests at heart. Hence, if you are storing data in SQLite that should remain confidential despite extreme measures to steal the data, you may wish to consider encrypting the database.
Perhaps the simplest way to encrypt a SQLite database is to use SQLCipher. SQLCipher is a SQLite extension that encrypts and decrypts database pages as they are written and read. However, SQLite extensions need to be compiled into SQLite, and the stock Android SQLite does not have the SQLCipher extension.
SQLCipher for Android,
therefore, comes in the form of a replacement implementation of
SQLite that you add as an NDK library to your project. It also ships
with replacement editions of the android.database.sqlite.* classes
that use the SQLCipher library instead of the built-in SQLite. This
way, your app can be largely oblivious to the actual database
implementation, particularly if it is hidden behind a
ContentProvider or similar abstraction layer.
SQLCipher for Android is a joint initiative of Zetetic (the creators of SQLCipher) and the Guardian Project (home of many privacy-enhancing projects for Android). SQLCipher for Android is open source, under the Apache License 2.0.
Understanding this chapter requires that you have read the chapter on database access.
So, why might you want to encrypt a database?
Some developers probably are thinking that this is a way of protecting the app’s content against “those pesky rooted device users”. In practice, this is unlikely to help. As with most encryption mechanisms, SQLCipher uses an encryption key. If the app has the key, such as being hard-coded into the app itself, anyone can get the key by reverse-engineering the app.
Rather, encrypted databases are to help the user defend their data against other people seeing it when they should not. The classic example is somebody leaving their phone in the back of a taxi — if that device winds up in the hands of some group with the skills to root the device, they can get at any unencrypted content they want. While some users will handle this via the whole-disk encryption available since Android 3.0, others might not.
If the database is going anywhere other than internal storage, there is all the more reason to consider encrypting it, as then it may not even require a rooted device to access the database. Scenarios here include:
SQLCipher is available from Zetetic. As of July 2016, the current shipping version was 3.5.0. It is very important for you to use 3.5.0 or higher, as earlier versions of SQLCipher for Android will not work on Android 7.0 or higher versions of Android.
In Android Studio, to add SQLCipher for Android to your project, just add the official AAR dependency:
dependencies {
implementation 'net.zetetic:android-database-sqlcipher:3.5.0@aar'
}
If you have existing code that uses classic Android SQLite, you will
need to change your import statements to pick up the SQLCipher for
Android equivalents of the classes. For example, you obtain
SQLiteDatabase now from net.sqlcipher.database.sqlcipher,
not android.database.sqlite. Similarly, you obtain SQLException
from net.sqlcipher.database instead of android.database.
Unfortunately, there is no complete list of which classes need this
conversion — Cursor, for example, does not. Try converting
everything from android.database and android.database.sqlite, and
leave alone those that do not exist in the SQLCipher for Android
equivalent packages.
Before starting to use SQLCipher for Android, you need to call
SQLiteDatabase.loadLibs(), supplying a suitable Context object as
a parameter. This initializes the necessary libraries. If you are
using a ContentProvider, just call this in onCreate() before
actually using anything else with your database. If you are not using
a ContentProvider, you probably will want to create a custom
subclass of Application and make this call from that class’
onCreate(), and reference your custom Application class in the
android:name attribute of the <application> element in your
manifest. Either of these approaches will help ensure that the
libraries are ready before you try doing anything with the database.
Finally, when calling getReadableDatabase() or
getWritableDatabase() on SQLiteDatabase, you need to supply the
encryption key to use. For the purposes of book examples, a hard-coded
passphrase is sufficient. However, those can be trivially
reverse-engineered, and so they offer little real-world protection.
But, they keep the code simple, which is useful when examining APIs.
The
Database/ConstantsSecure-AndroidStudio
sample app is yet another variation of the ConstantsBrowser
sample that we have been using for most of the database examples.
From the standpoint of the ConstantsBrowser activity and
ConstantsFragment UI, nothing is different. However,
DatabaseHelper uses SQLCipher, rather than SQLite.
In the DatabaseHelper constructor, we call loadLibs()
on the SQLiteDatabase class, which is a required initialization step
to get the native libraries set up:
public DatabaseHelper(Context context) {
super(context, DATABASE_NAME, null, SCHEMA);
SQLiteDatabase.loadLibs(context);
}
It also offers zero-argument getReadableDatabase() and
getWritableDatabase() methods, akin to those offered by the regular
SQLiteOpenHelper. However, the DatabaseHelper editions turn around
and invoke the one-argument equivalents on the SQLCipher edition of
SQLiteOpenHelper:
SQLiteDatabase getReadableDatabase() {
return(super.getReadableDatabase(PASSPHRASE));
}
SQLiteDatabase getWritableDatabase() {
return(super.getWritableDatabase(PASSPHRASE));
}
Here, the PASSPHRASE is just a hard-coded string:
private static final String PASSPHRASE=
"hard-coding passphrases is only for sample code;"+
"nobody does this in production";
That is all the changes that are needed to use SQLCipher.
Alas, SQLCipher for Android is not perfect.
It will add a few MB to the size of your APK file per CPU architecture.
For most modern
Android devices, this extra size will not be a huge issue, though it
will be an impediment for older devices with less internal storage, or
for apps that are getting close to the size limits imposed by the Play
Store or other distribution mechanisms. The chapter on the NDK contains
a section about a technology called libhoudini
that can help reduce this bloat, albeit with a significant performance
penalty.
However, the size is mostly from code, and that may cause a problem
for Eclipse users. Eclipse may crash with its own OutOfMemoryError
during the final build process. To address that, find your
eclipse.ini file (location varies by OS and installation method)
and increase the -Xmx value shown on one of the lines (e.g., change
it to -Xmx512m).
Other code that expects to be using native SQLite databases will
require alteration to work with SQLCipher for Android databases. For
example, the SQLiteAssetHelper described
elsewhere in this book
would need to be ported to use the
SQLCipher for Android implementations of SQLiteOpenHelper,
SQLiteDatabase, etc. This is not too difficult for an open source
component like SQLiteAssetHelper.
Given an encrypted database, there are several ways that an attacker can try to access the data, including:
The classic way to prevent the first approach is by having business logic that prevents lots of failed login attempts in a short period of time. This can be built into your login dialog (or the equivalent), tracking the number and times of failed logins and introducing delays, forced app exits, or something to add time and hassle for trying lots of passwords.
Since manually trying passwords is nasty, brutish, and long, many attackers would automate the process by copying the SQLCipher database to another machine (e.g., desktop) and running a brute-force attack on it directly. SQLCipher for Android has many built-in protections to help defend against this. So long as you are using a sufficiently long and complex encryption key, you should be fairly well-protected against such attacks.
Defending against wrenches is decidedly more difficult and is beyond the scope of this book.
Having a solid encryption algorithm, like the AES-256 used by default with SQLCipher for Android, is only half the battle. The other half is in using a high-quality passphrase, one that is unlikely to be guessed by anyone looking to break the encryption.
Suppose you have an app already out on the market, and you decide that you want to add the option for encryption. It is fairly likely that the user will be miffed if they lose all their data in the process of switching to an encrypted database. Therefore, you will want to try to retain their data.
SQLCipher for Android does not support in-place encryption of database. However, it does support working with unencrypted databases and encrypted databases simultaneously, giving you the option of migration.
The approach boils down to:
ATTACH statement to open the encrypted database inside the same
SQLCipher for Android sessionsqlcipher_export() function to migrate most of the dataDETACH the encrypted databaseSince both database files will exist at one time, you will find it simplest
to use separate names for them (e.g., stuff.db and stuff-encrypted.db).
To see how this works, take a look at the
Database/SQLCipherPassphrase-AndroidStudio,
which is a variation of the original, non-ContentProvider “constants”
sample app, this time using SQLCipher for Android and supporting an upgrade
from a non-encrypted database to an encrypted one.
The bulk of the logic for handling the encryption upgrade is in a static
encrypt() method on our DatabaseHelper:
static void encrypt(Context ctxt) {
SQLiteDatabase.loadLibs(ctxt);
File dbFile=ctxt.getDatabasePath(DATABASE_NAME);
File legacyFile=ctxt.getDatabasePath(LEGACY_DATABASE_NAME);
if (!dbFile.exists() && legacyFile.exists()) {
SQLiteDatabase db=
SQLiteDatabase.openOrCreateDatabase(legacyFile, "", null);
db.rawExecSQL(String.format("ATTACH DATABASE '%s' AS encrypted KEY '%s';",
dbFile.getAbsolutePath(), PASSPHRASE));
db.rawExecSQL("SELECT sqlcipher_export('encrypted')");
db.rawExecSQL("DETACH DATABASE encrypted;");
int version=db.getVersion();
db.close();
db=SQLiteDatabase.openOrCreateDatabase(dbFile, PASSPHRASE, null);
db.setVersion(version);
db.close();
legacyFile.delete();
}
}
First, we initialize SQLCipher for Android by calling loadLibs() on the
SQLCipher version of SQLiteDatabase. We could do this someplace else,
but for this sample, this is as good a spot as any.
We then create File objects pointing at the locations of the old, unencrypted
database (with a name represented by a LEGACY_DATABASE_NAME static data
member) and the new encrypted database (DATABASE_NAME). To get the File
locations of those databases, we use getDatabasePath(), a method on Context,
which returns the correct location for a database file given its name.
If the encrypted database exists, there is nothing that we need to do. Similarly,
if it does not exist but the unencrypted database also does not exist,
there is nothing that we can do. In either of those cases, we skip over
the rest of the logic. In the first case, we already did the conversion
(presumably); in the latter case, this is a new installation, and our
SQLiteOpenHelper onCreate() logic will handle that. But, in the case where
we do not have the encrypted database but do have the unencrypted one, we
can create the encrypted database from the unencrypted data, which is what
the bulk of the encrypt() method does.
To that, we:
openOrCreateDatabase() to open the already-existing unencrypted
database file in SQLCipher for Android, using "" as the passphrase.rawExecSQL() method available on the SQLCipher for Android version
of SQLiteDatabase to ATTACH the encrypted database, given its path,
to our database session, using the supplied passphrase. This means that we
can access the tables from both databases simultaneously, though we need
to prefix all references to the attached database via its handle, encrypted.rawExecSQL() to execute SELECT sqlcipher_export('encrypted'), which
copies most of our data from the unencrypted database (the database we have
open) into the encrypted database (the one we attached). The big thing that
sqlcipher_export() does not copy is the schema version number that
Android maintains.rawExecSQL() to DETACH the attached encrypted database, as we
no longer need it.getVersion() on the SQLiteDatabase representing the unencrypted
database, to retrieve the schema version number that Android maintains.openOrCreateDatabase().setVersion() on SQLiteDatabase to set the schema version of the
encrypted database to the value we had from the unencrypted database.deleteDatabase() method on SQLiteDatabase
to cleanly delete everything associated with SQLite.The combination of doing all of that migrates our data from an unencrypted database to an encrypted one.
Then, we simply need to call encrypt() before we try loading our constants,
from doInBackground() of our LoadCursorTask:
private class LoadCursorTask extends BaseTask<Void> {
private final Context ctxt;
LoadCursorTask() {
this.ctxt=getActivity().getApplicationContext();
}
@Override
protected Cursor doInBackground(Void... params) {
DatabaseHelper.encrypt(ctxt);
return(doQuery());
}
}
To test this upgrade logic, you will need to:
Database/Constants
sample applicationYou will see your added constant appear along with all of the standard ones,
yet if you examine /data/data/com.commonsware.android.constants/databases
on your ARM emulator via DDMS, you will see that your database is now named
constants-crypt.db instead of constants.db, as we have replaced the
unencrypted database with an encrypted one.
Another thing the user might wish to do is change their passphrase. Perhaps they fear that their existing passphrase has been compromised (e.g., a narrow escape from a $5 wrench). Perhaps they rotate their passphrases as a matter of course. Perhaps they simply keep typing in their current one incorrectly and want to switch to one they think they can enter more accurately.
SQLCipher for Android supports a rekey PRAGMA that can accomplish this.
Given an open encrypted database db — opened using the old passphrase –
you can change the password to a newPassword string variable via:
db.execSQL(String.format("PRAGMA rekey = '%s'", newPassword));
Note that this may take some time, as SQLCipher for Android needs to re-encrypt the entire database.
If you are starting with SQLCipher for Android with the 3.0.x release, all is good.
If you have been using SQLCipher for Android from previous releases, but you are still in development mode, all is still good, so long as you can wipe out your old databases.
If you have apps in production using SQLCipher for Android from previous
releases, you will have a small headache: the database structure has changed.
SQLCipher for Android provides us with a PRAGMA cipher_migrate that we can
run to upgrade the database in place to the new structure, once we have
opened the database with our passphrase. However:
SQLCipher for Android, in an attempt to help with this,
offers a modified version of methods like openOrCreateDatabase()
on SQLiteDatabase, ones that take a SQLiteDatabaseHook implementation
as the last parameter. This interface requires two methods:
preKey(), called after the database is opened but before the
passphrase is appliedpostKey(), called after the database is opened and after
the passphrase is applied, but before anything else is done (e.g., standard
SQLiteOpenHelper schema version checking)Both methods are passed the SQLiteDatabase as a parameter, for you
to do with as needed. So, for example, you could have a postKey() implementation
that does the postKey() call only if needed:
public class SQLCipherV3Hook implements SQLiteDatabaseHook {
private static final String PREFS=
"net.sqlcipher.database.SQLCipherV3Helper";
public static void resetMigrationFlag(Context ctxt, String dbPath) {
SharedPreferences prefs=
ctxt.getSharedPreferences(PREFS, Context.MODE_PRIVATE);
prefs.edit().putBoolean(dbPath, false).commit();
}
@Override
public void preKey(SQLiteDatabase database) {
// no-op
}
@Override
public void postKey(SQLiteDatabase database) {
SharedPreferences prefs=
getContext().getSharedPreferences(PREFS, Context.MODE_PRIVATE);
boolean isMigrated=prefs.getBoolean(database.getPath(), false);
if (!isMigrated) {
database.rawExecSQL("PRAGMA cipher_migrate;");
prefs.edit().putBoolean(database.getPath(), true).commit();
}
}
}
You can also pass a SQLiteDatabaseHook implementation into the
SQLiteOpenHelper constructor as the fifth parameter, which will
be used when SQLiteOpenHelper works with the underlying SQLiteDatabase.
Another way to effectively boost the strength of your security is to implement
your own multi-factor authentication. In this case, the passphrase is not
obtained solely through the user typing in the whole thing, but instead
is synthesized from two or more sources. So, in addition to some EditText
widget for entering in a portion of the passphrase, the rest could come from
things like:
You, in code, would concatenate the pieces together, possibly using delimiters that cannot be typed in (e.g., ASCII characters below 32) to denote the sources of each segment of the passphrase. The result would be the actual passphrase you would use with SQLCipher for Android.
The objective is to make it easier for users to have more complex passphrases, while not having to type in something complex every time. Tapping an NFC tag is much faster than tapping out a passphrase on a typical phone keyboard, for example. Also, the “something you know and something you have” benefit of multi-factor authentication can help with defending against $5 wrench attacks: if the NFC tag was destroyed, and the user never knew the portion of the passphrase stored on it, the user cannot divulge it.
Of course, this adds risks, such as the NFC tag being destroyed accidentally (e.g., “my dog ate it”). This can be mitigated in some cases by some “admin” being able to reset the password or supply a new NFC tag. In that case, getting the credentials requires two kidnappings and two $5 wrenches (or the serial application of a single $5 wrench, if budgets preclude buying two such wrenches), adding to the degree of difficulty for breaking the encryption by that means.
If you try to decrypt a database using the incorrect passphrase — whether an attempt by outsiders to use the app, or the user “fat-fingering” the passphrase and making a typo — you will get an exception:
11-19 09:17:22.700: E/SQLiteOpenHelper(1634): net.sqlcipher.database.SQLiteException: file is encrypted or is not a database
Alas, this is not a specific exception, making it a bit difficult to detect failed passphrases specifically. Your options are:
Some developers worry about the overhead that encryption will place on the database I/O, and therefore worry that SQLCipher for Android will make their app unacceptably slow.
The impact of SQLCipher is not that bad, particularly for hardware with faster CPUs. Encryption is CPU-intensive, so faster CPUs reduce the overhead of the encryption. Also, since the disk I/O is comparable between SQLite and SQLCipher, the fact that flash memory is slow will mean that disk I/O, not decryption speed, will be the primary determinant of the speed of your queries. Similarly, disk I/O will count for more than CPU speed for the encryption needed for INSERT/UPDATE/DELETE operations.
For example, porting one relatively crude benchmark to use SQLCipher for Android showed no statistically significant performance difference from the SQLite edition on a Nexus 5 running Android 4.4.2.
To the extent that encryption adds overhead, it will tend to magnify existing problems. For example, anything that involves a “table scan” (i.e., a non-indexed lookup of database contents) will need more pages to be decrypted and, therefore, more decryption time. If your database I/O is well-tuned for SQLite, such as adding appropriate indexes, then your SQLCipher for Android overhead should be nominal.
Of course, the worse the CPU, the worse the story, and so older/cheaper devices may fare worse with SQLCipher for Android by comparison.
There are effectively three forms of data storage in Android:
SharedPreferencesYou can encrypt SQLite via SQLCipher for Android, as seen in this chapter.
You can encrypt arbitrary files as part of your data format, such as via
javax.crypto.
What is not supported, out of the box, is a way to encrypt SharedPreferences.
There are two approaches for encrypting the contents of SharedPreferences:
SharedPreferences are storedSharedPreferences,
and decrypt it when you read the value back outSharedPreferences is an interface. Hence, you can create other implementations
of that interface that store their data in something other than unencrypted
XML files.
CWSharedPreferences is one such implementation. You can find it in the
cwac-prefs project on GitHub.
CWSharedPreferences handles the SharedPreferences and
SharedPreferences.Editor interfaces, along with the in-memory representations
of the preferences. It then delegates the work of storing the preferences
to a strategy object, implementing a strategy interface
(CWSharedPreferences.StorageStrategy). Two such strategy implementations
are supplied in the project: one using ordinary SQLite, and one using
SQLCipher for Android.
The basic recipe for using CWSharedPreferences is:
new SQLCipherStrategy(getContext(), NAME, "atestpassword", LoadPolicy.SYNC)
(here, NAME is the name of the set of preferences, "atestpassword" is your
passphrase, and LoadPolicy.SYNC indicates that the preferences should be loaded
from disk immediately, not on a background thread)
CWSharedPreferences that employs your chosen strategy:
new CWSharedPreferences(yourStrategyObjectGoesHere);
CWSharedPreferences as you would any other SharedPreferences
implementationclose() on the strategy object, to release any resources that it
might hold (e.g., open database connection)The big drawback to the custom SharedPreferences is the fact that you cannot
get the PreferenceScreen system to work with it. The preference UI is hard-wired
to use the stock implementation of SharedPreferences and does not appear to
support any way to substitute in some other implementation.
Hence, another approach is to keep things in standard SharedPreferences’ XML
files, but encrypt text values on a preference-by-preference basis. Since the
data type needs to remain the same, most likely you would restrict this to
encrypting strings (e.g., EditTextPreference, ListPreference) rather than
numbers, booleans, etc.
To do this, you would need to:
Preference classes of interest and override methods that
would deal with the raw preference data, like onDialogClosed(), to
encrypt the values you persist and decrypt the values you read in, using
the static methods mentioned abovePreference classes in your preference XML as neededSharedPreferences
The downsides to this approach include:
Preference
classes, forcing you to roll your ownSQLCipher for Android is also used as the backing store for
IOCipher. IOCipher is a virtual
file system (VFS) for Android, allowing you to write code that looks and works
like it uses normal file I/O, yet all of the files are actually saved as BLOBs
in a SQLCipher for Android database. The result is a fully-encrypted VFS,
inheriting all of SQLCipher’s security features, such as default AES-256
encryption. This may be easier for you to use than encrypting and decrypting
files individually via javax.crypto, for example.
IOCipher is considered to be in pre-alpha state as of November 2012.