Given that you are collecting metrics about bandwidth consumption, you can now start to determine ways to reduce that consumption. You may be able to permanently reduce that consumption (at least on a per-operation basis). You may be able to shunt that consumption to times or networks that the user prefers. This chapter reviews a variety of means of accomplishing these ends.
Understanding this chapter requires that you have read the core chapters and understand how Android apps are set up and operate, particularly the chapter on Internet access.
The best way to reduce bandwidth consumption is to consume less bandwidth.
(in other breaking news, water is wet)
In recent years, developers have been able to be relatively profligate in their use of bandwidth, pretty much assuming everyone has an unlimited high-speed Internet connection to their desktop or notebook and the desktop or Web apps in use on them. However, those of us who lived through the early days of the Internet remember far too well the challenges that dial-up modem accounts would present to users (and perhaps ourselves). Even today, as Web apps try to “scale to the Moon and back”, bandwidth savings becomes important not so much for the end user, but for the Web app host, so its own bandwidth is not swamped as its user base grows.
Fortunately, widespread development problems tend to bring rise to a variety of solutions — a variant on the “many eyes make bugs shallow” collaborative development phenomenon. Hence, there are any number of tried-and-true techniques for reducing bandwidth consumption that have had use in Web apps and elsewhere. Many of these are valid for native Android apps as well, and a few of them are profiled in the following sections.
Trying to get lots of data to fit on a narrow pipe — whether that pipe is on the user’s end or the provider’s end — has long been a struggle in Web development. Fortunately, there are a number of ways you can leverage HTTP intelligently to reduce your bandwidth consumption.
By default, HTTP requests and response are uncompressed. However, you can enable GZip encoding and thereby request that the server compress its response, which is then decompressed on the client. This trades off CPU for bandwidth savings and therefore needs to be done judiciously.
Enabling GZip compression is a two-step process:
Accept-Encoding: gzip header to the HTTP requestBear in mind that the Web server may or may not honor your GZip request, for whatever reason (e.g., response is too small to make it worthwhile).
Of course, avoiding a download offers near-100% compression. If you
are caching data, you can take advantage of HTTP headers to try to
skip downloads that are the same content as what you already have,
specifically If-Modified-Since and If-None-Match.
An HTTP response can contain either a Last-Modified header or an
ETag header. The former will contain a timestamp and the latter
will contain some opaque value. You can store this information with
the cached copy of the data (e.g., in a database table). Later on,
when you want to ensure you have the latest version of that file,
your HTTP GET request can include an If-Modified-Since header (with
the cached Last-Modified value) or an If-None-Match header (with
the cached ETag value). In either case, the server should return
either a 304 response, indicating that your cached copy is up to
date, or a 200 response with the updated data. As a result, you
avoid the download entirely (other than HTTP headers) when you do not
need the updated data.
While XML and JSON are relatively easy for humans to read, that very characteristic means they tend to be bloated in terms of bandwidth consumption. There are a variety of tools, such as Google’s Protocol Buffers and Apache’s Thrift, that allow you to create and parse binary data structures in a cross-platform fashion. These might allow you to transfer the same data that you would in XML or JSON in less space. As a side benefit, parsing the binary responses is likely to be faster than parsing XML or JSON. Both of these tools involve the creation of an IDL-type file to describe the data structure, then offer code generators to create Java classes (or equivalents for other languages) that can read and write such structures, converting them into platform-neutral on-the-wire byte arrays as needed.
If you are loading JavaScript or CSS into a WebView, you should
consider standard tricks for compressing those scripts, collectively
referred to as
“minification”.
These techniques eliminate all unnecessary whitespace and such
from the files, rename variables to be short, and otherwise create a
syntactically-identical script that takes up a fraction of the space.
A chunk of the overhead involved in HTTP operations is simply establishing the socket connection with the Web server. Advertising that you want the socket to be kept alive, in anticipation of upcoming follow-on requests, can reduce this overhead.
Using higher-level HTTP clients, like OkHttp, helps here, because usually they handle all the details of keeping the socket open.
With SSL, though, keep-alive was not an option, until Google released the SPDY specification. SPDY in turn formed the basis of HTTP/2, the new standard for Web communications (replacing the venerable HTTP/1.1). OkHttp supports SPDY and HTTP/2.
Another way to consume less bandwidth is to only make the requests when it is needed. For example, if you are writing an email client, the way to use the least bandwidth is to download new messages only when they exist, rather than frequently polling for messages.
Off the cuff, this may seem counter-intuitive. After all, how can we know whether or not there are any messages if we are not polling for them?
The answer is to use a low-bandwidth push mechanism. The
quintessential example of this is GCM, the Google Cloud Messaging
system, available for Android 2.2 and newer. This service from Google
allows your application to subscribe to push notifications sent out
by your server. Those notifications are delivered asynchronously to
the device by way of Google’s own servers, using a long-lived socket
connection. All you do is register a BroadcastReceiver to receive
the notifications and do something with them.
For example, Remember the Milk — a task management Web site and set of mobile apps — uses GCM to alert the device of task changes you make through the Web site. Rather than the Remember the Milk app having to constantly poll to see if tasks were added, changed, or deleted, the app simply waits for GCM events.
You could create your own push mechanism, perhaps using a WebSocket or MQTT. The downside is that you will need a service in memory all of the time to manage the socket and thread that monitors it. If you only need this while your service is in memory for other reasons, that is fine. However, keeping a service in memory 24x7 has its own set of issues, not the least of which is that users will tend to smack it down using a “task killer” or the Manage Services screen in the Settings app. Doze mode on Android 6.0+ will also cause problems with this approach.
A general rule of thumb is: don’t download it until you really need it.
Sometimes, you do not know if you really need a particular item until
something happens in the UI. Take a ListView displaying thumbnails
of album covers for a music app. Assuming the album covers are not
stored locally, you will need to download them for display. However,
which covers you need varies based upon scrolling. Downloading a
high-resolution album cover that might get tossed in a matter of
milliseconds (after an expensive rescale to fit a thumbnail-sized
space) is a waste of bandwidth.
In this case, either the album covers are something you control on
the server side, or they are not. If they are, you can have the
server prepare thumbnails of the covers, stored at a spot that the
app can know about (e.g., .../cover.jpg it is .../thumbnail.jpg).
The app can then download thumbnails on the fly and only grab the
full-resolution cover if needed (e.g., user clicks on the album to
bring up a detail screen). If you do not control the album covers,
this option might still be available to you if you can run your own
server for the purposes of generating such thumbnails.
You can see a similar effect with the map tiles in Google Maps. When zooming out, the existing map tiles are scaled down, with placeholders (the gridlines) for the remaining spots, until the tiles for those spots are downloaded. When zooming in, the existing map tiles are scaled up with a slight blurring effect, to give the user some immediate feedback while the full set of more-detailed tiles is downloaded. And, if the user pans, you once again get placeholders while the tiles for the newly uncovered areas are downloaded. In this fashion, Google Maps is able to minimize bandwidth consumption by giving users partial results immediately and back-filling in the final results only when needed. This same sort of approach may be useful with your own imagery.
Sometimes, you have no ability to reduce the bandwidth itself. Perhaps you do not control both ends of the communications pipeline. Perhaps the data you are trying to exchange is already compressed (e.g., downloading an MP4 video). Perhaps some of the techniques in the preceding section were unavailable to you (e.g., cannot route data through third-party servers like Google’s for GCM).
There still may be ways for you to help your users, by shaping your bandwidth use. Rather than just blindly doing whatever you want whenever you want, you learn what the user wants and what other applications want and tailor your bandwidth use on the fly to match those needs. The following sections outline some ways of achieving this.
If you are consuming enough bandwidth that this chapter is relevant to you, you probably are consuming enough bandwidth that you should be asking the user how best to consume that bandwidth. After all, they are the one paying the price — in time as well as money – for that consumption.
The following sections present some possible strategies for preference-based bandwidth shaping.
One strategy is for the user to give you a budget (e.g., 20MB/day) and for you to stick within that budget.
Collecting the budget is fairly easy — just use
SharedPreferences. Either use a ListPreference with likely budget
value or an EditTextPreference and a bit of validation for a
free-form budget amount.
Next, you will need to have some idea how much bandwidth any given
network operation will consume. For some things, this might be an
estimate based on your experiments as a developer, or perhaps it is
based on historical averages for this user and type of operation. For
example, a “podcatcher” (feed reader designed to download podcast
episodes) should have some idea how big a given RSS or Atom feed
download should be. In some cases, it might be worthwhile to get a
better estimate — for example, the podcatcher might use an HTTP
HEAD request to determine the size of the MP3 or OGG file before
deciding whether to download it.
Then, you need to be keeping track of your budget. This could be a
simple flat file with the initial TrafficStats bandwidth values for
your process. Re-initialize that file on the first network operation
of the day (or whatever period you chose for your budget). Before
doing another network operation, compare the current TrafficStats
values with the initial ones and see how close you are to the budget.
If the new network operation will exceed the budget, skip the
operation, perhaps putting it in a work queue to perform in the next
budget. You might even hold a reserve for certain types of
operations. For example, the podcatcher might ensure there is at
least 10% of the budget available for downloading the feeds, even if
it means putting a podcast on the queue for download tomorrow. That
way, you can present to the user the latest podcast information, with
icons indicating which are downloaded and which are queued for
download — the user might be able to then request to override
the budget and download something on demand.
For devices that lack per-UID TrafficStats support, you will have
to “fake it” a bit. Use your own calculations of how much bandwidth
each operation consumes and track that information, even if you wind
up missing out on some bytes here or there.
If the user might not care how much bandwidth you consume, so long as
it is un-metered bandwidth, you might include a CheckBoxPreference
to indicate if large network operations should be limited to WiFi and
avoid mobile data.
You could then use ConnectivityManager and getActiveNetworkInfo()
to see what connection you have before performing a network
operation. If it is a background operation (e.g., the podcatcher
checking for new podcasts every hour), if the network is not the
desired one, you can skip the operation or put it on a work queue for
re-trying later. If it is a foreground operation (e.g., the user
clicked a “refresh” menu choice), you could pop up a confirmation
AlertDialog to warn the user that they are on mobile data —
perhaps this time they are interested in doing the operation anyway.
Another approach for handling the background operations is to
register a BroadcastReceiver for the CONNECTIVITY_ACTION
broadcast (defined on ConnectivityManager). If the connectivity
switches to mobile data, cancel your outstanding AlarmManager
alarms; if connectivity switches to WiFi, re-enable those alarms.
Of course, you should also consider monitoring the background data
setting — the global Settings checkbox indicating whether
background network operations are allowed. On ConnectivityManager,
getBackgroundDataSetting() tells you the state of this checkbox,
and ACTION_BACKGROUND_DATA_SETTING_CHANGED allows you to set up a
BroadcastReceiver to watch for changes in its state.
If your user is less concerned about the bandwidth or the network,
but does care about the time of day (e.g., does not want your
application consuming significant bandwidth when they might be
getting a VOIP call), you could offer preferences for that as well.
Cook up a TimePreference and use that
to collect start and stop times for the high-bandwidth window. Then,
set up alarms with AlarmManager for those points in time. The alarm
for the start time of the window sets up a third alarm with your
regular polling interval. The alarm for the stop time of the window
cancels the polling interval alarm.
If your network I/O is part of a foreground application, one presumes that you are the most important thing in the user’s life right now. Or, at least, the most important thing on the user’s phone right now. Hence, what other applications might want to do with the Internet connection is not a major concern.
If, however, your network I/O is part of a background operation, it might be nice to try to avoid doing things that might upset the user. If the user is watching streaming video or is on a VOIP call or otherwise is aware of bandwidth changes, the bandwidth you use might impact the user in ways that the user will not appreciate very much. This is unlikely to be a big problem for small operations (e.g., downloading a 1KB JSON file), but larger operations (e.g., downloading a 5MB podcast) might be more noticeable.
You can use TrafficStats to help here. Before doing the actual
network I/O, grab the current traffic data, wait a couple of seconds,
and compare the latest to the previous values. If little to no
bandwidth was consumed during that period, assume it is safe and go
ahead and do your work. If, however, a bunch of bandwidth was
consumed, you might want to consider:
AlarmManager to give you
control again in a minute, with the current traffic data packaged as
an extra on the Intent, so you can make a decision after a bigger
sample size of bandwidth consumption, orYou could try to get more sophisticated, by using ActivityManager
and the per-UID values from TrafficStats to see if it is a
foreground application that is the one consuming the bandwidth. It is
unclear how reliable this will be, both in determining who is
consuming the bandwidth (again, per-UID traffic is not available on
many devices) and in avoid user angst. It may be simpler just to
assume the worst and side-step your I/O until the other apps have
quieted down.
Android 4.1 added isActiveNetworkMetered() as a method
on ConnectivityManager. In principle, this will return true if Android
thinks that the current data connection may involve bandwidth charges. You can
examine this value and steer your bandwidth consumption accordingly.
Android 5.0 added JobScheduler, as an alternative to AlarmManager
for arranging periodic work. One feature of JobScheduler is that you
can indicate that certain jobs require Internet access, in which case
Android will not bother giving you control unless such access is
available. A further refinement is that you can state that a job
requires an unmetered Internet connection, so you avoid doing bandwidth-hogging
work on an expensive connection.
Android has had a per-app “data saver” mode for some time, with an eye towards reducing bandwidth consumption when the device is using a known metered data plan. Android 7.0 extends this to a device-wide setting,
Apps can be in one of three states as a result:
The idea is that if the device is in normal mode, you can do what you want. If the device is in data-saver mode, you should restrict your bandwidth, even if the user whitelists you. Apps that are not whitelisted have no network access while in the background.
To that end, ConnectivityManager has three things for you.
First, isActiveNetworkMetered() will return true if the device
is on a metered data connection, false otherwise. This has been
around for years (API Level 16+), but has not been all that popular,
apparently.
Second, Android 7.0 has a getRestrictBackgroundStatus() method on
ConnectivityManager. This returns an int that resolves to one of
three values:
RESTRICT_BACKGROUND_STATUS_DISABLEDRESTRICT_BACKGROUND_STATUS_ENABLEDRESTRICT_BACKGROUND_STATUS_WHITELISTEDIf isActiveNetworkMetered() is true, and getRestrictBackgroundStatus()
returns RESTRICT_BACKGROUND_STATUS_ENABLED, any attempts to use the
network may fail, and so your app should plan accordingly.
If you want to try to react in real-time to changes in the data-saver
configuration, you can register a receiver for ACTION_RESTRICT_BACKGROUND_CHANGED
(defined on ConnectivityManager). This will be broadcast for any change
in data-saver settings, which means that your app’s state may not have
changed. You will need to call getRestrictBackgroundStatus() to find
out your current state. Also note that this broadcast is only sent to
receivers registered dynamically, via registerReceiver(). You cannot
register for this broadcast in the manifest.
To try to get on the whitelist, you might be tempted to try using
ACTION_IGNORE_BACKGROUND_DATA_RESTRICTIONS_SETTINGS to lead the
user to add your app to the Data Saver whitelist, so you have normal
background network access. However, bear in mind that Google has
a similar feature for the battery saver whitelist… and trying
to use that action
got apps banned from the Play Store.
At the moment, there is no similar language around the use of the
data saver whitelist… but, then again, they did not tell you they were
going to ban you for asking to be on the battery saver whitelist until
after Android 6.0 shipped.