Today, you have two widespread and well-supported mechanisms for storing data on the client: cookies and Web Storage. Many say that Web Storage is the evolution of cookies, but in reality, cookies may stick around a lot longer than people think. Mainly because they are much different than Web Storage and implicitly send data back to the server upon each request through HTTP headers. Cookies are always available on the server and can be read and written to freely, which is great for user session management and similar situations. The downside is that you only get 4kb of storage per cookie.
Web Storage is different from cookies in that the stored data is not shared with the server. You can currently store 5MB of data on the client device with Web Storage, and some browsers allow up to 10MB of storage with user approval. However, these limits can be a little misleading. If you try to store 5MB of pure HTML in local storage within WebKit-based browsers, such as Mobile Safari, you will see that it allows for a maximum of 2.6MB only. For this, you can thank section 7.4 of the first W3C Working Draft of the Web Storage specification, which states:
In general, it is recommended that
userAgents not support features that control how databases are stored on disk. For example, there is little reason to allow Web authors to control the character encoding used in the disk representation of the data, as all data in JavaScript is implicitly UTF-16.
Although section 7.4 was removed in following drafts, most WebKit browser vendors have stuck to the UTF-16 implementation of data encoding. Two exceptions, Firefox and IE, give you the actual 5MB of storage space.
Web Storage offers two storage objects—localStorage and sessionStorage—both of which are widely
supported from IE8 upward and in all modern browsers, including mobile devices. (For
browsers that don’t support Web Storage natively, it includes several polyfills.) Both
storage objects use exactly the same APIs, which means that anything you can do with
localStorage, you can also do with sessionStorage and vice versa. With sessionStorage, however, your data is stored only while that particular
browser window (or tab) is open. After the user closes the window, the data is purged. With
localStorage, your data will stay on the client
across browser sessions, device restart, and more. Any data stored is tied to the document
origin, in that it’s tied to the specific protocol like HTTP or HTTPS, the top-level domain
(for example html5e.org), and the port, usually port
80. One more caveat regarding sessionStorage: if the
browser supports resuming sessions after restart, then your sessionStorage object may be persisted unknowingly. This can be an issue if
your use case expects the sessionStorage data to be
destroyed upon closing of the browser.
Web Storage defines two APIs, Storage and StorageEvent, which either local or session storage can use. This means you have two ways of working with and managing local data. Whichever API you choose, remember with Web Storage, operations are synchronous: When you store or retrieve data, you are blocking the main UI thread, and the rest of the page won’t render until your data operations are finished.
As this usage example illustrates, the Storage API offers multiple ways of working with your data in a storage object:
localStorage.bookName='HTML5 Architecture';//orlocalStorage['bookName']='HTML5 Architecture';//orlocalStorage.setItem('bookName')='HTML5 Architecture';
You can treat your data like any other JavaScript object and add the key directly as a localStorage property, which calls setItem() behind the scenes. Your available functions are:
.length//returns the number of key/value pairs.key(n)//returns the name of the nth key in the list.getItem(key)//returns the current value associated with the key.setItem(key,value)//creates new or adds to existing key.removeItem(key)//you can probably guess what this does :).clear()//removes everything from storage associated with this domain
You might think that all of the methods for storing data have the same performance, but that would be crazy talk in web browser land, right? Figure 6-1 and Figure 6-2 provide a more realistic view and a performance analysis (http://jsperf.com/localstorage-getitem-setitem-vs-getter-setter/4) on which storage approach works the best.
Instead of using localStorage.setItem() and calling the object setter property directly, with localStorage.small or localStorage['small'], you can give your data storage a 50% speed boost in Chrome (Figure 6-1). The same performance test in the latest Firefox and Safari web browsers, however, reveals that localStorage.setItem() performs better than the others for small values (Figure 6-2).
As for most applications, you want your web app to perform at top speed across all browsers. Usually, a real-world application will store a larger JSON object, base64 image string, or HTML snippet in localStorage. As you can see with the largeValue tests in the figures, all Storage API options perform roughly the same.
Although Web Storage is considered to be “racy” (more on this in a moment), you can avoid most race conditions by using the StorageEvent API. If the user has the same site open in different tabs, this event can be used to synchronize the data. Here’s a sample usage:
window.addEventListener('storage',function(evt){alert(evt.key)},false);
.key//returns the value of the key being changed. It's null upon creation..oldValue//represents the old value of the key being changed.newValue//represents the new value of the key being changed.url//the address of the document whose key changed.storageArea//the Storage object which was affected. Either localStorage or sessionStorage
The storage event is fired only when the new value is not
the same as the old value. The storage event contains
the key, oldValue,
and newValue properties of data that has changed,
which you can access in code. This example creates the appropriate event listener, which
logs the oldValue and newValue across all open browser sessions:
window.addEventListener('storage',function(event){console.log('The value for '+event.key+' was changed from'+event.oldValue+' to '+event.newValue);},false);
Note
The storage event fires on the other windows only. It won’t fire on the window that did the storing.
Ultimately, a downside of Web Storage is the lack of transactions. For example, a user might have your app open in two tabs. Tab 1 starts writing several things to the database, then tab 2 starts reading it, getting the data when it has only partially been updated. Thus, Web Storage is racy, meaning it’s susceptible to race conditions. As a result, you need to take precautions to ensure the integrity of your data and the accuracy of any queries. As mentioned earlier, the only mechanism to prevent race conditions is the StorageEvent.
Race conditions can occur with multithreaded browsers, as well, because threads can cause problems when saving data. Here’s a good example:
varmyarray=[a,b];varfirst=myarray.splice(0,1);localStorage.first=first;
You would expect the following:
localStorage.first==a;//true
When a race condition occurs, we could find that this happens:
localStorage.first==b;//true
As one thread splices myarray and is de-scheduled, another
thread runs the same code segment and effectively reads myarray as only having one element, b, and as such, assigns it to first.
Bottom line, the exact same problem exists with cookies, which doesn’t seem to have bothered people much. If race conditions are a problem (or you think they’re going to be a problem), you need a more advanced storage mechanism than Web Storage, such as IndexedDB or a solution that can support transactions and write-locks.
To store a JavaScript object (or an array perhaps) in your localStorage or sessionStorage, you need to use JSON to encode and decode your data, as in:
varmydata={"Book":"HTML5 Architecture","Author":"Wesley Hales",};
Next, store the JavaScript object as a string:
localStorage.setItem("mydata",JSON.stringify(mydata));
When you’re ready to retrieve the data, use:
JSON.parse(localStorage.getItem("mydata"));
All this communication between client and server raises security issues. They come in two flavors: keeping your app secure and private browsing by users.
Because of the potential for DNS spoofing attacks, you cannot guarantee that a host claiming to be in a certain domain really is from that domain. To mitigate this and keep your app secure, you can use TLS (Transport Layer Security) for your pages. TLS and its predecessor, Secure Sockets Layer (SSL), are cryptographic protocols that provide communication security over the Internet. Pages using TLS can be sure that only the user, software working on behalf of the user, and other pages using TLS that have certificates identifying them as being from the same domain, can access their storage areas.
Web Storage, both localStorage and sessionStorage, is not secure and is stored in plain text with no way
to encrypt. If you’re worried about data security, don’t use localStorage. There are solutions like JCryption
(http://www.jcryption.org) for those unwilling to buy SSL
certificates or with hosting providers who do not support SSL. It’s no replacement
for SSL, because there is no authentication, but the jCryption plug-in offers a base
level of security while being very easy and quick to install.
Warning
Be aware that any form of JavaScript encryption is intrinsically vulnerable to man-in-the-middle (MITM) attacks, so it is not a recommended practice for storing sensitive data.
Within certain browsers, while the user is running in private or incognito browsing modes, your application will get an exception when trying to store anything in Web Storage. Every app that uses localStorage should check window['localStorage'].setItem for a rising QUOTA_EXCEEDED_ERR exception before using it. For example, the problem in Figure 6-3 is that the window object still exposes localStorage in the global namespace, but when you call setItem, this exception is thrown. Any calls to .removeItem are ignored.
Safari returns null for any item that is set within the localStorage or sessionStorage objects. So even if you set something before the user goes into private browsing mode, you won’t be able to retrieve until they come out of the private session.
Chrome and Opera will allow you to retrieve items set before going into
incognito mode, but once private browsing commences, localStorage is treated like sessionStorage (only items set on the localStorage by that session will be returned).
Firefox, like Chrome, will not retrieve items set on localStorage prior to a private session starting, but in private
browsing treats localStorage like sessionStorage.
To be safe, always do a series of checks before using localStorage or sessionStorage:
functionisLocalStorageSupported(){try{varsupported=('localStorage'inwindow&&window['localStorage']!==null);if(supported){localStorage.setItem("storage","");localStorage.removeItem("storage");}returnsupported;}catch(err){returnfalse;}}
Take a look at Table 6-1, an overview of the five most visited sites on the Internet, to get an idea of which sites are (or aren’t) using Web Storage to optimize.
Table 6-1. Web Storage Survey
Website | Desktop | Mobile |
|---|---|---|
Google Search | Yes, 87K | Yes, 160K |
Yahoo! | No | No |
Wikipedia | No | No |
Yes, less than 1K | Yes, 46K | |
Amazon | Yes, less than 1K | No |
For mobile, Google’s basic search page is making the most use of localStorage by storing base64 images and other CSS. For each subsequent page request, it uses JavaScript to insert <style> blocks just after the page title in the document head with the CSS values from localStorage (Figure 6-4).
For a basic Google search on the desktop, data is stored differently than on mobile. First, sessionStorage is used, so you know this will be temporary data. Looking at the raw JSON data stored by a simple Google search in Figure 6-5, you can see mostly CSS and HTML is stored along with some namespaced tracking data.
Twitter also makes heavy use of localStorage on mobile devices. Looking at the JSON saved on the device in Figure 6-6, you can see that Twitter stores all of the data required to present the user interface. The data isn’t a straight dump of HTML to localStorage, however, it’s organized in a JSON object structure with proper escaping and JavaScript templating variables.
Amazon’s use of sessionStorage is minimal tracking information related to “product likes.” But overall, it’s a bit surprising to see that the top sites on the Internet are still not leveraging Web Storage to speed up their site and reduce HTTP requests.
Efficient requests and zippier interfaces may not be a huge problem for desktop sites, but there’s no reason we shouldn’t have these storage enhancements on both mobile and desktop. Some of the reasons we’re seeing heavy Web Storage usage only on mobile are:
Data URIs (base64 encoding background images) used in CSS work with modern browsers only. There are limits and annoyances with this technique all the way up through IE9.
Mobile latencies are much higher (as you saw in Chapter 3), so caching on mobile devices can make the UI much faster.
Note
When using base64-encoded data URIs, be aware that the encoded data is one third larger in size than its binary equivalent. (However, this overhead is reduced to 2 to 3% if the HTTP server compresses the response using GZIP.)
As you have seen, there are a million different ways to use Web Storage within your application. It really comes down to answering a few questions:
How can I make the user experience better?
How can I reduce HTTP requests on mobile?
How can I efficiently reduce load on the server?
Of course, after seeing that Web Storage blocks the main JavaScript UI thread when accessing data, you must be considerate of how your page loads and use best practices for storing and retrieving data.
The best place to start with localStorage is using it where
your app requires user input. For example, if you have a comments input box within a
form, you could use localStorage to save a draft
on the user input in case the session times out or the form is submitted improperly.
The commenting service Disqus follows this practice and saves your draft comments in localStorage.
Another good use of localStorage is for automatic sign-in. Here is a recommendation from section 3.3.2 in the W3C Mobile Web Application Best Practices:
3.3.2.1 What it means
If an application requires user identity it is usual to prompt for user credentials (username and password) and provide the option to sign-in automatically on next usage session. This is especially important on a mobile device where data input is more difficult than on a desktop.
Note that if automatic sign-in is enabled, a sign-out link should also be provided.
3.3.2.2 How to do it
User credentials can be stored in a cookie or in local storage. However, it is important not to store unencrypted password information since this is insecure. Typically, a securely hashed token which, if necessary, can be revoked on the server, is stored locally in order to enable automatic sign-in.
Most web services allow you to hit their service a limited number of
times per day. By using localStorage with a
timestamp, you can cache results of web services locally and access them only
after a specified time to refresh the data.
A simple library that allows for this memcache-like behavior is lscache (https://github.com/pamelafox/lscache).
lscache emulates memcache functions using HTML5 localStorage, so that you can cache data on the client and
associate an expiration time with each piece of data. If the localStorage limit (about 5MB) is exceeded, it
tries to create space by removing the items that are closest to expiring anyway.
If localStorage is not available at all in
the browser, the library degrades by simply not caching, and all cache requests
return null.
All of the ways for Web Storage to speed up your web application discussed so far, are forms of one-way communication for which syncing, or transmitting modified JSON objects, back to the server is not required. Instead, you simply push data to the browser and use it as a cache.
Today, companies are just starting to leverage Web Storage to store and sync the object model back to the server-side database. Such functionality is useful to:
Allow a web app to function offline, then sync new client data to server
Allow a web app to function offline, then refresh client data on reconnect
Allow an offline web app and online server data to be changed, and then sync both datasets while handling conflicts
Some of these data management and versioning situations can get fairly complex. For example, LinkedIn recently posted its solution to managing RESTful JSON data with localStorage. The company’s main reasoning for bringing localStorage into the picture was to reduce latency and unneeded network requests on its latest iPad app. According to LinkedIn engineer Akhilesh Gupta:
LinkedIn just released a brand new iPad app built using HTML5, backbone.js, and underscore.js. The app includes a rich stream of information, which is a combination of network updates, group posts, and news articles. The user can also switch to specific streams like Co-Worker updates or news articles from a specific category.
For the full article, see http://engineering.linkedin.com/mobile/linkedin-ipad-using-local-storage-snappy-mobile-apps.
At its core, this particular application uses Backbone to manage client-side data models. The developers then wrote the necessary code to override the basic sync functionality to allow models and collections to be stored in localStorage. Again, this is clearly a performance move and doesn’t really address syncing data back to the server. But, it is a more complex use case that manages versioning and migration of the data to newer versions of the app. In the end, the iPad application gained the following performance improvements:
A more responsive application thanks to temporarily storing recently fetched data; users no longer have to wait for network requests to finish before moving around the application
Seamless sharing of fetched data among multiple web views in the native application
Independence from memory constraints in mobile devices;
localStoragecan store and populate temporary objects in memory when necessaryDecreased memory footprint and rendering time while scrolling because complicated HTML document fragments are stored in
localStorage
A few frameworks allow for data to be synced from localStorage back to the server. For example, Backbone.js comes with methods for fetching and saving data models to and from the server. Out of the box, however, it does not provide the advanced functionality required by an application that needs to work offline and synchronize with the server when online. To address this, Neil Bevis of the Dev Camp blog posted an excellent solution that I’ll summarize here. (For the complete blog post, see http://occdevcamp.wordpress.com/2011/10/15/backbone-local-storage-and-server-synchronization.)
Backbone-localstorage.js provides communication with localStorage by simply adding the JavaScript file to the project. By adding this file, however, you then cannot communicate between Backbone and the server with Backbone.sync. The first thing you must do is create a copy of the Backbone.sync method before it’s replaced by the inclusion of the backbone-localstorage.js JavaScript file:
<scriptsrc="backbone.js"></script><script>Backbone.serverSync=Backbone.sync;</script><scriptsrc="backbone-localstorage.js"></script>
Now, you’ll be able to save data to the server using:
Backbone.serverSync('update',model,options);
This gives the standard model.fetch() and model.save() functions the ability to use localStorage. Next, you must provide a synchronized flag with a Boolean value describing its
client-side status. When the client is ready to push local changes to the server
from a given collection, it sends model objects with synchronized=false on a model-by-model basis using:
Backbone.serverSync('update',model,{success:'foo',error:'bar'}).
If the server responds with a different ID than what is stored on the client, then that means you have a new object. If the IDs remain the same, however, then you simply have an update. When a new object comes from the server, the following code deletes the existing ID in localStorage and adds the new version:
for(vari=0;i<models.length;i++){varmodel=models[i];if(model.get('synchronized')){continue;}model.change();Backbone.serverSync('update',model,{success:function(data){varmodel=collection.get(data.ClientId);//if new server will return a different Idif(data.ServerId!=data.ClientId){//delete from localStorage with current IdBackbone.sync("delete",model,{success:function(){},error:function(){}});//save model back into localStoragemodel.save({Id:data.ServerId})}model.save({synchronized:true});collection.localCacheActive=false;},error:function(jqTHX,textStatus,errorThrown){console.log('Model upload failure:'+textStatus);collection.localCacheActive=false;}});}
When asked to pull server-side changes to a collection from the server, the client first uses
model.save() to save any unpushed client-side
changes into localStorage. It next requests the
entire collection from the server via the standard Backbone fetch
method:
tempCollection.sync=Backbone.serverSync;tempCollection.fetch({success:blah,error:blah});
In practice, you could reduce the associated data download to only items that require
updating. As it receives each model back from the server, the success function checks each one against its own list.
If the model is new, success adds it to the
collection that is updating and also uses model.save() to record it into local storage:
collection.add(tempModel);tempModel.change();tempModel.save({synchronized:true});
Finally, the success function updates the model with
revised data after the model has been synchronized:
model.set(tempModel.toJSON());model.set({synchronized:true});model.save();
The big issue with this approach is if the model already exists and the user has made localStorage-based modifications to it. In this code,
those models are not updated during the pull of server-side changes. Those objects
are pushed to the server to be updated in the database.
This is not an end-all solution, and there are many frameworks currently trying to address
this problem. Many of the solutions are just as mature as the one reviewed here. So
your use of localStorage and syncing to a
server-side database will be dictated by the complexity of your use case.
Although you can use localStorage safely within most modern web browsers, if your application must accommodate browsers without localStorage, you can use some easy-to-follow, lightweight polyfills. For example, the following example polyfill accommodates IE 6 and 7, as well as Firefox 2 and 3. With the exact same API as defined in the Web Storage spec, you can start using it today with roughly 90 lines of JavaScript included in your application. (For the full source, see https://raw.github.com/wojodesign/local-storage-js/master/storage.js.)
(function(){varwindow=this;// check to see if we have localStorage or notif(!window.localStorage){// globalStorage// non-standard: Firefox 2+// https://developer.mozilla.org/en/dom/storage#globalStorageif(window.globalStorage){// try/catch for file protocol in Firefoxtry{window.localStorage=window.globalStorage;}catch(e){}return;}// userData// non-standard: IE 5+// http://msdn.microsoft.com/en-us/library/ms531424(v=vs.85).aspxvardiv=document.createElement("div"),attrKey="localStorage";div.style.display="none";document.getElementsByTagName("head")[0].appendChild(div);if(div.addBehavior){div.addBehavior("#default#userdata");varlocalStorage=window["localStorage"]={"length":0,"setItem":function(key,value){div.load(attrKey);key=cleanKey(key);if(!div.getAttribute(key)){this.length++;}div.setAttribute(key,value);div.save(attrKey);},"getItem":function(key){div.load(attrKey);key=cleanKey(key);returndiv.getAttribute(key);},"removeItem":function(key){div.load(attrKey);key=cleanKey(key);div.removeAttribute(key);div.save(attrKey);this.length--;if(this.length<0){this.length=0;}},"clear":function(){div.load(attrKey);vari=0;while(attr=div.XMLDocument.documentElement.attributes[i++]){div.removeAttribute(attr.name);}div.save(attrKey);this.length=0;},"key":function(key){div.load(attrKey);returndiv.XMLDocument.documentElement.attributes[key];}},// convert invalid characters to dashes// http://www.w3.org/TR/REC-xml/#NT-Name// simplified to assume the starting character is validcleanKey=function(key){returnkey.replace(/[^-._0-9A-Za-z\xb7\xc0-\xd6\xd8-\xf6\xf8-\u037d\u37f-\u1fff\u200c-\u200d\u203f\u2040\u2070-\u218f]/g,"-");};div.load(attrKey);localStorage["length"]=div.XMLDocument.documentElement.attributes.length;}}})();
A few JavaScript frameworks address Web Storage needs on mobile devices. When evaluating Web Storage frameworks, look for a nice consistent storage API that works across all devices. Of course, this is what the spec itself does through a simple JavaScript API, but until all devices support this specification, you need a helper framework.
LawnChair (http://westcoastlogic.com/lawnchair) is
designed with mobile in mind. Supporting all major mobile browsers, it’s adaptive to
the mobile and desktop environments described in this book and gives you a
consistent API for accessing some form of localStorage. LawnChair allows you to store and query data on
browsers without worrying about the underlying API. It’s also agnostic to any
server-side implementations, enabling you to get started quickly with a simple,
lightweight framework.
The page setup is:
<!DOCTYPE html><html><head><title>my app</theitle></head><body><scriptsrc="lawnchair.js"></script></body></html>
To persist data, use:
Lawnchair(function(){this.save({msg:'hooray!'})})
Supporting all major mobile browser platforms, persistence.js (http://persistencejs.org) is an asynchronous JavaScript object-relational mapper. It integrates with Node.js and server-side MySQL databases and is recommended for server-side use, because using in-memory data storage seems to slow down filtering and sorting. The download size is much heavier than that of LawnChair.
For page setup, use:
<!DOCTYPE html><html><head><title>my app</title></head><body><scriptsrc="persistence.js"type="application/javascript"></script><scriptsrc="persistence.store.sql.js"type="application/javascript"></script><scriptsrc="persistence.store.websql.js"type="application/javascript"></script></body></html>
if(window.openDatabase){persistence.store.websql.config(persistence,"jquerymobile",'database',5*1024*1024);}else{persistence.store.memory.config(persistence);}persistence.define('Order',{shipping:"TEXT"});persistence.schemaSync();
Similar to Hibernate (JBoss’s persistence framework), persistence.js uses a tracking mechanism to determine which objects
changes have to be persisted to the database. All objects retrieved from the
database are automatically tracked for changes. New entities can be tracked and
persisted using the persistence.add
function:
varc=newCategory({name:"Main category"});persistence.add(c);
All changes made to tracked objects can be flushed to the database by using persistence.flush, which takes a transaction object and callback function as arguments. You can start a new transaction using persistence.transaction:
persistence.transaction(function(tx){persistence.flush(tx,function(){alert('Done flushing!');});});