JSON for JavaScript Clients

JavaScript clients, often written in a dialect such as jQuery, are now a major force in RESTful web services. Gone are the bad old days when a browser downloaded a page generated on the server and did no further client-side processing. A modern website typically integrates client-side and server-side processing to serve the overall goal of high performance combined with professional look and feel, and JavaScript—in a sense broad enough to encompass languages/frameworks such as Dojo, Enyo, Meteor, midori, jQuery, and SmartClient—is the dominant language for client-side processing.

At one time, the data displayed on a HTML page typically came from a database through a server-side application down to a browser. Web services are now a prevalent source of data, a source accessible directly from a client-side script embedded in an HTML page. Data flows into an HTML page need not originate, at least not immediately, in a server-side application connected to a database. These changes further blur the distinction between websites and web services because the HTML pages that remain a key part of any website embed scripts that can act as web service clients. Pioneering JavaScript frameworks such as Meteor aim at virtually collapsing the difference between client-side and server-side functionality by allowing client-side scripts direct access to a server-side database. These changes and trends together open new possibilities for web services and their clients; these changes likewise point to an even greater role for JavaScript processing and JSON representations in web services.

JSONP and Web Services

Contemporary web services regularly support not only JSON payloads but also JSONP, where the P stands for with padding. JSONP originally signified a way to work around the traditional same domain policy that prevents a page downloaded from a domain such as server.foo.org from communicating with a domain other than foo.org; JSONP still serves this purpose. The JSONP workaround involves script elements in a page, as these elements allow code to be downloaded from anywhere; the downloaded code can then perform arbitrary processing, which includes communicating with servers in arbitrary domains. JSONP works nicely with web services.

JSONP brings an event-driven API to client-side processing (Example 3-19). Using JSONP, the programmer can do the following:

  • Provide a URL to a data source.
  • Specify a callback function to be executed, in browser context, once the data from the specified source arrives.

Example 3-19. A JSONP call against the RESTful Twitter search

<!DOCTYPE html>
<html>
<head>
    <title>Twittering</title>
    <script type = "text/javascript"
            src = "http://code.jquery.com/jquery-latest.min.js">
    </script>
    <script type = 'text/javascript'>
       function cBack(data) { alert(JSON.stringify(data)); } 1
       (function($) {
          var url = "http://search.twitter.com/search.json?q=skiing&rpp=4";
          $.ajax({
             type: 'GET',
             url: url,
             async: false,
             jsonpCallback: 'cBack',                         2
             contentType: "application/json",
             dataType: 'jsonp',                              3
             success: function(json) { console.dir(json.sites); },
             error: function(e) { console.log(e.message); }
          });
       })(jQuery);
    </script>
</head>
<body></body>
</html>

The HTML document in Example 3-19 embeds jQuery code, which illustrates a JSONP call against Twitter’s RESTful search service. The three code lines of interest are numbered. The GET request to Twitter has jsonp as its dataType (line 3), and the jsonpCallback is a function named cBack (lines 1 and 2) that takes one argument: the data returned from the Twitter search. The Twitter response can be depicted in text as follows:

cBack({...})

This is the invocation syntax for a JavaScript function that takes one argument: a JavaScript object. (It should be noted that a JavaScript function is likewise a JavaScript object, as JavaScript treats functions as first-class objects.) The curly braces, { and }, mark the start and the end of the JavaScript argument to the function. This argument is a mix of metadata about the search and a list (that is, a JavaScript array) of information about skiing images. Here is a small slice of the argument, with personal data obscured with ellipses:

{"completed_in":0.009, "max_id":308446633115385860,
 ...
 "query":"skiing","refresh_url":"?
 ...
 [{"created_at": "Mon, 04 Mar 2013 05:19:42 +0000",
   "from_user": "...", "from_user_id": ...,
   "from_user_id_str": "...", "from_user_name" : "... ",
   "geo":null,
   ...
}

The JavaScript callback function cBack, upon receipt of the Twitter data, pops up an alert window that shows the text representation of the downloaded JSON object.

The JSONP exchange between the client-side script and the Twitter search service can be summarized as follows. The script, written in the jQuery dialect of JavaScript, makes a JSONP call that specifies a callback function; the Twitter service accommodates by returning, as the HTTP response, the name of the callback function together with the requested data as the argument to the function. The browser executes the function call, which displays the returned JSON in a pop-up window. This small example thus illustrates the event-driven character of a JSONP request.

With respect to RESTful web services, JSON is an ideal format for JavaScript clients because JSON is the text representation of a native JavaScript object. With two additional examples, this section covers the basics and sketches some possibilities. The first example involves a composed web service—a service that includes another service as a component. This example also illustrates a common task in web services—reformatting data, in this case reformatting XML as JSON. The second example focuses on Ajax calls: jQuery embedded in an HTML page regularly polls a server so that the partially updated page reflects the current state of the resource.

A Composed RESTful Service with jQuery

The first jQuery example, named cds, is a composed RESTful service. Here is an overview:

The composed cds service

Figure 3-2. The composed cds service

Perhaps the best way to clarify the architecture is to start with the jQuery client, which expects the data to come in JSON format. The catch is that the W3C service provides the data in XML format. The intermediary cds service takes on the job of format converter (Example 3-20).

Example 3-20. The jQuery RESTful client embedded in an HTML page

<!DOCTYPE html>
<html>
  <head>
    <title>JSON</title>
    <script type = "text/javascript"
            src = "http://code.jquery.com/jquery-latest.min.js">
    </script>
    <script type = "text/javascript">
      $.getJSON('http://localhost:8080/cds/getCDs.jsp', function(response) {
         var cds = response.catalog.cd;
         alert(JSON.stringify(cds));
         $.each(cds, function(ind, val) {                                      1
             $('#container').append('<li>' + cds[ind].title + ': ' +           2
                                             cds[ind].artist + '</li>'); } );  3
      });
    </script>
  </head>
  <body>
    <ul id = 'container'></ul>
  </body>
</html>

The HTML page with the embedded jQuery in Example 3-20 is short, and the embedded jQuery is likewise terse compared to the earlier Twitter sample (see Example 3-19). The current example shows a jQuery shortcut method, in this example getJSON, that can be used in place of the generic jQuery ajax method. Of particular interest here, however, is that the jQuery code does not parse the JSON document returned from the RESTful service but, rather, treats the JSON document as a native JavaScript array. In the body of the each iteration (line 1), for example, the jQuery extracts the title and the artist using regular JavaScript syntax:

cds[ind].title  /* line 2: ind is the index into the array */
cds[ind].artist /* line 3 */

No parsing is required.

In the composed cds service, the W3C service is the ultimate data source. The other service, which consists of the very short JSP script getCDs.jsp (see Example 3-21) and the backend POJO class FetchXML (see Example 3-22), is the fetch-and-convert module: a jQuery client hits the JSP script, which returns JSON from the FetchXML instance method getJson, and the JSON is the result of transforming XML from the W3C service into JSON.

Example 3-21. The getCDs.jsp script in the cds service

<jsp:useBean id    = "fetcher"
             type  = "cds.FetchXML"
             class = "cds.FetchXML">
</jsp:useBean>
<jsp:getProperty name = "fetcher" property = "json" /> 1

When a client request hits the getCDs.jsp script (technically, the servlet instance into which this JSP script is ultimately transformed), the client gets in response the value returned from the FetchXML method getJson, named in the JSP script as the property json (line 1).

Example 3-22. The backend POJO class FetchXML and its getJson method

package cds;

import org.json.JSONObject;
import org.json.XML;
import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.io.InputStream;
import java.net.URL;
import java.net.URLConnection;

public class FetchXML {
    public void setJson(String json) { }
    public String getJson() {
        JSONObject json = null;
        try {
            // Fetch the XML document from the W3C site.
            String xml = "";
            URL url = new URL("http://www.w3schools.com/xml/cd_catalog.xml");     1
            URLConnection conn  = url.openConnection();                           2
            BufferedReader in =
                new BufferedReader(new InputStreamReader(conn.getInputStream()));
            // Read the document records.
            String line = null;
            while ((line = in.readLine()) != null) xml += line;                   3
            in.close();
            xml = xml.replace("'", ""); // Clean up the XML.                      4
            // Transform the XML document into a JSON object.
            json = XML.toJSONObject(xml.toLowerCase());                           5
        }
        catch(Exception e) { throw new RuntimeException(e); }
        return json.toString(); // JSON document                                  6
    }
}

The FetchXML method getJson uses a URLConnection (line 2) to communicate with the W3C service at the designated URL (line 1). The W3C service returns the CD list as an XML document, which the FetchXML instance reads line by line (line 3). For readability, the XML document is converted to lowercase and otherwise cleaned up (lines 4 and 5) before being converted into a JSON document (line 6), which is returned to the jQuery script. A JSON library deployed in the WAR file converts the XML to JSON, and the jQuery script, which invokes the composed RESTful service to begin, benefits from receiving JSON rather than XML. If the jQuery script dealt directly with the W3C service, then this script would take on the task of either parsing the returned XML document or transforming the XML into JSON.

An Ajax Polling Example

The JAX-RS predictions3 web service in Chapter 2 supports all of the CRUD operations, which means that the resource can change state over time. How should clients against this RESTful service keep abreast of the changes? Two general ways are available:

The example in this section illustrates the second approach with a jQuery client that continually polls the predictions3 service with GET requests for a JSON response. The predictions3 service is unchanged from Chapter 2.

The HTML page poll.html (see Example 3-23) embeds a jQuery script that continually polls the predictions3 service. The workhorse function is fetch (line 1), which makes an Ajax call against the RESTful predictions3 service. The URL ends with /json, which signals that the response should be JSON rather than XML or plain text. As the key named method and its value indicate (line 2), the Ajax call is essentially a GET request. This example thus goes back to the generic ajax call rather than a shortcut call such as getJSON.

Example 3-23. The poll.html page with jQuery to poll the predictions3 service

<!DOCTYPE html>
<html>
  <head>
    <title>Ajax polling example</title>
    <script type = "text/javascript"
            src  = "http://code.jquery.com/jquery-latest.min.js">
    </script>
    <script type = "text/javascript">
      var url = 'http://localhost:8080/predictions3/resourcesP/json';
      function displayPreds(preds) {
         $('#container').empty(); // clear the old list, if any
         $.each(preds, function(ind, val) {
              $('#container').append('<li>' + preds[ind].who + ': ' +
                                              preds[ind].what + '</li>'); } );
      }
      function fetch() {
          $.ajax({                                                                1
                   url:         url,
                   method:      'GET',                                            2
                   dataType:    'json',
                   contentType: "application/json; charset=utf-8",
                   cache:       false,
                   success:     function(res) { displayPreds(res.predictions); },
                   error:       function(res) { console.log(res); },
                   complete:    function () {                                     3
                                   setTimeout(function() { fetch() }, 5000)}});   4
      }
      $(document).ready(fetch); // invoked after DOM is built and loaded          5
    </script>
  </head>
  <body>
    <ul id = 'container'></ul>
  </body>
</html>

The Ajax request from fetch has two possible outcomes:

In either case, the complete key (line 3) has as its value a function (lines 4 and 5) that invokes the jQuery setTimeout function, and the setTimeout function pauses for five seconds before invoking fetch yet again. The complete function executes after either the success or the error function has executed. The jQuery script thus polls the predictions3 service repeatedly, with brief pauses in between each polling operation; the poll.html page displays, to within five seconds or so, the current state of the predictions3 resource.

The deployed poll.html page (see the sidebar) can be tested as follows:

Why Deploy the poll.html Page as a One-Page Website?

The ZIP file with the sample code includes a simple website (ch3/poll) consisting of two files, poll.html and web.xml. The site can be deployed in the usual way:

% ant -Dwar.name=ajax deploy

After deployment, a browser hits the site with the URL:

http://localhost:8080/ajax/

It is tempting to avoid the deployment altogether and, instead, load the local file poll.html into the browser with a URL such as:

file:///home/kalin/jwsur2/ch3/poll/poll.html

In some browsers this may be all right, but in others (for example, Chrome), the same origin policy, discussed earlier with respect to JSONP, comes into play. In the case of Chrome, even two local files in the same directory are treated as having different origins. The upshot is that Ajax calls in an HTML page such as poll.html should be tested by deploying the page as part of a website. Accordingly, the poll.html page is deployed in the ajax.war file and a browser is then directed to this WAR file.

The Ajax polling example, like the earlier cds service, shows the benefit that a jQuery or other JavaScipt client enjoys from a JSON response. In the polling example, the who and the what attributes of a JSON prediction are extracted straightforwardly. Here, for review, is the code snippet that displays a prediction as an HTML element in an unordered list:

$('#container').append('<li>' + preds[ind].who + ': ' +
                        preds[ind].what + '</li>'); } );

In an expression such as:

preds[ind].who

preds refers to the JSON array, ind is an integer index into this array, and who is a field in the JSON representation of a JavaScript prediction object. The jQuery script can access the information of interest in a natural way precisely because the script deals with JSON rather than, say, XML.

JavaScript clients of RESTful web services, such as the jQuery clients illustrated in the three examples of this section, are becoming increasingly popular. RESTful services are sufficiently flexible to generate response payloads in whatever format a client prefers. In the case of JavaScript clients, the preferred format is clear: JSON.