The Java virtual machine (JVM) is a remarkably flexible engine. While it was originally developed purely for Java, it has spawned its own family of languages, just as Microsoft’s CIL supports multiple languages for the Windows platform. Some languages targeting the JVM as a runtime will work on Android, since the regular Java VM and Android’s Dalvik VM are so similar.
Understanding this chapter requires that you have read the core chapters of this book. Some of the sample code demonstrates JUnit test cases, so reading the chapter on unit testing may be useful.
Except for the handful of early language interpreters and compilers hand-constructed in machine code, every programming language is built atop earlier ones. C and C++ are built atop assembly language. Many other languages, such as Java itself, are built atop C/C++.
Hence, it should not come as much of a surprise that an environment as popular as Java has spawned another generation of languages whose implementations are in Java.
There are a few flavors of these languages. Some, like Scala and Clojure, are compiled languages whose compilers created JVM bytecodes, no different than would a Java compiler. These do not strictly qualify as a “scripting language”, however, since they typically compile their source code to bytecode ahead of time.
Some Java-based scripting languages use fairly simple interpreters. These interpreters convert scripting code into parsed representations (frequently so-called “abstract syntax trees”, or ASTs), then execute the scripts from their parsed forms. Most scripting languages at least start here, and some, like BeanShell, stick with this implementation.
Other scripting languages try to bridge the gap between a purely interpreted language and a compiled one like Scala or Clojure. These languages turn the parsed scripting code into JVM bytecode, effectively implementing their own just-in-time compiler (JIT). Since many Java runtimes themselves have a JIT to turn bytecode into machine code (“opcode”), languages with their own JIT can significantly outperform their purely-interpreted counterparts. JRuby and Rhino are two languages that have taken this approach.
Back in the beginning, the only way to write for the JVM was in Java itself. However, since writing language interpreters is a common pastime, it did not take long for people to start implementing interpreters in Java. These had their niche audiences, but there was only modest interest in the early days — interpreters made Java applets too large to download, for example.
Things got a bit more interesting in 1999, when IBM released the Bean Scripting Framework (BSF). This offered a uniform API for scripting engines, meaning that a hosting Java application could write to the BSF API, then plug in arbitrary interpreters at runtime. It was even possible, with a bit of extra work, to allow new interpreters to be downloaded and used on demand, rather than having to be pre-installed with the application. BSF also standardized how to inject Java objects into the scripting engines themselves, for access by the scripts. This allowed scripts to work with the host application’s objects, such as allowing scripts to manipulate the contents of the jEdit text editor.
This spurred interest in scripting. In addition to some IBM languages (e.g., NetREXX) supporting BSF natively, other languages, like BeanShell, created BSF adapters to allow their languages to participate in the BSF space. On the consumer side, various Web frameworks started supporting BSF scripting for dynamic Web content generation, and so forth.
Interest was high enough that Apache took over stewardship of
BSF in 2003. Shortly thereafter,
Sun and others started work on
JSR-223, which added the
javax.script framework to Java 6. The javax.script framework
advanced the BSF concept and standardized it as part of Java itself.
At this point, most JVM scripting languages that are currently
maintained support javax.script integration, and may also support
integration with the older BSF API as well.
Android does not include javax.script as part of its subset of the
Java SE class library from the Apache Harmony project. This certainly
does not preclude integrating scripting languages into Android
applications, but it does raise the degree of difficulty a bit.
Of course, JVM scripting languages do not necessarily work on Android without issue. There may be some work to get a JVM language going on Android.
Android is not Java SE, or Java ME, or even Java EE. While Android has many standard Java classes, it does not have a class library that matches any traditional pattern. As such, languages built assuming Java SE, for example, may have some dependency issues.
For languages where you have access to the source code, removing these dependencies may be relatively straightforward, particularly if they are ancillary to the operation of the language itself. For example, the language may come with miniature Swing IDEs, support for scripted servlets, or other capabilities that are not particularly relevant on Android and can be excised from the source code.
Android runs Dalvik bytecode, not Java bytecode. The conversion from Java bytecode to Dalvik bytecode happens at compile time. However, the conversion tool is rather finicky — it wants bytecode from Sun/Oracle’s Java 1.5 or 1.6, nothing else. This can cause some problems:
Again, if you have the source code, recompiling on an Android-friendly Java compiler should be a simple process.
The heyday of some JVM languages is in the past. As such, you may find that support for some languages will be limited, simply because few people are still interested in them. Finding people interested in those languages on Android — the cross-section of two niches – may be even more of a problem.
SL4A supports three JVM languages today:
You can use those within your SL4A environment no different than you can any other scripting language (e.g., Perl, Python, PHP). Hence, if what you are looking for is to create your own personal scripts, or writing small applications, SL4A saves you a lot of hassle. If there is a JVM scripting language you like but is not supported by SL4A, adding support for new interpreters within SL4A is fairly straightforward, though the APIs may change as SL4A is undergoing a fairly frequent set of revisions.
While SL4A will drive end users towards writing their own scripts or miniature applications using JVM languages, another use of these languages is for embedding in a full Android application. Scripting may accelerate development, if the developers are more comfortable with the scripted language than with Java. Also, if the scripts are able to be modified or expanded by users, an ecosystem may emerge for user-contributed scripts.
Embedding a scripting language is not something to be undertaken lightly, even on a desktop or server application. Mobile devices running Android will have similar issues.
One potential problem is that a script may take too long to execute. Android’s architecture assume that work triggered by buttons, menus, and the like will either happen very quickly or will be done on background threads. Particularly for user-generated scripts, the script execution time is unknowable in advance — it might be a few milliseconds, or it might be several seconds. Hence, any implementation of a scripting extension for an Android application needs to consider executing all scripts in a background thread. This, of course, raises its own challenges for reflecting those scripts’ results on-screen, since GUI updates cannot be done on a background thread.
Scripts in Android inherit the security restrictions of the process that runs the script. If an application has the right to access the Internet, so will any scripts run in that application’s process. If an application has the right to read the user’s contacts, so will any scripts run in that application’s process. And so on. If the scripts in question are created by the application’s authors, this is not a big deal — the rest of the application has those same permissions, after all. But, if the application supports user-authored scripts, it raises the potential of malware hijacking the application to do things that the malware itself would otherwise lack the rights to do.
One way to solve both of those problems is to isolate the scripting language in a self-contained low-permission APK — “sandboxing” the interpreter so the scripts it executes are less able to cause harm. This APK could also arrange to have the interpreter execute its scripts on a background thread. An even better implementation would allow the embedding application to decide whether or not the “sandbox” is important — applications with a controlled source of scripts may not need the extra security or the implementation headaches it causes.
With that in mind, let us take a look at the
JVM/InterpreterService
sample project, one possible implementation of the strategy described
above.
The InterpreterService can support an arbitrary number of
interpreters, via a common interface. This interface provides a
simplified API for having an interpreter execute a script and return
a result:
package com.commonsware.abj.interp;
import android.os.Bundle;
public interface I_Interpreter {
Bundle executeScript(Bundle input);
}
As you can see, it is very simplified, offering just a single
executeScript() method. That method accepts a Bundle (a key-value
store akin to a Java HashMap) as a parameter — that Bundle
will need to contain the script and any other objects needed to
execute the script.
The interpreter will return another Bundle from executeScript(),
containing whatever data it wants the script’s requester to have
access to.
For example, here is the implementation of EchoInterpreter, which
just returns the same Bundle that was passed in:
package com.commonsware.abj.interp;
import android.os.Bundle;
public class EchoInterpreter implements I_Interpreter {
public Bundle executeScript(Bundle input) {
return(input);
}
}
A somewhat more elaborate sample is the SQLiteInterpreter:
package com.commonsware.abj.interp;
import android.database.Cursor;
import android.database.sqlite.SQLiteDatabase;
import android.os.Bundle;
public class SQLiteInterpreter implements I_Interpreter {
public Bundle executeScript(Bundle input) {
Bundle result=new Bundle(input);
String script=input.getString(InterpreterService.SCRIPT);
if (script!=null) {
SQLiteDatabase db=SQLiteDatabase.create(null);
Cursor c=db.rawQuery(script, null);
c.moveToFirst();
for (int i=0;i<c.getColumnCount();i++) {
result.putString(c.getColumnName(i), c.getString(i));
}
c.close();
db.close();
}
return(result);
}
}
This class accepts a script, in the form of a SQLite database query.
It extracts the script from the Bundle, using a pre-defined key
(InterpreterService.SCRIPT). Assuming there is such a script, it
creates an empty in-memory
database and executes the SQLite query against that database.
The results come back in the form of a Cursor — itself a
key-value store. SQLiteInterpreter takes those results and pours
them into a Bundle to be returned.
The Bundle being returned starts from a copy of the input Bundle,
so the script requester can embed in the input Bundle any
identifiers it needs to determine how to handle the results from
executing this script.
SQLiteInterpreter is not terribly flexible, but you can use it for
simple numeric and string calculations, such as the following script:
SELECT 1+2 AS result, 'foo' AS other_result, 3*8 AS third_result;
This would return a Bundle containing a key of result with a
value of 3, a key of other_result with a value of foo, and a
key of third_result with a value of 24.
Of course, it would be nice to support more compelling interpreters, and we will examine a pair of those later in this chapter.
Of course, having a nice clean interface to the interpreters does
nothing in terms of actually executing them on a background thread,
let alone sandboxing them. The InterpreterService class itself
handles that.
InterpreterService is an IntentService, which automatically
routes incoming Intent objects (from calls to startService()) to
a background thread via a call to onHandleIntent(). IntentService
will queue up Intent objects if needed, and IntentService even
automatically shuts down if there is no more work to be done.
Here is the implementation of onHandleIntent() from
InterpreterService:
@Override
protected void onHandleIntent(Intent intent) {
String action=intent.getAction();
I_Interpreter interpreter=interpreters.get(action);
if (interpreter==null) {
try {
interpreter=(I_Interpreter)Class.forName(action).newInstance();
interpreters.put(action, interpreter);
}
catch (Throwable t) {
Log.e("InterpreterService", "Error creating interpreter", t);
}
}
if (interpreter==null) {
failure(intent, "Could not create interpreter: "+intent.getAction());
}
else {
try {
success(intent, interpreter.executeScript(intent.getBundleExtra(BUNDLE)));
}
catch (Throwable t) {
Log.e("InterpreterService", "Error executing script", t);
try {
failure(intent, t);
}
catch (Throwable t2) {
Log.e("InterpreterService",
"Error returning exception to client",
t2);
}
}
}
}
We keep a cache of interpreters, since initializing their engines may
take some time. That cache is keyed by the interpreter’s class name,
and that key comes in to the service by way of the action on the
Intent that was used to start the service. In other words, the
script requester tells us, by way of the Intent used in
startService(), which interpreter to use.
Those interpreters are created using reflection. This way,
InterpreterService has no compile-time knowledge of any
given interpreter class. Interpreters can come and go, but
InterpreterService remains the same.
Assuming an interpreter was found (either cached or newly created),
we have it execute the script, with the input Bundle coming from an
“extra” on the Intent. Methods named success() and failure()
are then responsible for getting the results to the script
requester… as will be seen in the next section.
Script requesters can get the results of the script back — in
the form of the interpreter’s output Bundle — in one of two ways.
One option is a private broadcast Intent. This is a broadcast
Intent where the broadcast is limited to be delivered only to a
specific package, not to any potential broadcast receiver on the
device.
The other option is to supply a PendingIntent that will be sent
with the results. This could be used by an Activity and
createPendingIntent() to have control routed to its
onActivityResult() method. Or, an arbitrary PendingIntent could
be created, to start another activity, for example.
The implementations of success() and failure() in
InterpreterService simply build up an Intent containing the
results to be delivered:
private void success(Intent intent, Bundle result) {
Intent data=new Intent();
data.putExtras(result);
data.putExtra(RESULT_CODE, SUCCESS);
send(intent, data);
}
private void failure(Intent intent, String message) {
Intent data=new Intent();
data.putExtra(ERROR, message);
data.putExtra(RESULT_CODE, FAILURE);
send(intent, data);
}
private void failure(Intent intent, Throwable t) {
Intent data=new Intent();
data.putExtra(ERROR, t.getMessage());
data.putExtra(TRACE, getStackTrace(t));
data.putExtra(RESULT_CODE, FAILURE);
send(intent, data);
}
These, in turn, delegate the actual sending logic to a send()
method that delivers the result Intent via a private broadcast or a
PendingIntent, as indicated by the script requester:
private void send(Intent intent, Intent data) {
String broadcast=intent.getStringExtra(BROADCAST_ACTION);
if (broadcast==null) {
PendingIntent pi=(PendingIntent)intent.getParcelableExtra(PENDING_RESULT);
if (pi!=null) {
try {
pi.send(this, Activity.RESULT_OK, data);
}
catch (PendingIntent.CanceledException e) {
// no-op -- client must be gone
}
}
}
else {
data.setPackage(intent.getStringExtra(BROADCAST_PACKAGE));
data.setAction(broadcast);
sendBroadcast(data);
}
}
There are three steps for integrating InterpreterService into an
application.
First, you need to decide what APK the InterpreterService goes in
– the main one for the application (no sandbox) or a separate
low-permission one (sandbox).
Second, you need to decide what interpreters you wish to support,
writing I_Interpreter implementations and getting the interpreters’
JARs into the project’s libs/ directory.
Third, you need to add the source code for InterpreterService along
with a suitable <service> entry in AndroidManifest.xml. This
entry will need to support <intent-filter> elements for each
scripting language you are supporting, such as:
<service
android:name=".InterpreterService"
android:exported="false">
<intent-filter>
<action android:name="com.commonsware.abj.interp.EchoInterpreter"/>
</intent-filter>
<intent-filter>
<action android:name="com.commonsware.abj.interp.SQLiteInterpreter"/>
</intent-filter>
<intent-filter>
<action android:name="com.commonsware.abj.interp.BshInterpreter"/>
</intent-filter>
<intent-filter>
<action android:name="com.commonsware.abj.interp.RhinoInterpreter"/>
</intent-filter>
</service>
From there, it is a matter of adding in appropriate startService()
calls to your application wherever you want to execute a script, and
processing the results you get back.
To use the InterpreterService, you need to first determine which
I_Interpreter engine you are using, as that forms the action for
the Intent to be used with the InterpreterService. Create an
Intent with that action, then add in an InterpreterService.BUNDLE
extra for the script and other data to be supplied to the
interpreter. Also, you can add an
InterpreterService.BROADCAST_ACTION, to be used by
InterpreterService to send results back to you via a broadcast
Intent. Finally, call startService() on the Intent, and the
results will be delivered to you asynchronously.
For example, here is a test method from the EchoInterpreterTests
test case:
package com.commonsware.abj.interp;
import android.os.Bundle;
public class EchoInterpreterTests extends InterpreterTestCase {
protected String getInterpreterName() {
return("com.commonsware.abj.interp.EchoInterpreter");
}
public void testNoInput() {
Bundle results=execServiceTest(new Bundle());
assertNotNull(results);
assert (results.size() == 0);
}
public void testWithSomeInputJustForGrins() {
Bundle input=new Bundle();
input.putString("this", "is a value");
Bundle results=execServiceTest(input);
assertNotNull(results);
assertEquals(results.getString("this"), "is a value");
}
}
The echo “interpreter” simply echoes the input Bundle into the
output. The execServiceTest() method is inherited from the
InterpreterTestCase base class:
protected Bundle execServiceTest(Bundle input) {
Intent i=new Intent(getInterpreterName());
i.putExtra(InterpreterService.BUNDLE, input);
i.putExtra(InterpreterService.BROADCAST_ACTION, ACTION);
getContext().startService(i);
try {
latch.await(5000, TimeUnit.MILLISECONDS);
}
catch (InterruptedException e) {
// just keep rollin'
}
return(results);
}
The execServiceTest() method uses a CountDownLatch to wait on the
interpreter to do its work before proceeding (or 5000 milliseconds,
whichever comes first). The broadcast Intent containing the
results, registered to watch for
com.commonsware.abj.interp.InterpreterTestCase broadcasts, stuffs
the output Bundle in a results data member and drops the latch,
allowing the main test thread to continue.
What if Java itself were a scripting language? What if you could just execute a snippet of Java code, outside of any class or method? What if you could still import classes, call static methods on classes, create new objects, as well?
That was what BeanShell offered, back in its heyday. And, since BeanShell does not use sophisticated tricks with its interpreter – like JIT compilation of scripting code — BeanShell is fairly easy to integrate into Android.
BeanShell is Java on Java.
With BeanShell, you can write scripts in loose Java syntax. Here, “loose” means:
BeanShell was originally developed in the late 1990’s by Pat Niemeyer. It enjoyed a fair amount of success, even being considered as a standard interpreter to ship with Java (JSR-274). However, shortly thereafter, BeanShell lost momentum, and it is no longer being actively maintained. That being said, it works quite nicely on Android… once a few minor packaging issues are taken care of.
BeanShell has two main problems when it comes to Android:
Fortunately, with BeanShell being open source, it is easy enough to overcome these challenges. You could download the source into an Android library project, then remove the classes that are not necessary (e.g., the servlet), and use that library project in your main application. Or, you could use an Android project for creating a JAR file that was compiled against the Android class library, so you are certain everything is supported.
However, the easiest answer is to use SL4A’s BeanShell JAR, since they have solved those problems already. The JAR can be found in the SL4A source code repository, though you will probably need to check out the project using Mercurial, since JARs cannot readily be downloaded from the Google Code Web site.
The BeanShell engine is found in the bsh.Interpreter class.
Wrapping one of these in an I_Interpreter interface, for use with
InterpreterService, is fairly simple:
package com.commonsware.abj.interp;
import android.os.Bundle;
import bsh.Interpreter;
public class BshInterpreter implements I_Interpreter {
public Bundle executeScript(Bundle input) {
Interpreter i=new Interpreter();
Bundle output=new Bundle(input);
String script=input.getString(InterpreterService.SCRIPT);
if (script != null) {
try {
i.set(InterpreterService.BUNDLE, input);
i.set(InterpreterService.RESULT, output);
Object eval_result=i.eval(script);
output.putString("result", eval_result.toString());
}
catch (Throwable t) {
output.putString("error", t.getMessage());
}
}
return(output);
}
}
BeanShell interpreters are fairly inexpensive objects, so we create a
fresh Interpreter for each script, so one script cannot somehow
access results from prior scripts. After setting up the output
Bundle and extracting the script from the input Bundle, we inject
both Bundle objects into BeanShell itself, where they can be
accessed like global variables, named _bundle and _result.
At this point, we evaluate the script, using the eval() method on
the Interpreter object. If all goes well, we convert the object
returned by the script into a String and tuck it into the output
Bundle, alongside anything else the script may have put into the
Bundle. If there is a problem, such as a syntax error in the
script, we put the error message into the output Bundle.
So long as the InterpreterService has an <intent-filter> for the
com.commonsware.abj.interp.BshInterpreter action, and so long as we
have a BeanShell JAR in the project’s libs/ directory,
InterpreterService is now capable of executing BeanShell scripts as
needed.
With our inherited execServiceTest() method handling invoking the
InterpreterService and waiting for responses, we can “simply” put
our script as the InterpreterService.SCRIPT value in the input
Bundle, and see what we get out. The first test script returns a
simple value; the second test script directly calls methods on the
output Bundle to return its results.
JavaScript arrived on the language scene hot on the heels of Java itself. The name was chosen for marketing purposes more so than for any technical reason. Java and JavaScript had little to do with one another, other than both adding interactivity to Web browsers. And while Java has largely faded from mainstream browser usage, JavaScript has become more and more of a force on the browser, and even now on Web servers.
And, along the way, the Mozilla project put JavaScript on Java and gave us Rhino.
If BeanShell is Java in Java, Rhino is JavaScript in Java.
As part of Netscape’s failed “Javagator” attempt to create a Web browser in Java, they created a JavaScript interpreter for Java, code-named Rhino after the cover of O’Reilly Media’s JavaScript: The Definitive Guide. Eventually, Rhino was made available to the Mozilla Foundation, which has continued maintaining it. At the present time, Rhino implements JavaScript 1.7, so it does not support the latest and greatest JavaScript capabilities, but it is still fairly full-featured.
Interest in Rhino has ticked upwards, courtesy of interest in using JavaScript in places other than Web browsers, such as server-side frameworks. And, of course, it works nicely with Android.
Similar to BeanShell, Rhino has a few minor source-level incompatibilities with Android. However, these can be readily pruned out, leaving you with a still-functional JavaScript interpreter. However, once again, it is easiest to use SL4A’s Rhino JAR, since all that work is done for you.
Putting an I_Interpreter facade on Rhino is incrementally more
difficult than it is for BeanShell, but not by that much:
package com.commonsware.abj.interp;
import android.os.Bundle;
import org.mozilla.javascript.*;
public class RhinoInterpreter implements I_Interpreter {
public Bundle executeScript(Bundle input) {
String script=input.getString(InterpreterService.SCRIPT);
Bundle output=new Bundle(input);
if (script != null) {
Context ctxt=Context.enter();
try {
ctxt.setOptimizationLevel(-1);
Scriptable scope=ctxt.initStandardObjects();
Object jsBundle=Context.javaToJS(input, scope);
ScriptableObject.putProperty(scope, InterpreterService.BUNDLE,
jsBundle);
jsBundle=Context.javaToJS(output, scope);
ScriptableObject.putProperty(scope, InterpreterService.RESULT,
jsBundle);
String result=
Context.toString(ctxt.evaluateString(scope, script,
"<script>", 1, null));
output.putString("result", result);
}
finally {
Context.exit();
}
}
return(output);
}
}
As with BshInterpreter, RhinoInterpreter sets up the output
Bundle and extracts the script from the input Bundle. Assuming
there is a script, RhinoInterpreter then sets up a Rhino Context
object, which is roughly analogous to the BeanShell Interpreter
object. One key difference is that you need to clean up the
Context, by calling a static exit() method on the Context
class, whereas with a BeanShell Interpreter, you just let garbage
collection deal with it.
Rhino has a JIT compiler, one that unfortunately will not work with
Android, since it generates Java bytecode, not Dalvik bytecode.
However, Rhino lets you turn that off, by calling
setOptimizationLevel() on the Context object with a value of -1
(meaning, in effect, disable all optimizations).
After that, we:
Bundle objects with JavaScript proxies via calls to
javaToJS(), then injecting those objects into the scope as _bundle and _result via putProperty() calls
evaluateString() on the
Context object, converting the resulting object into a String and
pouring it into the output Bundle
If our InterpreterService has an <intent-filter> for the
com.commonsware.abj.interp.RhinoInterpreter action, and so long as
we have a Rhino JAR in the project’s libs/ directory,
InterpreterService can now invoke JavaScript.
As mentioned previously, there are many languages that, themselves, are implemented in Java and can be ported to Android, with varying degrees of difficulty. Many of these languages are fairly esoteric. Some, like JRuby, have evolved to the point where they transcend a simple “scripting language” on Android.
However, there are two other languages worth mentioning, as they are fairly well-known in Java circles: Groovy and Jython.
Groovy is perhaps the most popular Java-based language that does not have its roots in some other previous language (Java, JavaScript, Python, etc.). Designed in some respects to be a “better Java than Java”, Groovy gives you access to Java classes while allowing you to write scripts with dynamic typing, closures, and so forth. Groovy has an extensive community, complete with a fair number of Groovy-specific libraries and frameworks, plus some books on the market.
At the time of this writing, it does not appear that Groovy has been successfully ported to work on Android, though.
Jython is an implementation of a Python language interpreter in Java. It has been around for quite some time, and gives you Python syntax with access to standard Java classes where needed. While the Jython community is not as well-organized as that of Groovy, there are plenty of books covering the use of Jython.
Jython’s momentum has flagged a bit in recent months, in part due to Sun’s waning interest in the technology and the departure of Sun employees from the project. One attempt to get Jython working with Android has been shut down, with people steered towards SL4A. It is unclear if others will make subsequent attempts.