Now, we finally have all the pieces of the puzzle. Over the last several chapters, we’ve accumulated a wide range of capabilities based around libuv’s event loop for a full asynchronous programming environment.
Since our curl client provides results as a Future[HttpResponse], it’s straightforward to plug in to our async request handler pattern:
| | def main(args:Array[String]):Unit = { |
| | Service() |
| | .getAsync("/fetch/example") { r => |
| | Curl.get(c"https://www.example.com").map { response => |
| | Response(200,"OK",Map(),response.body) |
| | } |
| | } |
| | uv_run(EventLoop.loop, UV_RUN_DEFAULT) |
| | } |
This makes writing a straightforward proxy server trivial. But we can make it even cleaner if we write some boilerplate to convert the ResponseState class used by our curl binding to the Response[String] that our server API expects:
| | def makeResponse(responseState:ResponseState):Response[String] = { |
| | Response(200,"OK",responseState.headers,responseState.body) |
| | } |
And we can also, again, use Argonaut to parse a curl response as any Scala class:
| | def makeJsonResponse[T](responseState:ResponseState) |
| | (implict d:DecodeJson[T]):Response[T] = { |
| | Parse.decodeOption[T](responseState.body) match { |
| | case Some(t) => |
| | Response(200,"OK",responseState.headers,t) |
| | case None => |
| | throw new Exception(s"parse error: couldn't decode {responseState.body}") |
| | } |
| | } |
This close integration makes it easy to provide a curl-based client for any service we write. If we were to revisit our curl client, in fact, we could probably adjust the API to consolidate on Response[String] as a result type and add JSON support directly to the library.
LMDB integration is a trickier case. As you may recollect from Chapter 9, Durability: An Embedded Key-Value Database with LMDB, our LMDB binding is synchronous, since it generally returns instantaneously, and single-threaded operation means that we don’t have to worry about simultaneous writes locking each other out. Although in theory, that does mean that an LMDB operation can block the event loop, LMDB’s latency in practical usage is much faster than any network operation, so it can be better thought of as “effectively synchronous.” This means we can use it with a simple synchronous handler, like this:
| | val fetchPatn = raw"/fetch/([^/]+)".r |
| | |
| | def main(args:Array[String]):Unit = { |
| | val env = LMDB.open(c"./db") |
| | |
| | Service() |
| | .get("/fetch/") { r => |
| | val key = r.url match { |
| | case fetchPatn(key) => key |
| | case _ => "route error; couldn't find key to lookup" |
| | } |
| | val data = LMDB.getJson[Map[String,String]](env,key) |
| | OK(data) |
| | } |
| | .run(9999) |
| | uv_run(EventLoop.loop, UV_RUN_DEFAULT) |
| | } |
This code is pretty minimally modified from the minimal web app we wrote in Chapter 9. One pain point here is that we have to inspect the request route twice, once to match the URL for the handler, and a second time to extract the key to look up. Many popular URL routers solve this by allowing URL patterns to contain wildcard or placeholder expressions for routing. In this case, something like /fetch/:key could extract the key for us and somehow pass it into the handler function.
However, this approach raises even more questions: how do we pass named parameters into functions in a typed, safe way? As an alternative approach, our HTTP POST handler can accept and parse any kind of JSON-structured data as a case class with validation and typed fields. Generally speaking, JSON-based patterns lend themselves to the sort of interservice communication scenarios we’ve been working with.
As such, we could rework the previous example to look like this:
| | case class FetchRequest(key:String) |
| | case class SetRequest(key:String,value:Map[String,String]) |
| | |
| | def main(args:Array[String]):Unit = { |
| | val env = LMDB.open(c"./db") |
| | |
| | Service() |
| | .post[FetchRequest,Map[String,String]]("/fetch/") { r => |
| | val data = LMDB.getJson[Map[String,String]](env,key) |
| | OK(data) |
| | .post[SetRequest,Map[String,String]]("/set/") { r => |
| | LMDB.putJson(env.r.body.key,r.body.value) |
| | OK(r.body.value) |
| | } |
| | // ... |
| | } |
This seems both simpler and more robust than a URL-based approach. That said, a robust URL matcher is still useful for many common use cases and would be a great addition to our framework.
We could integrate many more additional C libraries with our event loop for connectivity with real-world systems. For example, Redis[57] is a high-performance key-value store, popular for use as a persistent caching layer. It doesn’t offer the same level of durability or transaction support as LMDB, but it has great support for a wide variety of data structures and is widely used in cloud environments. Its official C library, hiredis,[58] has excellent synchronous and asynchronous APIs, and includes example C code[59] for libuv integration.
Postgres,[60] a full-featured relational database, is one of the most widely used database engines available and is always being updated with new features like JSON support and clustering. Its C client library, libpq,[61] likewise has thorough support for asynchronous processing. Although it’s somewhat more complex to use than hiredis, open-source projects do exist that demonstrate a full integration with libuv,[62] following an implementation style similar to our libcurl client.
Sqlite[63] is another popular relational database that has the benefit of running as a daemonless library, embedded in your application code. David Bouyssié has contributed a Scala Native binding for Sqlite[64] that can replace a traditional database for many common use cases.
In short, any C library that supports asynchronous, callback-based processing is a great candidate for integration with our event loop framework. And there are high-quality C libraries available for just about every problem domain under the sun.[65]