libuv’s C API provides a lot of functions. In many ways it serves as a replacement for much of the standard library of a traditional UNIX operating system, so that’s not too surprising. What’s fascinating, though, is how few basic concepts libuv relies upon to provide a consistent programming model over a wide variety of different I/O scenarios: clients and server, sockets and pipes, TCP and UDP. There are three fundamental building blocks to libuv:
They fit together like so: pipes and the like are represented as handles, and registered onto the event loop. The event loop is responsible for monitoring each handle. When an event occurs, a user-specified callback function is invoked with the data received. That’s it! The same pattern largely applies to reading and writing to all kinds of sources and destinations, except for files.
To make use of this elegant and consistent model for I/O, we’re going to need to look at a lot of different functions from the libuv API. We’re also going to need to look at how to work with functions from an external C library like libuv.
Before we can do anything else, we have to initialize an event loop. Although it’s possible to maintain multiple event loops with libuv, I don’t recommend it, even for experts—the library was developed for single-threaded programs running a single event loop, and that’s where support is strongest. As we’ll see, a single thread is capable of serving surprisingly heavy I/O workloads by avoiding blocking and utilizing every CPU cycle available. Since we only need a single loop, then, we can rely on libuv’s default loop, which we can retrieve with uv_default_loop:
def uv_default_loop():Loop
This is probably the simplest function we’ve seen so far, ironically. The complexity lies in the work we do with the Loop struct it returns. So far, all the structs we’ve worked with have been relatively simple, with no more than three or four fields. Loop is different. If we look at its definition in the C header file uv.h, we’ll see this:
| | struct uv_loop_s { |
| | /* User data - use this for whatever. */ |
| | void* data; |
| | /* Loop reference counting. */ |
| | unsigned int active_handles; |
| | void* handle_queue[2]; |
| | union { |
| | void* unused[2]; |
| | unsigned int count; |
| | } active_reqs; |
| | /* Internal flag to signal loop stop. */ |
| | unsigned int stop_flag; |
| | UV_LOOP_PRIVATE_FIELDS |
| | }; |
We haven’t seen any C code before in this book, and it’s not necessary to understand the details of this at all. However, there are two features worth calling out:
The first field in the struct is an untyped pointer that’s free for custom use.
The end of the struct consists of an unspecified number of nonpublic fields that may vary by OS.
This is a tricky problem. We haven’t had to deal with structs that vary their layouts in different environments before, and since Scala Native lacks C’s conditional compilation, we don’t have an obvious way to represent them. Fortunately, libuv’s designers have given us several ways to structure and allocate Loop objects. The basic technique we’re going to use is a type pun, much like we did with IP address casting in our TCP socket code. Since we never have to allocate a Loop instance ourselves, and we only care about data in its first field, we can represent Loop like this:
type Loop = Ptr[Ptr[Byte]]
Then we can get at that inner pointer, and its contents, something like this:
| | val loop:Loop |
| | val inner_data_ptr:Ptr[Data] = (!loop).cast[Ptr[Data] |
| | val inner_data:Data = !inner_data_ptr |
That’s it! And because all pointers are the same size, we can store any kind of data we want; even though the Loop code contains many more fields, our code doesn’t need to know about them. This technique would cause serious errors if we tried to allocate a Loop and populate it ourselves, but we have no reason to do so. And as you’ll see shortly, libuv gives us tools to help with memory allocation as well.
Once we have a Loop, we can start creating handles and assigning callbacks to them. To start, we’ll create something really simple: a Timer handle, which fires an event on the loop after a configurable timeout, with a separately configurable repeat duration. To plug it into our loop, we’ll need to do a few different things:
We’ll do all of these, plus write our callback, step by step. First, though, we have to figure out how to represent the Timer handle as a struct. Unfortunately, the C header definition for uv_timer_t is even gnarlier than uv_loop_t, and we don’t have an out—we have to figure out how to allocate it correctly. Fortunately, libuv provides a utility function for this: uv_handle_size, with this signature:
def uv_handle_size(uv_handle_t:Int):Int
uv_handle_size will tell us, at runtime, exactly how many bytes we need to allocate for any kind of handle, so it’s perfect for passing into malloc. The only catch is that we have to look up the uv_handle_t values, which are constants defined in a C header file. I’ve translated them all from C to Scala:
| | val UV_PIPE_T = 7 // Pipes |
| | val UV_POLL_T = 8 // Polling external sockets |
| | val UV_PREPARE_T = 9 // Runs every loop iteration |
| | val UV_PROCESS_T = 10 // Subprocess |
| | val UV_TCP_T = 12 // TCP sockets |
| | val UV_TIMER_T = 13 // Timer |
| | val UV_TTY_T = 14 // Terminal emulator |
| | val UV_UDP_T = 15 // UDP sockets |
We’ll only use a small fraction of these, but they’re all extremely well documented on the libuv website.[36] Likewise, the Timer (and all other handles) has a leading data field for application use, so we can structure it exactly the same way as we did Loop:
type Timer = Ptr[Ptr[Byte]]
Once we have space for a Timer handle allocated, we can initialize it, which associates it with a loop and gives the memory chunk meaningful values. The function to do this, uv_timer_init, is largely self-explanatory:
def uv_timer_init(loop:Ptr[Loop], handle:Ptr[Timer]):Int
uv_timer_init takes a loop and a handle, initializes the handle, and returns 0 if successful or an error code otherwise. Many libuv functions can return error codes and messages, so it’s worth taking a moment to write a small wrapper function to check those errors; it will itself use the helper functions uv_strerr and uv_err_name:
| | def check_error(v:Int, label:String):Int = { |
| | if (v == 0) { |
| | println(s"$label returned $v") |
| | v |
| | } else { |
| | val error = fromCString(uv_err_name(v)) |
| | val message = fromCString(uv_strerror(v)) |
| | println(s"$label returned $v: $error: $message") |
| | v |
| | } |
| | } |
Now we’re almost ready to run a function on our event loop. To start the Timer and associate a callback function with it, call uv_timer_start:
def uv_timer_start(handle:Timer, callback:uv_timer_cb, timeout:Long, repeat:Long)
uv_timer_start takes a Timer handle, a CFunctionPtr callback, and two Long values: timeout, which determines the initial delay before the timer fires, and repeat, which determines the delay for each invocation after the first (both are in milliseconds). The callback is worth special attention. Like we saw with qsort back in Chapter 2, Arrays, Structs, and the Heap, the callback function must itself follow a particular signature:
type uv_timer_cb = CFunctionPtr1[Timer,Unit]
If we pass a function that takes a Timer handle as an argument and returns Unit (in other words, nothing), the event loop will invoke it and pass in our handle every time. Since we can store custom data inside the handle’s internal data pointer, we could even keep custom data structures in there if we needed to. We also mustn’t forget to use CFunctionPtr.fromFunction1 to convert a regular Scala method into the sort of statically scoped function pointer we can pass to a C API, just like we did with qsort().
Finally, once our loop, its handle, and all of the callbacks are set up, we can run the loop with uv_run:
def uv_run(loop:Loop, runMode:Int): Int = extern
uv_run takes the loop itself and a runMode—I won’t go deep into the details, but each value of runMode determines at what point in the future uv_run will return. In this book, I’ll always call it with the constant UV_RUN_DEFAULT, which will run the loop continuously until there are no outstanding active handles. In other words, all files are closed, connections are closed, and timers are canceled or completed. (For convenience, I’ve defined this and other useful constants in a LibUVConstants object that you can see in the accompanying source code.)
With these preliminaries out of the way, we’re ready to write a short program to test out libuv. To start, we’ll do something really simple: create a short, one-second timer delay, print out a message, complete the loop, and exit. Ironically, we now know that println is itself a blocking, synchronous I/O function, but it remains indispensable for logging, so we’ll keep it around a bit longer.
| | def main(args:Array[String]):Unit = { |
| | println("hello, world!") |
| | val loop = uv_default_loop() |
| | val timer = stdlib.malloc(uv_handle_size(UV_TIMER_T)) |
| | val ret = uv_timer_init(loop, timer) |
| | println(s"uv_timer_init returned $ret") |
| | uv_timer_start(timer, timerCB, 1000, 0) |
| | println("invoking loop") |
| | uv_run(loop,UV_RUN_DEFAULT) |
| | println("done") |
| | } |
| | |
| | val timerCB = new TimerCB { |
| | def apply(handle:TimerHandle):Unit = { |
| | println("timer fired!") |
| | } |
| | } |
But there’s a catch. This code won’t compile yet because we haven’t prepared bindings for the libuv C API. As far as the Scala compiler is concerned, these functions that we’re calling don’t exist, just as though we had failed to add a dependency to our build.sbt file. In fact, if we want to use a nonstandard C library in Scala Native, there are two separate steps we need to take. We have to first ensure that the library is compiled and installed on the system, and then we have to supply definitions for all of the functions we need to invoke.
If you’re using the Docker-based build environment, libuv is already installed, so you don’t have that to worry about it. But if you’re working in a noncontainer environment, see Appendix 1, Setting Up the Environment, for notes on installing libuv. Writing the bindings themselves will take just a little more work, but we’ve already done the hard part.
Once a library is installed on our system, we can use its functions by translating their signatures from C into Scala, which is exactly what we’ve done as we’ve introduced each function throughout the chapter. Then, we place the function definitions into a special object literal in a Scala file, with three unusual characteristics:
All together the bindings for the libuv functions we have used so far look like this:
| | @link("uv") |
| | @extern |
| | object LibUV { |
| | type TimerHandle = Ptr[Byte] |
| | type PipeHandle = Ptr[Ptr[Byte]] |
| | |
| | type Loop = Ptr[Byte] |
| | type TimerCB = CFuncPtr1[TimerHandle,Unit] |
| | |
| | def uv_default_loop(): Loop = extern |
| | def uv_loop_size(): CSize = extern |
| | def uv_is_active(handle:Ptr[Byte]): Int = extern |
| | def uv_handle_size(h_type:Int): CSize = extern |
| | def uv_req_size(r_type:Int): CSize = extern |
| | |
| | def uv_timer_init(loop:Loop, handle:TimerHandle):Int = extern |
| | def uv_timer_start(handle:TimerHandle, cb:TimerCB, timeout:Long, |
| | repeat:Long):Int = extern |
| | def uv_timer_stop(handle:TimerHandle):Int = extern |
| | |
| | def uv_run(loop:Loop, runMode:Int): Int = extern |
| | |
| | def uv_strerror(err:Int): CString = extern |
| | def uv_err_name(err:Int): CString = extern |
| | } |
Now, our program can run! If you run it from the console, you should see output like this:
| | $ ./target/scala-2.11/async_timer-out |
| | hello, world! |
| | uv_timer_init returned 0 |
| | invoking loop |
| | timer fired! |
| | done |
So far, so good. Next, we’ll extend this event-driven programming style to TCP sockets.