LMDB’s design provides extraordinarily fast access to data by mapping file-based storage directly into process memory, avoiding the cost of intermediate copies. Although that does generally limit us to an amount of data we can fit into working memory, it provides many other benefits, including transactionality. And as you’ll see, the fast, synchronous data access patterns LMDB provides are an exceedingly good fit for the nonblocking libuv server we’ve constructed over the last several chapters. Compared to libuv or libcurl, LMDB’s API surface is refreshingly compact, consisting of a handful of functions and types. That said, it does introduce a few concepts that are worth defining before we get into the weeds.
In LMDB, an environment is the global, top-level object, corresponding to a directory somewhere on the filesystem. LMDB will not create a directory for us, and trying to open a directory that doesn’t yet exist is a common error for beginners.
Inside a directory there are one or many databases. If an environment contains a single database, it may be anonymous, otherwise each database is distinguished by name.
Each database contains keys. Keys are unique within a single database, but the same key may exist in multiple databases in a single environment. Keys are shortish bytestrings—the default upper limit on length is 512 bytes.
In a default configuration, each key has exactly one value, which may be a bytestring of variable length. However, there are ways to configure LMDB to allow a single key to have multiple distinct values; in this case, the values are sorted in some deterministic order, which is defined by a user-supplied function pointer. Multiple values may be traversed one at a time by a cursor, as can ranges of keys.
Cursors, multiple values, and custom sorting are incredibly powerful techniques for designing custom data structures, but they are outside of the scope of this book. Instead, we’ll focus on simple, transactional updates of a few straightforward data structures, and you’ll discover just how powerful these simple techniques can be when integrated with what we’ve already built.