Chapter 4. Kong architecture
The Kong website displays a great getting started section that includes in three steps how to register a new API and start requesting it. You can really consider (read buy ) Kong API gateway as a blackbox:
And that is a very good thing about the project. Being able sometimes to outsource most of the work to focus on more different features for your business makes a lot of sense.
But you might also want to understand what’s going on with your requests. Perhaps you are not comfortable with blackboxes manipulating all of your sensitive requests. Or maybe you need more than the available plugins to develop a new one. Or perhaps your application is under edge conditions and it makes sense to understand the underlying logic to make a better usage of the project.
The following sections should give you the framework you need, whatever the use case.
All of the code for the sample project in this book can be found here .
Overall approach and black box description
If you know how to bootstrap and configure Kong, you basically have a magic service that takes requests as input, routes it to the right service, and gives you back a response.
Along the way the gorilla gets full access and control over this request, transforming or analyzing its attributes.
From this very high level perspective you can see the shape of Kong’s benefits for cloud architectures. All of the common features you can expect, or wish, from modern APIs can be refactored into one separate layer. No need to reimplement your JWT authentification , given both your payment API in Python and your events server in Scala. Do it once, deploy once. And remove some duplicate boilerplate. No more rate limiting code close to business logic or data modelization issues.
Kong unlocks very powerful architectural patterns. But that’s exactly what we won’t develop or study. That is a topic for another time because we have much to explore into the architecture of Kong itself. Indeed we are going back to lower levels and will dive into the details of how Kong handles requests. No matter where it goes, and from where it came, we will try to understand the how and why of what happens to those requests.
I hope it will provide you with the wisdom to move forward and understand under the right lights the other sections of this book. There’s no blackbox, no magic, just cautious choices and considerations for real-life challenges.
Now let’s move on and dissect the project like hardware hackers.
The HTTP server blocks
When tackling new frameworks or technologies in 2018 you are often offered the choice to either use the project right away, thanks to Docker , or to build it from source. The latter option is usually barely documented and barely useful, except for some low level customization that needs to recompile the project. But I believe this is actually a powerful approach to learn more about the project internals.
The project is 99.8% made of lua scripts, and you can quickly understand why reaching to the official install from source section . Step 1 states right away that Kong is an OpenResty application, which is itself an extension of Nginx. So Kong stands on a giant’s shoulders. Or more precisely it is composed of giants. Let’s start from the beginning and understand this design.
Nginx can be extended by Lua scripts through the ngx_lua module , which you can compile to native C code that can integrate with Nginx Sources. OpenResty builds on top of this and bundles a consistent set of modules along with Nginx itself to offer a powerful and extensible webserver for HTTP intensive applications. The two projects maintain a close relationship, with Nginx merging sometimes OpenResty modules into its upstream source. As Thibault Charbonnier explains it on Quora :
At Mashape, we wanted to build a modulable reverse proxy (on top of Nginx) that would handle features that are common between APIs (authentication, logging, rate-limiting...). We had two ways to do that: by bundling Nginx with our own flavour of 3rd party modules (the said plugins), or by developing on top of OpenResty.
We admired the work done on OpenResty and thought of a way to build a modulable core, on which plugins could be added. Hence, Kong is mostly a collection of Lua scripts that allow it to add services (APIs...) at runtime, and the execution of plugins on top of those.
The answer was written on Oct 17, 2015 and 2 years later it proved to be a successful approach. The project is thriving in demanding production environments, building upon Nginx strengths and OpenResty extensibility. Core maintainers and developers can focus on developing features that are common between APIs , while the foundations handle high performance non-blocking HTTP traffic handling.
Relying on OpenResty also helps developers writing Kong plugin because their Lua API is accessible within the script. The bot-detector plugin for example imports a very common LRU Cache with components directly from [OpenResty library][modules].
local lrucache = require "resty.lrucache"

-- per-worker cache of matched UAs
-- we use a weak table, index by the `conf` parameter, so once the
plugin config
-- is GC'ed, the cache follows automatically
local ua_caches = setmetatable ({}, { __mode = "k" })
local UA_CACHE_SIZE = 10 ^ 4

-- [ ... ]

function BotDetectionHandler : access ( conf )
    -- [ ... ]
    local cache = ua_caches [ conf ]
  if not cache then
      cache = lrucache . new ( UA_CACHE_SIZE )
    ua_caches [ conf ] = cache
  end

    local match   = cache : get ( user_agent )
    if not match then
        match = examine_agent ( user_agent , conf )
        cache : set ( user_agent , match )
    end
    -- [ ... ]
end

return BotDetectionHandler
A few lines of scripting and you just implemented a useful caching system for your plugin.
But Kong also brings this HTTP admin interface that makes the whole proxy dynamic and transforms the static webserver into an adaptable API Gateway.
Look back at the installation page and after compiling Kong lua sources, you’re asked to prepare a database. Cassandra 3.x.x and Postgresql 9.5+ are supported but for what purpose exactly? Let’s move on to the next section and figure this out.
Database and high level blocks
Put simply, Kong needs to remember a lot of details along its lifecycle and store them between requests to provide a consistent behavior. We can distinguish between three kinds of scenarios where data ends up stored in a relational database:
Let’s take a concrete look at what’s inside an initialized Kong cluster (i.e. Kong started and the database was migrated).
As a convenient example we use below a Postgres datastore assuming the reader is comfortable with basic SQL queries. It is important to remember, like mentioned above, that Kong supports a second datastore backend: Apache Cassandra.
Given your infrastructure and production constraints you might consider using one or the other. When Kong is used in a multi-region, multi data-center, high-availability environment, Cassandra is usually the preferred choice, because a distribute setup is built into its core.
In a nutshell, the columnar database is indeed designed to use many commodity servers distributed across several data centers to handle large amounts of data without a single point of failure. Every single node of the cluster is considered identical (masterless) and data is replicated automatically to multiple nodes, for fault-tolerance.
To tease your curiosity and illustrate, here are a few examples of CQL (Cassadra Query Language):
cqlsh> create keyspace demo with replication = {'class':'SimpleStrategy','replication_factor' : 2};
cqlsh> use demo;
cqlsh:demo> create table api (apiid int primary key, apiname varchar);
cqlsh:demo> insert into api (apiid, apiname) values (1, 'Mortgages');
cqlsh:demo> update api set apiname = 'Loans' where apiid = 1;
cqlsh:demo> select * from api where apiid = 1;
 apiid | apiname
-------+---------
     1 |   Loans

(1 rows)
Back to our initial setup with a Kong instance and a PostgreSQL data backend. Let’s inspect the internal memory of our fresh Gateway.
(kpler) kong :: (tech/kong*) » pgcli -h localhost --user=kong --password
Password:
Version: 1.4.0
Chat: https://gitter.im/dbcli/pgcli
Mail: https://groups.google.com/forum/#!forum/pgcli
Home: http://pgcli.com
kong@localhost:kong> \d
+----------+-------------------------------+--------+---------+
| Schema   | Name                          | Type   | Owner   |
|----------+-------------------------------+--------+---------|
| public   | acls                          | table  | kong    |
| public   | apis                          | table  | kong    |
| public   | basicauth_credentials         | table  | kong    |
| public   | cluster_events                | table  | kong    |
| public   | consumers                     | table  | kong    |
| public   | hmacauth_credentials          | table  | kong    |
| public   | jwt_secrets                   | table  | kong    |
| public   | keyauth_credentials           | table  | kong    |
| public   | oauth2_authorization_codes    | table  | kong    |
| public   | oauth2_credentials            | table  | kong    |
| public   | oauth2_tokens                 | table  | kong    |
| public   | plugins                       | table  | kong    |
| public   | ratelimiting_metrics          | table  | kong    |
| public   | response_ratelimiting_metrics | table  | kong    |
| public   | schema_migrations             | table  | kong    |
| public   | ssl_certificates              | table  | kong    |
| public   | ssl_servers_names             | table  | kong    |
| public   | targets                       | table  | kong    |
| public   | ttls                          | table  | kong    |
| public   | upstreams                     | table  | kong    |
+----------+-------------------------------+--------+---------+
SELECT 20
At this stage everything is pretty much empty, but we can spot the three mentioned categories. Let’s try to add a new API like the quickstart guide suggests.
$ # let's register a new API
$ curl -i -X POST \
  --url http://localhost:8001/apis/ \
  --data 'name=example-api' \
  --data 'hosts=example.com' \
  --data 'upstream_url=http://mockbin.org'

$ # and enable a 'basic auth' plugin for this API
And without any surprise, Kong stores this information to handle future requests.
kong@localhost:kong> SELECT name, hosts, upstream_url FROM apis
+-------------+-----------------+--------------------+
| name        | hosts           | upstream_url       |
|-------------+-----------------+--------------------|
| example-api | ["example.com"] | http://mockbin.org |
+-------------+-----------------+--------------------+

kong@localhost:kong> SELECT a.name as api, a.upstream_url, p.name as plugin, p.enabled
FROM plugins p, apis a
WHERE p.api_id = a.id
+-------------+--------------------+----------+-----------+
| api         | upstream_url       | plugin   | enabled   |
|-------------+--------------------+----------+-----------|
| example-api | http://mockbin.org | key-auth | True      |
+-------------+--------------------+----------+-----------+
Everything you can control through the admin API is stored. It allows multiple Kong instances to share the same state of the Gateway and therefore to scale horizontally (it is actually more complex than that, but we will get there). Kong can even crash and restart without losing its configuration.
Kong’s data layer is a good way to wrap up the high level components that collaborate to handle incoming requests.
Let’s take the schema above to help us walk through the tech once again. Assuming a production setup, we start several instances of Kong for failover and redundancy. We still need a load balancer to distribute traffic among the nodes, so we also start Nginx or Haproxy in front of them.
Of course, we have APIs distributed around an operator configuring Kong to route the HTTP traffic, sent by consumers. All of this setup is stored in Cassandra or Postgres by the Gateway and every nodes share it. It makes the data store a single pont of failure so a realistic and cautious production environment will deploy them with failover capabilities.
So we went from a Kong blackbox to a composition of blocks that should give you a better sense of the underlying technologies and philosophies. In the next section we will dig deeper into how Kong (the library) uses those elements to decide what to do with incoming traffic.
Requests/response path
We saw in the previous sections the critical technological choices made and part of their rational. Something Kong brags a lot about is its real life deployment in high-intensive environments. Although relying on very concurrent and fast stacks (like NGINX), an API Gateway is a critical path by nature. It should be as reliable, as best known practices suggest, and that is why Kong’s authors implemented some high-availibility and failover mechanisms.
But before forcing users to horizontally scale their fleet of gateways (and their bill), Mashape implemented some sound optimizations to avoid unecessary computing or i/o roundtrips.
Kong uses a cache in memory to minimize access to the database. All the entities such as APIs, Plugins or Consumers are cached in memory.
Previous versions of the software used Serf as a mechanism to notify to other members of the Kong cluster that a change was done in the datastore so those nodes were able to update their caches. With older versions, it was common to use ports such as 7946 and 7373 to notify invalidation events.
Now, Kong uses a completely different strategy. All nodes perform a periodic background job to check if configuration changes have been received and processed by other Kong nodes.
This table summarizes the parameters that control how the invalidation mechanism should work:
Parameter
Default
Description
db_update_frequency
5
Controls the frequency that will use Kong to search for changes in the database. The smaller the value, the more often the different nodes will apply the configuration changes.
db_update_propagation
0
Useful when Cassandra is used as a datastore. As Cassandra is eventually consistent, that means that not all Cassandra nodes will contain the latest changes of configuration. When this parameter is set, Kong nodes will wait for db_update_propagations seconds to update their cache.
db_cache_ttl
3600
This parameter controls how long a value is stored on the cache. This mechanism acts as a safeguard to guarantee that in case a node misses an invalidation event, that node will end reading again the configuration from the database after db_cache_ttl seconds.
Summary
Running Kong internals through X-Ray gave us some useful clues. The project is a composition of battle-tested technologies, and some of them can be swapped, like the datastore, leaving some flexibility to production deployments.
Already this is something constructive for sceptical DevOps. We also had a glimpse at some performance optimization the Gorilla implements, giving us the keys to evaluate the project against business needs. The time invested in understanding what happens to HTTP requests will also shine when debugging complex network scenarios, or scaling APIs to high workloads (the very worst we can wish for a growing a business).
A lot of promises... but right now, it should help us tackle the rest of this book.