Using Redis to cache the API data

Redis is an in-memory database that can store key/value pairs. It best suits the use case of storing heavy read-intensive data. For example, news agencies such as the BBC and The Guardian show the latest news articles on their dashboard. Their traffic is high and, if documents are to be fetched from the database, they have to maintain a huge cluster of databases at all times.

Since the given set of news articles does not change (for hours), an agency can maintain a cache of articles. When the first customer visits the page, a copy is pulled from the DB, placed in the Redis cache, and then sent to the browser. Then, for another customer, the news agency server reads content from Redis instead of hitting the DB. Since Redis runs in the primary memory, latency is minimal. As a result, the customer sees faster page loads. The benchmarks on the web can tell us more about how efficiently a site can optimize its contents.

What if data is no longer relevant in Redis? (For example, the agency updated its top stories.) Redis provides a way to expire the keys:values stored in it. We can run a scheduler that updates Redis whenever the expiration time has passed.

Similarly, we can cache the third-party API responses for the given request (GET). We need to do this because third-party systems such as GitHub have a rate limit (telling us to be conservative). For a given GET URL, we can store the URL as a key and the Response as a value. Whenever the same request is given within the next time (before key expiration), just pull the response out of Redis instead of hitting the GitHub servers.

This method is applicable to our REST API, too. The most frequent and unchanged REST API responses can be cached in order to reduce the load on the primary database.

There is a wonderful library available for Go that can talk to Redis. It can be found at https://github.com/go-redis/redis. It is a well-known library that many developers recommend. The following diagram illustrates this concept very well:

One caveat here is the expiration of the API. A real-time API should not be cached because of its dynamic nature. Caching brings performance optimization to our plate, as well as a few headaches regarding data syncing.

Be careful while caching. Always implement a robust cache-busting method. There are many better practices available globally. Please go through them to get an understanding of the various architectures.

We'll discuss Redis in more detail in the next chapter, where we'll discuss strategies that can be used to develop asynchronous APIs.