Caching

When some of your application functions takes too long to compute, a useful technique to consider is caching. Caching saves the return values of function calls, database queries, HTTP requests, and so on for future reference. The result of a function or method that is expensive to run can be cached as long as the following prerequisites are met:

• The function is deterministic and the results have the same value every time, given the same input
• The return value of the function is nondeterministic but continues to be useful and valid for some period of time

In other words, a deterministic function always returns the same result for the same set of arguments, whereas a nondeterministic function returns results that may vary in time. Caching both types of results usually greatly reduces the time of computation and allows you to save a lot of computer resources.

The most important requirement for any caching solution is a storage system that allows you to retrieve saved values significantly faster than it takes to calculate them. Good candidates for caching are usually as follows:

• Results from callables that query databases
• Results from callables that render static values, such as file content, web requests, or PDF rendering
• Results from deterministic callables that perform complex calculations
• Global mappings that keep track of values with expiration times, such as web session objects
• Results that needs to be accessed often and quickly

Another important use case for caching is when saving results from third-party APIs served over the web. This may greatly improve application performance by cutting off network latencies, but it also allows you to save money if you are billed for every request to an API.

Depending on your application architecture, the cache can be implemented in many ways and with various levels of complexity. There are many ways of providing a cache, and complex applications can use different approaches on different levels of the application's architecture stack. Sometimes, a cache may be as simple as a single global data structure (usually a dict) that is kept in the process memory. In other situations, you may want to set up a dedicated caching service that will run on carefully tailored hardware. This section will provide you with basic information on the most popular caching approaches, guiding you through some common use cases as well as the common pitfalls.

So, let's move on and see what deterministic caching is.