Originally formalized as a specification to be approved by the World Wide Web Consortium (W3C) back in 2001, ESI was thought to be a way of stepping up to the challenge of web infrastructure scaling by applying edge computing to it. Edge computing is a method of optimizing cloud computing by doing data processing near the source of the data instead of centralizing all data processing in the datacenter. In the case of ESI, the idea was to decentralize web page content to the logical extremes of the network in order to avoid having all content requests being sent to the web server every time.
The specification called for new HTML tags that would allow HTTP cache servers to determine if certain parts of a page needed to be fetched from the original web server or if cached versions of those parts could be sent back to the client without having to query the server for it. It is possible to think of ESI as a sort of HTML include feature that is used to assemble a web page's dynamic content from different external sources.
Many HTTP cache servers started using the new markup tags. Some Content Delivery Networks (CDN), such as Akamai, and many HTTP Proxy Servers, such as Varnish, Squid and Mongrel ESI, started implementing the specification over the years, although most did not implement the entire specification. Also, some of these servers, such as Akamai, added additional features that were not in the original specification.
Moreover, important PHP frameworks, such as Symfony, started adding ESI functionality within their core configurations, thus allowing the PHP developer to immediately start thinking of ESI when developing an application.
Also, browsers started encouraging ESI usage by keeping a local cache of all files that were fetched on the web and reusing them when a different website requested the same file, for example. Thus, using a CDN-hosted JavaScript file on your website brought the advantage of reducing the number of client requests to one's web server just to get that same file over and over again.
It is very easy to start using esi:include tags within your HTML in order to cache parts of your web pages. For example, you could use it in this way:
<!DOCTYPE html>
<html>
<body>
... content ...
<!-- Cache part of the page here -->
<esi:include src="http://..." />
... content continued ...
</body>
</html>
Another example would be to use PHP and the Symfony framework to automatically generate the ESI include tags. This is easily accomplished by having Symfony trust the Varnish Cache server, enabling ESI in your YAML configuration file, setting the shared maximum age limit of the web page within its controller's method and adding the needed rendering helper methods within the corresponding templates. Let's go through these steps one at a time.
Start by having Symfony trust the Varnish Cache server. In the most recent version of Symfony, you must add a call to the static setTrustedProxies() method of the Request class. In the public/index.php file of your Symfony installation, add the following lines:
# public/index.php
[...]
$request = Request::createFromGlobals();
// Have Symfony trust your reverse proxy
Request::setTrustedProxies(
// the IP address (or range) of your proxy
['192.0.0.1', '10.0.0.0/8'],
// Trust the "Forwarded" header
Request::HEADER_FORWARDED
// or, trust *all* "X-Forwarded-*" headers
// Request::HEADER_X_FORWARDED_ALL
// or, trust headers when using AWS ELB
// Request::HEADER_X_FORWARDED_AWS_ELB
); }
[...]
Depending on the version of Symfony and the version of Varnish that you are using, you might have different steps to follow in order to do so. Please consult the following page of the Symfony documentation in order to complete this first step: https://symfony.com/doc/current/http_cache/varnish.html.
Then, add the following lines to your Symfony configuration file:
# config/packages/framework.yaml
framework:
# ...
esi: { enabled: true }
fragments: { path: /_fragment }
Once done, modify a couple of controllers like so:
# src/Controller/SomeController.php
namespace App\Controller;
...
class SomeController extends Controller
{
public function indexAction()
{
$response = $this->render('static/index.html.twig');
$response->setSharedMaxAge(600);
return $response;
}
}
And, the second one should be modified as follows:
# src/Controller/OtherController.php
namespace App\Controller;
...
class OtherController extends Controller
{
public function recentAction($maxPerPage)
{
...
$response->setSharedMaxAge(30);
return $response;
}
}
Finally, perform the following modifications within your Twig template:
{# templates/static/index.html.twig #}
{{ render_esi(controller('App\Controller\OtherController::recent', { 'maxPerPage': 5 })) }}
You should now be able to see the effects of ESI when loading the pages of your Symfony application.
In order to get a better grasp of the inner workings of ESI, let's try installing and running an HTTP reverse proxy server that partially implements the ESI specification.