Source: The Benefits of Microcaching with NGINX
What: Microcaching is a technique that caches dynamic content for a very short period of time (as little as 1 second).
So what? Dynamic data, such as content coming from a CMS (Content Management System) or direct from a database, doesn't need to be queried/fetched/calculated every time someone requests for it.
Isn't that a benefit of caching in general? Yes, but longer-term caching techniques (e.g. cache that expires after a few days or weeks) may require sophisticated cache invalidation at the right time.
Who cares? If you have content that changes frequently (e.g. news site, marketing site, forum), have many concurrent users (active users online at the same time), and you don't want to pay your hosting provider hundreds of dollars a month for a supposedly simple and straightforward site, then microcaching has its own set of benefits that long-term caches don't offer.
What benefits? Microcaching, especially with very short period of time, like 1 second, allows your site to be up to date without relying on any cache invalidation mechanisms. Of course, caching in general helps you reduce CPU usage (e.g. no unnecessary re-calculation of the same results), lower response time (i.e. no need to wait for re-calc), and handle more requests (5 with no cache to 3300 requests per second with optimized microcaching).
OK, I'm sold. How do I implement it?
The source article is from NGINX's blog, so the implementation will be using NGINX.
Just in case you are not familiar with NGINX: it's a very popular open source load balancer, web server and reverse proxy (according to their website).
In the hypothetical example, requests are handled directly by an Apache server running Wordpress. We need to have NGINX handle the traffic and act as the content cache.
So, first step is to set up a reverse proxy so NGINX proxies the client requests to the Apache server:
server {
listen external-ip:80; # External IP address
location / {
proxy_pass http://wordpress-upstreams;
}
}
upstream wordpress-upstreams {
server localhost:80;
}
I took the snippet out of the original article and remove all unnecessary (but good to have) setup for microcaching.
Next we turn on short-term caching:
Snippet directly copied from original article
proxy_cache_path /tmp/cache keys_zone=cache:10m levels=1:2 inactive=600s max_size=100m;
server {
proxy_cache cache;
proxy_cache_valid 200 1s;
# ...
}
At this point, we are already able to dramatically increase the requests-per-second our server can handle and process. Going from 5 requests per second to 600, according to the original article!
And we don't need to manually invalidate cache as we update our CMS or other content source because the cache will expire anyway after a second.
$$$
There is a small problem...
Even though we drastically increase the requests per second, the CPU usage is still 100% and processing time varies a lot.
This is because as the short-term 1-second cache expires, new incoming requests will be cache miss and will go directly through to the original source (Apache) again, until the next short-cache takes effect.
It's the gap of time between the cache being expired and refreshed.
So NGINX provides a fix for us: serve up stale (old) cache until cache gets refreshed.
This is simple to enable. Just replace the snippet before with this:
server {
proxy_cache cache;
proxy_cache_valid 200 1s;
proxy_cache_lock on;
proxy_cache_use_stale updating;
# ...
}
I reordered some stuff in this snippet from original article and corrected a typo (proxy_cache cache instead of proxy_cache one. It refers to the name of the zone).
And there you have it! Potentially 3300 requests per second with optimized microcaching.
3300, as the original article explains, is a theoretical number in an environment that doesn't have network limitations. The more realistic number is around 2200, given the network transfer rate is around 1Gbps.