I've done sysadmin type work in addition to general web development for a long time. I've built many a server stack and helped solve a lot of performance issues.
I've heard the term microcaching a lot recently. I had never heard of it for http web caches until then. If that is a new term for this process then it's a term for something people have been doing for a long time. If I just somehow missed that it was called that then ooops :p
I don't know if there is a technical definition for it specifically but essentially it's the process of caching things for unusually short lifetimes. It helps a ton on sites with specific dynamic pages but also are highly trafficked.
Twice in the last 6 months I've been contacted by clients who were in similar situations. Steadily growing and been asked by their hosting companies to upgrade because they were hitting limits or impacting other users.
When I was contacted they had upgraded in the past as well from shared to VPS hosting, one of the sites had even employed someone previously to tweak caching plugins and apache configs.
The usual big win caching items were all enabled and tweaked, in-memory caches were available on the local host. The problem was that most of those were done for only logged out users.
Logged in users often need fresher content – or the rules of caching are more complicated than is viable to offer through a plugin interface – so only minor improvements could be made by the site through the plugin.
A reverse proxy with some specially tailored caching rules was an incredible fix in areas that the plugins couldn't handle.
Even though both sites had previously been looked into by their hosts and even at one point a performance specialist somehow they missed something…
Logs showed that both sites were getting 50-75% of all their traffic on a single page type from logged in users. Looking at the pages there were obvious reasons for the high number of requests to them. Speaking with the site owners though I discovered they didn't need to be generated all the time.
There was a social network where the main group channel was the culprit for most of the server load.
The page was long, with many short messages each requiring it's own query and loop to get the data and output it to the page, new messages were posted every 5 seconds during peak times and there was no client-side loading. It was an expansive operation compared to other pages on the site.
It turns out that the group channel showed the same content regardless of who viewed it so long as they were logged in.
Short term caching of this page made a huge difference. ~85% cache hit rate on a highly dynamic page that changes roughly every 5 seconds dropped the server load to less than half what it was before.
Globally cached for all users who visit it with freshness lifetime of 5 seconds, stale pages allowed but only for up to 30 seconds.
The other site was an online store, it had a slightly different problem. The page it was getting most hits on was unique to each user and each order. It was the each customer which each customer visited directly after ordering. They would sit on the page and refresh it over and over again until status changed from pending to success.
Since this site was items with customization options available through drop shipping it took between 5-10 minutes for the order to be fully confirmed.
Refreshing between order and confirmation many times created dozens and dozens of needlessly generated pages. To make it worse the pages were also making Ajax calls to check for updated shipping info on each request. So all requests essentially = 2 requests.
Orders took between 300-600 seconds to confirm because of the drop shipping and no amount of refreshing is going to change what is returned within that window.
There was at least 50 times the requests to that page than there were unique sales.
Unique cached item for each user with 60 second freshness, allow stale for up to 90 seconds.
Microcaching is a thing. It brings big wins on highly dynamic pages with lots of requests per second.
Ultimately in these 2 specific cases a single rule was able to drop server load by more than half and prevent the need for these companies to upgrade to more expensive hosting plans.
Bonus: Users get much pages must faster.