I suffer from that paradoxical form of laziness peculiar to computer geeks where I will save myself save myself 15 minutes of work on something by spending 4 hours creating a shortcut. As such, the menus on this site are dynamically generated by traversing the category tree in PHP and laying out menus and submenus from categories and subcategories, sparing me the trouble of updating them manually as I add new content. This took some effort.

I created a shortcode that does this (well, modified a shortcode, originally from the Hierarchical HTML Sitemap plugin, by Alexandra Vovk & WP Puzzle) and soon my site was happily generating dynamic menus on the fly and keeping up with my work as I added pages, edited titles, and rearranged categories.

And, along the way, getting slower.

And slower. And slower.

Finally the other night, due to a confluence of circumstance, it took me over a minute to load the front page. “This,” I said, “has got to go.”

Fortunately WordPress includes something called “transients”, which allow you to quickly store and retrieve temporary values. I figured, the shortcode that generates the menus—as well as a number of others, such as the Recent Kwits section that appears in the sidebar of internal site pages—really don’t change that often. The menus in particular are very calculation intensive… in addition to traversing the category tree and listing the pages, they calculate the number of words on each page (you did notice that mousing over a menu item shows you how many words the page is, didn’t you?) which means not just loading and counting the page, but expanding every shortcode in the page body, to count the words they generate too. It’s a lot. It certainly doesn’t all need to be recalculated on the fly every single time any page loads. So, I thought, if I could keep track of when the last content update was that actually required them to be updated (such as changes to a page, post, category, or tag), I could otherwise store the value in a transient. Transients are not reliable, they can get wiped out occasionally, but considering that the worst that that can cause is having to recalculate them one time, it’s not a huge worry. (As a bonus, for you serious wordpress geeks, if you have an object cache installed, transients are stored in memory, rather than in the database, for even better performance.)

Long story short, that’s what I did. I used the Query Monitor plugin to time my page loads, and with various optimizations to the server I run this site on, I had gotten the time “time to first byte”—the delay for the server to calculate everything before it could start serving the front page to the browser—down to about 4-5 seconds.

After the below optimizations, time to first byte immediately dropped to a little over a second.

And there was much rejoicing.

It works like this. I’ve got a plugin I wrote, imaginatively called “MK Custom Plugins”, which contains all my custom plugins. At the top of that file is this:

/*
* Plugin Name: MK Custom Shortcodes
* Plugin URI:        https://michaelkupietz.com/plugins/the-basics/
* Description:       My custom shortcodes.
* Version:           1
* Requires at least: 5.2
* Requires PHP:      7.2
* Author:            Michael Kupietz
* Author URI:        https://michaelkupietz.com/
* License:           GPL v2 or later
* License URI:       https://www.gnu.org/licenses/gpl-2.0.html
* Update URI:        https://michaelkupietz.com/my-plugin/
* Text Domain:       mk-plugin
* Domain Path:       /languages
*/

// first, some useful functions
add_action( 'save_post', 'setLastUpdateTimestamp' );
add_action( 'create_category', 'setLastUpdateTimestamp' );
add_action( 'edited_category', 'setLastUpdateTimestamp' );
add_action( 'delete_category', 'setLastUpdateTimestamp' );
add_action( 'create_post_tag', 'setLastUpdateTimestamp', 10, 2 );
add_action( 'edited_terms', 'setLastUpdateTimestamp', 10, 2 );
add_action( 'delete_term', 'setLastUpdateTimestamp', 10, 3 );
add_action( 'trash_post', 'setLastUpdateTimestamp' );

function setLastUpdateTimestamp() {
     if ( defined( 'DOING_AUTOSAVE' ) && DOING_AUTOSAVE ) {
          return;
     }
     // If this is a revision, don't do anything.
     if ( wp_is_post_revision( $post_id ) ) {
          return;
     }

     set_transient( 'mkFuncTransient_LastUpdateCheck',microtime(true),2592000);
}

function latestPostUpdate() {
     $theDate = get_transient( 'mkFuncTransient_LastUpdate_all');
     if ( ! (isset($theDate) && $theDate > 0)) {
          $theDate=microtime(true);
          set_transient( 'mkFuncTransient_LastUpdate_all',$theDate,2592000); //dont expire for 1 month                    
          }
      }
     return $theDate;
}

function getFunctionTransient ($functionName,$arguments=[]) {
     //call with: $transientData = getFunctionTransient(__FUNCTION__,$arguments);
     if ( $transientData != null) {return $transientData;}
     //the $arguments term in the call is optional. It's only for functions that return different values based on their arguments, so you can cache and retrieve the values of different calls with different arguments.
     $data = get_transient( 'mkFuncTransient'.$functionName . hash( 'sha512',json_encode( $arguments ) ) ); //per https://daext.com/blog/improve-the-shortcode-performance-with-the-wordpress-transients-api/
     $lastUpdate = get_transient( 'mkFuncTransient_LastUpdate' . $functionName . hash( 'sha512', json_encode( $arguments ) ) );
     $lastPostUpdate = latestPostUpdate();

     if ( $data !== false and $lastUpdate < $lastPostUpdate ) {
          return $data;
     } else {
          return null;
     }
}

function setFunctionTransient ($functionName, $value = null, $arguments=[]) {
     //call this right before returning value of function with as: setFunctionTransient(__FUNCTION__,$returnValue,$arguments);
     //you can leave out returnvalue, or set it to null, to delete the transient.
     //the $arguments term in the call is optional. It's only for functions that return different values based on their arguments, so you can cache and retrieve the values of different calls with different arguments.
     if ( $value === null) {
          delete_transient( 'mkFuncTransient'.$functionName . hash( 'sha512',json_encode( $arguments ) ) );
          delete_transient( 'mkFuncTransient_LastUpdate' . $functionName . hash( 'sha512', json_encode( $arguments ) ));
          return $value;
     } else {
          set_transient( 'mkFuncTransient'.$functionName . hash( 'sha512',json_encode( $arguments ) ) , $value , 2592000); //dont expire for 1 month
          set_transient( 'mkFuncTransient_LastUpdate' . $functionName . hash( 'sha512', json_encode( $arguments ) ), latestPostUpdate(), 2592000 );
          return $value;
     }
}

[ ... the rest of my shortcode routines follow here ... ]

OK! Clear as mud, right? Here’s all you need to know. I have a menu generation shortcode that originally looked something like this:

function hierarchicalsitemap_shortcode_htmlmap($atts, $content = null) {
.... a bunch of fancy PHP to generate the menu HTML from the categories goes here ....
$out = [result of above code];
return $out;
}

add_shortcode(“htmlmap”, “hierarchicalsitemap_shortcode_htmlmap” );

So, once I had written the functions to handle transients, the only changes I had to make were to add three lines to the existing function, two right at the beginning and one before the “return” statement:

function hierarchicalsitemap_shortcode_htmlmap($atts, $content = null) {
     $origAtts=array($atts,$content);
     $data = getFunctionTransient(__FUNCTION__,  $origAtts);
     if ( $data != null) {return $data;}
[.... a bunch of fancy PHP to generate the menu HTML from the categories goes here ....]
     setFunctionTransient(__FUNCTION__, $out, $origAtts);
     return $out;
}

And that’s it. 75% faster load time, just like that.

If you inspect this page code, before the first menu up top is an invisible <LI> for reasons I can’t recall off the top of my head right this moment, but if you look at it, you will see output from the shortcode transient caching function.

If this is the very first page load since I made a site update, that LI will look something like this:

But if the menus have loaded at least once since my last update, you’ll see this:

and the page will have loaded in about a quarter of the time.

If you’re the sort of geek who likes this kind of stuff, come on over, we’ll have a beer and I’ll update the site in front of you and let you see this live. Wait, no, we should meet out in a coffee shop or something, my place is a mess.

Obviously I could get much fancier with this. No doubt recreational programmers will immediately see further optimizations.

I’m going to get this cleaned up, better documented, and posted to my github so it’s easier to understand. For, you know, some people.