WP_Bootstrap_Navawalker Updates and Unit Testing PHP

Covering days 16 through 20 and culminating in 1 month and 1/5th of the way through 100 days.

This week I had spent a substantial portion of my time working on a project I've supported for last 5+ years – the WP Bootstrap Navwalker.

I've used the walker since version 2 of Bootstrap and been a contributer all of BS3 lifespan. With Bootstrap 4 finally in a stable release (after around 3 years in development!) and I have been working through a substantial rework of the entire navwalker.

I've focused on a few things through the rebuild.

  • Simplifying the codebase and improving readability.
  • Better link modifier (disabled, header, divider) and icon handling.
  • Handling long standing issues that were difficult to fix or required a large rework in the previous iteration.

Simplifying The Codebase And Improving Readability

To simplify things I have focused on a few things.

Too many paths!

Reducing the number of possible paths in the start_lvl function has been the first focus. This function has grown to contain lots of conditionals that result in different processing or outputs. At one point in Scrutinizer I seen it say that this function had over 20000 paths!

Lots of paths result in code that becomes hard to follow so to help improve that I decided to externalize some parts of the code into their own functions.

Prime candidates were some decision logic that didn't need to be cluttering up the main thread. A single call with a well named function does a better job of explaining what is happening than the large conditional code blocks – even through those codeblocks are well commented.

Better Link Modifier and Icon Handling

Bootstrap navigation has always required the use of specific classes on the container, the link and the wrapper elements for the dropdown elements to function. WordPress provides a way to set classes for individual nav items in the Menu Editor UI.

In previous versions the Title Attribute input was used to handle special classes for modifiers and icons.

The complexity here is that most of the time Walkers add the classes to the wrapper element – in this walker that is an <li> – however some of the time you need those classes to get applied directly to the link.

Icon classes need to be placed separately from the wrapper and the link. They are added to their own <i> element with aria-hidden="true" so that screen readers don't try read an icon character.

$icon_classes = array(); // This array would contain valid icon classes.

// Join any icon classes plucked from $classes into a string.
$icon_class_string = join( ' ', $icon_classes );

/**
 * Initiate empty icon var, then if we have a string containing any
 * icon classes form the icon markup with an <i> element. This is
 * output inside of the item before the $title (the link text).
 */
$icon_html = '';
if ( ! empty( $icon_class_string ) ) {
	// append an <i> with the icon classes to what is output before links.
	$icon_html = '<i class="' . esc_attr( $icon_class_string ) . '" aria-hidden="true"></i> ';
}

To make this happen I needed to loop through arrays of classnames and strip specific classes from the main array and save them in a new array for later use. The $icon_classes array is filled with that function.

I decided to handle link modifiers and icon classes separately as the link modifiers are used for setting a typeflag for decision making later in the execution.

/**
 * Find any custom linkmod or icon classes and store in their holder
 * arrays then remove them from the main classes array.
 *
 * Supported linkmods: .disabled, .dropdown-header, .dropdown-divider
 * Supported iconsets: Font Awesome 4/5, Glypicons
 *
 * NOTE: This accepts the linkmod and icon arrays by reference.
 *
 * @since 4.0.0
 *
 * @param array   $classes      an array of classes currently assigned to the item.
 * @param array   $link_classes an array to hold linkmod classes. Passed by reference.
 * @param array   $icon_classes an array to hold icon classes. Passed by reference.
 * @param integer $depth        an integer holding current depth level.
 *
 * @return array  $classes      a maybe modified array of classnames.
 */
function seporate_linkmods_and_icons_from_classes( $classes, &$linkmod_classes, &$icon_classes, $depth ) {
	// Loop through $classes array to find linkmod or icon classes.
	foreach ( $classes as $key => $class ) {
		// If any special classes are found, store the class in it's
		// holder array and and unset the item from $classes.
		if ( preg_match( '/disabled/', $class ) ) {
			// Test for .disabled.
			$linkmod_classes[] = $class;
			unset( $classes[ $key ] );
		} elseif ( preg_match( '/dropdown-header|dropdown-divider/', $class ) && $depth > 0 ) {
			// Test for .dropdown-header or .dropdown-divider and a
			// depth greater than 0 - IE inside a dropdown.
			$linkmod_classes[] = $class;
			unset( $classes[ $key ] );
		} elseif ( preg_match( '/fa-(\S*)?|fas(\s?)|fa(\s?)/', $class ) ) {
			// Font Awesome.
			$icon_classes[] = $class;
			unset( $classes[ $key ] );
		} elseif ( preg_match( '/glyphicons-(\S*)?|glyphicons(\s?)/', $class ) ) {
			// Glyphicons.
			$icon_classes[] = $class;
			unset( $classes[ $key ] );
		}
	}

	return $classes;
}

Unit Testing My Code

When tests are successful and all is green 🙂

While I was going through the code and dealing with long standing issues I also took care of an issue which was opened not long ago on the early dev version of the v4 walker. The issue explained that the fallback method was broken and outputting nothing when it should have been outputting something.

The fix was easy – I had mistakenly used assign instead of append so swapping the operator on all of the markup to be output made it work.

// incorrect use:
$output = 'somehtmlcontent';
// fixed:
$output .= 'somehtmlcontent';

Once I'd fixed it I realised that the project has Unit Tests that can be run locally and are run by Travis but all they do is check that the file, class and methods exist. Those tests could be a lot more useful.

If there was a test in place that made sure the fallback output when it was supposed to and didn't when it wasn't I would have caught this mistake early and fixed it quick. Instead it escaped me till someone else pointed it out.

Testing That Functions Produce The Output They Should

The project has PHPUnit already available so all I had to do was add some tests that could check the output.

This fallback function should produce 2 different types of output. For a logged out user it should output nothing. If a user is logged in with edit_theme_options capability it should output markup with a link. There's also a flag that determines if it echos or returns.

I wrote some tests that do a few things:

  • Test that logged out users get empty strings and that both 'echo' and 'return' are the same.
  • Test that logged in users get html that looks to be valid and that 'echo' and 'return' are both the same.

When testing output of a function there are a few methods that PHPUnit has available. Those methods are useful but somewhat limited so many times – for output that may be complicated or subject to slight changes over time – you will want to check for traits of the valid output rather than the exact output to save you reworking tests all the time. PHPUnit already runs tests inside of a content buffer so all you need to do is get your output with ob_get_contents;.

One thing that got me stuck for a few minutes was how exactly could I get logged in output when this is run in CLI mode where no user is set. Turns out it's quite easy. Make a new user with the capability we need – I made an admin that does have all core capabilities – and set them to the current user.

// make an admin user and set it to be the current user.
$user_id = $this->factory->user->create( array( 'role' => 'administrator' ) );
wp_set_current_user( $user_id );

The full pair of test ended up coming out like this:

/**
 * Test Fallback method output for logged out users.
 *
 * Expects that for logged out users both echo and return requests should
 * produce empty strings.
 */
function test_fallback_function_output_loggedout() {

	// default is to echo reults, buffer.
	ob_start();
	WP_Bootstrap_Navwalker::fallback( $this->valid_sample_fallback_args );
	$fallback_output_echo = ob_get_clean();

	// empty string expected when not logged in.
	$this->assertEmpty(
		$fallback_output_echo,
		'Fallback output for logged out user created a non-empty string in echo mode.'
	);

	// set 'echo' to false and request the markup returned.
	$fallback_output_return = WP_Bootstrap_Navwalker::fallback( array_merge( $this->valid_sample_fallback_args, array(
		'echo' => false,
	) ) );

	// return and echo should result in the same values (both empty).
	$this->assertEquals(
		$fallback_output_echo,
		$fallback_output_return,
		'Fallback output for logged out user created a non-empty string in return mode.'
	);
}

/**
 * Test Fallback method output for logged in users.
 *
 * Expects strings to be produced with html markup and that they match when
 * requesting either a return or defaulting to echo.
 */
function test_fallback_function_output_loggedin() {

	// make an admin user and set it to be the current user.
	$user_id = $this->factory->user->create( array( 'role' => 'administrator' ) );
	wp_set_current_user( $user_id );

	// default is to echo results, buffer.
	ob_start();
	WP_Bootstrap_Navwalker::fallback( $this->valid_sample_fallback_args );
	$fallback_output_echo = ob_get_clean();

	// rudimentary content test - confirm it opens a div with 2 expected
	// values and ends by closing a div.
	$match = ( preg_match('/^(<div id="a_container_id" class="a_container_class">)(.*?)(<\/div>)$/', $fallback_output_echo ) ) ? true : false;
	$this->assertTrue(
		$match,
		'Fallback method seems to create unexpected html for logged in users in echo mode.'
	);

	// set 'echo' to false and request the markup returned.
	$fallback_output_return = WP_Bootstrap_Navwalker::fallback( array_merge( $this->valid_sample_fallback_args, array(
		'echo' => false,
	) ) );

	// return and echo should both produce the same strings.
	$this->assertEquals(
		$fallback_output_echo,
		$fallback_output_return,
		'Fallback method seems to create unexpected html for logged in users in return mode.'
	);
}

Testing Private Methods of a Class

When I was writing other tests I needed to call private methods and test their output. Private methods are not intended to be called from outside of the function so to make that happen we need to use a technique called Reflection.

We use a reflector to duplicate the class and methods we want and then set the methods to be publicly accessible from the reflector.

$wp_bootstrap_navwalker = $this->walker;

// since we're working with private methods we need to use a reflector.
$reflector = new ReflectionClass( 'WP_Bootstrap_Navwalker' );

// get a reflected method for the opener function and set to public.
$method = $reflector->getMethod( 'linkmod_element_open' );
$method->setAccessible( true );

Once you have the reflected method setup and accessible you can then use invokeArgs on the $method – passing the real class object followed by an array containing the args to pass to it.

// test openers for headers and dividers.
$header = $method->invokeArgs( $wp_bootstrap_navwalker, array( $this->valid_linkmod_typeflags[0], 'stringOfAttributes' ) );
$this->assertNotEmpty( $header, 'Got empty string for opener of ' . $this->valid_linkmod_typeflags[0] );

Using Some ES6 JavaScript

This is something I worked on as part of my 100 days of code challenge during week 3.

I invested a bit of development time into a project I've been thinking of for a long time but left it neglected at the beginning stages.

It's a Javascript split-testing library.

I'm perfectly capable of reading and writing JS code with plenty of real world experience working with it. It's not one of my strongest skills though so I took this as an opportunity to improve and learn some more things. 

Javascript and ES6

When people say modern Javascript what they often mean is ES6 Javascript. The 6th edition was finalized in 2015 and is called ECMAScript 2015.

Sadly even though it's been around for some time browser implementation was slow to start and there is no backporting being done in many cases for older browser versions.

Modern browsers support ES6 code fairly well. Chrome, Opera, Safari, Firefox and even Edge have good levels of support – older browser versions do not and there's essentially no support for it on any version of IE.

Developers want to take advantage of the newer features that ES6 offers. They write ES6 code then compile it to ES5 compatible code – systems like babel make that easy. I wanted to take advantage of the new paradgrims too..

New Methods and Features in ES6 I Was Excited About

Deciding to go full ES6 in this project allowed me to take advantage of some features that I had previously only made minimal or no use of. There are 3 things I discovered in ES6 that made writing JavaScript a vastly more pleasant experience.

  • Native Classes to build with.
  • Variable and Function scoping inside blocks with let.
  • Default value assignments for function declarations.

I'm excited to work with those features all the time. Classes especially.

In ES5 you could use variables with objects, custom prototypes and anonymous functions to get similar Classes-like behaviors. To me that method always seemed to create very convoluted and hard to read code so native Classes are very welcome.

There is a couple of other very nice additions that I'm looking forward to using in future and in code refactors.

  • Rest and Spread Operators – ...variable.
  • Template Literals with raw string access.
  • Constants – Immutable Variables
  • String Searching.

Doing Things the ES6 Way

There are a few things in this project that I done differently to in the past and in ES6 they are much cleaner.

Merging Objects

While working I wanted to merge some objects. Previously I'd done that with a custom function that iterates through all prototypes of an object and merged them into another object. ES6 has a method for doing just that in a single line.

let defaults = {the defaults};
/**
 * Merge 2 (or several) objects together.
 *
 * This merges myObject into the object containing the defaults.
 * and returns the updated destination object only.
 */
this.myObject = Object.assign( defaults, this.myObject );

Better Sub-String Methods

A lot of work takes place with strings and string manipulation. In my latest project I had the pleasure of searching for values inside strings using much cleaner methods.

let myString = 'a string';
// there's a `startsWith` method now and it includes an index spot. This returns true because we start at index 4. There is also an `endsWith` that behaves similarly.
if ( myString.startsWith( 'ring', 4 ) ) {
    console.log( 'found "ring" at end of ' + myString );
}
// my favorite is `includes`. Previously I would have used indexOf or a regex for complex strings for this.
if ( myString.includes( 'tri' ) {
    console.log( 'Found "tri" inside of . + myString );
}

Colorize Terminal Output

This is part 2 of the 2nd week of my 100 Days of Code Challenge.

I'm somewhat of a plain styles kind of person. Color conveys a meaning that I don't often care for when it comes to things I create for myself. A white background with black for me is perfectly ok in most cases. When I'm particularly adventurous I may opt for a dark mode.

This week I decided that I'd add some color to some of my scripts. Since I was working on the backup script I decided to put some color into the info that the script output. It outputs simple pieces of text with echo just like this:

echo -e 'files archived'
echo -e 'starting db dump'

Different terminals use different colorization methods and I've colorized text a few times by Googling to find out how to make it happen. This time I done the same lol.

When I looked up the codes this time I got to a page with a very thorough breakdown of what can be done and how. It had this note at the top.

In Bash, the <Esc> character can be obtained with the following syntaxes:

  • \e
  • \033
  • \x1B

Seeing the list of escape characters reminded me of a time previously when I had added color in a python script. It output color in my terminal and in a terminal running on a Windows 10 machine. At the time I thought nothing about it.

Suddenly realizing that the color codes must be somewhat standardized I decided to look more closely. I discovered the codes come from a system called ANSI/VT100. A lot of terminals support it in some way or another. Windows added support for it in version 10. Prior to windows 10 there was no support.

Escape Color Sequences

Colorizing terminal output is done by using an escape sequence followed by a color code and then closed. Reset follows the same method so the codes end up looking something like these.

\e[1m # bold
\e[7m # inverted
\e[0m # reset

# OR with different escape sequences that bash understands. 

\033[1m # bold
\x1B[7m # inverted
\e[0m # reset

I have no idea exactly what the escape sequences that windows terminal supports however with my python script these were the colors I had used with the \033 escape sequence.

# http://stackoverflow.com/a/287944/2375493
class tcolors:
    HEADER = '\033[95m'
    BLUE = '\033[94m'
    GREEN = '\033[92m'
    WARNING = '\033[93m'
    FAIL = '\033[91m'
    ENDC = '\033[0m'
    BOLD = '\033[1m'
    UNDERLINE = '\033[4m'

You can see a large list of color codes and support for them on this page.

Color Escaped Text in a Bash Script

I decided to invert the colors of some of the info messages to make them stand out from the other messages that may scroll past. For a success messge I also turned the text green as well as doing the invert.

Those echo commands end up starting with the color code, outputting the text and then end with a reset code.

echo -e '\e[7m\e[92mfiles archived\e[0m'
echo -e '\e[7mstarting DB dump\e[0m'

Of course helper text and colorization is only useful in a manual run of this script. Since it's primary purpose is to be run as a cron job this info is going to be put to /dev/null. But I wanted to learn how it's done properly so that's what I did.

Backup WordPress and MySQL Docker Containers

As part of my 100 Days of Code Challenge this is something I worked on in week 2.

The start of this week with the sniffles as my wife calls it. I had the cold and I spent a lot of time reading on my phone, not a lot of time working unfortunately.

One of the things I did work on in that downtime was improving a backups setup that I had been using.

I wrote a script to run through each of the sites I wanted backed up and while writing it I decided I'd put a little color in it for a change to make it easier to see the status of the execution.

I run a lot of WordPress sites using docker and multi-container setups. It's mainly based on similar ideas I wrote about when I first put WordPress Containers into production.

What Needs Backup

There are 2 things we need to back up for WordPress sites. There's the site files and there's the database. Both of those are run inside of different containers.

  1. The Web Server Container – Apache with PHP and Memcached installed.
  2. The Database Container – Mariadb.

The content of these sites isn't updated often – many of them are essentially static at this point. Slow weekly/monthly backups are enough to secure what they store.

All of the sites and their databases get included in different system level backups at different times of the month. It's not ideal but it's been enough so far.

In a more tailored system the file and database backups would occur more regularly – and be more targeted towards each individual site.

Backup WordPress Docker Container Files

The container that runs the WordPress files holds them inside of a directory on the host that is mounted inside of the the container. Technically – and in the default home directory backups I run as part of a different system – they can be backed up from the host by backing up those directories. 

I wanted to perform the backup without relying on the host system. Instead I came up with this one liner to fire up a different container that's using the ubuntu image and archive the files through it for the backup. It's a little complex but I'll explain each bit below.

docker run --rm --volumes-from pattonwebzinfo_wordpress_1 -v $(pwd)/backup:/backup/ ubuntu tar -cvf /backup/pattonwebz_info.tar /var/www/
  • docker run --rm – Run a container that will be removed when it exits.
  • --volumes-from pattonwebz_info_wordpress_1 – Mount the volumes FROM another container INTO this container too.
  • -v $(pwd)/backup:/backup/ create a volume that's found at [current working directory]/backup on the host and make it accessible inside the container from /backup/
  • ubuntu – the container we're using is a base ubuntu instance which will have the command we need available.
  • tar -cvf /backup/pattonwebz_info.tar /var/www/ – using tar command we backup the /var/www/ directory (which is a directory mounted from the wordpress container) and store the archive inside of this containers mounted volume – backup.

Backup a Mysql Database Inside of A Docker Container

To backup the database I considered spinning up a different mysql container and mounting the directories inside of it in a read-only mode to prevent from 2 systems accessing the same set of files.

I worried that file locking might be a problem with this and it seemed like a custom image may be needed – or for custom configs to be written in advance and mounted into the db backup container.

There was also authentication issues to deal with. I don't want to be bundling credentials into scripts if I don't need to.

I opted to take a different approach and use docker exec to execute a mysqldump command inside of the container and redirect the output to a file inside of the host filesystem. Inside of [current working directory]/backup/wp_database.sql– this puts it right next to the sites files archive.

docker exec pattonwebzinfo_mysql_1 sh -c 'exec mysqldump -u"$MYSQL_USER" -p"$MYSQL_PASSWORD" "$MYSQL_DATABASE"' > $(pwd)/backup/wp_database.sql

The database name, user and password are available inside the container as environment variables so can be used easily.

Script it up to make it cycle through all the containers I need to backup and then trigger that script via cron. Targeted scheduled backups for WordPress docker containers.

100 Days of Code Challenge

Week 1 – Day 1-5

For the first week of my 100 Days of Code challenge I didn't go too far out of my way to work with code. It's part of my job, I do it day-to-day.

I spent some time looking for a code bug to discover it was simple typesetting issue. I learned, once again, the value of explicit typesetting and the pitfalls of allowing an operation to do it's own type juggling.

When you're in need of specific operations or types to be the outcome it's best to strictly define those types instead of allowing the system to do it for you.

Another thing that I was amazed by is how much can be done, in a far more efficient manner, when you spend time only in the command line and semi-automate tasks.

There are times when it's far more time effective and less error prone to do that over manually processing data and verifying the results of your work.

Juggling & Defensive Typecasting

Be defensive when it comes to casting variables to the types you expect when data comes from somewhere you're not in control of. Sometimes it's what you expect – sometimes it's not.

William Patton

Wait… What type is this??

Strict typesetting is a good idea when working with variable types in a loosely declared language because sometimes types are juggled in a way that you don't expect and when you've got a type you're not expecting your results may not be what you're expecting as well.

Example of this happening with strings and numbers in JavaScript:

// Data comes from an object.
var data = {
    'id': '1452',
}
// Try increment the ID number by 5.
let new_id = data.id + 5;
/**
 * Instead of incrementing the starting data was a string 
 * and appended our number to the end like a string concat.
 */
console.log( new_id ); // new_id = 14525
/**
 * What we really wanted to happen was to increment the number
 * by 5.
 */
let new_id = parseInt( data.id ) + 5; // parsed the `id` and made sure it was int.
console.log( new_id ); // new_id = 1457

There is also a different operator specifically made for doing type conversions from string to number in javascript and is written in a more easily flowing manner. Both methods create the same output when successful.

var data = {
    'id': '1452',
}
let new_id = data.id;
new_id = + 5;
console.log( new_id ); // new_id = 1457

Command Line Automation

Working in command line and creating automation scripts beats manual tagging and sorting for large datasets.

Every. Single. Time.

William Patton

With a directory of .html files to have articles exported and a database of thousands of posts already imported: Find what files were not already converted and imported to the site as articles.

  • The list of content that was intended to include came from thousands of .html files – in various directories arranged mostly by year.
  • Using mysql queries to extract data about articles then filter it so it had only a value easily compared against the files needing import – that value will be a match with the filename that's to be imported.
  • With the data use BASH with some loops and a direct comparison logic to find matches.
  • Use git diff reporting to determine what posts were missing from the list of imported content.

BASH Scripts That Loop and Compare

At the heart of things were bash scripts that mostly looped within a loop.

Iterate through lines of a file while looping through lines of another file then output the results to different file.

It's a blunt object approach. A lot of overheads, not at all efficient in operations.

Regardless of inefficiencies it didn't take long to run through. It was simple and incredibly effective to highlight the differences between items present and items expected.



I called the script it match.sh then ran it passing 3 .txt files. One filled with the files that are expected to be in the list, another filled with the list of articles present for this category and the 3rd file to store the results.

./match.sh filename-list.txt metavalue-present.txt matched.txt 
#!/bin/bash

echo "Matching $1 inside of $2 and saving to $3"
echo "Do the above values look correct?"
sleep 5s

# loop through all the lines in the first file passed.
while IFS='' read -r line || [[ -n "$line" ]]; do
	# loop through all the lines in the second file passed.
	while IFS='' read -r linein || [[ -n "$linein" ]]; do
		# if line from f2 matches the line from f1...
		if [[ $linein == *$line* ]]; then
			# echo line 1 (filename) to a file.
			echo "$line" >> ./$3
			# we found a match, break from loop 2.
			break
		fi
	# loop 2 is passed contents of file passed as arg2.
	done < "$2"
# loop 1 is passed the contents of file passed as arg1.
done < "$1"

The next step was a repeat of this except to compare the list of expected items against the list of matches and save that to new file too.

./match.sh filename-list.txt matched.txt missing.txt 

The final step was to use git diff to compare the differences between the matched items and the original list of items expected.

One thing that helped here to improve processing speed by several minutes per run was to split the large groups of data into smaller, more manageable, sets or categories and then handle comparing them in them in batches. Since my compare method was so inefficient this helped a lot.

A 100 Days of Code Challenge

I've been mulling over this idea for a long time. 100 days of code. The original idea is essentially spend 1 hour a day coding, document the progress and share on code platforms or on social.

On the surface that sounds easy as I spend almost every single day with code in some form or another. Generally it's an hour or more.

The fact is that it's difficult to have 100 consecutive days to work on an idea without interruptions. Even if it's just 1 hour a day – sometimes you might just not have it. There's work and family life to prioritize.

30 days ok, 60 maybe. 100 full days in a row working directly with code and no deviations: That's more of a challenge than I can commit to.

An Outline of a More Forgiving 100 Days of Code Challenge

It's not just about writing code – there is plenty of value to be found in working with code it in other ways.

It is not 100 consecutive days and you never need to commit 7 days a week.

Improving documentation for code or reading and reviewing someone else's code are both high value tasks in some cases.

  • It's 100 working days, no minimum per day.
  • Code review or other improvements are included and highly encouraged.
  • Documentation isn't necessary everyday – once a week is a good target.
  • Daily tracking isn't required – track ideas or projects over the space of a week or more. 

The 100 days could start from any point. I've decided to start January 1st.

  • Between January 1st and June 1st there are a little over 100 working days.
  • That's 5 days in a week – or every day minus weekends and major holidays. Spread over around 5 months of time.

I will be documenting some things I work on each week, write some example code and schedule posts to publish throughout the span of the challenge.

Documentation Format Suggestions

Sharing of knowledge in the programming and development world is done in a few formats. Most obviously the sharing of code. That's a great way let someone see the work that's been done or methods that were used.

Another common way to share is via articles, blog posts and books. This is useful for sharing overall process, detailing setup steps or explaining some thought processes behind what's been done.

  • Anything that can be shared in a blogpost. Text, screenshots or examples could be included here.
  • Links to code sharing platforms like a GitHub repo or a Codepen.
  • Social shares like tweets.

Consider helping someone to solve a coding problem as being a valid exercise as well. Working through an issue tracker like in GitHub repo or similar counts. StackOverflow is a great place to offer assistance and to share what you know with someone in need of help. 

Important Notes

With an emphasis on working with code there's some things I think will be relatively important to keep in mind throughout the 100 days. It's about working with code not necessarily writing code all the time.

  • Looking at and learning from some code is a valid exercise.
  • Adding inline docblocks or improving existing comments is valid. 
  • Participating in an open source project of any kind will be valid.

Remember if it's working with code then it can be considered part of the challenge. Some other things to note throughout the challenge:

  • Sharing of example code is always good, sharing of full source when possible is encouraged too.
  • Creation of actual code documentation – either external documentation or inline code doc block addition or improvements is a very big part of writing code that is for sharing. 

Two Cases Where Frontend Microcaching Saved The Day

I've done sysadmin type work in addition to general web development for a long time. I've built many a server stack and helped solve a lot of performance issues.

I've heard the term microcaching a lot recently. I had never heard of it for http web caches until then. If that is a new term for this process then it's a term for something people have been doing for a long time. If I just somehow missed that it was called that then ooops :p

What is Microcaching?

I don't know if there is a technical definition for it specifically but essentially it's the process of caching things for unusually short lifetimes. It helps a ton on sites with specific dynamic pages but also are highly trafficked.

The Semi-Unusual Cases Recently Where Microcaching Was A Big Win

Twice in the last 6 months I've been contacted by clients who were in similar situations. Steadily growing and been asked by their hosting companies to upgrade because they were hitting limits or impacting other users.

When I was contacted they had upgraded in the past as well from shared to VPS hosting, one of the sites had even employed someone previously to tweak caching plugins and apache configs.

The usual big win caching items were all enabled and tweaked, in-memory caches were available on the local host. The problem was that most of those were done for only logged out users.

Logged in users often need fresher content – or the rules of caching are more complicated than is viable to offer through a plugin interface – so only minor improvements could be made by the site through the plugin.

A reverse proxy with some specially tailored caching rules was an incredible fix in areas that the plugins couldn't handle.

Even though both sites had previously been looked into by their hosts and even at one point a performance specialist somehow they missed something…

Logs showed that both sites were getting 50-75% of all their traffic on a single page type from logged in users. Looking at the pages there were obvious reasons for the high number of requests to them. Speaking with the site owners though I discovered they didn't need to be generated all the time. 

Group Social Freshness

There was a social network where the main group channel was the culprit for most of the server load. 

The page was long, with many short messages each requiring it's own query and loop to get the data and output it to the page, new messages were posted every 5 seconds during peak times and there was no client-side loading. It was an expansive operation compared to other pages on the site. 

It turns out that the group channel showed the same content regardless of who viewed it so long as they were logged in. 

Short term caching of this page made a huge difference. ~85% cache hit rate on a highly dynamic page that changes roughly every 5 seconds dropped the server load to less than half what it was before.

Globally cached for all users who visit it with freshness lifetime of 5 seconds, stale pages allowed but only for up to 30 seconds.

Online Store Confirmation/Delivery Details

The other site was an online store, it had a slightly different problem. The page it was getting most hits on was unique to each user and each order. It was the each customer which each customer visited directly after ordering. They would sit on the page and refresh it over and over again until status changed from pending to success.

Since this site was items with customization options available through drop shipping it took between 5-10 minutes for the order to be fully confirmed.

Refreshing between order and confirmation many times created dozens and dozens of needlessly generated pages. To make it worse the pages were also making Ajax calls to check for updated shipping info on each request. So all requests essentially = 2 requests.

Orders took between 300-600 seconds to confirm because of the drop shipping and no amount of refreshing is going to change what is returned within that window. 

There was at least 50 times the requests to that page than there were unique sales.

Unique cached item for each user with 60 second freshness, allow stale for up to 90 seconds.

Takeaway: Win-Win

Microcaching is a thing. It brings big wins on highly dynamic pages with lots of requests per second.

Ultimately in these 2 specific cases a single rule was able to drop server load by more than half and prevent the need for these companies to upgrade to more expensive hosting plans.

Bonus: Users get much pages must faster.

Correct Way To Create WordPress Theme Frontpage Templates

People can make several versions of the exact same front page and for them to be accessible at several different of URLs.

Not good for Theme users, Site Visitors or for SEO

Making a page template (with a file containing an opening comment something like: Template Name: Front Page) coded specifically for use as the frontpage is the wrong way to do it. People can make several versions of the exact same front page and for them to be accessible at several different of URLs. Not good.

WordPress has specially designated templates for use as the front page and as the blog page. They are front-page.php and home.php respectively. If these templates are in place and static page is defined then they will be used automatically.

Open Source Projects That I Rely On To Effectively Do My Job – Part 3

There are a number of things that exist in the open source world without which I do no think I could do my Job. I am a Web Developer. I work on a range of projects using different systems, languages and processes. I work a lot with WordPress as well.

Many aspects of my work revolve around scanning logs, writing and reading code in a text editor and browsing the internet. I have my prefered programs for doing each of those tasks.

This is a set of articles that look at a lof of the open source projects that I rely on to do my job and do it effectively.

Online Applications

Some of the tools I use are online services or applications. In the open source world people build things and they share them. Since I am in the web developer sphere that means a lot of the circles I am in people build online software.

Online software is convenient because they are more portable and often accessible from a variety of devices. A lot of online services are powered by open source software (and that's not counting the unlaying OS or the fact that it probably uses Apache or NGINX to respond to people's browsers).

WordPress

A lot of the work that I do relates back to WordPress in some way. It powers a huge amount of the publicly accessible internet. Sometimes I build for WP or extend it, other times I build things to work alongside it. Sometimes I just build server stacks capable of running it.

If WordPress was closed source, or did not exist, a god portion of my work would not come in.

GitHub – And Git

GitHub is a giant when it comes to source code management. GitHub manages code using an unlaying software called git. That software was started by the same man who started the linux kernel.

GitHub itself is not an open source application. I can't download a copy of it and run my own private version of it (but you can have private instances setup and managed by them, either hosted in the cloud or in-house). It is powered by an open software and also values open source greatly. Most projects hosted there are under some kind of open licence.

Other Online Git Services – BitBucket, GitLab

There are other repo hosts available. Bitbucket is a good choice. GitLab is also a good choice.

GitLab is an online service where you can host your code as well but it's an open software too. You can download it to run on your own server managed by yourself. It is extremely full featured – offering much of the same as github and bitbucket – as well as a lot of integrated CI and tooling.

Communications – Slack

Even talking to yourself can be useful at times, communication is better when more people can be involved and the conversations can be archived and searched. Slack lets that happen. It's actually not an open source project as such but a tool for communication that isn't email is essential when working online with others. 

Conversations happen in Chat Rooms. Slack provides nice rooms to have those conversations.

Developer Log: Best-Reloaded Theme Update

I've been thinking that I should write some development logs for some work that I do because it may be useful for others. Plus it gets me to writing more which is something I'm trying my hardest to make a habit.

This log is about some updates I'm making to a theme I have hosted in the .org theme repo.

Best Reloaded in the WordPress Theme Directory

Bootstrap Header Nav Improvements

This theme uses Bootstrap 4 for a framework. It has a top navigation bar with a menu using the navwalker class that I help maintain. It also has a search bar and is styled with a custom theme specific colored button.

The search bar is always on, first I want to make it possible to turn it off if users do not want it. Then I plan to offer color choice selections.

Adding an on/off toggle to theme options. This is easy. A checkbox in the customizer and a test for it's value at page generation.

A Checkbox On/Off Toggle In Customizer

Start with adding a section for the header nav options.

$wp_customize->add_section( 'best_reloaded_navbar', array(
	'title' => __( 'Header Navbar', 'best-reloaded' ),
	'priority' => 100,
) );

Then add a control and a setting for the checkbox.

$wp_customize->add_setting( 'display_navbar_search', array(
	'default' => 1,
	'sanitize_callback' => 'best_reloaded_sanitize_checkbox',
) );
$wp_customize->add_control( 'display_navbar_search', array(
	'label' => __( 'Toggle on/off the navbar search form. Checked = on', 'best-reloaded' ),
	'section' => 'best_reloaded_navbar',
	'settings' => 'display_navbar_search',
	'type' => 'checkbox',
) );

This uses a custom sanitization callback that simply checks value is either 1 or 0 – TRUE or FALSE.

/**
 * Sanitization for checkbox input
 *
 * @param booleen $input	we either have a value or it's empty to depeict
 *                       	a checkbox state.
 * @return booleen $output
 */
function best_reloaded_sanitize_checkbox( $input ) {
	if ( $input ) {
		$output = true;
	} else {
		$output = false;
	}
	return $output;
}

The final part of this is testing the value of the option and outputting the search form when it's set to 'on'.

// if the navbar search is on then output search form.
if ( get_theme_mod( 'display_navbar_search', true ) ) {
	get_search_form();
}

This is a screenshot of it in action in the customizer.

Navbar Brand Options

Next thing I was wanting to add was the ability to add a small branding icon to the navbar. Bootstrap has some styles and classes that allow this so let's look at what we need for this.

  • An On/Off toggle for navbar brand.
  • Option to select an image from media library
  • Checkbox to include the site title as text.

This time there are 3 options to add to the customizer. It's 2 checkboxes again and another for image upload. Sanitization for an image upload is a little different than with checkboxes.

Sanitizing Values With Image Uploads

When it comes to sanitizing the values from image uploads what you are actually working with is text strings. Urls in fact.

You get a string with the url to the file. First you want to check that you have a valid extension for the file it points to. WP has a function to do this – wp_check_filetype()

Once you're sure it's the right filetype then you can escape it as a url at return.

/**
 * Santization for image uploads.
 *
 * @param  string $input	This should be a direct url to an image file..
 * @return string          	Return an excaped url to a file.
 */
function best_reloaded_sanitize_image( $input ) {

	// allowed file types.
	$mimes = array(
		'jpg|jpeg|jpe' => 'image/jpeg',
		'gif'          => 'image/gif',
		'png'          => 'image/png',
	);

	// check file type from file name.
	$file_ext = wp_check_filetype( $input, $mimes );
	// if filetype matches the allowed types set above then cast to output,
	// otherwise pass empty string.
	$output = ( $file_ext['ext'] ? $input : '' );
	// if file has a valid mime type return it as valud url.
	return esc_url_raw( $output );
}

Controls and Settings for Branding Options and Image Upload

There's 3 sets of controls and settings here for each of the options we need set above. The most complicated one is the image upload control as it's building it's control from the class of a core control. It's a little more complicated to look at but works essentially the same.

// on/off toggle.
$wp_customize->add_setting( 'display_navbar_brand', array(
	'default' => 0,
	'sanitize_callback' => 'best_reloaded_sanitize_checkbox',
) );
$wp_customize->add_control( 'display_navbar_brand', array(
	'label' => __( 'Enable the navbar branding options which can be a small image and the site-title.', 'best-reloaded' ),
	'section' => 'best_reloaded_navbar',
	'settings' => 'display_navbar_brand',
	'type' => 'checkbox',
) );

// brand image.
$wp_customize->add_setting( 'brand_image', array(
	'default' => '',
	'sanitize_callback' => 'best_reloaded_sanitize_image',
) );
$wp_customize->add_control(
	new WP_Customize_Image_Control(
		$wp_customize,
		'brand_image',
		array(
			'label'      => __( 'Add a brand image to the navbar.', 'best-reloaded' ),
			'section'    => 'best_reloaded_navbar',
			'settings'   => 'brand_image',
			'description' => __( 'Choose an image to use for brancd image in navbar. Leave empty for no image.', 'best-reloaded' ),
		)
	)
);

/ toggle text on/off in brand.
$wp_customize->add_setting( 'display_brand_text', array(
	'default' => 0,
	'sanitize_callback' => 'best_reloaded_sanitize_checkbox',
) );
$wp_customize->add_control( 'display_brand_text', array(
	'label' => __( 'Select the checkbox to display the site title in the navbar as brand text.', 'best-reloaded' ),
	'section' => 'best_reloaded_navbar',
	'settings' => 'display_brand_text',
	'type' => 'checkbox',
) );

Outputting Navbar Brand in a Bootstrap Theme

Now at this point I realised that output would be slightly more complicated than just echoing values. I also spotted that very long titles could break layout of navbar quite easily so I needed to account for that.

When the brand is turned on you can output 3 things.

  • The Brand Image
  • The Site Title
  • Brand Image + Site Title

Some logic for deciding what is output is needed at runtime so instead of echoing values to in the template file I added an action hook instead. The hook will trigger, check if we should output a brand, try to build the brand and then ultimately output it if we have a brand to use.

The Hook & Action

The hook is a standard action hook for WP.

/**
 * Fires the navbar-brand action hook.
 *
 * @since 1.2.0
 */
function best_reloaded_do_navbar_brand() {
	/**
	 * Used to output whatever featured content is desired in for the navbar brand.
	 */
	do_action( 'best_reloaded_do_navbar_brand' );
}

The action calls a function to perform the output logic and stores the value. It then tests if it has a value, sanitizes it against a list of accepted html tags and attributes then echoes it to the page.

/**
 * Echos the markup output by navbar branding function.
 *
 * @return void
 */
function best_reloaded_output_navbar_brand() {
	// try get the branding markup.
	$output = best_reloaded_navbar_branding();
	// if we have output to use then sanitize and echo it.
	if ( $output ) {
		$allowed_brand_tags = array(
			'span' => array(
				'class' => array(),
			),
			'img' => array(
				'id'	=> array(),
				'class'	=> array(),
				'src' => array(),
				'alt' => array(),
				'width' => array(),
				'height' => array(),
				'style' => array(),
			),
		);
		echo wp_kses( apply_filters( 'best_reloaded_filter_navbar_brand', best_reloaded_navbar_branding() ), $allowed_brand_tags );
	}
}
add_action( 'best_reloaded_do_navbar_brand', 'best_reloaded_output_navbar_brand' );

Function to Generate Navbar Brand Markup

The function that generates the markup also handles the logic of what is output and deals with the issue of long titles breaking things.

I added a character cap by default of 30 chars and another customizer option for an override to allow long titles if the site owner wants to.

$wp_customize->add_setting( 'allow_long_brand', array(
	'default' => 0,
	'sanitize_callback' => 'best_reloaded_sanitize_checkbox',
) );
$wp_customize->add_control( 'allow_long_brand', array(
	'label' => __( 'Very long titles break the default navbar layout, if you want to allow very long titles here then check this box. NOTE: You can also turn off the search form for more space.', 'best-reloaded' ),
	'section' => 'best_reloaded_navbar',
	'settings' => 'allow_long_brand',
	'type' => 'checkbox',
) );

The function that returns the markup looks like this:

/**
 * Builds out a .navbar-brand based on options set in the theme.
 *
 * @return string containing html markup for brand
 */
function best_reloaded_navbar_branding() {
	// initial value for the output is false.
	$brand_output = false;
	// check for image set in theme options theme options.
	$brand_image = get_theme_mod( 'brand_image', '' );
	// Did we get an image or is the brand text turned on?
	if ( $brand_image || get_theme_mod( 'display_brand_text', false ) ) {
		// since we have at least 1 of the items then start the output.
		$brand_output = '<span class="h1 navbar-brand mb-0">';
		if ( $brand_image ) {
			// we have an image.
			$brand_output .= '<img id="brand-img" class="d-inline-block align-top mr-2" src="' . esc_url( $brand_image ) . '" >';
		}
		if ( get_theme_mod( 'display_brand_text' ) ) {
			// text is toggled on, get site title.
			$site_title = get_bloginfo( 'name', 'display' );
			// very long site titles break the navbar so cap it at a generous 50 chars.
			if ( strlen( $site_title ) <= 50 || get_theme_mod( 'allow_long_brand', false ) ) {
				$brand_output .= esc_html( $site_title );
			}
		}
		$brand_output .= '</span>';
	}
	// this will return the markup if we have any or it will return false.
	return $brand_output;
}

Next Steps

Now that this works and I've tested it I will push the update to the .org repo and think about my next set of tweaks and changes.