Categories
100 Days of Code

WP_Bootstrap_Navawalker Updates and Unit Testing PHP

Covering days 16 through 20 and culminating in 1 month and 1/5th of the way through 100 days.

This week I had spent a substantial portion of my time working on a project I've supported for last 5+ years – the WP Bootstrap Navwalker.

I've used the walker since version 2 of Bootstrap and been a contributer all of BS3 lifespan. With Bootstrap 4 finally in a stable release (after around 3 years in development!) and I have been working through a substantial rework of the entire navwalker.

I've focused on a few things through the rebuild.

  • Simplifying the codebase and improving readability.
  • Better link modifier (disabled, header, divider) and icon handling.
  • Handling long standing issues that were difficult to fix or required a large rework in the previous iteration.

Simplifying The Codebase And Improving Readability

To simplify things I have focused on a few things.

Too many paths!

Reducing the number of possible paths in the start_lvl function has been the first focus. This function has grown to contain lots of conditionals that result in different processing or outputs. At one point in Scrutinizer I seen it say that this function had over 20000 paths!

Lots of paths result in code that becomes hard to follow so to help improve that I decided to externalize some parts of the code into their own functions.

Prime candidates were some decision logic that didn't need to be cluttering up the main thread. A single call with a well named function does a better job of explaining what is happening than the large conditional code blocks – even through those codeblocks are well commented.

Better Link Modifier and Icon Handling

Bootstrap navigation has always required the use of specific classes on the container, the link and the wrapper elements for the dropdown elements to function. WordPress provides a way to set classes for individual nav items in the Menu Editor UI.

In previous versions the Title Attribute input was used to handle special classes for modifiers and icons.

The complexity here is that most of the time Walkers add the classes to the wrapper element – in this walker that is an <li> – however some of the time you need those classes to get applied directly to the link.

Icon classes need to be placed separately from the wrapper and the link. They are added to their own <i> element with aria-hidden="true" so that screen readers don't try read an icon character.

$icon_classes = array(); // This array would contain valid icon classes.

// Join any icon classes plucked from $classes into a string.
$icon_class_string = join( ' ', $icon_classes );

/**
 * Initiate empty icon var, then if we have a string containing any
 * icon classes form the icon markup with an <i> element. This is
 * output inside of the item before the $title (the link text).
 */
$icon_html = '';
if ( ! empty( $icon_class_string ) ) {
	// append an <i> with the icon classes to what is output before links.
	$icon_html = '<i class="' . esc_attr( $icon_class_string ) . '" aria-hidden="true"></i> ';
}

To make this happen I needed to loop through arrays of classnames and strip specific classes from the main array and save them in a new array for later use. The $icon_classes array is filled with that function.

I decided to handle link modifiers and icon classes separately as the link modifiers are used for setting a typeflag for decision making later in the execution.

/**
 * Find any custom linkmod or icon classes and store in their holder
 * arrays then remove them from the main classes array.
 *
 * Supported linkmods: .disabled, .dropdown-header, .dropdown-divider
 * Supported iconsets: Font Awesome 4/5, Glypicons
 *
 * NOTE: This accepts the linkmod and icon arrays by reference.
 *
 * @since 4.0.0
 *
 * @param array   $classes      an array of classes currently assigned to the item.
 * @param array   $link_classes an array to hold linkmod classes. Passed by reference.
 * @param array   $icon_classes an array to hold icon classes. Passed by reference.
 * @param integer $depth        an integer holding current depth level.
 *
 * @return array  $classes      a maybe modified array of classnames.
 */
function seporate_linkmods_and_icons_from_classes( $classes, &$linkmod_classes, &$icon_classes, $depth ) {
	// Loop through $classes array to find linkmod or icon classes.
	foreach ( $classes as $key => $class ) {
		// If any special classes are found, store the class in it's
		// holder array and and unset the item from $classes.
		if ( preg_match( '/disabled/', $class ) ) {
			// Test for .disabled.
			$linkmod_classes[] = $class;
			unset( $classes[ $key ] );
		} elseif ( preg_match( '/dropdown-header|dropdown-divider/', $class ) && $depth > 0 ) {
			// Test for .dropdown-header or .dropdown-divider and a
			// depth greater than 0 - IE inside a dropdown.
			$linkmod_classes[] = $class;
			unset( $classes[ $key ] );
		} elseif ( preg_match( '/fa-(\S*)?|fas(\s?)|fa(\s?)/', $class ) ) {
			// Font Awesome.
			$icon_classes[] = $class;
			unset( $classes[ $key ] );
		} elseif ( preg_match( '/glyphicons-(\S*)?|glyphicons(\s?)/', $class ) ) {
			// Glyphicons.
			$icon_classes[] = $class;
			unset( $classes[ $key ] );
		}
	}

	return $classes;
}

Unit Testing My Code

When tests are successful and all is green 🙂

While I was going through the code and dealing with long standing issues I also took care of an issue which was opened not long ago on the early dev version of the v4 walker. The issue explained that the fallback method was broken and outputting nothing when it should have been outputting something.

The fix was easy – I had mistakenly used assign instead of append so swapping the operator on all of the markup to be output made it work.

// incorrect use:
$output = 'somehtmlcontent';
// fixed:
$output .= 'somehtmlcontent';

Once I'd fixed it I realised that the project has Unit Tests that can be run locally and are run by Travis but all they do is check that the file, class and methods exist. Those tests could be a lot more useful.

If there was a test in place that made sure the fallback output when it was supposed to and didn't when it wasn't I would have caught this mistake early and fixed it quick. Instead it escaped me till someone else pointed it out.

Testing That Functions Produce The Output They Should

The project has PHPUnit already available so all I had to do was add some tests that could check the output.

This fallback function should produce 2 different types of output. For a logged out user it should output nothing. If a user is logged in with edit_theme_options capability it should output markup with a link. There's also a flag that determines if it echos or returns.

I wrote some tests that do a few things:

  • Test that logged out users get empty strings and that both 'echo' and 'return' are the same.
  • Test that logged in users get html that looks to be valid and that 'echo' and 'return' are both the same.

When testing output of a function there are a few methods that PHPUnit has available. Those methods are useful but somewhat limited so many times – for output that may be complicated or subject to slight changes over time – you will want to check for traits of the valid output rather than the exact output to save you reworking tests all the time. PHPUnit already runs tests inside of a content buffer so all you need to do is get your output with ob_get_contents;.

One thing that got me stuck for a few minutes was how exactly could I get logged in output when this is run in CLI mode where no user is set. Turns out it's quite easy. Make a new user with the capability we need – I made an admin that does have all core capabilities – and set them to the current user.

// make an admin user and set it to be the current user.
$user_id = $this->factory->user->create( array( 'role' => 'administrator' ) );
wp_set_current_user( $user_id );

The full pair of test ended up coming out like this:

/**
 * Test Fallback method output for logged out users.
 *
 * Expects that for logged out users both echo and return requests should
 * produce empty strings.
 */
function test_fallback_function_output_loggedout() {

	// default is to echo reults, buffer.
	ob_start();
	WP_Bootstrap_Navwalker::fallback( $this->valid_sample_fallback_args );
	$fallback_output_echo = ob_get_clean();

	// empty string expected when not logged in.
	$this->assertEmpty(
		$fallback_output_echo,
		'Fallback output for logged out user created a non-empty string in echo mode.'
	);

	// set 'echo' to false and request the markup returned.
	$fallback_output_return = WP_Bootstrap_Navwalker::fallback( array_merge( $this->valid_sample_fallback_args, array(
		'echo' => false,
	) ) );

	// return and echo should result in the same values (both empty).
	$this->assertEquals(
		$fallback_output_echo,
		$fallback_output_return,
		'Fallback output for logged out user created a non-empty string in return mode.'
	);
}

/**
 * Test Fallback method output for logged in users.
 *
 * Expects strings to be produced with html markup and that they match when
 * requesting either a return or defaulting to echo.
 */
function test_fallback_function_output_loggedin() {

	// make an admin user and set it to be the current user.
	$user_id = $this->factory->user->create( array( 'role' => 'administrator' ) );
	wp_set_current_user( $user_id );

	// default is to echo results, buffer.
	ob_start();
	WP_Bootstrap_Navwalker::fallback( $this->valid_sample_fallback_args );
	$fallback_output_echo = ob_get_clean();

	// rudimentary content test - confirm it opens a div with 2 expected
	// values and ends by closing a div.
	$match = ( preg_match('/^(<div id="a_container_id" class="a_container_class">)(.*?)(<\/div>)$/', $fallback_output_echo ) ) ? true : false;
	$this->assertTrue(
		$match,
		'Fallback method seems to create unexpected html for logged in users in echo mode.'
	);

	// set 'echo' to false and request the markup returned.
	$fallback_output_return = WP_Bootstrap_Navwalker::fallback( array_merge( $this->valid_sample_fallback_args, array(
		'echo' => false,
	) ) );

	// return and echo should both produce the same strings.
	$this->assertEquals(
		$fallback_output_echo,
		$fallback_output_return,
		'Fallback method seems to create unexpected html for logged in users in return mode.'
	);
}

Testing Private Methods of a Class

When I was writing other tests I needed to call private methods and test their output. Private methods are not intended to be called from outside of the function so to make that happen we need to use a technique called Reflection.

We use a reflector to duplicate the class and methods we want and then set the methods to be publicly accessible from the reflector.

$wp_bootstrap_navwalker = $this->walker;

// since we're working with private methods we need to use a reflector.
$reflector = new ReflectionClass( 'WP_Bootstrap_Navwalker' );

// get a reflected method for the opener function and set to public.
$method = $reflector->getMethod( 'linkmod_element_open' );
$method->setAccessible( true );

Once you have the reflected method setup and accessible you can then use invokeArgs on the $method – passing the real class object followed by an array containing the args to pass to it.

// test openers for headers and dividers.
$header = $method->invokeArgs( $wp_bootstrap_navwalker, array( $this->valid_linkmod_typeflags[0], 'stringOfAttributes' ) );
$this->assertNotEmpty( $header, 'Got empty string for opener of ' . $this->valid_linkmod_typeflags[0] );
Categories
100 Days of Code

Using Some ES6 JavaScript

This is something I worked on as part of my 100 days of code challenge during week 3.

I invested a bit of development time into a project I've been thinking of for a long time but left it neglected at the beginning stages.

It's a Javascript split-testing library.

I'm perfectly capable of reading and writing JS code with plenty of real world experience working with it. It's not one of my strongest skills though so I took this as an opportunity to improve and learn some more things. 

Javascript and ES6

When people say modern Javascript what they often mean is ES6 Javascript. The 6th edition was finalized in 2015 and is called ECMAScript 2015.

Sadly even though it's been around for some time browser implementation was slow to start and there is no backporting being done in many cases for older browser versions.

Modern browsers support ES6 code fairly well. Chrome, Opera, Safari, Firefox and even Edge have good levels of support – older browser versions do not and there's essentially no support for it on any version of IE.

Developers want to take advantage of the newer features that ES6 offers. They write ES6 code then compile it to ES5 compatible code – systems like babel make that easy. I wanted to take advantage of the new paradgrims too..

New Methods and Features in ES6 I Was Excited About

Deciding to go full ES6 in this project allowed me to take advantage of some features that I had previously only made minimal or no use of. There are 3 things I discovered in ES6 that made writing JavaScript a vastly more pleasant experience.

  • Native Classes to build with.
  • Variable and Function scoping inside blocks with let.
  • Default value assignments for function declarations.

I'm excited to work with those features all the time. Classes especially.

In ES5 you could use variables with objects, custom prototypes and anonymous functions to get similar Classes-like behaviors. To me that method always seemed to create very convoluted and hard to read code so native Classes are very welcome.

There is a couple of other very nice additions that I'm looking forward to using in future and in code refactors.

  • Rest and Spread Operators – ...variable.
  • Template Literals with raw string access.
  • Constants – Immutable Variables
  • String Searching.

Doing Things the ES6 Way

There are a few things in this project that I done differently to in the past and in ES6 they are much cleaner.

Merging Objects

While working I wanted to merge some objects. Previously I'd done that with a custom function that iterates through all prototypes of an object and merged them into another object. ES6 has a method for doing just that in a single line.

let defaults = {the defaults};
/**
 * Merge 2 (or several) objects together.
 *
 * This merges myObject into the object containing the defaults.
 * and returns the updated destination object only.
 */
this.myObject = Object.assign( defaults, this.myObject );

Better Sub-String Methods

A lot of work takes place with strings and string manipulation. In my latest project I had the pleasure of searching for values inside strings using much cleaner methods.

let myString = 'a string';
// there's a `startsWith` method now and it includes an index spot. This returns true because we start at index 4. There is also an `endsWith` that behaves similarly.
if ( myString.startsWith( 'ring', 4 ) ) {
    console.log( 'found "ring" at end of ' + myString );
}
// my favorite is `includes`. Previously I would have used indexOf or a regex for complex strings for this.
if ( myString.includes( 'tri' ) {
    console.log( 'Found "tri" inside of . + myString );
}
Categories
100 Days of Code

Backup WordPress and MySQL Docker Containers

As part of my 100 Days of Code Challenge this is something I worked on in week 2.

The start of this week with the sniffles as my wife calls it. I had the cold and I spent a lot of time reading on my phone, not a lot of time working unfortunately.

One of the things I did work on in that downtime was improving a backups setup that I had been using.

I wrote a script to run through each of the sites I wanted backed up and while writing it I decided I'd put a little color in it for a change to make it easier to see the status of the execution.

I run a lot of WordPress sites using docker and multi-container setups. It's mainly based on similar ideas I wrote about when I first put WordPress Containers into production.

What Needs Backup

There are 2 things we need to back up for WordPress sites. There's the site files and there's the database. Both of those are run inside of different containers.

  1. The Web Server Container – Apache with PHP and Memcached installed.
  2. The Database Container – Mariadb.

The content of these sites isn't updated often – many of them are essentially static at this point. Slow weekly/monthly backups are enough to secure what they store.

All of the sites and their databases get included in different system level backups at different times of the month. It's not ideal but it's been enough so far.

In a more tailored system the file and database backups would occur more regularly – and be more targeted towards each individual site.

Backup WordPress Docker Container Files

The container that runs the WordPress files holds them inside of a directory on the host that is mounted inside of the the container. Technically – and in the default home directory backups I run as part of a different system – they can be backed up from the host by backing up those directories. 

I wanted to perform the backup without relying on the host system. Instead I came up with this one liner to fire up a different container that's using the ubuntu image and archive the files through it for the backup. It's a little complex but I'll explain each bit below.

docker run --rm --volumes-from pattonwebzinfo_wordpress_1 -v $(pwd)/backup:/backup/ ubuntu tar -cvf /backup/pattonwebz_info.tar /var/www/
  • docker run --rm – Run a container that will be removed when it exits.
  • --volumes-from pattonwebz_info_wordpress_1 – Mount the volumes FROM another container INTO this container too.
  • -v $(pwd)/backup:/backup/ create a volume that's found at [current working directory]/backup on the host and make it accessible inside the container from /backup/
  • ubuntu – the container we're using is a base ubuntu instance which will have the command we need available.
  • tar -cvf /backup/pattonwebz_info.tar /var/www/ – using tar command we backup the /var/www/ directory (which is a directory mounted from the wordpress container) and store the archive inside of this containers mounted volume – backup.

Backup a Mysql Database Inside of A Docker Container

To backup the database I considered spinning up a different mysql container and mounting the directories inside of it in a read-only mode to prevent from 2 systems accessing the same set of files.

I worried that file locking might be a problem with this and it seemed like a custom image may be needed – or for custom configs to be written in advance and mounted into the db backup container.

There was also authentication issues to deal with. I don't want to be bundling credentials into scripts if I don't need to.

I opted to take a different approach and use docker exec to execute a mysqldump command inside of the container and redirect the output to a file inside of the host filesystem. Inside of [current working directory]/backup/wp_database.sql– this puts it right next to the sites files archive.

docker exec pattonwebzinfo_mysql_1 sh -c 'exec mysqldump -u"$MYSQL_USER" -p"$MYSQL_PASSWORD" "$MYSQL_DATABASE"' > $(pwd)/backup/wp_database.sql

The database name, user and password are available inside the container as environment variables so can be used easily.

Script it up to make it cycle through all the containers I need to backup and then trigger that script via cron. Targeted scheduled backups for WordPress docker containers.

Categories
100 Days of Code

100 Days of Code Challenge

Week 1 – Day 1-5

For the first week of my 100 Days of Code challenge I didn't go too far out of my way to work with code. It's part of my job, I do it day-to-day.

I spent some time looking for a code bug to discover it was simple typesetting issue. I learned, once again, the value of explicit typesetting and the pitfalls of allowing an operation to do it's own type juggling.

When you're in need of specific operations or types to be the outcome it's best to strictly define those types instead of allowing the system to do it for you.

Another thing that I was amazed by is how much can be done, in a far more efficient manner, when you spend time only in the command line and semi-automate tasks.

There are times when it's far more time effective and less error prone to do that over manually processing data and verifying the results of your work.

Juggling & Defensive Typecasting

Be defensive when it comes to casting variables to the types you expect when data comes from somewhere you're not in control of. Sometimes it's what you expect – sometimes it's not.

William Patton

Wait… What type is this??

Strict typesetting is a good idea when working with variable types in a loosely declared language because sometimes types are juggled in a way that you don't expect and when you've got a type you're not expecting your results may not be what you're expecting as well.

Example of this happening with strings and numbers in JavaScript:

// Data comes from an object.
var data = {
    'id': '1452',
}
// Try increment the ID number by 5.
let new_id = data.id + 5;
/**
 * Instead of incrementing the starting data was a string 
 * and appended our number to the end like a string concat.
 */
console.log( new_id ); // new_id = 14525
/**
 * What we really wanted to happen was to increment the number
 * by 5.
 */
let new_id = parseInt( data.id ) + 5; // parsed the `id` and made sure it was int.
console.log( new_id ); // new_id = 1457

There is also a different operator specifically made for doing type conversions from string to number in javascript and is written in a more easily flowing manner. Both methods create the same output when successful.

var data = {
    'id': '1452',
}
let new_id = data.id;
new_id = + 5;
console.log( new_id ); // new_id = 1457

Command Line Automation

Working in command line and creating automation scripts beats manual tagging and sorting for large datasets.

Every. Single. Time.

William Patton

With a directory of .html files to have articles exported and a database of thousands of posts already imported: Find what files were not already converted and imported to the site as articles.

  • The list of content that was intended to include came from thousands of .html files – in various directories arranged mostly by year.
  • Using mysql queries to extract data about articles then filter it so it had only a value easily compared against the files needing import – that value will be a match with the filename that's to be imported.
  • With the data use BASH with some loops and a direct comparison logic to find matches.
  • Use git diff reporting to determine what posts were missing from the list of imported content.

BASH Scripts That Loop and Compare

At the heart of things were bash scripts that mostly looped within a loop.

Iterate through lines of a file while looping through lines of another file then output the results to different file.

It's a blunt object approach. A lot of overheads, not at all efficient in operations.

Regardless of inefficiencies it didn't take long to run through. It was simple and incredibly effective to highlight the differences between items present and items expected.



I called the script it match.sh then ran it passing 3 .txt files. One filled with the files that are expected to be in the list, another filled with the list of articles present for this category and the 3rd file to store the results.

./match.sh filename-list.txt metavalue-present.txt matched.txt 
#!/bin/bash

echo "Matching $1 inside of $2 and saving to $3"
echo "Do the above values look correct?"
sleep 5s

# loop through all the lines in the first file passed.
while IFS='' read -r line || [[ -n "$line" ]]; do
	# loop through all the lines in the second file passed.
	while IFS='' read -r linein || [[ -n "$linein" ]]; do
		# if line from f2 matches the line from f1...
		if [[ $linein == *$line* ]]; then
			# echo line 1 (filename) to a file.
			echo "$line" >> ./$3
			# we found a match, break from loop 2.
			break
		fi
	# loop 2 is passed contents of file passed as arg2.
	done < "$2"
# loop 1 is passed the contents of file passed as arg1.
done < "$1"

The next step was a repeat of this except to compare the list of expected items against the list of matches and save that to new file too.

./match.sh filename-list.txt matched.txt missing.txt 

The final step was to use git diff to compare the differences between the matched items and the original list of items expected.

One thing that helped here to improve processing speed by several minutes per run was to split the large groups of data into smaller, more manageable, sets or categories and then handle comparing them in them in batches. Since my compare method was so inefficient this helped a lot.

Categories
100 Days of Code

A 100 Days of Code Challenge

I've been mulling over this idea for a long time. 100 days of code. The original idea is essentially spend 1 hour a day coding, document the progress and share on code platforms or on social.

On the surface that sounds easy as I spend almost every single day with code in some form or another. Generally it's an hour or more.

The fact is that it's difficult to have 100 consecutive days to work on an idea without interruptions. Even if it's just 1 hour a day – sometimes you might just not have it. There's work and family life to prioritize.

30 days ok, 60 maybe. 100 full days in a row working directly with code and no deviations: That's more of a challenge than I can commit to.

An Outline of a More Forgiving 100 Days of Code Challenge

It's not just about writing code – there is plenty of value to be found in working with code it in other ways.

It is not 100 consecutive days and you never need to commit 7 days a week.

Improving documentation for code or reading and reviewing someone else's code are both high value tasks in some cases.

  • It's 100 working days, no minimum per day.
  • Code review or other improvements are included and highly encouraged.
  • Documentation isn't necessary everyday – once a week is a good target.
  • Daily tracking isn't required – track ideas or projects over the space of a week or more. 

The 100 days could start from any point. I've decided to start January 1st.

  • Between January 1st and June 1st there are a little over 100 working days.
  • That's 5 days in a week – or every day minus weekends and major holidays. Spread over around 5 months of time.

I will be documenting some things I work on each week, write some example code and schedule posts to publish throughout the span of the challenge.

Documentation Format Suggestions

Sharing of knowledge in the programming and development world is done in a few formats. Most obviously the sharing of code. That's a great way let someone see the work that's been done or methods that were used.

Another common way to share is via articles, blog posts and books. This is useful for sharing overall process, detailing setup steps or explaining some thought processes behind what's been done.

  • Anything that can be shared in a blogpost. Text, screenshots or examples could be included here.
  • Links to code sharing platforms like a GitHub repo or a Codepen.
  • Social shares like tweets.

Consider helping someone to solve a coding problem as being a valid exercise as well. Working through an issue tracker like in GitHub repo or similar counts. StackOverflow is a great place to offer assistance and to share what you know with someone in need of help. 

Important Notes

With an emphasis on working with code there's some things I think will be relatively important to keep in mind throughout the 100 days. It's about working with code not necessarily writing code all the time.

  • Looking at and learning from some code is a valid exercise.
  • Adding inline docblocks or improving existing comments is valid. 
  • Participating in an open source project of any kind will be valid.

Remember if it's working with code then it can be considered part of the challenge. Some other things to note throughout the challenge:

  • Sharing of example code is always good, sharing of full source when possible is encouraged too.
  • Creation of actual code documentation – either external documentation or inline code doc block addition or improvements is a very big part of writing code that is for sharing.