Colorize Terminal Output

This is part 2 of the 2nd week of my 100 Days of Code Challenge.

I'm somewhat of a plain styles kind of person. Color conveys a meaning that I don't often care for when it comes to things I create for myself. A white background with black for me is perfectly ok in most cases. When I'm particularly adventurous I may opt for a dark mode.

This week I decided that I'd add some color to some of my scripts. Since I was working on the backup script I decided to put some color into the info that the script output. It outputs simple pieces of text with echo just like this:

echo -e 'files archived'
echo -e 'starting db dump'

Different terminals use different colorization methods and I've colorized text a few times by Googling to find out how to make it happen. This time I done the same lol.

When I looked up the codes this time I got to a page with a very thorough breakdown of what can be done and how. It had this note at the top.

In Bash, the <Esc> character can be obtained with the following syntaxes:

  • \e
  • \033
  • \x1B

Seeing the list of escape characters reminded me of a time previously when I had added color in a python script. It output color in my terminal and in a terminal running on a Windows 10 machine. At the time I thought nothing about it.

Suddenly realizing that the color codes must be somewhat standardized I decided to look more closely. I discovered the codes come from a system called ANSI/VT100. A lot of terminals support it in some way or another. Windows added support for it in version 10. Prior to windows 10 there was no support.

Escape Color Sequences

Colorizing terminal output is done by using an escape sequence followed by a color code and then closed. Reset follows the same method so the codes end up looking something like these.

\e[1m # bold
\e[7m # inverted
\e[0m # reset

# OR with different escape sequences that bash understands. 

\033[1m # bold
\x1B[7m # inverted
\e[0m # reset

I have no idea exactly what the escape sequences that windows terminal supports however with my python script these were the colors I had used with the \033 escape sequence.

class tcolors:
    HEADER = '\033[95m'
    BLUE = '\033[94m'
    GREEN = '\033[92m'
    WARNING = '\033[93m'
    FAIL = '\033[91m'
    ENDC = '\033[0m'
    BOLD = '\033[1m'
    UNDERLINE = '\033[4m'

You can see a large list of color codes and support for them on this page.

Color Escaped Text in a Bash Script

I decided to invert the colors of some of the info messages to make them stand out from the other messages that may scroll past. For a success messge I also turned the text green as well as doing the invert.

Those echo commands end up starting with the color code, outputting the text and then end with a reset code.

echo -e '\e[7m\e[92mfiles archived\e[0m'
echo -e '\e[7mstarting DB dump\e[0m'

Of course helper text and colorization is only useful in a manual run of this script. Since it's primary purpose is to be run as a cron job this info is going to be put to /dev/null. But I wanted to learn how it's done properly so that's what I did.


Protected: My Review Process

This content is password protected. To view it please enter your password below:


One Post A Week In Gutenberg

I don't recall who said it or where I read it (and I frantically scrolled back through many days of social feeds to find it) but at some point in the last 2 weeks I'd heard someone say to try writing a post every week in the new Gutenberg editor.

I'm going to try do just that and see how it goes. There are going to be regular updates and tweaks so it may be fun to see a full evolution from now till the merge with core.

Why I Think This Is A Good Idea

There are two reason why I think this could be a good experiment. Especially for myself.

  1. Following development, reporting bugs, creating feature requests. The more I use Gutenberg the more I can be sure that the final product works how I want to use it.
  2. I've been incredibly lax on writing blog posts and creating new content in general in recent years. Partly due to time demands of a growing family but some other factors played a part in that too.

I would like to change the fact that I haven't done nearly as much writing as I could have. Encouraging myself to write something at least once a week I hope will build a habit that I can easily return to.

Ready For Production…?

I don't think Gutenberg is ready to use in production just yet. Perhaps in a month it could be closer but currently a lack of content blocks (or options for blocks) I feel would be useful – and many missing or broken styles – make is so usage is limited to personal sites or side-projects.

Posts like this one will be how I use Gutenberg and for the most part I expect primarily to use it for text. Paragraphs, headers, lists and links.


Best WordPress Plugins For A Successful Blog

Originally I wrote this answer as a first draft to a question I read on Quora. Figured it’d be worth posting here as well because it’s unlikely to get much views on the question – but I still wanted to answer it anyhow 🙂

In my experience there is, honestly, no plugins required to make a successful blog. In fact, more often than not, my advice to clients is usually about removing plugins not required rather than adding more.

WordPress, right out-of-box, is an excellent platform for content management. In terms of being purely a means to share content online (like a blog, as opposed to being an online store or some other product/service provider) there is nothing that fits the bill for as many uses as WordPress does without modification. Personally I think there are a couple of shortfalls, which I’ll detail a little later in the answer, but those are easily filled by a small collection of plugins.

  • Form builder/Form processor – WordPress has no form builder in core. You can certainly write the markup yourself and use sanitization and validation functions from WordPress during form processing but that’s custom code and not a feature available out the box. My recommendation is Gravity Forms (premium) but free alternatives are available. Contact Forms 7 is an excellent free plugin that works similarly.
  • Caching – WordPress, on it’s own, provides an excellent base for database caching – Transients. What the Transients API provides is essentially an object cache that stores the results of certain database queries (querying the database is often one of the slowest operations of sending the end-user the page they requested) so that the query only needs run one time and the query results can be obtained with a single lookup. I see this as both a benefit and one of the shortfalls – because it stores the objects in the database! It does speed up getting the data on second request but it still needs a DB lookup all the same. The best extension to this is to put that object cache into RAM using an in-memory cache – such as MemCached. My choice of plugin for doing this (and other cache/performance related tweaks) is W3 Total Cache. The other popular choice is WP Super Cache. Both are good, and have very expansive options. WP Rocket is also an incredible caching plugin but it’s a premium plugin. Another plugin which is recommended to me by another WP developer is Simple Cache. It was described as having an on/off switch and no complicated options and can put your object cache into Redis/MemCached.
  • Security – before any recommendations are made here it’s worth noting that WordPress core is extremely secure and the core team are incredibly fast acting when it comes to security exploits. When you hear about WordPress site comprises it’s rarely, if ever, the fault of WP core and almost always the fault of code that extends it – such as that in plugins or themes. When it comes to security and plugins what you’re looking at is enhancement. Things like temporarily locking an account with too many failed login attempts. Temp or perma-ban on IP addresses and hosts that repeatedly fail logins. Scanning for file changes when you haven’t changed any files. You can do these things with the free version of WordFence.

In addition to Form builder/processing, Caching and Security plugins it’s certainly a good idea to take backups. Plugins are available for backing up your site files, uploads and database. Personally I can’t make a recommendation amongst the best of bunch backup plugins because I don’t use them on my own sites. I favour a server side solution for backups because it’s usually easier to handle a restore. We all know backups aren’t about storing your data – they’re about restoring it, right?


These 2 plugins are here in the bonus section because many people consider them to be overkill.

Jetpack is a massive plugin, offering many features. Most notably the functions it provides are simple off-site stats gathering, social publishing and a 1-click image optimization CDN. It might be said that Jetpack is overkill for these features since the plugin is so huge and offers so much more. There’s a lot of truth to that however I see Jetpack as a relatively good way to get these features easily without any need to worry about complex setup or config – a real bonus if your focus is primarily on creating content rather than spending a lot of time setting up features.

Akismet – for vetting comments and form fills to check it for potential spam. Since Akismet has such a massive database of known spam, IP addresses and identification patterns it’s one of the better choices. Some people find certain rules applied by Akismet does block legitimate comments because they look a little bit like spam according to their rules (and no rules are ever perfect).

Akismet is a large (in terms of the shear amount of code it adds) plugin for what it does and some consider this overkill. If you find it’s giving false positives on your site or want a more lightweight solution one lesser used option is Growmap Anti-Spambot Plugin. It hasn’t been updated in 2 years but I’m certain it still works. It essentially adds a honeybot type block that is able to block unsophisticated spambot (which is probably 90% of them or more).


Converting to Bootstrap 4

I’ve been a fan of Bootstrap since before I knew it existed. The theme I had on my site several years ago was built with a very early version of Bootstrap 2 and I’ve used it in dozens of projects myself since I found out about it. I also updated that theme to work with Bootstrap 3.

Changes in Bootstrap 4 – From LESS to SASS

Now that Bootstrap 4 is in alpha it’s time to get ready and make a start using it. It’s sort of a re-thought again from the ground up change so it’ll take some time to get used to and to learn the new .classnames but another major change is that it has swapped from less to sass for css. For me that’s a big change – I’ve never used sass. I’ve only recently learned to use less and the reason why was because css pre-processors are more efficient and modular – I had to start using one – and I picked less purely based on the fact that Bootstrap used it.

Well that’s changed and I’ll happily move over to sass, but it’s one more thing in a long list of things that I need to learn in a short space of time.

Getting Bootstrap 4 Files

If you’ve worked with Bootstrap and git before you’ve probably cloned the repo at some point. If you have just go into the directory and run a git pull to grab any updates and then checkout one of the v4 branches. There’s an official alpha branch but I chose to use the v4-dev branch as it’ll likely be updated frequently and I’ll get upcoming changes between releases as soon as they are merged.

If you don’t have repo cloned grab it from github or get precompiled files from the site. You could even install it with npm or as a ruby gem.

Bootstrap 4 Migration Time

So I already have a theme that is powered by Bootstrap 3 that I like a lot. I’m going to convert it to work with Bootstrap 4. It’ll take a lot of work (I remember doing this for v3 and it did…) but mostly it’ll involve .classname changes and possible structure reworks but it’s doable. I’ll probably do a search and replace to make a lot of it happen, particularly when it comes to the layout and grid classes. I’ll may use grep to do it but if I have the project open for anything I will likely just use the search and replace in my editor -Atom.

FYI Atom is awesome, you should try it.


IWC – Honeypot comment spam filter

So I see spam bots buzzing around my sites all the time. There’s one particular site that gets hit more than others and it’s causing problems.

Every now and then a spambot tries to post hundreds or thousands of comments one street the other and it ties up server resources. That shows the whole server down and eventually if it continues can cause crashes and other problems – like memory exhaustion.

You can clearly see the offending spam bots in your access logs because you’ll see several lines that contain:

POST /xmlrpc.php

Note: the IP in the line above is from a real spambot that hit my site. It got past most of the protection systems and in the end fail2ban had to step in and ban the offending IP.

I have other systems in place to restart on crash, to rate limit spam bots and to eventually block them. The problem is that they’re not working 100% effectively even when properly configured.

The new idea is to prevent then being able to submit the comment form in the first place.

A honeypot might not be the best solution but it will defiantly help to stop automated submissions so it’s a start.


Chatting About Open Source Software

I’m of the opinion that open source software is better software. I know that’s a broad statement and I suppose it’s not entirely factual – but that’s my opinion. It’s free to use, modify or sell and anyone can browse the code to see how it works for whatever purpose.

Open source software is found everywhere. WordPress (and all the themes and plugins in the repo) is open source under the GPLv2 or greater licence and my entire server stack uses open source software to power it.

In general the entire web relies heavily on open source software to make it work. The most common web server softwares (Apache and NGINX) are open source and another great example is how WordPress powers about 23% of all websites online. Google uses (and creates) open source software for many of it’s systems and even Facebook created – and then open sourced – the server software they use.

It’s crazy how much the web relies on open source to work. It’s part of the very heart of how we make and deliver what we build. I think that every web developer should take a few minutes every now and then to honestly consider if what they are doing could be contributed back to the community.

If it could be contributed back that doesn’t necessarily mean that it should be but it is worth considering.

With open source software it can be very easy for you to contribute to projects or make new ones. Many projects are hosted at GitHub – which is a free to use code and project hosting site that uses the Git software to manage repositories of code and provide distributed and centralized management capabilities.

GitHub is by far the largest open source code hosting site of all time so learning a bit about using Git is probably in your best interest if you use, or want to contribute to, open source projects.

With Git the learning curve feels very steep but it quickly levels off to a comfortable progression only needing to learn more if you want to have more power over your repository.

Since I’m a big user of open source software and I feel strongly about it from this moment onwards I’m committing myself to finding at least 1 hour in my week so I can work on open source projects.

I will be contributing to existing projects as well as releasing some of my own personal ones for anyone to use. I actually started to work on making a split testing plugin for WordPress a long time ago and never finished it so I’ve been working a bit on that. I recently pushed the MVTS plugin to GitHub if you’re interested.


Varnish CDN

A while ago I wrote about an idea I’d had to make use of the Varnish forward caching proxy as a CDN. As it happens I’m not the only one who’s had the idea. There are companies offering Varnish as a CDN in SaaS (or IaaS) form out there but the prices I seen for the minimum offerings were definitely above the average WordPress site’s CDN requirements and budget.

If other companies are using a similar system and charging for it then it seems like the idea is a good one. As it happens it’s not all that hard to implement. In fact this very site is using Varnish as a CDN right now (inspect the page and look at one of the static resources in the network tab and you should see some Varnish Headers).

A bit of testing testing and configuration and 2 hours later it was ready to be put in production. A single DNS change to point it towards the Varnish Server and it was live.


Programmers Effieciency

The efficiency that a programmer demonstrates in their code could be misconstrued in the real world as laziness.

As a programmer one of the things that we try to do is solve problems or perform a task in the most efficient way possible. In computing terms efficiency generally bogs down to doing it in less code or doing it with less CPU cycles.

Knowing how programmer efficiency could be perceived by other people I tend not to point out the basic though process I go through whenever my wife asks me to do something. It’s best if that never gets declared globaly lol.

“What’s the quickest way possible to get this done?”, “Do I need to do all of these steps or are some unnecessary?” or

  • “Would looping small sections of tasks instead of doing it one big block be better?”
  • It’s not lazyness it’s efficiency… at least that’s what I tell myself in my head.