Peer Review On Stack Overflow – it’s What Makes it Awesome!

stackoverflow stickers with and brand logo and name text

Stack Overflow has to be the number 1 place to visit when facing programming challenges. I visit it regularly and every other developer I know has been to it at least once during their career.

There are a swath of different network sites each with a focus on a specific set of challenges faced by a particular group. The point of all the sites is to get help, find answers and to get peer review for your ideas, questions or solutions.

The reason that Stack Exchange sites are so prevalent, and so useful, is because they follow some guidelines that ensure only high quality, useful, content.

Stack Exchange Questions

As a general rule Questions on Stack Overflow should be directly related to programming and Answers should be direct answers to the questions posed. Questions on other SE network sites may not be related to programming – that depends on the site but it should always be directly related to the site topic.

This way other people with the same question can find the solution they need a lot quicker than if questions, or answers, are too broad and do not include good examples.

Questions should follow a format that includes a minimum, complete, and verifiable example of the issue at hand. Ensuring questions have examples, capable of being reproduced, detailing the problem gives a greater chance of that question, and it’s answers, being valuable to people with similar issues.

Peer Review at Stack Overflow

I often run quickly through the question scenarios to verify issues or to give additional clarity to a question. I comment often but answer infrequently.

That is the essence of the peer review system. You provide whatever input or help you can in a given situation and are rewarded with Rep points for useful insight. You do not need to be providing the answers – all you need to do is provide some helpful input.

Rep Points

You gain Rep points from various positive site actions. Reaching certain milestones unlock some privileges.

The Rep system was not built to be treated as a status symbol. Benefit unlocks are infrequent, and often negligible, but you consistently unlock news ways to be helpful to other users and the site as a whole.

The points you accrue unlock milestones that allow you to be helpful in different ways.

Recently I crossed the 500 Rep mark. It’s not a very high milestone but it’s an important one.

At the 500 point mark you unlock access to some review queues. What I learned from looking through these queues is just how much peer review goes into everything you see on the Stack Exchange network of sites.

Peer Review – Review Queues

All questions from new users are reviewed. First answers are reviewed. New answers added to old questions are reviewed.

Almost every question is checked over to make sure it’s appropriate and that it meets a minimum standard expected for creating good questions.

One of the earliest review queues you can access is the new Documentation review queue. You can access it at 100 Rep. New docs are checked and edits to existing docs are checked to ensure that they add value in some way. This section needs more eyes to build it into a valuable resource but has less activity than some of the other queues.

At 500 points you can access:

  • Triage – to help identify good posts from ones that need work.
  • First Posts – you can use this as an opportunity to teach new users how to ask questions that result in good answers.
  • Late Answers – used to spot hidden Gems that may be missed due to the age of a question. This is also used to help filter out people adding answers purely for the Rep points.

People with review privileges are able to look through these posts and do quality checking on them. Several users check each post so a consensus can be met and people are encouraged to edit or comment on posts if they feel they can be improved.

This gives existing users a way to filter out low quality posts while helping improve useful content. It goes beyond simply spotting spam and removing answers that have no value. It’s about improving things as a whole.

Teach Users to Use Stack Overflow and They’ll Make It a Better Place for Everyone.

Aside from being filled with great answers one of the most instrumental things in making SO so useful is the help that users provide. It’s what keeps SO the dominant source of Questions and Answers to specific programming questions. Users ensure the site contains high quality content – and remove or edit content that is not useful or otherwise considered low quality.

Teach users to ask good questions and they’ll ask them better.

Show users the correct network to ask their question on and they’ll get a better chance at a good answer.

Point out an inefficient method in a question or solution.

Provide links to proper documentation to flesh out a fuller answer.

The point is improve things in a way that makes it more useful for everyone. Users might be asking the question or providing solutions – it doesn’t matter. The end result is a system that enables people to, find, provide or ask the right questions and answers to any given problem.

Peer review is instrumental in making that happen.

Best WordPress Plugins For A Successful Blog

Originally I wrote this answer as a first draft to a question I read on Quora. Figured it’d be worth posting here as well because it’s unlikely to get much views on the question – but I still wanted to answer it anyhow 🙂

In my experience there is, honestly, no plugins required to make a successful blog. In fact, more often than not, my advice to clients is usually about removing plugins not required rather than adding more.

WordPress, right out-of-box, is an excellent platform for content management. In terms of being purely a means to share content online (like a blog, as opposed to being an online store or some other product/service provider) there is nothing that fits the bill for as many uses as WordPress does without modification. Personally I think there are a couple of shortfalls, which I’ll detail a little later in the answer, but those are easily filled by a small collection of plugins.

  • Form builder/Form processor – WordPress has no form builder in core. You can certainly write the markup yourself and use sanitization and validation functions from WordPress during form processing but that’s custom code and not a feature available out the box. My recommendation is Gravity Forms (premium) but free alternatives are available. Contact Forms 7 is an excellent free plugin that works similarly.
  • Caching – WordPress, on it’s own, provides an excellent base for database caching – Transients. What the Transients API provides is essentially an object cache that stores the results of certain database queries (querying the database is often one of the slowest operations of sending the end-user the page they requested) so that the query only needs run one time and the query results can be obtained with a single lookup. I see this as both a benefit and one of the shortfalls – because it stores the objects in the database! It does speed up getting the data on second request but it still needs a DB lookup all the same. The best extension to this is to put that object cache into RAM using an in-memory cache – such as MemCached. My choice of plugin for doing this (and other cache/performance related tweaks) is W3 Total Cache. The other popular choice is WP Super Cache. Both are good, and have very expansive options. WP Rocket is also an incredible caching plugin but it’s a premium plugin. Another plugin which is recommended to me by another WP developer is Simple Cache. It was described as having an on/off switch and no complicated options and can put your object cache into Redis/MemCached.
  • Security – before any recommendations are made here it’s worth noting that WordPress core is extremely secure and the core team are incredibly fast acting when it comes to security exploits. When you hear about WordPress site comprises it’s rarely, if ever, the fault of WP core and almost always the fault of code that extends it – such as that in plugins or themes. When it comes to security and plugins what you’re looking at is enhancement. Things like temporarily locking an account with too many failed login attempts. Temp or perma-ban on IP addresses and hosts that repeatedly fail logins. Scanning for file changes when you haven’t changed any files. You can do these things with the free version of WordFence.

In addition to Form builder/processing, Caching and Security plugins it’s certainly a good idea to take backups. Plugins are available for backing up your site files, uploads and database. Personally I can’t make a recommendation amongst the best of bunch backup plugins because I don’t use them on my own sites. I favour a server side solution for backups because it’s usually easier to handle a restore. We all know backups aren’t about storing your data – they’re about restoring it, right?

BONUS

These 2 plugins are here in the bonus section because many people consider them to be overkill.

Jetpack is a massive plugin, offering many features. Most notably the functions it provides are simple off-site stats gathering, social publishing and a 1-click image optimization CDN. It might be said that Jetpack is overkill for these features since the plugin is so huge and offers so much more. There’s a lot of truth to that however I see Jetpack as a relatively good way to get these features easily without any need to worry about complex setup or config – a real bonus if your focus is primarily on creating content rather than spending a lot of time setting up features.

Akismet – for vetting comments and form fills to check it for potential spam. Since Akismet has such a massive database of known spam, IP addresses and identification patterns it’s one of the better choices. Some people find certain rules applied by Akismet does block legitimate comments because they look a little bit like spam according to their rules (and no rules are ever perfect).

Akismet is a large (in terms of the shear amount of code it adds) plugin for what it does and some consider this overkill. If you find it’s giving false positives on your site or want a more lightweight solution one lesser used option is Growmap Anti-Spambot Plugin. It hasn’t been updated in 2 years but I’m certain it still works. It essentially adds a honeybot type block that is able to block unsophisticated spambot (which is probably 90% of them or more).

A Raspberry Pi Twitter Bot In Python

NOTE: This post was sitting unpublished for almost exactly 1 year. I went ahead and gave it database storage and implemented scheduled posting You can find the tweetbot on GitHub and I even have a working version that is deamonized.

I’ve wanted to build a Twitter bot for some time. Mostly just something to send the occasional tweet. That could easily be extended to something that would become a scheduled tweet bot and a database could even be added to store future tweets.

I also wanted to monitor for mentions and notify me of them. Watching for something to occur and then running an action could also be extended in many ways, especially if a live search stream were to be added to the mix.

The basics of what the bot does is relatively simple. It needs to be able to access various streams (my notifications, a search stream). It has to be able to parse them and invoke something based on a given result. It needs to be capable of posting a tweet from my account.

Since I plan on using my Raspberry Pi for this and Python is a popular language to use on it I looked around for some reference points. There’s a very nice Python library written that is capable of doing the heavy lifting of sending requests to the Twitter API for me. It’s called Tweepy and I found it through GitHub.

Using Tweepy I should be able to easily connect and post/get to the Twitter API. Let’s see how that goes.

You will need to create an app and get some access credentials from Twitter to make your API calls – especially since the plan is to make it actually post to accounts.

Installing Tweepy

First I need to install Tweepy. You can run pip install tweepy to do it – and I did on my laptop and that worked just fine. On my RPi though I will be cloning it from Github and installing manually. There are certain base level dependencies of Tweepy, or of it’s dependencies, that are probably already installed on most systems. They were not available on my Pi though and the setup.py script doesn’t handle those. A quick Google of the problem told me to run pip install --upgrade pip to get them. That worked.

git clone https://github.com/tweepy/tweepy.git
cd tweepy
sudo python setup.py install

Since I also plan to eventually use a database to store things in I also installed mysql-server but that’s not absolutly necessary for right now.

sudo apt-get install mysql-server

Writing the Bot Script

After that I used the code I found on this site to make a bot that was able to tweet things out that it read from a text file. I called the script bot.py and the text file with the tweets tweets.txt.

#!/usr/bin/env python
# -*- coding: utf-8 -*-
# from: http://www.dototot.com/how-to-write-a-twitter-bot-with-python-and-tweepy/
import tweepy, time, sys

argfile = str(sys.argv[1])

#enter the corresponding information from your Twitter application:
CONSUMER_KEY = '123456'#keep the quotes, replace this with your consumer key
CONSUMER_SECRET = '123456'#keep the quotes, replace this with your consumer secret key
ACCESS_KEY = '123456-abcdefg'#keep the quotes, replace this with your access token
ACCESS_SECRET = '123456'#keep the quotes, replace this with your access token secret
auth = tweepy.OAuthHandler(CONSUMER_KEY, CONSUMER_SECRET)
auth.set_access_token(ACCESS_KEY, ACCESS_SECRET)
api = tweepy.API(auth)

filename=open(argfile,'r')
f=filename.readlines()
filename.close()

for line in f:
api.update_status(line)
time.sleep(60)#Tweet every 1 minute

The script needs to be given a text file containing the tweets you want it to post. Make a .txt file in the same directory containing some tweets. Then call the script passing the .txt file. Assuming the script is called ‘bot.py’ and the tweets are in a file called ‘tweets.txt’ this is the command.

python bot.py tweets.txt

It’ll run for as long as it takes to post all the tweets from your file and it’ll wait 60 seconds between posting each one. When I ran it myself I got an InsecurePlatformWarning. It seems that’s down to the version of Python that I ran it with and the version of requests that it uses. To fix it I ran installed the requests[security] package as per this StackOverflow answer.

As of now you should be totally up and running with a Twitter Bot that can post tweets for you. It’s not the most useful of things considering it’ll only post through a list from a text file at a fixed interval.

Next steps in this project will be to add database support and time scheduling into the system.

Could I Teach a Machine to Learn?

It’s pretty obvious to anyone who knows me that computers fascinate me. The hardware, the software, their uses. Everything about them intrigues me.

What tells packets where to go once they are out on the open web? How does a computer generate a random number? What allows memory to hold a persistent electrical signal? I encourage you to find out the answers to each of those in your spare time – everything about it is fascinating.

One of the particular things that I am interested in is Artificial Intelligence. It just so happens that one of my favorite YouTube channels Computerphile has several recent videos that are extremely informative on AI. They also have videos about Machine Learning and Search Engines in videos from recent months. All worth watching. Each of the topics are somewhat related to each other and yet each is distinctly different.

After watching them it got me to thinking about Structured Data and how exactly the structure is given or defined. At small scale you can take a dataset find common attributes and organize it by that criteria.

You manually set the criteria and the amount of categories then sort them into each pile. It’s easy.

How exactly would that be done with data that has no labels or clear set of common attributes? Taking unorganized data and indexing it, assigning labels, working out attributes. Finding better and more efficient ways of doing that is part of the improvement process of Machine Learning.

That’s exactly what I’m going to investigate doing in a long running project. Extremely efficient indexation and giving structure to random data is kind of how search engines work. There’s a strong correlation between the kind of thing I want to do and how search engines provide the most relevant result for a given terms.

I’m going to grab my data from Twitter and store it, index it, categorize it and learn from it. The data from Twitter already has somewhat of a structure to start with but that exact structure might not be what I’m after. I want to structure it in many more ways.

I’m going to make use of what I learn in… maybe no ways at all but I’m gonna do it anyhow haha!

  • Make a Twitter Bot with search capabilities.
  • Store Tweets in a database.
  • Index them.
  • Categorize the data.
  • Learn and Enjoy!

I hope that I’ll learn an awful lot from doing this. Probably not directly from the data I gather but definitely in terms of skills. Plus everyone needs a project to keep them focused. Some of the elements of this have been on my project list for a long time, now is as good a time as any to make some headway.

 

Converting to Bootstrap 4

I’ve been a fan of Bootstrap since before I knew it existed. The theme I had on my site several years ago was built with a very early version of Bootstrap 2 and I’ve used it in dozens of projects myself since I found out about it. I also updated that theme to work with Bootstrap 3.

Changes in Bootstrap 4 – From LESS to SASS

Now that Bootstrap 4 is in alpha it’s time to get ready and make a start using it. It’s sort of a re-thought again from the ground up change so it’ll take some time to get used to and to learn the new .classnames but another major change is that it has swapped from less to sass for css. For me that’s a big change – I’ve never used sass. I’ve only recently learned to use less and the reason why was because css pre-processors are more efficient and modular – I had to start using one – and I picked less purely based on the fact that Bootstrap used it.

Well that’s changed and I’ll happily move over to sass, but it’s one more thing in a long list of things that I need to learn in a short space of time.

Getting Bootstrap 4 Files

If you’ve worked with Bootstrap and git before you’ve probably cloned the repo at some point. If you have just go into the directory and run a git pull to grab any updates and then checkout one of the v4 branches. There’s an official alpha branch but I chose to use the v4-dev branch as it’ll likely be updated frequently and I’ll get upcoming changes between releases as soon as they are merged.

If you don’t have repo cloned grab it from github or get precompiled files from the site. You could even install it with npm or as a ruby gem.

Bootstrap 4 Migration Time

So I already have a theme that is powered by Bootstrap 3 that I like a lot. I’m going to convert it to work with Bootstrap 4. It’ll take a lot of work (I remember doing this for v3 and it did…) but mostly it’ll involve .classname changes and possible structure reworks but it’s doable. I’ll probably do a search and replace to make a lot of it happen, particularly when it comes to the layout and grid classes. I’ll may use grep to do it but if I have the project open for anything I will likely just use the search and replace in my editor -Atom.

FYI Atom is awesome, you should try it.

A Default Kinda Guy – Installing TwentySixteen

This year’s WordPress release schedule puts the next major release around the end of the year. In that release will come the new default theme – TwentySixteen. The development version is up on GitHub for you to test and contribute if your inclined.

On this site I’ve been a default theme user. When I started I used TwentyFifteen and wrote a couple of thoughts. Now that the development version of next years default is available I couldn’t resist giving it a try.

There’s a few ways that you could get this installed on your site. The easiest may be to download the zip from Github and upload it to your site. Alternatively you could clone it to your site with git clone or use WP-CLI to install and activate it for you.

Cloning The Theme into Your Site

Navigate to the themes directory on your site – /wp-content/themes/ – and run the clone command:

git clone https://github.com/WordPress/twentysixteen.git

Updating the theme as it develops is easy by running git pull from the theme directory.

Installing a theme From Github via WP-CLI

From anywhere in the WordPress installation you can run the theme install command, pass it a theme-slug or a url to a zip file and it’ll install it in the correct location.

wp theme install https://github.com/WordPress/twentysixteen/archive/master.zip

add --activate at the end of the command to activate it right away and --force if you want to suppress any warnings about overwrites.

Note: If you install the theme like this it’ll be located in the twentysixteen-master folder instaed of the twentysixteen folder.

Contributing to the Next Default Theme

You can contribute to the theme and shape its development moving forward by creating issues and pull requests in the GitHub repo.

You will need to know a little bit about using git to contribute but it’s nothing really complicated. You should be able to find out everything you need to know by taking a look at my WordPress and Git Workflow post.

If you’re interested in contributing you should also check out the CONTIBUTING.md file in the repo.

Automate All The things With Grunt

I’ve recently become a fan of Grunt. It’s a JavaScript task runner that get installed within a project to handle jobs for the project. For me it’s extremely useful for a whole range of things – even in just a short 2 weeks I’ve got so much more value from using it than I ever expected.

If you’re a developer of any kind then Grunt is something that you could probably benefit from using. The reason? This is the reason they give on their homepage:

In one word: automation.

On it’s own Grunt doesn’t do a lot – it runs tasks but you need to define those tasks for it to be useful.

Tasks are defined by plugins installed alongside Grunt. There’s already thousands of plugins created that do many of the things that you might need (4,403 of them at time of writing). Many of them are officially maintained by the team at Grunt, so you can guarantee a decent level of support or documentation will be available.

Grunt for WordPress Development

I mostly work on WordPress projects. PHP, JavaScript, HTML and CSS is what I deal with daily. Grunt has MANY tasks available for working with those. There’s 2 specific things that are extremely useful.

Linting

Linting is the process of parsing your code and making sure it complies with a set of predefined rules. It can parse for errors as well as coding standards and all of the major code types used in WordPress can be linted with readily available tools.

Being notified of errors in code as they are introduced can save many hours of debugging.

Combine + Minify

Making sure you have comments in your files so you know what’s happening at a given point is good practice. It’s also good practice to serve optimized files to users – in as few requests and bytes as possible.

Combining your styles and scripts reduces the amount of requests needed and eliminates any overheads associated with additional requests. Minfying them as well will make sure you’re sending only the data that you need.

Making sure your images are optimized or compressed as well can save a lot of unnecessary transfer. You can have Grunt handle that for you too.

Watching for Changes

Since automation is the name of the game you can make use of the Watch task. It watches for changes to your files and when it detects them goes ahead and runs certain tasks. As a starting point watching your styles and scripts for changes then recompiling them could save you hours of time to spend on doing more useful things in your project.

Re-evaluating the Use of Certain Domains

I have many domain names. So do most other web developers. Some of them serve production sites, others are used for testing and some even get used just as placeholders. There’s even a few I just don’t use.

This domain was registered alongside the .com variant – where I run my main business. I’ve used it for all kinds of things over the space of several years but mostly it’s been a playspace.

I recently re-evaluated it’s use and decided it was a good place for a blog to live.I still use it for a few things but at the moment it’s the place where I keep this personal blog. I’m going to be doing that with a few of my domains and it’s going to result in a shake-up of the services I offer.

Firstly I’ll be taking a look at my hosting site and seeing what I can turn that into that’s more useful. Currently it’s the front-end for my managed hosting service but there has been a slow uptake in the offer and managing a growing cluster of servers, tracking what runs where and rejigging things as necessary has become a pain.

It often takes me more time to keep it running smoothly than I get paid for by the users.

In short – it was an idea I tested and it isn’t viable for 2 reasons. The traffic I send to it isn’t interested in managed hosting – they are technical enough to do that themselves. The pricing point is hard to compete with larger companies that offer ‘shared hosting’ because my target market can’t quite see the difference between shared and managed.

I’ll be shutting up shop on the hosting and using the domain for something different.

Similarly I’ve had another domain that I planned to use as a split testing/conversion optimization service. It never got anywhere because it evolved to the point where 2 server-side services I created for it continued to grow to the point where it would have been an inefficient million lines of code. I vastly underestimated what would be needed to make it work like I pictured so the project – while it’s still been worked on in development – is stagnant till I finish. Eventually this service will exist but it’s a long time off.

Also I’ve tested the water with split testing content on my site and there’s been some interest but not as much as I’d like. That means that to get traffic to the service I’d need to invest a lot more time in research and content creation before I could get enough people who would be interested. That’s time that I don’t currently have considering that my family is about to grow and I’ll have a new baby to look after.

I’m still not sure what I’ll be doing with the repurposed domains but I suspect that I’ll narrow the audience I was targeting and focus purely on the audience I do have traffic from – other web developers like me.

IWC – Honeypot comment spam filter

So I see spam bots buzzing around my sites all the time. There’s one particular site that gets hit more than others and it’s causing problems.

Every now and then a spambot tries to post hundreds or thousands of comments one street the other and it ties up server resources. That shows the whole server down and eventually if it continues can cause crashes and other problems – like memory exhaustion.

You can clearly see the offending spam bots in your access logs because you’ll see several lines that contain:

POST /xmlrpc.php

Note: the IP in the line above is from a real spambot that hit my site. It got past most of the protection systems and in the end fail2ban had to step in and ban the offending IP.

I have other systems in place to restart on crash, to rate limit spam bots and to eventually block them. The problem is that they’re not working 100% effectively even when properly configured.

The new idea is to prevent then being able to submit the comment form in the first place.

A honeypot might not be the best solution but it will defiantly help to stop automated submissions so it’s a start.

Chatting About Open Source Software

I’m of the opinion that open source software is better software. I know that’s a broad statement and I suppose it’s not entirely factual – but that’s my opinion. It’s free to use, modify or sell and anyone can browse the code to see how it works for whatever purpose.

Open source software is found everywhere. WordPress (and all the themes and plugins in the WordPress.org repo) is open source under the GPLv2 or greater licence and my entire server stack uses open source software to power it.

In general the entire web relies heavily on open source software to make it work. The most common web server softwares (Apache and NGINX) are open source and another great example is how WordPress powers about 23% of all websites online. Google uses (and creates) open source software for many of it’s systems and even Facebook created – and then open sourced – the server software they use.

It’s crazy how much the web relies on open source to work. It’s part of the very heart of how we make and deliver what we build. I think that every web developer should take a few minutes every now and then to honestly consider if what they are doing could be contributed back to the community.

If it could be contributed back that doesn’t necessarily mean that it should be but it is worth considering.

With open source software it can be very easy for you to contribute to projects or make new ones. Many projects are hosted at GitHub – which is a free to use code and project hosting site that uses the Git software to manage repositories of code and provide distributed and centralized management capabilities.

GitHub is by far the largest open source code hosting site of all time so learning a bit about using Git is probably in your best interest if you use, or want to contribute to, open source projects.

With Git the learning curve feels very steep but it quickly levels off to a comfortable progression only needing to learn more if you want to have more power over your repository.

Since I’m a big user of open source software and I feel strongly about it from this moment onwards I’m committing myself to finding at least 1 hour in my week so I can work on open source projects.

I will be contributing to existing projects as well as releasing some of my own personal ones for anyone to use. I actually started to work on making a split testing plugin for WordPress a long time ago and never finished it so I’ve been working a bit on that. I recently pushed the MVTS plugin to GitHub if you’re interested.