Tag: BASH

Categories
100 Days of Code

Backup WordPress and MySQL Docker Containers

As part of my 100 Days of Code Challenge this is something I worked on in week 2.

The start of this week with the sniffles as my wife calls it. I had the cold and I spent a lot of time reading on my phone, not a lot of time working unfortunately.

One of the things I did work on in that downtime was improving a backups setup that I had been using.

I wrote a script to run through each of the sites I wanted backed up and while writing it I decided I'd put a little color in it for a change to make it easier to see the status of the execution.

I run a lot of WordPress sites using docker and multi-container setups. It's mainly based on similar ideas I wrote about when I first put WordPress Containers into production.

What Needs Backup

There are 2 things we need to back up for WordPress sites. There's the site files and there's the database. Both of those are run inside of different containers.

  1. The Web Server Container – Apache with PHP and Memcached installed.
  2. The Database Container – Mariadb.

The content of these sites isn't updated often – many of them are essentially static at this point. Slow weekly/monthly backups are enough to secure what they store.

All of the sites and their databases get included in different system level backups at different times of the month. It's not ideal but it's been enough so far.

In a more tailored system the file and database backups would occur more regularly – and be more targeted towards each individual site.

Backup WordPress Docker Container Files

The container that runs the WordPress files holds them inside of a directory on the host that is mounted inside of the the container. Technically – and in the default home directory backups I run as part of a different system – they can be backed up from the host by backing up those directories. 

I wanted to perform the backup without relying on the host system. Instead I came up with this one liner to fire up a different container that's using the ubuntu image and archive the files through it for the backup. It's a little complex but I'll explain each bit below.

docker run --rm --volumes-from pattonwebzinfo_wordpress_1 -v $(pwd)/backup:/backup/ ubuntu tar -cvf /backup/pattonwebz_info.tar /var/www/
  • docker run --rm – Run a container that will be removed when it exits.
  • --volumes-from pattonwebz_info_wordpress_1 – Mount the volumes FROM another container INTO this container too.
  • -v $(pwd)/backup:/backup/ create a volume that's found at [current working directory]/backup on the host and make it accessible inside the container from /backup/
  • ubuntu – the container we're using is a base ubuntu instance which will have the command we need available.
  • tar -cvf /backup/pattonwebz_info.tar /var/www/ – using tar command we backup the /var/www/ directory (which is a directory mounted from the wordpress container) and store the archive inside of this containers mounted volume – backup.

Backup a Mysql Database Inside of A Docker Container

To backup the database I considered spinning up a different mysql container and mounting the directories inside of it in a read-only mode to prevent from 2 systems accessing the same set of files.

I worried that file locking might be a problem with this and it seemed like a custom image may be needed – or for custom configs to be written in advance and mounted into the db backup container.

There was also authentication issues to deal with. I don't want to be bundling credentials into scripts if I don't need to.

I opted to take a different approach and use docker exec to execute a mysqldump command inside of the container and redirect the output to a file inside of the host filesystem. Inside of [current working directory]/backup/wp_database.sql– this puts it right next to the sites files archive.

docker exec pattonwebzinfo_mysql_1 sh -c 'exec mysqldump -u"$MYSQL_USER" -p"$MYSQL_PASSWORD" "$MYSQL_DATABASE"' > $(pwd)/backup/wp_database.sql

The database name, user and password are available inside the container as environment variables so can be used easily.

Script it up to make it cycle through all the containers I need to backup and then trigger that script via cron. Targeted scheduled backups for WordPress docker containers.

Categories
100 Days of Code

A 100 Days of Code Challenge

I've been mulling over this idea for a long time. 100 days of code. The original idea is essentially spend 1 hour a day coding, document the progress and share on code platforms or on social.

On the surface that sounds easy as I spend almost every single day with code in some form or another. Generally it's an hour or more.

The fact is that it's difficult to have 100 consecutive days to work on an idea without interruptions. Even if it's just 1 hour a day – sometimes you might just not have it. There's work and family life to prioritize.

30 days ok, 60 maybe. 100 full days in a row working directly with code and no deviations: That's more of a challenge than I can commit to.

An Outline of a More Forgiving 100 Days of Code Challenge

It's not just about writing code – there is plenty of value to be found in working with code it in other ways.

It is not 100 consecutive days and you never need to commit 7 days a week.

Improving documentation for code or reading and reviewing someone else's code are both high value tasks in some cases.

  • It's 100 working days, no minimum per day.
  • Code review or other improvements are included and highly encouraged.
  • Documentation isn't necessary everyday – once a week is a good target.
  • Daily tracking isn't required – track ideas or projects over the space of a week or more. 

The 100 days could start from any point. I've decided to start January 1st.

  • Between January 1st and June 1st there are a little over 100 working days.
  • That's 5 days in a week – or every day minus weekends and major holidays. Spread over around 5 months of time.

I will be documenting some things I work on each week, write some example code and schedule posts to publish throughout the span of the challenge.

Documentation Format Suggestions

Sharing of knowledge in the programming and development world is done in a few formats. Most obviously the sharing of code. That's a great way let someone see the work that's been done or methods that were used.

Another common way to share is via articles, blog posts and books. This is useful for sharing overall process, detailing setup steps or explaining some thought processes behind what's been done.

  • Anything that can be shared in a blogpost. Text, screenshots or examples could be included here.
  • Links to code sharing platforms like a GitHub repo or a Codepen.
  • Social shares like tweets.

Consider helping someone to solve a coding problem as being a valid exercise as well. Working through an issue tracker like in GitHub repo or similar counts. StackOverflow is a great place to offer assistance and to share what you know with someone in need of help. 

Important Notes

With an emphasis on working with code there's some things I think will be relatively important to keep in mind throughout the 100 days. It's about working with code not necessarily writing code all the time.

  • Looking at and learning from some code is a valid exercise.
  • Adding inline docblocks or improving existing comments is valid. 
  • Participating in an open source project of any kind will be valid.

Remember if it's working with code then it can be considered part of the challenge. Some other things to note throughout the challenge:

  • Sharing of example code is always good, sharing of full source when possible is encouraged too.
  • Creation of actual code documentation – either external documentation or inline code doc block addition or improvements is a very big part of writing code that is for sharing. 
Categories
Development Tools

Open Source Projects That I Rely On To Effectively Do My Job – Part 2

There are a number of things that exist in the open source world without which I do no think I could do my Job. I am a Web Developer. I work on a range of projects using different systems, languages and processes. I work a lot with WordPress as well.

Many aspects of my work revolve around scanning logs, writing and reading code in a text editor and browsing the internet. I have my prefered programs for doing each of those tasks.

This is a set of articles that look at a lof of the open source projects that I rely on to do my job and do it effectively.

Software And Tool Choices

My job consists of 3 primary task types and I have my preference of what software to use for each of the tasks.

  • Analysing log files.
  • Reading and writing code.
  • Browsing the internet.

Most of the time I opt for open source over closed and choose cross-platform options where available.

Browser Choice – Chrome/Firefox

As a browser I want to say I use a fully open software. I do not. I use Google Chrome primarily (Firefox secondary which is open source though. Half a point for that maybe???).

Chrome is based on the open source Chromium so it's origins are open. It may also still follow Chromium as upstream. I use Chromium on minimal virtual machines but not often.

There is tracking and closed systems built into Chrome which I make use of. Cloud syncing is useful for me.

Chrome is not fully open but it was forked from open software and for me the closed source parts are an acceptable drawback.

Plus it's the most popular browser choice from users. I need to see the web in the same way that most people see it.

Reading and Writing Code – Atom 

Reading and writing code I do in Atom Editor. It's fully open, started internally at Github and is built by them and others to be the best open source editor it can be. 

For anyone working in with code and do not need a special proprietary IDE (most people working with code) for a given purpose I highly recommend Atom. It's well maintained, constantly developed and improved based on the needs of developers using it.

Atom is built with a framework called Electron (again open, from Github) which helps compile and run JavaScript (Node) as desktop applications and allows building for desktop to be very akin to building for the web meaning transferable skills for developers.

If Atom didn't exist I would use Lime Text (OSS variant of Sublime Text) or Notepad++.

Scanning Logs – Terminal and BASH

I do a lot of work in the terminal. Often in several terminals at the same time. Working with them using CLI is actually an incredible way to multi-task and effectively monitor progress. Most of the time when on command line I'm using BASH syntax. Sometimes it's powerShell… let's avoid that conversation lol!

I use Ubuntu as my main dev machine. Ubuntu ships with terminals that run BASH. Most Linux OS run BASH as well so connection to another machines command line is familiar regardless of what machine.

Logs are usually files containing plain text. Many command line tools exist to read through text files. An incredibly useful tool is called grep. It is used to search input for strings or regex matches.