JonBlog
Thoughts on website ideas, PHP and other tech topics, plus going car-free
15.Apr.2018
Dockerising simple VPS machines
Categories: Docker

Introduction

When I first started maintaining “play” servers on the internet, in order to run my own blog and various software projects, the build process was long and laborious. That would have been around 2009, and at that time I was working for corporate logistics provider whose Disaster Recovery time was measured in days, not minutes. This was before serious server orchestration tools came onto the horizon (Ansible, Salt, Chef, Puppet) and well before the popularisation of lightweight containerisation, in which Docker now plays a central role.

However, the Linux technologies that Docker makes use of under the hood (namespaces and cgroups) have been around for years in similar guises elsewhere: LXC in Linux, and before that, Jails in FreeBSD. It’s interesting to ponder why Docker has only just exploded in popularity, but I’d wager it is the tools that Docker has now build around containerisation, and not container technologies themselves. The Dockerfile, the layered filing system, the caching of layers, easy networking, all the stuff you get automatically in Docker Compose — the list goes on.

Perhaps the mass availability of cloud server virtualisation has contributed to where we are today. I used to rent a 512M server for 15GBP a month, with rotating disk storage, and now I can have a machine with double the RAM, free additional cloud features, and significantly better performance, with SSD storage, for around 6USD a month. We’re at the point now where running a number of applications in Docker costs the same as two cups of café coffee per month, and I recommend it to every web software engineer (of course, one can sign up to free VPS offers from the likes of AWS, but they tend to be time-limited).

Plan

From the start, I planned several things:

  • My WordPress blog (with a traditional TLS certificate)
  • The system-monitoring software Netdata (via a Let’s Encrypt TLS certificate)
  • MySQL server, primarily for WordPress
  • An SMTP server for WordPress outbound messages
  • As much as possible to be Dockerised

There are a few other web apps to be moved from my previous VPS, but this is a good start.

Solution

Let me first present the solution I ended up with, and then I’ll walk the reader through it. There was not actually much of an architecture plan to start with – it was fairly organic, with a few wrong rabbit holes! It finally ended up looking like this:I knew I wanted more than one HTTPS site: this blog, some server monitoring software, and the ability to add more in the future. That would require that they share a web server, since only one process can bind to port 443, and I only have one IP address. Since there’s more than one Docker process with its own web server, this was not possible, and so I needed a frontend proxy to forward traffic to the correct subsystem.

While Apache/mod_proxy and NginX are capable of doing this, I previously had a good experience of Traefik. This is a highly configurable proxy device, written in Go, and capable of interfacing with all sorts of orchestration and configuration stacks. It will also set up and renew Lets Encrypt TLS certificates automatically. Of course, this can be used within a Docker container itself, with the external web ports bound to the public IP on the VPS.

For now, I’ve put MySQL on the host itself. In the early days of Docker, there was some worries about data corruption in containers, though I expect they’re resolved now. I suspect that while the data ought to stay on a host volume, the server itself would be just fine in a container. The server set-up is pretty simple as it is, but moving the MySQL configuration to a buildable Docker repo would be even nicer.

I wanted all of these containers to have a restart policy (so they respawn if they crash, and on server boot) and also to delete the container on exit, since none of them should store any state. However, it turns out specifying both --restart and --rm is not permitted, so I had to work around it. This is the purpose of Docker Tidy: each container has a Docker restart policy, but containers persist after stopping. Thus, this process sweeps up dead containers daily, to keep things clean, and to ensure the disk is not overwhelmed.

For my own containers, I maintain a Dockerfile, an automatic image build at CircleCI, and a private image registry at GitLab. These are the images:

  • WordPress – my build pipeline and my image registry
  • Docker Tidy – my build pipeline and my image registry
  • Traefik – from public registry
  • Netdata – from public registry
  • SMTP – from public registry

Here’s my Dockerfile for WordPress. I have lightly edited it for security reasons, but broadly it gives a good idea of how this platform can be containerised:

# Docker build script for my WordPress blog
#
# Requires Docker 17.05 to build (uses multistage feature)

FROM alpine:3.6 AS build

# Required to do Git clone operation
RUN apk --update add git openssh-client

WORKDIR /root

# Install the private SSH key to be able to fetch private, low-value repos
#
# The known hosts can be generated by turning off StrictHostKeyChecking temporarily,
# then cloning the private repo in the container manually, and copying the resulting
# known_hosts to this repository (see https://stackoverflow.com/a/29380672).
COPY config/ssh-keys/bitbucket_rsa /root/.ssh/id_rsa
RUN chmod 600 /root/.ssh/id_rsa
COPY config/ssh-keys/known_hosts /root/.ssh/known_hosts

RUN mkdir themes
RUN git clone git@bitbucket.org:halferbits/threattocreativity-forked.git themes/threattocreativity && \
    git clone git@bitbucket.org:halferbits/jonblog-theme.git themes/jonblog

# Stock unzip won't work
RUN apk add unzip openssl

# This will decompress to /root/wordpress
RUN cd /root && \
    wget https://wordpress.org/latest.tar.gz && \
    gunzip latest.tar.gz && \
    tar -xf latest.tar && \
    rm latest.tar

# Install plugins
# We don't need to install Akismet since it is installed already
#
# See this for why the wildcarded zip expression needs quotes:
# https://superuser.com/q/563215
RUN mkdir plugins
RUN cd plugins && \
    wget https://downloads.wordpress.org/plugin/plugin-one.zip && \
    wget https://downloads.wordpress.org/plugin/plugin-two.zip && \
    wget https://downloads.wordpress.org/plugin/plugin-three.zip && \
    wget https://downloads.wordpress.org/plugin/plugin-four.zip && \
    unzip -q "*.zip" && \
    rm *.zip
RUN cd plugins && \
    wget https://downloads.wordpress.org/plugin/plugin-five.zip && \
    wget https://downloads.wordpress.org/plugin/plugin-six.zip && \
    wget https://downloads.wordpress.org/plugin/plugin-seven.zip && \
    wget https://downloads.wordpress.org/plugin/plugin-eight.zip && \
    unzip -q "*.zip" && \
    rm *.zip
# This plugin may be abandoned, so using a patched fork
RUN cd plugins && \
    wget https://github.com/halfer/plugin-nine/archive/patch-1.zip && \
    unzip -q "*.zip" && \
    mv plugin-nine-patch-1 plugin-nine && \
    rm *.zip

FROM alpine:3.6

# Do a system update
RUN apk update
# Need ca-certificates and openssl to fetch WP
RUN apk --update add \
    ca-certificates openssl \
    php7-apache2
# php7-mysqli is required for the database
# php7-gd is required for ?
# php7-zlib is required for the admin UI
# mysql-client is for backup
# php7-opcache is for performance
RUN apk --update add \
    php7-mysqli php7-gd php7-json php7-zlib \
    mysql-client php7-opcache

# Set up PHP to have a smaller memory limit
RUN sed -i -r 's/memory_limit = \d+M/memory_limit = 30M/g' /etc/php7/php.ini

# Prep Apache
RUN mkdir -p /run/apache2
RUN echo "ServerName localhost" > /etc/apache2/conf.d/server-name.conf
COPY config/apache/rewrite.conf /etc/apache2/conf.d/rewrite.conf
COPY config/apache/total-cache.conf /etc/apache2/conf.d/total-cache.conf

# Disable Apache modules we don't need
COPY config/unload-mods.sh /tmp/
RUN sh /tmp/unload-mods.sh

# Overwrite the MPM default config
COPY config/apache/mpm.conf /etc/apache2/conf.d/mpm.conf

WORKDIR /var/www/localhost/htdocs

# Wipe Apache folder
RUN rm -rf /var/www/localhost/htdocs/*

# Copy WP from first build stage
COPY --from=build /root/wordpress /var/www/localhost/htdocs

# Delete stock themes
RUN rm -rf wp-content/themes/*

# COPY themes from build stage
COPY --from=build /root/themes/threattocreativity wp-content/themes/threattocreativity
COPY --from=build /root/themes/jonblog wp-content/themes/jonblog

# Delete demo plugin
RUN rm wp-content/plugins/hello.php

# Install plugins
COPY --from=build /root/plugins wp-content/plugins

# Set up config
COPY config/wp-config.php .
RUN rm wp-config-sample.php
RUN chown apache *
COPY config/.htaccess .

# Here's a copy of the cache script I generated earlier...
COPY config/advanced-cache.php wp-content/advanced-cache.php

# Make some things writable
RUN chmod u+x    .htaccess wp-content/advanced-cache.php && \
    chown apache .htaccess wp-content/advanced-cache.php

# Port to run service on
EXPOSE 80

ENTRYPOINT ["/usr/sbin/httpd", "-DFOREGROUND"]

Each of these services has an associated start shell script, which does the following:

  • Remove the restart policy from old containers of the same type
  • If it needs the host’s LAN IP, obtain that in an environment var
  • Start the Docker container, specifying the necessary volume/network/restart options
  • Once the container starts, run any initialisation in the container using docker exec

An example of a start script is given in this answer.

When running, the memory consumption is pretty lean:

IMAGE         CONTAINER        CPU %       MEM USAGE / LIMIT       MEM %      NET I/O             BLOCK I/O           PIDS
wordpress     9b133c635e74     0.00%       81.52 MiB / 992.2 MiB   8.22%      301 MB / 73.3 MB    28.3 MB / 0 B       5
netdata       7a68aacf846d     1.87%       42.47 MiB / 992.2 MiB   4.28%      10.5 MB / 12.1 MB   33.5 MB / 4.1 kB    14
traefik       4b87ccbabab6     0.00%       46.44 MiB / 992.2 MiB   4.68%      114 MB / 131 MB     55.4 MB / 16.4 kB   9
docker-tidy   8a105dd13d72     0.00%       800 KiB / 992.2 MiB     0.08%      828 B / 648 B       1.51 MB / 4.1 kB    1
smtp          0fa34f7756af     0.00%       7.027 MiB / 992.2 MiB   0.71%      31.1 kB / 648 B     20.9 MB / 41 kB     1

Together with MySQL on the host, and the OS itself, it takes around 0.5-0.6G RAM, which on my 1G RAM machine at Vultr is pretty comfortable. I’ve made some tweaks in WordPress’s Apache so that it doesn’t spin up too many worker processes (the default was taking 250M of RAM, so I’ve reduced the number of workers, and removed some unused Apache modules; it looks like it’s maxing out at ~150M now, and usually it lurks below 100M).

I’ll shortly be adding a couple more web apps, and it’ll be interesting to see how the memory consumption responds.

Future improvements

When building this server, I considered using Docker Compose, especially since that helps deal with some of the dependencies (e.g. WordPress needs SMTP, and Traefik needs WordPress). However, it didn’t feel quite right at the time, since DC strikes me as being suitable for an application (where everything has a dependency) rather than server configurations (where only some services are independent). Nevertheless, I may look into that that again.

One of the main deficiencies of my set-up is a strict versioning of images, which I will tackle next. This may start off as a script to tag new images pulled from my private registry, but at some point, I expect implementing a Swarm or Kubernetes stack will give me that feature. That will allow me to roll back to earlier images if a newer one fails.

I’d also like to set up memory limits for containers. Although the tweaked Apache configuration seems to be behaving itself, it would be nice to add some belt-and-braces. Docker should have some options to set this in the run command.

I also plan to move some containers to continuous deployment, direct from CircleCI. Presently the process is based on a manual pull, retag and restart operation, and that should be fairly trivial to automate. Watch this space!

Leave a Reply