Wilson Talks Tech

Migrating 14 years of personal websites and legacy cruft to Docker in a week

Note: This took place in 2020, I’m just slow at writing it up. References to current are referring to the server setup pre-docker in 2020.

This, plus all of my other sites, run on a 1U server in a colocation facility. While the price and speed is great, traveling to the facility and doing any sort of maintenance on the server is a pain.

I needed to upgrade Debian Jessie. Support is completely done. My usual OS upgrade path is either:

VPS

  • Spin up new VPS running updated OS
  • Install what packages I think I need
  • Slowly migrate each site over, checking to see if they work for the most part
  • Cancel original VPS

Physical server

  • Move services to spare server running same OS
  • Buy new SSD and install fresh new OS on it
  • Install what packages I think I need
  • Migrate all sites from spare server back, finding out what is broken along the way

But this time, I don’t have a spare server. And the upgrade path is anything but reliable. Since this is running multiple separate projects where pinning dependencies was a necessity, containerization seemed to fit the need perfectly.

The planned setup:

  • Each website is a separate set of docker containers
  • Managed by docker-compose
  • The database is in a separate container for each site
  • Something will sit in front of this to route the containers

We have a few website setups we’re running:

  • Mostly static PHP websites
  • PHP + MySQL (WordPress)
  • PHP + Postgres (various bespoke websites I wrote over the years)
  • Django + Postgres

Most sites are PHP with a few Django sites in there. Some are completely static, but still usually relying on PHP because that’s just how I did it. This is over 14 years of cruft that keeps coming along. Databases are Postgres and MySQL.

For PHP sites, the current setup is an nginx web server backed by FastCGI and PHP-FPM. There are nginx configs that will route the domain to the specific /var/www/domain.tld folders for each site. Cloudflare sits in front of everything and right now, we’re just using HTTP between the server and Cloudflare.

In order to make this easy, the plan is to replicate this setup as close as possible and improve it later.

—

In order to mitigate downtime and ease rolling back/out, the docker containers will be created and deployed to the same outdated server running everything else. Once everything is containerized, we can migrate as we need to.

To do this, we need something to sit in front of our docker web servers and non-docker nginx instance. I ended up choosing Varnish. The idea being that adding docker will add overhead so adding a good caching layer in between will help reduce this. One drawback with varnish, we don’t have SSL support, but since I’m routing everything with Cloudflare and wasn’t using SSL currently, this seemed fine.

Looking back, I would not have chosen Varnish and should’ve used nginx or traefik. Later, when I do want to use SSL, I bring nginx back into the mix anyway. But I used what I knew and it turned out fine.

Setup varnish in docker and configure it to route everything to localhost. I flip the server’s nginx to port 81, put varnish up on 80. Everything is still working.

Once varnish is working, we can now route to specific containers or fall back to our host. What I should’ve done is put everything in the same docker network and route based on hostname. What I did was use host networking and route everything based on port number.

So site1 would be running on port 12345 for example. This would mean the site was accessible to the world on that port without going through varnish, but I don’t see any major concerns this would cause.

—

Now that we can route to each container, we need to actually build them. Since I’m having to do this on 20+ different sites, I want to keep the containers as close to their current non-container state as possible.

I also wanted to keep the containers as self contained as possible, since the main purpose of this was migration and reproducibility. This means all the source files should be in the container.

To reproduce my nginx + php-fpm setup, the easiest route seemed to be running 2 separate containers for each part. Which meant, in order to keep the source files contained, they needed to be matched in both separate containers. So while this felt bad and flaky, these sites rarely, if ever, get updated and this seemed like a fair tradeoff, since this also lets me copy the current nginx site config and string replace a single line.

nginx Dockerfile +

—

—

Migrated around 50% of the sites (smaller, lower traffic sites) to Docker. Everything is working. This seems like it will work great.

1AM that night. Server down. Attempt to SSH in. Not responding. Uh oh. Outage at the colo or machine is hard down. Keep trying flipping the power and eventually wait long enough it boots up.

Spooky, but maybe a fluke.