Develop and Deploy Using the Stack

Our next generation WordPress hosting infrastructure based on Kubernetes is reaching beta, and we’re excited at the thought of being so close to officially launching the Stack.

Although we started spreading the news about the Stack last year at WordCamp Vienna, it’s only now we’re getting closer to our mission to democratize the WordPress hosting infrastructure.

This year, we went one step forward. We applied to speak at WordCamp London — a new milestone for us and a great opportunity to gather feedback from highly skilled professionals that have a genuine interest in WordPress. My theme was ‘Developing locally, deploying globally using the Stack‘ — a 40-minute presentation and demo aimed at developers who want to get things done without sacrificing the quality of their work, all the while applying a set of good practices.

If you’re too busy to read, you can watch

So let’s dive into the juicy part of the presentation.

#The Stack

We started building the Stack because we identified a real need. We wanted to develop our new version of the product in order to truly scale our managed hosting platform and also to better manage operations around it.

It all started with a blueprint. We knew right from the beginning we needed something that could run everywhere: bare metal servers, public clouds, as well as developers’ laptops. We also wanted to factor out the human errors as much as possible, which implied that we had to automate the operations around managing WordPress websites in general. Additionally, built-in security was a must, not something left to think about later. So it was clear to us — running a reliable platform requires the ability to observe what happens inside that particular platform, even down to single requests levels.

Some of the key beneficiaries we identified for our platform, would be:

• Enterprise environments which want to take complete control over their data, their processes and the way they deploy and host their websites.
• WooCommerce shops which need to be able to scale with their demands as they have particular needs like events and so on.
• Development agencies which need a reliable solution to be able to deploy/manage clients’ websites.
• Hosting companies which can benefit from having low-level open-source building blocks to assist in growing their businesses.

The Stack was born.

While still in development after more than 20,000 hours put into building it, the Stack is available today. But Why open-source? you may wonder. First of all, at Presslabs we’ve always regarded ourselves as keepers of customer data rather than something else. We really believe site owners and businesses should own their data, their processes, and their stack. By doing so you reduce liabilities, increase compliance with various rules and regulations (like GDPR and HIPPA) and helps standardize migrations from one provider to another.

By open-sourcing the Stack, we had the whole wide world being able to scrutinize what’s under the hood and how things happen for a WordPress website running on our platform

But we also open-sourced the project so that high-end technology becomes available to as many people and businesses, as possible, lifting the entry barrier for quality WordPress hosting that is secure, scalable and reliable.

#Architecture

The Stack is basically a collection of Kubernates operators glued together. Kubernetes is an abstraction layer on top of virtual or physical servers, which, to put it simply, pulls together the CPU, the memory, the disk and the networking from many machines together.

We’ve chosen Kubernetes to be the base for our platform because it can run anywhere. It runs on as many servers as you need, on many Cloud providers; you can even run it yourself into your datacenter or onto your laptop, pretty easily.

So what’s a Kubernetes operator?

A Kubernetes operator is a software that automates packaging, deployment and manages the application’s life cycle. For example, if you need to manage WordPress, you could create an operator that manages the way you deploy and unprovision a WordPress site.

The Stack has 5 components by now, two of them are open-sourced by us, and the other 3 are already open-sourced by the community.

Presslabs Operator for WordPress

The first operator we open-sourced is the Presslabs Operator for WordPress, which automates the common operations around WordPress (for example how you make deployments). It currently supports Git-based deployment, which means it can take a certain commit from a git repository and put it into production. Or it can do the classical FTP deployment.

On top of this, our operator manages the updates and the back-ups of a WordPress website. It consists of mainly two parts: the operator itself, which automates the operations, and the WordPress Runtime, which is responsible to provide a PHP environment in which a site runs.

Presslabs Operator for MySQL

The second operator we open-sourced is the MySQL operator. As WordPress developers, we all know that the better half of WordPress is MySQL and for that, we have invested a lot of time into developing and integrating it with the Stack.

We open-sourced the MySQL Operator last year in March and surprisingly, now it’s the solution of choice for many companies that choose to run MySQL databases on top of Kubernetes. The operator allows the Stack to run scalable, replicated MySQL nodes, if you happen to need that. For example, if you need to scale your reads or your writes, you might need more than one MySQL database server.

NGINX Ingress

We needed an automated way to route a certain request down to a site, deployed within the Stack. The decision was actually a no-brainer for us, as we’re heavy users of NGINX—we were using it before it was cool, around version 0.4 or 0.5 and we are using it also in production. So, we’ve integrated the nginx controller built around the Kubernetes Ingress resource into our Stack infrastructure.

Cert Manager for Let’s Encrypt integration

When building the Stack, we drew a hard line and made a decision that each site that runs on the Stack should run on top of HTTPS. We are using the Jetstack’s Cert Manager for provisioning SSL certificates, which also gives us out-of-the-box Let’s encrypt integration.

Prometheus Operator

Last but not least, if you run an operation which involves hosting multiple WordPress sites, you need to be able to observe what happens (at all times!). For that, we’re using Prometheus and Grafana, integrating Prometheus with all the components, even with WordPress. To automate the ops around it, we’ve chosen the CoreOS’ Prometheus Operator.

#Challenges

Trying to accomplish so many great things comes with plenty of challenges, so here are some of the ones we encountered down the road:

• Kubernetes is still a moving target. It has a release cycle of three months and things can break between releases.
• The WordPress patch process is slow. Contributing to the WordPress core is quite frustrating, as patches go into the core very slowly, taking from months to even years.
• The WordPress core follows very old development patterns. For example, it’s missing out of the box support for composer, has a hard time using symlinks and has a pretty inflexible folder structure.

#Development and deploying

Software development is not just coding, it’s actually a business process. During my time as a developer and manager, I observed that many sources of errors are not from the code you’re writing, but rather from this process. So when building the Stack, we wanted to integrate as many good practices as possible. We wanted to improve by providing you with a set of good practices that form an end-to-end development workflow.

#Development good practices

The first development good practice we integrated is code versioning. Versioning the code gives the accountability for the exact set of changes that goes into production.

Another good practice we consider of high importance is to have your project’s dependencies declared somewhere, and to have their version pinned. I don’t believe that just installing WordPress plugins through the WordPress interface is good practice.

Having this sort of dependency management it’s like having the bill of materials for when your’re building a house, it gives the new people, (read new developers) coming to a project an overview about what this project requires.

Also, it’s very important to provide a reproducible build. A build does not only mean simply taking your source code and building it into something like an archive or an executable file, a docker container, but you also need to have a process which takes your source and builds it into something. These reproducible builds are a requirement for having immutable build artifacts.

It’s essential to have the same environment from production in your development environment. How many of you have encountered the situation when you develop something and you test it locally, but when you push it to production it breaks? This is called the“works on my machine” syndrome.

#Deployment good practices

For a good deployment process, it’s very important to have immutable build artifacts. Immutable means it won’t change while running; this is very important because you always know exactly what runs on your site.

It’s also important to have reproducible deployment steps, which is like having fire drills. If you repeat your deployment over and over again, you can reduce the response times, for example in case something goes wrong.

It’s critical that the deployment process is not all or nothing. The deployment should be done gradually. Let’s say you have an immutable artifact and you are moving it to production, you should gradually move traffic towards this new version, and if you observe that something goes wrong (this is where the observability features rise on the stage), you should be able to stop that and to move back to a previously stable version.

Data migrations should be integrated into the deployment process. Data migrations mean, for example, adding a new table into the database. This also needs to be repeatable and should be part of the deployment pipeline.

#Existing solutions compared

We’ve evaluated some of the alternatives when it comes to development or deployment. First is the now-classic Vagrant plus FTP, where you can get code versioning pretty easily and it’s fairly simple to have dependency management. However, by just copying your files to a target, that’s not a reproducible pipeline, that’s not a building pipeline. Also, it’s very hard to have a similar local Vagrant environment to the destination remote server.

Next, when it comes to managed WordPress hosting, pretty much of the WordPress hosting ecosystem offers some form of code versioning, which is pretty cool, but I’m unaware of it offering dependency management, a build pipeline or a way to reproduce a build. By using a managed WordPress hosting, it’s very difficult to have the same environment both locally and remotely, because it’s impossible to bring your host’s infrastructure to your laptop.

#Demo time!

The key part in any technical presentation is the demo, where you show practical examples and use cases. I’ve prepared an easy to follow demo on how you can use the Stack to deploy a WordPress site, install some plugins, as well as configure a local environment.

You can easily reproduce the demo yourself, just follow the steps from github.com/presslabs/wordcamp-london-2019.

#Prerequisites

• A local Kubernetes cluster for local development (e.g. Docker for Desktop). With Docker for Desktop, you just have to click “Enable Kubernetes” and that’s it.
• A remote Kubernetes for production (e.g. Google Kubernetes Engine, we’ve been using it succesfully for our current dashboard)
• Local toolchain: composer, wp-cli, helm, skaffold; every WordPress developer should be familiar already with composer and wp-cli; helm and skaffold are “Kubernetes related”, but are easily installable.

#Demo steps

1. Install Stack on the local cluster and on the production cluster

For the demo at WordCamp London, I started with a pre-installed Stack locally and on Google Kubernetes Engine to keep the demo short, but all the steps on how to set these up are detailed on the demo on GitHub.

2. Create a project locally, based on Bedrock

Using roots.io’s Bedrock, create a project into the wclondon-2019 folder. You get a WordPress setup in a folder with a single command line:

$ composer create-project roots/bedrock wclondon-2019
$ cd wclondon-2019

You also need to install the WordPress version of the WordPress runtime you are going to use in production. Like I mentioned before when describing the WordPress Operator, there is a WordPress Runtime, which is essentially the PHP environment that your site runs in.

Before installing the WordPress Runtime, you need to remove roots.io’s WordPress, and then replace it with the Stack version:

composer remove roots/wordpress
composer require presslabs-stack/wordpress ^5.1

Now that you are set up, do the git init and consider this the initial commit.

git init
git add .
git commit -m "Initial commit"

You also need to initialize the Stack environment locally. It’s already provided as a wp-cli command:

wp stack init

You will need to insert the following:

• Docker image repository: you need to have a docker repository created, where you are going to publish images to, for example docker.io/USERNAME/wplondon-2019
• Your site’s development domain or where do you want your local site served; we own the domain localstack.pl, and we always point it to your localhost, so whatever you put in front of it will point to your localhost, for example wclondon-2019.presslabs.pl.
• Your site’s production domain
• Your site’s production kubeconfing context, in other words, what’s your Kubernetes configuration for your global cluster. You can obtain it by running:

 kubectl config get-context

Now you are good to go!

3. Install some plugins by using composer require

composer require wpackagist-plugin/debug-bar

4. Launch and configure a project locally using skaffold dev

To see how your newly created site run on your local environment, just run:

skaffold dev

If you browse to your selected local path, you should see a fresh WordPress install ready to be configured. After you configure it, go to the Plugins section to see your newly installed plugin. You’ll also notice you can’t install plugins from the WordPress interface, this is disabled by the Stack’s WordPress Runtime.

If you try to install another plugin, for example:

composer require rarst/laps

Skaffold detects that you installed a new plugin and it will rebuild the local Docker image you’re seeing on your site.

5. Build an image and deploy to GKE using skaffold deploy

To deploy to production, switch the context you’re deploying to and then run skaffold deploy:

kubectl config use-context MY_PROD_CONTEXT
skaffold deploy

This will push your container up to your registry, and it’s going to deploy it.

To check out the production environment, type:

watch kubectl get po

You will see that it deployed your WordPress, an instance for memcached and the MySQL. It’s also running the WordPress cron in the background.

#Developer’s benefits of using Stack

• Standard dependency manager: Composer
• Testable, with a build pipeline: Docker
• Automated deployment pipeline: Skaffold
• Immutable deployment: containers on Kubernetes

#Future plans

First, we admit that in the current beta state, the Stack is still a little too complex to deploy quickly in production, but are working hard to improve that by integrating it with various market places around the web. We’re starting with the Google Cloud Marketplace.

We plan to create flavoured bundles such as for independent journalism where privacy and data ownership are important and for WooCommerce sites who do mega-sales campaigns, where elasticity and auto-scaling are essential.

We are also building a dashboard on top of the Stack, so you get a graphical overview on everything.

We invite you to contribute to the project on GitHub with code, ideas, feedback, whatever you think it’s not there yet.

If I got your interest, we can schedule a free guided tour of the Stack, simply register on www.presslabs.com/stack/