Continuous Integration

The Basics The Basics

Before you actually add unit tests, work out what type of testing you’re adding. Is it style checks (i.e. phpcs) only? Or do you want functional, unit, or integration tests (i.e. PHPUnit) too?

You’ll also need to decide on a testing service. Open source projects should usually be on Travis, while private projects can use either Travis or CircleCI. There are minor differences between the two… [TODO]

Setting Up Your Tests Setting Up Your Tests

The first real step to adding CI is to have something to test. It’s best to set up your tests first on a VM, as this means you don’t have to simulatenously debug your tests and the service you’re using.

This is a very basic set up for the purposes of this guide. Want to go more in-depth? Check out the Writing Tests guide.

For this guide, let’s run a basic PHPUnit test. We’ll first create a tests directory for our unit tests to live in, and add a very basic unit test. Here’s the test we’ll use:

<?php

namespace HM\Example\Tests;

use PHPUnit\Framework\TestCase;

class ExampleTest extends TestCase {
    public function test_this_is_loaded() {
        $this->assertTrue( true );
    }
}

Save this to tests/class-exampletest.php.

We then need to create a minimal PHPUnit configuration.

<phpunit>
    <testsuites>
        <testsuite>
            <directory prefix="class-" suffix=".php">tests</directory>
        </testsuite>
    </testsuites>
</phpunit>

Save this to phpunit.xml.dist. (The extra .dist allows local overrides with a regular phpunit.xml instead.)

This will run all PHP files in the tests directory starting with class-. (We don’t want to run namespace.php or bootstrap files.)

You should then be able to run the tests with phpunit. Ensure this runs correctly before continuing.

Configuring the Test Service Configuring the Test Service

You should have picked a testing service, so follow the guide for the appropriate service.

Travis Travis

Travis requires a YAML configuration file called .travis.yml in your project’s root directory. Create that, and add the following to it:

language: php
php:
  - 5.6

Important: Do not use tabs in the YAML file, as this may cause it to fail parsing.

By default, Travis will install any Composer dependencies automatically, then run phpunit to run your tests.

You then need to tell Travis to run this repository. Head to travis-ci.com (.com, not .org) and log in via GitHub. Once you’ve connected your account, head to your profile, then to the Human Made organisation profile.

For repositories hosted under client organisations, they will need their own paid Travis account. Head to billing.travis-ci.com to set this up.

Hit the switch to turn on testing for the repository you want to test. This will automatically set up the required webhooks to integrate. Your first build will start after you push for the first time after activating Travis.

For more information about Travis configuration, check out the Travis documentation.

CircleCI CircleCI

CircleCI doesn’t require configuration by default, but can be specified if you want to override the defaults. If you don’t specify configuration, CircleCI will automatically infer the settings.

By default, CircleCI will detect Composer and npm dependencies, and run phpunit.

To set up a repository, head to circleci.com and log in via GitHub. Once you’ve connected your account, head to the Add Projects page, find the project you want to test, and hit the “Build project” button.

For more information about CircleCI configuration, check out the CircleCI documentation.

Adding Additional Tests Adding Additional Tests

We have PHPUnit running, but what if we want to add extra tests (such as phpcs)? Let’s step through setting up the coding standards style checks. To do this, we’ll first install the coding standards into the repo:

composer require --dev humanmade/coding-standards

We then have a command we need to run to test: vendor/bin/phpcs --standard=vendor/humanmade/coding-standards .

Travis Travis

Extra build steps can be specified by overriding the script configuration in your .travis.yml file. By default, this is just phpunit, but you can add as many steps as you want here:

script:
  - phpunit
  - vendor/bin/phpcs --standard=vendor/humanmade/coding-standards .

Note that even if one step fails, all steps will be run; in this case, this means that phpcs will still be run even if your PHPUnit tests fail.

CircleCI CircleCI

Extra build steps can be specified by overriding the test.override configuration in your circle.yml file. By default, CircleCI will attempt to infer this from your project layout, but you can set this instead:

test:
  override:
    - phpunit
    - vendor/bin/phpcs --standard=vendor/humanmade/coding-standards .

Private Dependencies Private Dependencies

If the repository you want to test is a private one, chances are high that you also have private dependencies. These could exist, for example, in the form of Git submodules, or as dependencies that you pull in via Composer. This means that the CI service needs to have (read) access to these private repositories to be able to test the whole project codebase.

Travis Travis

For Travis, there is extensive documentation on private dependencies.

Assuming the private repository is hosted on GitHub, there is a dedicated GitHub machine user account, humanmade-travis, that is already a member of the Human Made organisation. To be able to run Travis on a repository with private dependencies, you first need to add that user account to the repository you want to test, as well as all the private repositories. Please note that granting read access is enough.

CircleCI CircleCI

[TODO]

Debugging and Testing Your Builds Debugging and Testing Your Builds

It’s pretty common for things to go wrong with your build setup, as there are lots of moving parts. Debugging can also be tough since the tests are run on a remote machine which you typically can’t access. Thankfully, there are ways to debug these.

Travis Travis

Travis publishes the Docker containers used to run the builds, and you can run these locally if you have Docker installed.

The Docker images are available on quay.io, and can be run through the docker CLI:

# Download and run the Docker image
docker run -it quay.io/travisci/travis-php /bin/bash

# Run as the travis user
su - travis

# Clone your project
git clone git@github.com:humanmade/test-project.git
cd test-project

# Run your build task
phpunit

CircleCI CircleCI

CircleCI allows debugging through two different methods: Docker images, or direct SSH. The Docker setup is similar to the Travis setup above, but you’ll almost always want to use direct SSH.

After running your build, head to the “Debug via SSH” tab, and enable SSH for your build. This will boot the image and allow access for 30 minutes.

You can then copy the SSH command it gives you, and run the build steps:

# SSH into the machine. Copy this from your CircleCI build output, not from here.
ssh -p 64568 ubuntu@54.87.108.38

# Head into the project directory.
cd test-project

# Run your build task
phpunit

CircleCI sets up the machine using your GitHub SSH keys. If you use different sets of keys locally, pass -i ~/.ssh/your_github_key with the SSH command to set the identity file manually.