Continuous Integration for Unity 5.6.1f1 with Docker and Bitrise

I bet you know already the productivity boost you get by adopting continuous integration processes, so I will roughly skip that part. The moment you stop doing things manually you will realize how well you can focus on the work that really matters.

Introduction

CI processes are very well known in most areas of software development, although difficultly applied to Unity projects. So what makes CI hard to implement in Unity projects?

• Multiplatform support.Sure, Unity does marketing very well on insisting on how easy it is to develop for many platforms at once. In practice it is, however, not as easy as “one click and go”. Every platform has its own workflows that have to be individually considered (especially iOS).
• Limited command line support.Unity offers a CLI for building; feature that is theoretically pretty awesome for headless systems. However, it tends to break and it is not well documented. Everyone who had to deal with batchmode, no graphics, force opengl etc. may confirm you that.
• Limited Linux support.Most CI tools run very well in Linux, but sadly the Linux version that Unity offers is still experimental. They work well enough and are very often up to date but the possibility exist that you might come to edge cases that break your build. It may be reasonable to use a powerful Mac machine, since there you can build most common targets. However, considering the lower prices for more powerful linux machines, we decided to go for Linux so as to shorten the build times.
• License management.For building a project you have to activate a license, limiting the amount of parallel builds you may have.

I will keep this document short; my primary goal is to show you that such a CI system is possible, feasible and powerful.

Process overview

A quick summary of the process might prove to be helpful before we begin, so here we go:

1. A developer pushes commits to a branch.
2. The push triggers a build job in a tool such as Jenkins.
3. A docker container is instantiated based on an image that was built offline.
4. Bitrise CLI is invoked to perform the build steps defined in a yaml configuration file.

The basic idea is to have a service that executes the builds whenever a branch is pushed or we trigger a build through REST API calls (through the Unity editor, for instance). For that we went for Jenkins. It was as easy as setting the repository and adding a unique build step that triggers our build process with a one-line shell script. What is that magic line?

The line I am talking about looks like docker-compose run android. Basically it tells docker to instantiate a container based on a concrete image that has everything pre-installed and run the android build process that is described in a config file. The image used to spawn a container contains a specific version of Unity (5.6.1fx1 at the moment of writing), the correct SDK and NDK versions and every tool that is necessary for the build pipeline. That way, we avoid run-time installs that would only slow down the process.

Inside the container, Bitrise CLI is ran. In our case, it does the following:

1. Activate Unity license. The data is injected per environment variables.
2. Trigger the Unity build based on Unity’s command line.
3. Deactivate Unity license.
4. Store and upload artifacts to third-party providers such as S3, HockeyApp, FTP, etc..
5. Notify team members (Slack).

And that is it.

About Docker

Shortly explained, Docker is a technology that allows the user to create, run and destroy containers on the fly. Containers are lightweight virtual machines that are based on images. Those images are like templates that provide the environment needed for our purposes. The spawning process is really fast in comparison to virtual machines, usually taking less than a second. That way we isolate the build processes from each other so they can run in parallel and even in different machines.

The biggest advantage of this paradigm is the flexibility offered to parallelize, scale and relocate those containers without having to manually set up the operating system environments each time in every build node. You create an image once and you deploy it endlessly in different machines, since the configuration is stored in the image. And since they are containers, we don’t incur in a noticeable performance overhead.

I actually built two images for our purposes:

• unity-base-image: it contains all the common software Unity needs to run on Linux, such as xvfb, the correct versions of android sdk/ndk, etc..
• unity-image-5.6.1f1: that image is based on unity-base-image and all it does is downloading Unity and installing it.

A part of my docker-compose.yml file looks like this:

android:
  image: unity-image-5.6.1f1:v1.1
  command: bitrise run android
  privileged: true
  environment:
   - UNITY_EMAIL
   - UNITY_PASSWORD
   - UNITY_SERIAL
   - HOCKEYAPP_ANDROID_APP_ID
   - HOCKEYAPP_API_TOKEN
 volumes:
   - ./:/bitrise/src
   - /var/run/docker.sock:/var/run/docker.sock

As you may notice, I am passing some (secret) parameters by environment variables set by Jenkins so they don’t get versioned in the config files.

About Bitrise CLI

Bitrise CLI is a command line application that executes build processes. You define your different workflows and steps in a Yaml file and then they are executed sequentially. One of the most powerful concepts behind it is the speed and simplicity of adding steps.

The bitrise build steps are open-source and you may create your own. There are very useful ones, such as uploading to HockeyApp/S3/FTP, deploying with Fastlane, archiving and signing with XCode, downloading and extracting compressed files and surely shell scripts can also be executed. For instance, I created custom steps to activate and deactivate the Unity licenses and also to trigger Unity builds. The steps are meant to be reusable and they support versioning.

The Android part of my bitrise.yml looks like this:

  android:
  steps:
  - git::https://github.com/rubentorresbonet/activate-unity-license@master:
    title: Activate Unity License
  - git::https://github.com/rubentorresbonet/unity-build-ubs@master:
    title: Trigger Android build
    inputs:
      - project_path: $PROJECT_PATH
      - build_collection: $BUILD_COLLECTION
  - git::https://github.com/rubentorresbonet/deactivate-unity-license@master:
    title: Deactivate Unity License
  - git::https://github.com/bitrise-io/steps-hockeyapp-deploy@master:
    title: Upload APK to HockeyApp
    inputs:
    - ipa_path: $BUILDS_PATH/Android.apk
    - app_id: $HOCKEYAPP_ANDROID_APP_ID
    - api_token: $HOCKEYAPP_API_TOKEN
    - notify: "0"
  - git::https://github.com/bitrise-io/steps-slack-message@master:
    title: Post build link to slack
    inputs:
      - webhook_url: $SLACK_WEBHOOK_URL
      - message: "Android Build: $HOCKEYAPP_DEPLOY_PUBLIC_URL"
      - emoji: ":panda_face:"
      - emoji_on_error: ":cryingpanda:"
envs:
  - BUILD_COLLECTION: Assets/BuildCollections/Android.asset
  - PROJECT_PATH: $BITRISE_SOURCE_DIR/MyUnityProject
  - BUILDS_PATH: $PROJECT_PATH/Builds
  - SLACK_WEBHOOK_URL: https://hooks.slack.com/services/.....
  - SLACK_CHANNEL: "#ci"

Other platforms can be targeted similarly, with iOS being different at the end since we are running this process in Linux. In the iOS case and when the xcode project has been generated in Linux, I upload it to the online version of the bitrise services that takes care of archiving and signing it, as well as posting slack notifications, etc.. The online service has the advantage of running the builds in a mac virtual machine that has xcode preinstalled and also understands the bitrise yaml file.

Why the trouble?

It might look difficult. Because it is. But well, build pipelines are never easy with Unity. So, why bother?

However, surely there are some disadvantages!

Conclusions

I am pretty satisfied with the results so far. Not only does it run very quickly (3 to 4 minutes for a medium-sized WebGL and Android project) but also it is very flexible and scalable. I successfully build our game for iOS, WebGL and Android (il2cpp).

Let me know if you have any questions!