← Zur Übersicht

Automating Mobile app Deployment

  1. Introduction and Scope
  2. Accounts and Certificates
    1. Apple
    2. Google
  3. GUI-less building and deployment
    1. Preparing your dev machine
    2. Preparing the app to be buildable
    3. Setting up fastlane
    4. Testing the Setup
  4. Automating the Process
    1. Setting up the Jenkins environment
    2. Configuring Jenkins
    3. Setting up the build Pipeline
    4. Testing the Pipeline
  5. Conclusion

1. Introduction and Scope

This is a guide on how to automate the building and deployment of (multi platform) mobile apps. And this is something that really needs automation, because deploying an app manually can be a real pain and cost a lot of time and money:

  • Xcode can't find your distribution certificate or provisioning profile?
  • Preparing a new release takes hours every time?
  • Everything is broken after switching computers, because of some missing magic configuration from month ago?
  • The build works on one computer, but has bugs when building on another?
  • You can't release because "the guy with the working build setup" is on vacation?

These are all hassles that can be fixed with an automated, consistent and reproducible setup that works the same everytime and for everyone! And while there is a lot to configure and set up, the process itself is quite simple:

  1. Get all the necessary credentials
  2. Make building and deploying GUI-less
  3. Set up building/deploying on commit

When we're done, the whole process from "merging changes" to "the testers getting update notifications" will be fully automated. This blogpost assumes that you already have a Jenkins (or other continuous integration) server and an app. We're just here to connect the two.

I set up a Jenkins and a demo application that we'll automate in this post. Use it as a reference for a working example.

In this guide we will use the following tools*:

Tool Description
Git + GitHub Version control system that hosts the source code
Docker Containerization software for consistent build environments
fastlane Tool to simplify mobile app deployment with the command line
Jenkins Automation server

*The steps should be adaptable to other tooling, but I'll be using and catering to these specific tools and their quirks.

2. Accounts and Certificates

We need a couple of credentials to build/sign and upload the app, that might or might not already exist. This section is about what credentials are needed and how to get or create them.

2.1 Apple

For the Distribution of apps to app store connect we need 3 things:

  1. Set up an Apple-Id for Jenkins if you don't already have one
    • Make sure the email address doesn't contain a +, as this will prevent you from inviting this account to app store connect
    • Make sure 2-Factor-Authentication is disabled for this account
    • Don't use special characters in the password, otherwise this happens!
    • Go to developer.apple.com > account > people and invite this account as Member
    • Go to appstoreconnect > Users and Access and invite this account as Developer for your app
  2. Obtain a distribution certificate and its password for iOS App
    • Go to developer.apple.com > account > certificates, IDs & profiles
    • Click on Certificates > Production
    • If there already is a certificate of type iOS Distribution:
      click on it, see who created it and ask that person to export the certificate + private key from their keychain as .p12 file (see below for instructions)
    • If there is no certificate of type iOS Distribution:
      create a new one and then export it from your keychain (see below for instructions)
  3. Obtain a provisioning profile for the app
    • Go to developer.apple.com > account > certificates, IDs & profiles
    • Click on Provisioning Profiles > Distribution
    • If there aleady is a provisioning profile for the certificate you just obtained select it and download it
    • If there is no provisioning profile for the certificate you just obtained, create a new one (see below for instructions)
Creating and installing a distribution certificate
  1. Go to developer.apple.com > account > certificates, IDs & profiles and click on Certificates > Production
  2. Click on the + in the top right
  3. Select Production > App Store and Ad Hoc as certificate type
  4. Open your Keychain Access app
  5. Select the login-keychain
  6. Under Category select key
  7. Search for the public key with your Apple ID and highlight it
  8. Select Keychain Access > Certificate Assistant > Request a Certificate From a Certificate Authority...
  9. Enter the email address of your apple-ID (not the one for Jenkins) as User Email Address
  10. Enter some common name like "5minds ios distribution"
  11. Leave the CA Email Address empty
  12. Select Save to Disk and click "Continue"
  13. Go back to the browser, click "Continue" and upload the csr-file you just created
  14. Download the certificate and click "Done"
  15. Doubleclick the certificate to install it
Exporting the distribution certificate
  1. Open your Keychain Access app
  2. Select the keychain that contains the distribution certificate
  3. Under Category select My Certificates (This is important!)
  4. Search for the distribution certificate and click on the little triangle to the left of it
  5. Shift-Select both, the certificate and the key to it
  6. Rightclick and select "Export 2 Items..."
  7. Save the certificate somewhere as .p12
  8. Select a password for the certificate (and remember it!)
Creating a provisioning profile
  1. Go to developer.apple.com > account > certificates, IDs & profiles and click on Provisioning Profiles > Distribution
  2. Click on the + in the top right
  3. Select Distribution > App Store as profile type
  4. Select your app or Wildcard as AppID and click "Continue"
  5. Select the certificate that you have access to and click "Continue"
  6. Select a profile name like ci_ios_distribution and click "Continue"
  7. Download the profile and click "Done"

2.2 Google

For the Distribution of apps to the play store we'll need two things:

  1. Set up a Google Service Account for Jenkins if you don't already have one As described here
    1. Open the Google Play Console
    2. Click the "Settings" menu entry, followed by "API access"
    3. Click the "CREATE SERVICE ACCOUNT" button
    4. Follow the Google Developers Console link in the dialog, which opens a new tab/window:
      1. Click the "CREATE SERVICE ACCOUNT" button at the top of the "Google Developers Console"
      2. Provide a Service account name
      3. Click "Select a role" and choose "Service Accounts > Service Account User"
      4. Check the "Furnish a new private key" checkbox
      5. Make sure "JSON" is selected as the Key type
      6. Click "SAVE" to close the dialog
      7. Make a note of the file name of the JSON file downloaded to your computer
    5. Back on the "Google Play Console", click "DONE" to close the dialog
    6. Click on "Grant Access" for the newly added service account
    7. Choose "Release Manager" from the Role dropdown
    8. Click "ADD USER" to close the dialog
  2. Obtain the keystore and its credentials
    • you need the keystore file, its password, the alias of the private key and the password of that private key
    • If the app was ever uploaded at all, then someone has to have this keystore. Go and get it from that person
    • If the app was not yet uploaded to the play console, you need to create a new keystore containing a singing key
Creating a Keystore with signing key A Keystore ist a file that can contain multiple private/public keypairs. the private keys can be password protected, and the keystore itself can also be password protected, but as described here, they have to be the same password!

Create the keystore as described here:

  1. Go to a trusted computer with Java installed
  2. Create the keystore with this command
    keytool -genkeypair -v -keystore my-release-key.keystore -alias alias_name -keyalg RSA -keysize 2048 -validity 10000
    1. Replace my-release-key.keystore with your desired name for the keystore file
    2. Replace alias_name with a name for the singing key within the keystore
    3. You will be asked for a password for the keystore file and for the key within it. Choose a strong one (but without special characters), and don't lose it!
  3. Store the keystore, the name of the key and the password safely!

3. GUI-less building and deployment

To be able to automate the build and deployment process, we need it to work fully non-interactive, and without any GUI. fastlane is our friend here, as it will allow us to manipulate, build and upload the app from the command line.

We'll also already be using Docker here for our android builds. This has a couple of benefits:

  • You don't need the Android SDK to be installed
  • The build environment is always identical (!)
  • It's closer to what we'll later use in the Jenkins pipeline

We unfortunately cannot use Docker for the iOS build, as you can't run Xcode in a Docker container.

3.1 Preparing your dev machine

3.1.1 For iOS

Install these applications:

Application Install with
Xcode from the app store
Xcode cli tools xcode-select --install
Homebrew see the homebew website
Ruby brew install ruby
fastlane brew cask install fastlane

Start Xcode once after its installation! It will install additional components, and without starting it once your distribution certificate won't be trusted. You don't have to open any project.

Ruby and fastlane are installed now, but their binaries are not in the PATH-variable, so you can't just call them from everywhere. This can be set up by creating adding this to your ~/.bash_profile or ~/.zshrc.

.bash_profile for ruby and fastlane

export PATH="$HOME/.fastlane/bin:$PATH"
export PATH="/usr/local/lib/ruby/gems/2.5.0/bin:$PATH"
export PATH="/usr/local/opt/ruby/bin:$PATH"

You need to source the file you just edited, or open a new terminal window for the changes to take effect.

3.1.2 For Android

You literally just need to install Docker with

  • brew cask install docker (on a mac)
  • or as described here (on linux)

3.2 Preparing the app to be buildable

Skip this step if your app doesn't need preparation to be buildable, but make sure your android apps build.gradle contains the settings versionCode and versionName!

Because the demo app is an Ionic app, we need to transform it into an Android project and an Xcode project before being able to build it. You'd usually use
ionic cordova build <platform> for this, but that would also try to run the app.

Because of that we use ionic cordova prepare <platform> instead. It does the same without trying to run the app. We add these scripts to the package.json:

package.json > prepare scripts

"scripts": {
  "prepare_android": "ionic cordova prepare android --prod --release",
  "prepare_ios": "ionic cordova prepare ios --prod --release"
}

We will later set the app version and build number through environment variables before building the app. Using fastlane, this is easy for iOS apps, but a little tricky for android apps. To do so, you usually set the values for versionName and versionCode in your build.gradle (As described here), but the platforms/android/build.gradle is auto-generated by cordova and cordova doesn't add the versionName config. We need to somehow add these configs, so they can be edited before actually building the app.

The cordova way to manipulate the build.gradle is using a build-extras.gradle, that defines these configs. So we add that file to the root of the ionic project with the following content (don't worry about the actual numbers, they will later be replaced by fastlane):

build-extras.gradle

ext.postBuildExtras = {
  android.defaultConfig.versionCode = 1
  android.defaultConfig.versionName = "0.0.1"
}

Cordova expects this file to be next to the build.gradle in the platforms folder. However, it doesn't automatically put it there, so we have to do it ourselves using a hook that we set up in our config.xml (in the android-node) that looks like this:

config.xml > before_prepare hook

<platform name="android">
    <hook src="android_before_prepare.js" type="before_prepare" />
    ...
</platform>

As you can see, this will call android_before_prepare.js before preparing the android project. In this file, we'll copy over our build-extras.gradle into the platforms folder. Add this file to the root of the ionic project, with the following content (Don't omit the shebang! It's important, as described here):

android_before_prepare.js

#!/usr/bin/env node

const fs = require('fs')
const path = require('path');

module.exports = function(context) {
  const Q = context.requireCordovaModule('q');
  const deferral = new Q.defer();

  const rootFolder = context.opts.projectRoot;
  const source = path.join(rootFolder, 'build-extras.gradle')
  const target = path.join(rootFolder, 'platforms', 'android', 'build-extras.gradle')
  fs.copyFile(source, target, (error) => {
    if (error !== undefined && error !== null) {
      console.log(error);
      return deferral.reject(error);
    }

    deferral.resolve();
  });

  return deferral.promise;
}

After that treatment, the app is ready to be handled by fastlane!

3.3 Setting up fastlane

3.3.1 Initializing fastlane

If you're using Ionic, make sure you currently have no platforms, plugins or node_modules folder (they confuse the initial fastlane setup, because fastlane will think that there already are Xcode projects, and will try to use these)

Go into your project folder and run fastlane init. It might nag about not being in an android or ios project, and ask if you want to manually set up a fastlane config in the current folder. Answer with yes.

fastlane will create this file structure in your project:

├── fastlane
│   ├── Appfile
│   └── Fastfile
├── Gemfile
├── Gemfile.lock

fastlane is also sort of a task runner, but as we'll later handle most of the task running with Jenkins, you can safely delete the Appfile, and the content of the Fastfile.

For android projects, you'll need a plugin to set the apps version, so install it with:

bundle exec fastlane add_plugin versioning_android

If you're using Ionic you'll also need to install a plugin to upgrade your xcodeproj file to a newer format, because the Xcode project generated by cordova is in an old format:

bundle exec fastlane add_plugin upgrade_super_old_xcode_project

If fastlane asks you to update your Gemfile, allow it to do so.

3.3.2 Creating the fastlane steps

We will define 6 Lanes in the Fastfile, 3 per platform (Android/iOS):

  1. One to prepare the project (set the version, define code signing etc.)
  2. One to actually build the app
  3. One to upload the build

We'll control most of the values used in the Fastfile with environment variables. This way Jenkins can have a lot of control over the build, and it is easier to adapt to other projects. The Fastfile content for this demo is therefore very generic, so you can probably copy/paste most of it, with only few adjustments required

iOS lanes:

Here are the lanes used for the demo application. If your fastlane folder is directly next to your xcodeproj, then you can remove the path or xcodeproj value from most of these steps.

Fastfile > iOS lanes

lane :prepare_ios do
  # xcodeprojects created by ionic use some old format, so we need to update it.
  # You probably don't need this, if you don't use ionic
  upgrade_super_old_xcode_project(
    path: "platforms/ios/Deploydemo 5Minds.xcodeproj",
    team_id: ENV["APPLE_TEAM_ID"]
  )

  # To maintain maximum reproducability, we handle code signing ourselves.
  # This is harder to set up, but this way we don't rely on some magic services
  # that we don't really understand and that could change without our knowledge
  disable_automatic_code_signing(
    path: "platforms/ios/Deploydemo 5Minds.xcodeproj",
    code_sign_identity: "iPhone Distribution",
    team_id: ENV["APPLE_TEAM_ID"]
  )

  # Tell xcode to use our specific provisioning profile (obtained from jenkins)
  update_project_provisioning({
    xcodeproj: "platforms/ios/Deploydemo 5Minds.xcodeproj",
    profile: ENV["PROVISIONING_PROFILE_FILE"],
    build_configuration: "Release"
  })

  # the combination version + build must be unique, so set them here
  increment_version_number(
    xcodeproj: "platforms/ios/Deploydemo 5Minds.xcodeproj",
    version_number: ENV["APP_VERSION"]
  )

  # set the build number using set_info_plist_value, because
  # increment_build_number isn't working (it always says "Apple Generic
  # Versioning is not enabled in this project"), see https://github.com/fastlane/fastlane/issues/9506
  set_info_plist_value(
    path: "platforms/ios/Deploydemo 5Minds/Deploydemo 5Minds-Info.plist",
    key: "CFBundleVersion",
    value: ENV["BUILD_NUMBER"]
  )

  # If you'd like to build multiple apps with identical settings, you could add this
  # to set name and identifier based on environment variables:
  #update_info_plist(
  #  xcodeproj: "platforms/ios/Deploydemo 5Minds.xcodeproj",
  #  plist_path: "Deploydemo 5Minds/Deploydemo 5Minds-Info.plist",
  #  app_identifier: ENV["IOS_APP_IDENTIFIER"],
  #  display_name: ENV["IOS_APP_NAME"]
  #)

  # Tell apple that we do use don't use any **non-exepmt** encryption. If you DO
  # use non-exempt ecryption, set this to yes and enable the following block.
  # Setting this value allows us to skip the manual export compliance step in
  # App Store Connect, which in turn allows us to directly push the build to testers.
  set_info_plist_value(
    path: "platforms/ios/Deploydemo 5Minds/Deploydemo 5Minds-Info.plist",
    key: "ITSAppUsesNonExemptEncryption",
    value: false
  )

  # If you DO use non-exempt encryption enable this block and enter you compliance
  # code, which you can find in App Store Connect.
  #
  #set_info_plist_value(
  #  path: "platforms/ios/Deploydemo 5Minds/Deploydemo 5Minds-Info.plist",
  #  key: "ITSEncryptionExportComplianceCode",
  #  value: "YOUR_COMPLIANCE_CODE_HERE"
  #)
end

lane :build_ios do
  # Tell xcode to use the provisioning provile we provide to build the app
  build_ios_app(
    workspace: "platforms/ios/Deploydemo 5Minds.xcworkspace",
    configuration: "Release",
    clean: true,
    export_options: {
      export_method: "app-store",
      provisioningProfiles: {
        "de.fiveminds.deploydemo" => ENV["PROVISIONING_PROFILE_ID"]
        # use the following if you build mutliple apps with this fastfile:
        #ENV["IOS_APP_IDENTIFIER"] => ENV["PROVISIONING_PROFILE_ID"]
      },
      skip_profile_detection: true
    }
  )
end

lane :upload_ios do
  upload_to_testflight(
    username: ENV["APPSTORECONNECT_USER"]
  )
end 

Android lanes:

Here are the lanes used for the demo application. If your fastlane folder is within your android project root, then you can remove the project_dir or gradle_file value from most of these steps.

Also remember to adjust the package name and the apk-path to wherever gradle puts your apk-file

Fastfile > Android lanes

lane :prepare_android do
  # the combination version + build must be unique, so set them here
  android_set_version_name(
    version_name: ENV["APP_VERSION"],
    # use your build.gradle here for non-cordova projects
    gradle_file: "platforms/android/build-extras.gradle"
  )

  android_set_version_code(
    version_code: ENV["BUILD_NUMBER"],
    # use your build.gradle here for non-cordova projects
    gradle_file: "platforms/android/build-extras.gradle"
  )

  # If you'd like to build multiple apps with identical settings, you could add this
  # to set name and identifier based on environment variables:
  # (Remember adjust the project_path, the (old) package_name and set the ANDROID_APP_IDENTIFIER environment variable)
  # this runs from the fastlane-folder, so go to root first.
  #sh(
  #  'project_path="../platforms/android"; '\
  #  'package_name="de.fiveminds.deploydemo"; '\
  #  'new_package_name="' + ENV["ANDROID_APP_IDENTIFIER"] + '"; '\
  #  ''\
  #  'folder=$(echo "${package_name}" | sed "s/\./\//g"); '\
  #  'new_folder=$(echo "${new_package_name}" | sed "s/\./\//g"); '\
  #  'new_folder_path="${project_path}/app/src/main/java/${new_folder}"; '\
  #  ''\
  #  'mkdir --parents ${new_folder_path}; '\
  #  'mv ${project_path}/app/src/main/java/${folder}/*.java "${new_folder_path}/"; '\
  #  'find ${project_path}/app/src -name \'*.java\' -type f -exec sed -i "s/${package_name}/${new_package_name}/" {} \\;; '\
  #  'find ${project_path}/app/src -name \'AndroidManifest.xml\' -type f -exec sed -i "s/${package_name}/${new_package_name}/" {} \\;; '\
  #  'find ${project_path}/app -name \'build.gradle\' -type f -exec sed -i "s/${package_name}/${new_package_name}/" {} \\; '
  #)
  #
  # this runs from the fastlane-folder, so go to root first.
  # (Remember to correct this path to point to your strings.xml and set the ANDROID_APP_NAME environment variable )
  #sh(
  #  'sed -i '\
  #  '\'s/"app_name"\([^>]*>\)[^<]*/"app_name"\1' + ENV["ANDROID_APP_NAME"] + '/g\' '\
  #  '../platforms/android/app/src/main/res/values/strings.xml'
  #)
end

lane :build_android do
  build_android_app(
    task: "assemble",
    build_type: "Release",
    project_dir: "platforms/android/",
    properties: {
      "android.injected.signing.store.file" => ENV["KEYSTORE_FILE"],
      "android.injected.signing.store.password" => ENV["KEYSTORE_PASSWORD"],
      "android.injected.signing.key.alias" => ENV["SIGNING_KEY_ALIAS"],
      "android.injected.signing.key.password" => ENV["SIGNING_KEY_PASSWORD"]
    }
  )
end

lane :upload_android do
  upload_to_play_store(
    json_key: ENV["GOOGLE_PLAY_SERVICE_ACCOUNT"],
    package_name: "de.fiveminds.deploydemo",
    # use the following if you build mutliple apps with this fastfile:
    #package_name: ENV["ANDROID_APP_IDENTIFIER"]
    apk: "platforms/android/app/build/outputs/apk/release/app-release.apk",
    track: "alpha"
  )
end

3.4 Testing the Setup

If you've installed the distribution certificate on your dev machine (only required for the iOS build, see 2.1 for more info), we should now be able to build and upload our app using almost no GUI! (You need to adjust ids, logins, passwords and file paths of course)

For the process to be completely GUI-less, some keychain-access-trickery is required. We will later do this when setting up the jenkins pipeline, but it would be overkill for local testing.

When uploading the initial build with fastlane, this error could happen. If it does, just try again. For android builds: The company part of the app-id and the project name MUST NOT begin with a number. You build will fail if they do.

iOS

To test the upload, we need the App Store Connect TeamID. This is NOT your Apple Developer TeamID! To find out your TeamID run the following on a machine that has fastlane installed, and use the AppleID that was set up in 2.1:

Find App Store Connect TeamId

irb
irb> require "spaceship"
irb> Spaceship::Tunes.login("iTunesConnect_username", "iTunesConnect_password")
irb> Spaceship::Tunes.select_team

Now to actually test the setup:

Test iOS lanes locally

# prepare some environment variables
export PROVISIONING_PROFILE_FILE="/Users/heiko/Downloads/ci_ios_distribution.mobileprovision"
export PROVISIONING_PROFILE_ID=$(/usr/libexec/PlistBuddy -c 'Print :UUID' /dev/stdin <<< $(security cms -D -i "${PROVISIONING_PROFILE_FILE}")_)

# install the provisioning profile
cp "${PROVISIONING_PROFILE_FILE}" ~/Library/MobileDevice/Provisioning\ Profiles/${PROVISIONING_PROFILE_ID}.mobileprovision

# cleanup the build environment (ionic specific!)
rm -rf platforms node_modules www
npm install

# prepare the apps project (ionic specific!)
npm run prepare_ios

# prepare the app
PROVISIONING_PROFILE_FILE=${PROVISIONING_PROFILE_FILE} \
APPLE_TEAM_ID="53524734CS" \
APP_VERSION="0.0.4" \
BUILD_NUMBER="10" \
bundle exec fastlane prepare_ios

# build the app (this probably prompts you for your login password a couple of times)
PROVISIONING_PROFILE_ID="${PROVISIONING_PROFILE_ID}" \
bundle exec fastlane build_ios

# upload the app
APPSTORECONNECT_USER="heiko.mathes2@gmail.com" \
FASTLANE_PASSWORD="*****" \
FASTLANE_ITC_TEAM_ID="1310680" \
bundle exec fastlane upload_ios
You should now be able to see the new build in app store connect!

Android

Test Android lanes locally

# prepare some environment variables
export CURRENT_USER=$(id -u)
export CURRENT_GROUP=$(id -g)
export ANDROID_PROJECT_FOLDER="${PWD}/platforms/android"

# cleanup the build environment (ionic specific!)
rm -rf platforms node_modules www
npm install

# prepare the apps project (ionic specific!)
npm run prepare_android

# fastlane requires a gradlew-file that cordova only creates on 'build', not on 'prepare'. So we create this file ourselves
docker run \
  --tty \
  --user=0:0 \
  --workdir "${ANDROID_PROJECT_FOLDER}" \
  --volume "${ANDROID_PROJECT_FOLDER}":"${ANDROID_PROJECT_FOLDER}":rw,z \
  amsitoperations/ams-android-gradle \
  gradle wrapper

# prepare the app
docker run \
  --tty \
  --user=0:0 \
  --workdir ${PWD} \
  --volume ${PWD}:${PWD}:rw,z \
  --env="APP_VERSION=0.0.4" \
  --env="BUILD_NUMBER=10" \
  bigoloo/gitlab-ci-android-fastlane \
  fastlane prepare_android

# build and sign the app
docker run \
  --tty \
  --user=0:0 \
  --workdir ${PWD} \
  --volume ${PWD}:${PWD}:rw,z \
  --env="KEYSTORE_FILE=${PWD}/android.keystore" \
  --env="KEYSTORE_PASSWORD=*****" \
  --env="SIGNING_KEY_ALIAS=deploydemo" \
  --env="SIGNING_KEY_PASSWORD=*****" \
  bigoloo/gitlab-ci-android-fastlane \
  fastlane build_android

sudo chown -R ${CURRENT_USER}:${CURRENT_GROUP} ./platforms/android/*
sudo chown -R ${CURRENT_USER}:${CURRENT_GROUP} ./platforms/android/.*

# upload the app
docker run \
  --tty \
  --user=0:0 \
  --workdir ${PWD} \
  --volume ${PWD}:${PWD}:rw,z \
  --env="GOOGLE_PLAY_SERVICE_ACCOUNT=${PWD}/service_account.json" \
  bigoloo/gitlab-ci-android-fastlane \
  fastlane upload_android
You should now be able to see the new build in the Google Play Console!

4. Automating the Process

Now that we have an app that can be built non-interactively, we need to set up Jenkins to do just that when merges happen.

This is mostly a lot of setup and configuration, but it's all described in this section.
You'll need access to 3 "computers". These can however be the same machine:

  1. One that runs the Jenkins master
  2. One that acts as an Android build node
  3. One that acts as an iOS build node (must run macOS)
Example setups
  • If you don't build iOS apps, having a linux machine that runs the Jenkins master, with another user on that same machine that will act as android build node is fine.

  • If you do need iOS builds, You could use a single mac to run the Jenkins master, with another user on that mac that can be used as android build node and as iOS build node.

  • In the demo setup I run a dockerized Jenkins on a linux server as master, have a user called "jenkins" on that same server as android build node, and my macbook as iOS build node.

4.1 Setting up the Jenkins environment

4.1.1 Setting up the iOS build node

For this we need a mac that can run the latest Xcode and that is available via a fix ip or domain. You ideally choose one that isn't in active use by anyone or for anything. You'll need to setup port forwarding for ssh (port 22) to that machine.

Prepare the mac

  • Update to the latest possible macOS
  • Add a dedicated (admin) user (in this example called jenkins)
  • Enable SSH for that user
    • Go to System Preferences > Sharing
    • Enable Remote Login
    • Select "Only these users"
    • Remove "Administrators" from the list
    • Add the jenkins user to that list
  • Enable public key authentication so you can remote access the jenkins user
    • Login as jenkins on the build-mac
    • create the folder ~/.ssh if it doesn't exist
    • add your personal public key to ~/.ssh/authorized_keys, creating the file if it doesn't already exist
  • Enable public key authentication for the Jenkins-Server (see here for more detailed instructions)
    • create a private/public keypair on a trusted computer of your choosing with
      ssh-keygen -t rsa -m PEM -b 4096 -C "Jenkins agent key" -f "jenkins_agent_rsa"
      (the -m PEM option is important on newer openssh-versions to generate the key in a compatible format)
    • Select a passphrase and remember it (could be left blank, but setting one is more secure)
    • Add the new public key (jenkins_agent_rsa.pub) to the ~/.ssh/authorized_keys of the jenkins user on the build mac
    • The new private key (jenkins_agent_rsa) will later be added to the Credentials available to Jenkins
  • Disable SSH-Login with password (see here)
    • Login as jenkins on the build-mac
    • Edit the sshd-config as root with sudo nano /etc/ssh/sshd_config and uncomment and set these settings:
      • PasswordAuthentication no
      • ChallengeResponseAuthentication no
      • UsePAM no
    • Restart the ssh-service by unchecking and rechecking System Preferences > Sharing > Remote Login

Install the necessary applications (as jenkins user):

Application Install with
Xcode from the app store
Xcode cli tools xcode-select --install
Homebrew see the homebew website
Ruby brew install ruby
fastlane brew cask install fastlane
Java 8 brew tap caskroom/versions
brew cask install java8
Caffeine* brew cask install caffeine
*Remember to start and configure caffeine after the installation
  • Start caffeine (using spotlight)
  • Set it up to auto start, auto launch and don't show the welcome-message
  • Click the little coffee-icon in the menu bar to activate it

Finalizing the Setup:

Start Xcode once after its installation! It will install additional components, and without starting it once your distribution certificate won't be trusted and this happens. You don't have to open any project, just letting it install its additional components is enough.

Ruby and fastlane are installed now, but their binaries are not in the PATH-variable, so you can't just call them from everywhere. fastlane also needs some locale-settings. All this can be set up by creating a ~/.bashrc as the jenkins user on the build-mac with this content:

.bashrc for ruby and fastlane

export PATH="$HOME/.fastlane/bin:$PATH"
export PATH="/usr/local/lib/ruby/gems/2.5.0/bin:$PATH"
export PATH="/usr/local/opt/ruby/bin:$PATH"
export LC_ALL=en_US.UTF-8
export LANG=en_US.UTF-8

The .bashrc will be sourced when the jenkins server logs in with its non-interactive shell.

4.1.2 Setting up the Android build node

For this we need a linux machine or a mac with Docker installed. This could be done without Docker, but using Docker allows us to have a consistent build environment that is easier to replicate locally, and doesn't require any additional installations on the host machine.

Prepare a mac build node

  • Set up ssh access as described in 4.1.1 > Prepare the mac
  • Install Caffeine* and Docker with brew cask install caffeine docker
  • Add Docker your path by adding this to your ~/.bashrc:
    export PATH="/usr/local/opt/ruby/bin:$PATH"
  • Because you can't start Docker on boot without logging into the gui (as described here), you must login to the jenkins user when using a mac with Docker as android build node!
*Remember to start and configure caffeine after the installation
  • Start caffeine (using spotlight)
  • Set it up to auto start, auto launch and don't show the welcome-message
  • Click the little coffee-icon in the menu bar to activate it

Prepare a linux build node

This assumes that the linux machine is a debian or ubuntu system and you already have ssh setup for it.

  • Be logged in as root
  • Add a Jenkins user with adduser jenkins
  • Enable public key authentication so you can remote access the jenkins user
    • Login as the jenkins user on the build-node
    • create the folder ~/.ssh if it doesn't exist
    • add your personal public key to ~/.ssh/authorized_keys, creating the file if it doesn't already exist
  • Enable public key authentication for the Jenkins-Server (see here for more detailed instructions)
    • create a private/public keypair on a trusted computer of your choosing with
      ssh-keygen -t rsa -m PEM -b 4096 -C "Jenkins agent key" -f "jenkins_agent_rsa"
      (the -m PEM option is important on newer openssh-versions to generate the key in a compatible format)
    • Select a passphrase and remember it (could be left blank, but setting one is more secure)
    • Add the new public key (jenkins_agent_rsa.pub) to the ~/.ssh/authorized_keys of the jenkins user on the build node
    • The new private key (jenkins_agent_rsa) will later be added to the Credentials available to Jenkins
  • Disable SSH-Login with password for the jenkins user
    • Login as root on the build-node
    • Edit the sshd-config (/etc/ssh/sshd_config) and add this to the end of the file (see here for more info):
      Match User jenkins
        PasswordAuthentication no
    • Restart the ssh-server with systemctl restart sshd
  • Install the necessary applications (as root user)
    • Java 8 with apt install openjdk-8-jre
    • Docker-CE as described here
      • Add the jenkins user to the docker group with usermod -aG docker jenkins

4.2 Configuring Jenkins

With the credentials and the build nodes setup, we have everything we need to make the necessary Jenkins configurations. We need to:

  • Set up the on-commit webhook
  • Give Jenkins the credentials from 2.
  • [optional] Set up some globally available build tools
  • Install some Jenkins plugins
  • Add the build nodes we just set up
  • Add the build job for the app

4.2.1 Configuring the credentials

GitHub:

Make sure the jenkins server has access to the repository with your app. In this setup i'm using a Personal API Access Token to access the repository.

Make sure to set up the webhook, so that jenkins builds when commits are pushed. See here on how to do that (for github).

Apple:

Go to Jenkins > Credentials > System > Global credentials and add the following credentials (all of which were created in 2.1):

Credential as type example Id
distribution certificate Secret file ios_distribution_certificate_and_key
password for the certificate Secret text ios_distribution_certificate_key_password
provisioning profile Secret file ios_provisioning_profile
Apple account Username and Password ios_distribution_appstore_user

Google:

Go to Jenkins > Credentials > System > Global credentials and add the following credentials (All of which were created in 2.2):

Credential as type example Id
keystore file Secret file android_keystore
password for the keystore Secret text android_keystore_password
key alias Secret text android_signing_key_alias
password for the key Secret text android_singing_key_password
Google service account Secret file google_play_service_account

4.2.2 Configuring the buildtools

Here we make sure that Jenkins has the required tools to prepare the app, so that it becomes buildable. If your app doesn't need preparation to be buildable, you can skip this step.

Ionic/Cordova applications however require NodeJS to generate a buildable project, so i'll go to Jenkins > Manage Jenkins > Manage Plugins and make sure the NodeJS Plugin is installed. After that i go to
Jenkins > Manage Jenkins > Global Tool Configuration and add a new NodeJS-Installation for the current LTS version and call it node-lts

4.2.3 Configuring the jenkins plugins

Go to Jenkins > Manage Jenkins > Manage Plugins and make sure you have the following Plugins installed:

  • Pipeline (So we can define Build Pipelines in Jenkinsfiles in our codebase)
  • Blue Ocean (you really do want this to view parallel tasks!)

4.2.4 Adding the iOS and Android build nodes

This process is nearly identical for both, the iOS build node and the Android build node

  • Go to Jenkins > Manage Jenkins > Manage Nodes and select "New Node"
  • Give it a name (for example macos-node or android-node) and mark it as "Permanent Agent"
  • Click "OK"
  • Define a "Remote root directory", for example /Users/jenkins/ (mac) or /home/jenkins/ (linux)
  • Define a label like fastlane-ios or docker. This will later be used to define what part of the build-pipeline is executed on what machine
  • Tell it to only build jobs with label expressions matching this node. With this we have full control over what gets executed where
  • Set the build servers fix ip or domain as "Host"
  • For the credentials click "Add" and Add new credentials:
    • Choose "SSH Username with private key" as type
    • Set the jenkins user from 4.1.1 (mac) or 4.1.2 (linux) as "Username" (in this example: jenkins)
    • Activate "Enter directly" next to "Private Key"
    • Copy the jenkins-agent-private-key from 4.1.1 (mac) or 4.1.2 (linux) into the textarea
    • Enter the passphrase of the private key if you chose to set one
    • Select a sensible id like ssh_jenkins_macos or ssh_jenkins_linux
    • Click "Add"
  • Select the newly added credentials in the "Credentials" dropdown
  • Select "Manually trusted Verification Strategy" as "Host Key Verification Strategy"
  • If the SSH Port to the build-node is mapped to a port other than 22 (for example when using port forwarding), then click on "Advanced" and set the SSH port
  • Click "Save"
  • Click "Launch agent"

If everything was done correctly, Jenkins will now connect to the machine and set it up!

4.2.5 Adding the build job

Go to Jenkins and create a "New Item" of type "Multibranch Pipeline" and give it a name, for example the name of the app or of the repo.

Add a Branch Source of type "Github", Select your Github credentials, set the repository owner and choose the repository that contains your app from the dropdown.

Set "Discover branches" to "All branches" and remove "Discover pull requests from origin" and "Discover pull requests from forks".

If the "Build Configuration" is set to Jenkinsfile, you can leave the rest as it is. Click "Save".

4.3 Setting up the build Pipeline

Here comes the part where we define what Jenkins actually does when a build is triggered. Everything in this section is solely to create the Jenkinsfile in our project root, as this file is what defines the build pipeline.

When we're done, we'll have a pipeline with a structure that looks about like this:

Pipeline parts

  1. General preparation for the build (like setting up environment variables based on the branch that is being built)
  2. [Optional] A test step that will run tests or try a simple build just to flag commits as buildable or broken
  3. [Optional] Parallel steps to generate the Android project and the Xcode project from our source files. Can be omitted if your source files already are a buildable Android and/or Xcode project
  4. Preparing, building and deploying the Android project on the android build node
  5. Preparing, building and depoying the Xcode project on the iOS build node
  6. Deleting the workspace folders, so that we don't spam the servers hard drive

A Jenkinsfile that represents this structure looks like this (see it in action here) (: is the noop-operator in bash):

Jenkinsfile > base structure

pipeline {
  agent any
  stages {
    // 1
    stage('prepare') {
      steps {sh ':'}
    }

    // 2
    stage('test build') {
      steps {sh ':'}
    }

    stage('prepare build') {
      parallel {
        // 3
        stage("build base android app") {
          steps {sh ':'}
        }

        stage("build base ios app") {
          steps {sh ':'}
        }
      }
    }

    stage('build and deploy') {
      parallel {

        // 4
        stage('Android app') {
          stages {
            stage("setup build dependencies") {
              steps {sh ':'}
            }
            stage("prepare project") {
              steps {sh ':'}
            }
            stage("build") {
              steps {sh ':'}
            }
            stage("upload") {
              steps {sh ':'}
            }
          }
        }

        // 5
        stage('iOS app') {
          stages {
            stage("setup build dependencies") {
              steps {sh ':'}
            }
            stage("setup keychain and profile") {
              steps {sh ':'}
            }
            stage("prepare xcode project") {
              steps {sh ':'}
            }
            stage("build") {
              steps {sh ':'}
            }
            stage("upload") {
              steps {sh ':'}
            }
            stage("reset keychain") {
              steps {sh ':'}
            }
          }
        }
      }
    }

    // 6
    stage('cleanup') {
      steps {sh ':'}
    }
  }
}

So let's fill these steps with content!

4.3.1 General pipeline setup

We'll need to later clean up the workspace a lot, because we're running multiple parallel steps on multiple computers. To make this easier, we'll define the following function at the beginning of the Jenkinsfile:

Jenkinsfile > cleanup

def cleanup_workspace() {
  cleanWs()
  dir("${env.WORKSPACE}@tmp") {
    deleteDir()
  }
  dir("${env.WORKSPACE}@script") {
    deleteDir()
  }
  dir("${env.WORKSPACE}@script@tmp") {
    deleteDir()
  }
}

Because the demo app is an ionic app, and it will also be built for iOS, we need to setup some tooling and environment variables at the beginning of the pipeline.

We also want to cleanup after every build, so we add the clenaup to the end of the pipeline

Jenkinsfile > environment

pipeline {
  agent any
  // only required if you actually use node in the build process
  tools {
    nodejs 'node-lts'
  }
  environment {
    APPSTORECONNECT_TEAMID = '1310680'// only required if build for iOS
  }

  stages {
    ...
  }

  post {
    always {
      script {
        cleanup_workspace();
      }
    }
  }
}

The App Store Connect TeamID is NOT your Apple Developer TeamID! To find out your TeamID run the following on a machine that has fastlane installed, and use the AppleID that was set up in 2.1:

Find App Store Connect TeamId

irb
irb> require "spaceship"
irb> Spaceship::Tunes.login("iTunesConnect_username", "iTunesConnect_password")
irb> Spaceship::Tunes.select_team

At this point, the Jenkinsfile looks like this, and here it is in action

4.3.2 Platform agnostic and preparatory pipeline stages

These are basically all the steps that will run on the Jenkins master, as they neither require a special build node, nor would outsourcing them improve build times. When these are setup, the Jenkinsfile will look like this, the develop-build like this and the master build like this.

prepare

In this stage we will:

  • Get the version for the app (in case of the demo app, from the package.json)
  • Define that the app will only be built from the master branch
  • install and stash base-app-build dependencies (skip this if your projects don't need extra preparation to be buildable. The ionic example app needs this though)

To do so, the stage looks like this:

Jenkinsfile > prepare

stage('prepare') {
  steps {
    script {
      // get the package version
      PACKAGE_VERSION = sh(
        script: 'node --print --eval "require(\'./package.json\').version"',
        returnStdout: true
      ).trim();

      echo("Package version is '${PACKAGE_VERSION}'");

      // Define when to build the app
      BRANCH_IS_MASTER = env.BRANCH_NAME == 'master';
      BUILD_APP = BRANCH_IS_MASTER;

      // prepare dependencies to use when setting up the android/ios projects
      sh('npm install');
      stash(includes: 'node_modules/', name: 'node_modules');
    }
  }
}

[optional] test build

We'll only run this stage if we're not building the app later on anyway. This way every commit can be checked if it at least builds, without hogging Jenkins with resource intensive app build jobs:

Jenkinsfile > test build

stage('test build') {
  when {
    expression {!BUILD_APP}
  }
  steps {
    unstash('node_modules');
    sh ('npm run build');
  }
}

prepare build

Both parallel steps in "prepare build" are only required, if we want to actually build the app. So to only run them if we do build the app, add this:

Jenkinsfile > prepare build

stage('prepare build') {
  when {
    expression {BUILD_APP}
  }
  ...
}

[optional] prepare build > build base android app

Skip this step if your app doesn't need preparation to be buildable. What happens in this step highly depends on how you make sure your app is buildable. In the Ionic example, we want this step to:

  • Run on the Jenkins master
  • Create the android project
  • Save the result of that preparation so it can later be used on the android build node
  • Clean up after we're done preparing the android app

Jenkinsfile > build base android app

stage("build base android app") {
  agent {
    label "master"
  }

  steps {
    // we need the platform so that the ng run app:ionic-cordova command
    // can produce platform sepcific cordova code in www, and correctly
    // setup this code in the platform-folder
    unstash('node_modules');
    sh('npm run add_android');
    sh('npm run prepare_android');

    // these folders are all we need to later build the actual app
    stash(includes: 'www/, platforms/', name: 'base_android_build');
  }
  post {
    always {
      cleanup_workspace();
    }
  }
}

[optional] prepare build > build base ios app

Skip this step if your app doesn't need preparation to be buildable. What happens in this step highly depends on how you make sure your app is buildable. In the Ionic example, we want this step to:

  • Run on the Jenkins master
  • Create our Xcode project
  • Save the result of that preparation so it can later be used on the iOS build node
  • Clean up after we're done preparing the iOS app

Jenkinsfile > build base ios app

stage("build base ios app") {
  agent {
    label "master"
  }

  steps {
    // we need the platform so that the ng run app:ionic-cordova command
    // can produce platform sepcific cordova code in www, and correctly
    // setup this code in the platform-folder
    unstash('node_modules');
    sh('npm run add_ios');
    sh('npm run prepare_ios');

    // these folders are all we need to later build the actual app
    stash(includes: 'www/, platforms/', name: 'base_ios_build');
  }
  post {
    always {
      cleanup_workspace();
    }
  }
}

build and deploy

Only run these stages, if we actually want to build the app. To do so, add this:

Jenkinsfile > build and deploy

stage('build and deploy') {
  when {
    expression {BUILD_APP}
  }
  ...
}

4.3.3 Android specific pipeline stages

These are the stages that have to run on the android build node, because they require Docker. They could be run on the Jenkins master, if Docker is installed there, but we might want to not hog the resources of the Jenkins master, especially if the android build node is on a different machine.

Android app

Here we just define that:

  • the substages will run on the android/docker build node
  • The build environment should be cleaned up when the build is done

Jenkinsfile > build and deploy > Android app

stage('Android app') {
  agent {
    label "docker"
  }
  stages {
    ...
  }
  post {
    always {
      cleanup_workspace();
    }
  }
}

[optional] Android app > setup build dependencies

fastlane requires a gradlew-file in the root of your android project. If you have that, then you can skip this step.

For our ionic example however, the android project folder is created new every build, and cordova only adds the gradlew-file when actually building the app. This is a problem, because the building will be handled by fastlane, not cordova. Because of that, we'll need to create the gradlew-file ourselves by calling gradle wrapper.

This requires an installed gradle and android sdk tough. To not clutter our android build node, and so we can easily use other machines (like a mac) to build our android app, we'll be using a Docker container that has gradle and the android-sdk setup:

Jenkinsfile > build and deploy > Android app > setup build dependencies

stage("setup build dependencies") {
  steps {
    unstash('base_android_build');
    // the docker plugin automatically mounts the current folder als working directory,
    // so we need to neither mount it ourselves, nor define the workdir ourselves.
    // see for mor info: https://github.com/jenkinsci/docker-plugin/issues/561
    dir("${env.WORKSPACE}/platforms/android") {
      script {
        // create gradlew file in the android project folder. This is needed by fastlane
        docker
          .image('amsitoperations/ams-android-gradle')
          /**
          * we run as root inside the docker container, otherwise this step
          * will fail if additional android sdk components need to be
          * installed. This makes some files in the android folder have
          * incorrect ownership, but that will be corrected in the build step
          */
          .inside('--user=0:0') { c ->
          sh 'gradle wrapper';
        }
      }
    }
  }
}

Android app > prepare project

Here we'll unstash our Android project (but only if we didn't already do this in the setup build dependencies step!) and use fastlane to make some final adjustments to it, like setting the version- and build numbers:

Jenkinsfile > build and deploy > Android app > prepare project

stage("prepare project") {
  steps {
    // unstash('base_android_build'); //uncomment if build-dependency step was skipped
    script {
      docker
        .image('bigoloo/gitlab-ci-android-fastlane')
        // we run as root inside the docker container, otherwise the installed tools won't be accessible
        .inside('--user=0:0') { c ->
          sh ("""
            APP_VERSION=${PACKAGE_VERSION} \
            BUILD_NUMBER=${BUILD_NUMBER} \
            fastlane prepare_android
          """);
      }
    }
  }
}

Android app > build

Now that our Android project is set up and all the settings are configured, we can finally tell Jenkins to tell fastlane to tell gradle to build and sign our app!

Jenkinsfile > build and deploy > Android app > build

stage("build") {
  steps {
    script {
      // We need these later to correct file ownership
      CURRENT_USER = sh (script: "id -u", returnStdout: true).trim();
      CURRENT_GROUP = sh (script: "id -g", returnStdout: true).trim();

      withCredentials([
        file(credentialsId: 'android_keystore', variable: 'KEYSTORE_FILE'),
        string(credentialsId: 'android_keystore_password', variable: 'KEYSTORE_PASSWORD'),
        string(credentialsId: 'android_signing_key_alias', variable: 'SIGNING_KEY_ALIAS'),
        string(credentialsId: 'android_singing_key_password', variable: 'SIGNING_KEY_PASSWORD'),
      ]) {
        /**
         * The ${KEYSTORE_FILE}-path points to a file in ${env.WORKSPACE}@tmp/secretFiles.
         * fastlane or gradle don't seem to handle the "@" in that path correctly,
         * so we copy it over to our regular workspace folder, that doesn't contain an "@"
         */
        sh "cp ${KEYSTORE_FILE} ${env.WORKSPACE}/android.keystore";
        docker
          .image('bigoloo/gitlab-ci-android-fastlane')
          // we run as root inside the docker container, otherwise the installed tools won't be accessible
          .inside('--user=0:0') { c ->
            sh("""
              KEYSTORE_FILE="${env.WORKSPACE}/android.keystore" \
              KEYSTORE_PASSWORD=${KEYSTORE_PASSWORD} \
              SIGNING_KEY_ALIAS=${SIGNING_KEY_ALIAS} \
              SIGNING_KEY_PASSWORD=${SIGNING_KEY_PASSWORD} \
              fastlane build_android
            """)

            // make the build is accessible for the user outside the docker container.
            // without this, uploading and workspace-cleanup won't work
            sh "chown -R ${CURRENT_USER}:${CURRENT_GROUP} ./platforms/android/*";
            sh "chown -R ${CURRENT_USER}:${CURRENT_GROUP} ./platforms/android/.*";
        }
      }
    }
  }
}

Android app > upload

We have the build, time to share it with the world (or at least the testers)!

Jenkinsfile > build and deploy > Android app > upload

stage("upload") {
  steps {
    script {
      withCredentials([
        file(credentialsId: 'google_play_service_account', variable: 'GOOGLE_PLAY_SERVICE_ACCOUNT'),
      ]) {
        docker
          .image('bigoloo/gitlab-ci-android-fastlane')
          // we run as root inside the docker container, otherwise the installed tools won't be accessible
          .inside('--user=0:0') { c ->
            sh ("""
              GOOGLE_PLAY_SERVICE_ACCOUNT=${GOOGLE_PLAY_SERVICE_ACCOUNT} \
              fastlane upload_android
            """);
        }
      }
    }
  }
}

4.3.4 iOS specific pipeline stages

These are the stages that have to run on the ios build node, because they require Ruby, fastlane and Xcode.

iOS app

Here we just define that:

  • The substages will run on the iOS build node
  • The build environment should be cleaned up when the build is done

Jenkinsfile > build and deploy > iOS app

stage('iOS app') {
  agent {
    label "fastlane-ios"
  }
  stages {
    ...
  }
  post {
    always {
      cleanup_workspace();
    }
  }
}

iOS app > setup build dependencies

Here we just install/update dependencies that are installed systemwide and that are required to build the app (fastlane plugins in this case)

Jenkinsfile > build and deploy > iOS app > setup build dependencies

stage("setup build dependencies") {
  steps {
    sh("bundle install");
  }
}

iOS app > setup keychain and profile

Here comes the tricky part of the iOS build. Xcode requires a couple of things to be able to build and upload an app:

  • A valid distribution certificate incl. its private key in the current default keychain
  • A provisioning profile for that app and certificate

Xcode (and fastlane) can in theory create/fetch these on the fly and set them up for us, but that process is not reliable at all!

We however need reliable and reproducable builds. Because of that, we need a setup that forces Xcode and fastlane to use the certificates and profiles we provide them, and only these. (The ones created in 2.1)

To do so, we do the following in this stage:

  1. Delete all currently installed provisioning profiles
  2. Delete a possibly existing temporary keychain, which makes sure we're not using an old certificate
  3. Create a temporary keychain just for this build
  4. Set the temporary keychain as the current default keychain
  5. Import our distribution certificate into that keychain
  6. Unlock the keychain and make sure it stays open long enough (the build might take a while)
  7. Make sure codesign works in a non-interactive shell (and doesn't try to prompt the user for a password)
  8. Install our provisioning profile

Jenkinsfile > build and deploy > iOS app > setup keychain and profile

stage("setup keychain and profile") {
  steps {
    // cleanup distribution environment
    // make sure the provisioning profile folder exists
    sh("mkdir -p ~/Library/MobileDevice/Provisioning\\ Profiles");

    // uninstall all provisioning profiles and previous build certificates
    sh("rm -f ~/Library/MobileDevice/Provisioning\\ Profiles/*");
    sh("security delete-keychain deploy_demo_build.keychain || :");

    withCredentials([
      file(credentialsId: 'ios_provisioning_profile', variable: 'PROVISIONING_PROFILE_FILE'),
      file(credentialsId: 'ios_distribution_certificate_and_key', variable: 'DISTRIBUTION_CERTIFICATE_FILE'),
      string(credentialsId: 'ios_distribution_certificate_key_password', variable: 'DISTRIBUTION_CERTIFICATE_PASSWORD'),
    ]) {
      script {
        // setup the singing certificate
        // see https://stackoverflow.com/a/19550453 on using a new keychain just for the build
        def TEMP_KEYCHAIN_PASSWORD = 'sandbox-gondola-majority';
        sh("security create-keychain -p ${TEMP_KEYCHAIN_PASSWORD} deploy_demo_build.keychain");
        sh("security list-keychains -s ~/Library/Keychains/deploy_demo_build.keychain");
        sh("security default-keychain -s ~/Library/Keychains/deploy_demo_build.keychain");

        // install the certificate and keep the keychain for this build unlocked
        sh("security import ${DISTRIBUTION_CERTIFICATE_FILE} -k deploy_demo_build.keychain -P ${DISTRIBUTION_CERTIFICATE_PASSWORD} -A");
        sh("security unlock-keychain -p \"${TEMP_KEYCHAIN_PASSWORD}\" deploy_demo_build.keychain");
        sh("security set-keychain-settings -t 3600 -l deploy_demo_build.keychain");

        // make it so that xcode can use the keychain non-interactively
        // see https://apple.stackexchange.com/a/285320
        def certificate_identity = sh(script: "security find-identity -v -p codesigning \"deploy_demo_build.keychain\" | head -1 | grep '\"' | sed -e 's/[^\"]*\"//' -e 's/\".*//'", returnStdout: true).trim(); // Programmatically derive the identity
        sh("security set-key-partition-list -S apple-tool:,apple: -s -k ${TEMP_KEYCHAIN_PASSWORD} -D \"${certificate_identity}\" -t private deploy_demo_build.keychain"); // Enable codesigning from a non user interactive shell

        // install the provisioning profile
        // see https://gist.github.com/benvium/2568707 for how to install provisioning profiles from command line
        PROVISIONING_PROFILE_ID = sh(script: "/usr/libexec/PlistBuddy -c 'Print :UUID' /dev/stdin <<< \$(security cms -D -i ${PROVISIONING_PROFILE_FILE})_", returnStdout: true).trim();
        sh("cp \"${PROVISIONING_PROFILE_FILE}\" ~/Library/MobileDevice/Provisioning\\ Profiles/${PROVISIONING_PROFILE_ID}.mobileprovision");
      }
    }
  }
}

iOS app > prepare xcode project

Here we'll unstash our Xcode project and use fastlane to make some final adjustments to it, like:

  • Disabling of automatic code signing
  • Setting the provisioning profile to use
  • Setting the version- and build number
  • Configuring our encryption export compliance settings

Jenkinsfile > build and deploy > iOS app > prepare xcode project

stage("prepare xcode project") {
  steps {
    unstash('base_ios_build');

    withCredentials([
      file(credentialsId: 'ios_provisioning_profile', variable: 'PROVISIONING_PROFILE_FILE'),
    ]) {
      script {
        def apple_team_id = sh(script: "/usr/libexec/PlistBuddy -c 'Print :TeamIdentifier:0' /dev/stdin <<< \$(security cms -D -i ${PROVISIONING_PROFILE_FILE})_", returnStdout: true).trim();

        sh("""\
          PROVISIONING_PROFILE_FILE="${PROVISIONING_PROFILE_FILE}" \
          APPLE_TEAM_ID=${apple_team_id} \
          APP_VERSION=${PACKAGE_VERSION} \
          BUILD_NUMBER=${BUILD_NUMBER} \
          bundle exec fastlane prepare_ios
        """);
      }
    }
  }
}

iOS app > build

Now that our build node is set up, all the dependencies are installed and all the settings are configured, we can finally tell Jenkins to tell fastlane to tell Xcode to build the app!

Jenkinsfile > build and deploy > iOS app > build

stage("build") {
  steps {
    script {
      sh("""\
        PROVISIONING_PROFILE_ID="${PROVISIONING_PROFILE_ID}" \
        bundle exec fastlane build_ios
      """);
    }
  }
}

iOS app > upload

We have the build, time to share it with the world (or at least the testers)!
("ITC" stands for "iTunes Connect", the old name of "App Store Connect")

Jenkinsfile > build and deploy > iOS app > upload

stage("upload") {
  steps {
    withCredentials([
      usernamePassword(credentialsId: 'ios_distribution_appstore_user', usernameVariable: 'APPSTORE_USER', passwordVariable: 'APPSTORE_PASSWORD'),
    ]) {

      script {
        sh("""\
          APPSTORECONNECT_USER=${APPSTORE_USER} \
          FASTLANE_PASSWORD="${APPSTORE_PASSWORD}" \
          FASTLANE_ITC_TEAM_ID=${APPSTORECONNECT_TEAMID} \
          bundle exec fastlane upload_ios
        """);
      }
    }
  }
}

iOS app > reset keychain

In "Setup keychain and profile" we changed the default keychain of the jenkins user, so Xcode uses our distribution certificate only. We need to change the default keychain back to the regular default keychain:

Jenkinsfile > build and deploy > iOS app > reset keychain

stage("reset keychain") {
  steps {
    // revert to the regular default keychain
    sh("security list-keychains -s ~/Library/Keychains/login.keychain");
    sh("security default-keychain -s login.keychain");
  }
}

4.4 Testing the Pipeline

Now that the pipeline is complete, your Jenkinsfile should look somewhat like this. All that's left to do is merge everything into master and see how Jenkins builds the app and publishes it to the testers. Zero interaction required!

5. Conclusion

If you made it all the way to here, congratulations! This was a lot to setup and configure, but it's definitely worth the hassle, because every step that gets automated is a step you no longer have to worry about.

I learned that the tools we used are for themselves excellent at solving their own niche problems, and that you can build great things by combining them in just the right way, to solve a completely different problem.

I'm not entirely happy with the two Docker images used for the android builds though. They work for now, but are not that up to date and not really built for this specific workflow. A possible next step would be to create Docker images that are more up to date, only have the exact tooling required and are setup in a way that doesn't force us to update the file ownership after the build.

Thank you for reading this post, and i hope you too learned a thing or three :)