GitHub Actions' Strategy Matrix

Yesterday, I was introduced to a useful feature of GitHub Actions which we’ll refer to as strategy matrix. I’m more famliar with Google Cloud Build but, to my knowledge, Cloud Build does not provide this feature.

The challenge is in providing an iterator for steps in e.g. a CI/CD platform.

Below is a summarized version of what I had. My (self-created) problem was that I had 4 container images to build, but the Dockerfile names didn’t exactly match the desired repository names. I had e.g. grpc.broker for the Dockerfile name and I wanted e.g. grpc-broker. The principle though is more general than my challenge naming things. The YAML delow describes the same step multiple times and what I would like to do is range over some set of values.

name: "Build Apps"

on:
  push:
    branches:
      - http-extensibility

jobs:
  device:
    name: device
    runs-on: ubuntu-latest
    steps:
      - name: checkout
        uses: actions/checkout@v2
      - name: setup
        uses: docker/setup-buildx-action@v1
      - name: login
        uses: docker/login-action@v1
        with:
          registry: ghcr.io
          username: ${{ secrets.crUsername }}
          password: ${{ secrets.crPassword }}
      - name: build-push
        uses: docker/build-push-action@v2
        with:
          context: .
          file: ./samples/apps/http-apps/Dockerfiles/device
          tags: ghcr.io/deislabs/akri/http-apps-device:latest
          push: true

  discovery:
    name: discovery
    runs-on: ubuntu-latest
    steps:
      - name: checkout
        uses: actions/checkout@v2
      - name: setup
        uses: docker/setup-buildx-action@v1
      - name: login
        uses: docker/login-action@v1
        with:
          registry: ghcr.io
          username: ${{ secrets.crUsername }}
          password: ${{ secrets.crPassword }}
      - name: build-push
        uses: docker/build-push-action@v2
        with:
          context: .
          file: ./samples/apps/http-apps/Dockerfiles/discovery
          tags: ghcr.io/deislabs/akri/http-apps-discovery:latest
          push: true

  grpc-broker:
    name: grpc-broker
    runs-on: ubuntu-latest
    steps:
      - name: checkout
        uses: actions/checkout@v2
      - name: setup
        uses: docker/setup-buildx-action@v1
      - name: login
        uses: docker/login-action@v1
        with:
          registry: ghcr.io
          username: ${{ secrets.crUsername }}
          password: ${{ secrets.crPassword }}
      - name: build-push
        uses: docker/build-push-action@v2
        with:
          context: .
          file: ./samples/apps/http-apps/Dockerfiles/grpc.broker
          tags: ghcr.io/deislabs/akri/http-apps-grpc-broker:latest
          push: true

  grpc-client:
    name: grpc-client
    runs-on: ubuntu-latest
    steps: The names are 
        uses: docker/setup-buildx-action@v1
      - name: login
        uses: docker/login-action@v1
        with:
          registry: ghcr.io
          username: ${{ secrets.crUsername }}
          password: ${{ secrets.crPassword }}
      - name: build-push
        uses: docker/build-push-action@v2
        with:
          context: .
          file: ./samples/apps/http-apps/Dockerfiles/grpc.client
          tags: ghcr.io/deislabs/akri/http-apps-grpc-client:latest
          push: true

Enter strategy matrix Tada!

Using strategy matrix, the YAML becomes:

name: "Build Apps"

on:
  push:
    branches:
      - http-extensibility

jobs:
  apps:
    name: apps
    runs-on: ubuntu-latest
    strategy:
      matrix:
        include:
          - component: device
            dockerfile: device
          - component: discovery
            dockerfile: discovery
          - component: grpc-broker
            dockerfile: grpc.broker
          - component: grpc-client
            dockerfile: grpc.client
    steps:
      - name: checkout
        uses: actions/checkout@v2
      - name: setup
        uses: docker/setup-buildx-action@v1
      - name: login
        uses: docker/login-action@v1
        with:
          registry: ghcr.io
          username: ${{ secrets.crUsername }}
          password: ${{ secrets.crPassword }}
      - name: build-push
        uses: docker/build-push-action@v2
        with:
          context: .
          file: ./samples/apps/http-apps/Dockerfiles/${{ matrix.dockerfile }}
          tags: ghcr.io/deislabs/akri/http-apps-${{ matrix.component }}:latest
          push: true

Now there’s a single step and it’s repeated 4 times thanks to:

strategy:
  matrix:
    include:
    - component: device
      dockerfile: device
    - component: discovery
      dockerfile: discovery
    - component: grpc-broker
      dockerfile: grpc.broker
    - component: grpc-client
      dockerfile: grpc.client

Essentially, we define a matrix (array|slice) named include (this is a reserved word).

In my case it’s a 4,2 matrix (although the row values are named) and I assume there’s a reasonable limit to the size.

In my case each row contains keys of component and dockerfile.

NOTE To reference these keys, the syntax is ${{ matrix.component }} and ${{ matrix.dockerfile }}; the include is not referenced.

I could have used the simpler form of strategy matrix had I just needed to range over a list (device, discovery, broker, client) and not compounded my problem by varying the names of the Dockerfiles and the image names.

protoc

One trick I use with Google Cloud Build is to share go modules and e.g. protoc-generated files once during a build and then references these static files multiple times when building the container, something like:

+ step #1: go mod download
+ step #2: protoc ...
+ step #3: build broker
+ step #4: build client
+ step #5: push images

I’ve not worked out how to do this using GitHub Actions.

That’s all!