Github composites added feature

As we know Github is the desired ecosystem for Microsoft as it comes to building and deploying your code. Wildly popular among developers in the open-source community. So with a background in Azure DevOps Pipelines, I was surprised by the lack of support for templates. This just is not possible, …. well until two weeks ago.

TL;DR

A much-needed improvement to the GitHub ecosystem. Benefit from the work of existing actions instead of figuring out everything yourself, while preventing copy past patterns. See a working example of GitHub composite action inside your own repository here.

Treat your pipeline as your code

When building your GitHub workflow you start by creating your build step. In most repositories, you do not think much about it because you will have a single workflow anyway. Just start a new pipeline and in your first job give it a build name and copy in the actions that you require.

jobs:
  build_team:
    if: github.event_name == 'push' || (github.event_name == 'pull_request' && github.event.action != 'closed') || github.event_name == 'repository_dispatch' || github.event_name == 'workflow_dispatch'
    env:
      buildConfiguration: 'Release'
      buildPlatform: 'Any CPU'
      buildNumber: '1.1.0.0'
    runs-on: ubuntu-latest
    defaults:
      run:
        working-directory: services/detection/src
    name: Build & Test Team
    steps:
    - uses: actions/checkout@v2
    - name: Setup .NET
      uses: actions/setup-dotnet@v1
      with:
        dotnet-version: 5.0.x
    - name: Setup .NET 3.1.x 
      uses: actions/setup-dotnet@v1
      with:
        dotnet-version: '3.1.x'
    - name: Restore dependencies
      run: |
        dotnet restore ./detection.api
        dotnet restore ../detection.logic.tests
    - name: Build
      run: |
        dotnet build --configuration ${{ env.buildConfiguration }} --no-restore ./detection.api
        dotnet build --configuration ${{ env.buildConfiguration }} --no-restore ../detection.logic.tests
    - name: Test
      run: dotnet test --no-build --configuration ${{ env.buildConfiguration }} --verbosity normal ../detection.logic.tests
    - name: Publish
      run: dotnet publish --no-build --configuration ${{ env.buildConfiguration }} --output ./published ./detection.api/detection.api.csproj
    - name: Package
      run: (cd ./published && zip -r ../api.zip .)
    - uses: actions/upload-artifact@v2
      with:
        name: drop
        #separate upload to enforce flattening
        path: ./services/detection/src/api.zip
    - uses: actions/upload-artifact@v2
      with:
        name: drop
        #separate upload to enforce flattening
        path: ./shared/workflows/**

While this does your job perfectly it becomes more complex when you are adding additional jobs to the pipeline to deploy. Because you want the deploy steps to development and all the way up to production to be the same. Up to 2 weeks ago the only option was to entirely write it in scripting or use a smelly copy-paste pattern making it hard to maintain and just guarantees it will drift eventually. Just create a PowerShell or bash script that does the deployment for you. That PowerShell would be part of your deployment artifact so you can reuse it on any environment with different parameters. You could use a GitHub composite action but you would be limited to the use of just scripts within the composite. At that point, it would have no additional benefit to using it over your own script files. For example, the underlying workflow could not be transformed into a GitHub composite because we use the actions/download-artifact@v2 and azure/loging@v1 action.

  deploy:
    needs: build_provision
    if: github.event_name == 'push' ||  github.event_name == 'repository_dispatch' || github.event_name == 'workflow_dispatch'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/download-artifact@v2
        with:
          name: drop
          path: drop
      
      - name: Azure Login
        uses: azure/login@v1
        with:
          creds: ${{ secrets.AZURE_CREDENTIALS }}
      - shell: pwsh
        run: |
            ./drop/deploy.ps1 -resourceGroupName aegis-apis-provision `
                              -packageSource './drop/api.zip' `
                              -webAppName aegis-apis-provision `
                              -serviceName provision `
                              -environmentCode dev

Github actions composites v2

Two weeks ago GitHub announced a major improvement in your GitHub composites, now supporting the use of other actions besides script actions in your composite actions. Giving you the possibility to finally create proper templates with the reuse of by GitHub and the marketplace (community) offered actions. Those actions actually have a benefit over just your own scripting cause most of them do the heavy lifting for you, knowing endpoints of third-party systems, validating for required parameters instead of you figuring it all out on your own, and offering the support to change in advance when a breaking change is imminent.

Create the composite action

In an empty repository, you can create the following structure. Adding the workflows, and local actions.

.github/workflows/main.yml
.github/actions/<action-name>/action.yml
iac/main.bicep

For this example, I choose to create an action (action-name: build-infra-action) that will build a simple bicep file and upload the generated ARM templates as an artifact to the workflow. It requires two parameters and uses two other actions knowing; azure/CLI@v1, to build and generate bicep into ARM and the actions/upload-artifact@v2 to upload the generated ARM template as an immutable artifact to the workflow. To make this work as a composite action you need to add the using: “composite”.

name: "Build infra artifact"
description: "Build arm template from bicep and upload it as artifact"

inputs:
  bicepFilePath:
    description: “Path to the bicep file”
    required: true
  armFilePath:
    description: “Path to the generated arm file”
    required: true

runs:
  using: "composite"
  steps:
      - name: Build bicep
        uses: azure/CLI@v1
        with:
          azcliversion: 2.21.0
          inlineScript: |
            az bicep build --files ${{inputs.bicepFilePath}}

      - name: Upload artifacts
        uses: actions/upload-artifact@v2
        with:
          name: drop
          path: |
            ${{inputs.armFilePath}}

Calling it from the workflow

In the workflows main.yml, we can now refer to the local composite action passing in the parameters needed by the action to actually build and upload the infrastructure. remember to use the local actions you need to also use the actions/checkout@v2 to have them available during your pipeline.

name: CI

on:
  workflow_dispatch:

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v2
      - uses: ./.github/actions/build-infra-action
        name: build infra structure
        with:
          bicepFilePath: "./iac/main.bicep"
          armFilePath: "./iac/main.json"

For a complete and working example take a look here, this also contains the simple bicep file.

Have fun Erick!

Update 13 september 2021

As a reaction to this blog post, my colleague Rob Bos reached out that he was curious if and to what extent you can nest composite actions. He tested it and …

Thnx Rob