# Running Your BATS Tests Against Multiple Bash Versions in GitHub

In this post I'll go over the github workflow I created that allows me to test bash scripts using BATS against multiple versions of Bash.

TL;DR: See my github workflow : [bats-multi-bash.yml](https://github.com/TekWizely/bash-tpl/blob/main/.github/workflows/bats-multi-bash.yml)

### My Bash Script

The script in question is my [Bash-TPL](https://github.com/TekWizely/bash-tpl) project, which is a light-weight shell-scripting template engine.

### My Testing Framework

I chose [BATS](https://github.com/bats-core/bats-core) as the testing framework, as its comprehensive, tests are easy to write, and its Bash all the way down !

I had already written an extensive suite of tests, so I was confident they would reveal any compatibility issues once I figured out how to run them against older Bash versions.

### Finding Older Bash Versions

In researching _how_ I was going to run my tests against older bash versions, I discovered the [Official Bash Docker Repository](https://hub.docker.com/_/bash)

#### Local Testing

This made running the tests on a specific bash version very easy to do on my local machine:

```
$ docker --rm -it -v"${PWD}:/bash-tpl" bash:3.2

bash-3.2# apk add bats diffutils
# ...
OK: 8 MiB in 21 packages

bash-3.2# cd /bash-tpl
bash-3.2# bats test

1..92
ok 1 BASH_TPL_TAG_DELIMS: Should do nothing when unset or empty
ok 2 BASH_TPL_TAG_DELIMS: Should error on invalid input
ok 3 BASH_TPL_TAG_DELIMS: Should process valid input
# OK looking good !
# ...
# DOH!
not ok 42 misc-function: escape_regex
not ok 43 misc-function: normalize_directive
# ...
```

#### Hate Compatibility Much?

So it turned out that my script **only** worked on Bash 5.1 :(

#### Checking Multiple Bash Versions

Once I identified all of the compatibility issues, I was able to work on fixing them, using the above technique to test against multiple Bash versions.  Specifically, I ran tests against:

* Bash Version 3.2 (`bash:3.2`)
* Bash Version 4.4 (`bash:4.4`)
* Bash Version 5.0 (`bash:5.0`)
* Bash Version 5.1 (`bash:5.1`)

### Auto-Detecting Future Issues

Having successfully fixed the compatibility issues, and [making a new release](https://github.com/TekWizely/bash-tpl/releases/tag/v0.4.0), I set out to create a github workflow to run these tests for me in the future.

### Workflow Research

#### Bash Docker Images

I had to determine **if** I could utilize the previously discovered Bash docker images within a github workflow.

##### Run a Single Bash Image

My first goal was to determine if I could run the workflow within just a _single_ docker image.

Thanks to the well-documented [Workflow Syntax For Github Guide](https://docs.github.com/en/actions/learn-github-actions/workflow-syntax-for-github-actions), I discovered the [jobs.\<job_id\>.container](https://docs.github.com/en/actions/learn-github-actions/workflow-syntax-for-github-actions#jobsjob_idcontainer) configuration.

```
jobs:
  my_job:
    container:
      image: "bash:3.2"
```

This will run your job steps inside the specified container !

#### Configuring BATS

Next I needed to install BATS into my fresh new Bash container.

I knew I could issue an `apk add bats` command, but I wanted to have more control over the actual version of BATS being used.

##### GitHub Actions Marketplace

I figured I probably wasn't the first person who wanted to run BATS in a workflow, so I searched the [github actions marketplace](https://github.com/marketplace?type=actions).

Sure enough, there was already an action for configuring BATS:
* https://github.com/marketplace/actions/setup-bats-testing-framework

Most importantly, it allows you to declare _which_ version of BATS you want to install.

For me, that was the shiny new [1.5.0](https://github.com/bats-core/bats-core/releases/tag/v1.5.0):

```
jobs:
  my_job:
    steps:
      - name: Setup BATS
        uses: mig4/setup-bats@v1
        with:
          bats-version: 1.5.0
```

#### Multiple Bash Versions

Now that I could successfully run my tests using a single container, it was time to figure out how to use multiple containers.

##### Enter The Matrix

From the github workflow documentation of the [jobs.\<job_id\>.strategy.matrix](https://docs.github.com/en/actions/learn-github-actions/workflow-syntax-for-github-actions#jobsjob_idstrategymatrix) strategy:

> You can define a matrix of different job configurations. A matrix allows you to create multiple jobs by performing variable substitution in a single job definition. For example, you can use a matrix to create jobs for more than one supported version of a programming language, operating system, or tool. A matrix reuses the job's configuration and creates a job for each matrix you configure.

##### Not Exactly New

I first learned about the [github workflow matrix]() strategy when implementing [github-repo-stats](https://github.com/jgehrcke/github-repo-stats#tracking-multiple-repos) to save stat history for all of my repositories.

##### But What About Containers?

But I couldn't find any examples of a matrix being used for `container` setup.

##### ... Favors The Bold

I decided to give a try anyway - And It Worked !

```
jobs:
  my_job:
    strategy:
      matrix:
        bash: ["bash:3.2", "bash:4.4", "bash:5.0", "bash:5.1"]
    container: 
      image: ${{ matrix.bash }}
```

### Putting It All Together

Now that I had all the pieces worked out, I could _finally_ implement my workflow.

Below is the version of the workflow at the time of posting:

_bats-multi-bash.yml_
```
name: BATS Tests

on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]

  workflow_dispatch:

jobs:
  bats:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        bash: ["bash:3.2", "bash:4.4", "bash:5.0", "bash:5.1"]
    container: 
      image: ${{ matrix.bash }}
    steps:
      - name: Install packages
        run: apk add diffutils

      - name: Setup BATS
        uses: mig4/setup-bats@v1.2.0
        with:
          bats-version: 1.5.0

      - name: Checkout code
        uses: actions/checkout@v2

      - name: Run bash-tpl tests
        run: bats test/

      - name: Run template tests
        run: bats test/tpl
```

### Results

The BATS tests run successfully on all specified Bash versions, **and they're fast!**

![BATS Test Results Against Multiple Bash Versions](https://cdn.hashnode.com/res/hashnode/image/upload/v1656540867563/UScK_IZrB.png)

### Conclusion

Thank you for reading - I hope you found this post helpful.  Please let me know if you have any comments or questions.

-TekWizely