Untitled Publication

Running Your BATS Tests Against Multiple Bash Versions in GitHub

TekWizely — Fri, 12 Nov 2021 22:50:11 GMT

In this post I'll go over the github workflow I created that allows me to test bash scripts using BATS against multiple versions of Bash.

TL;DR: See my github workflow : bats-multi-bash.yml

My Bash Script

The script in question is my Bash-TPL project, which is a light-weight shell-scripting template engine.

My Testing Framework

I chose BATS as the testing framework, as its comprehensive, tests are easy to write, and its Bash all the way down !

I had already written an extensive suite of tests, so I was confident they would reveal any compatibility issues once I figured out how to run them against older Bash versions.

Finding Older Bash Versions

In researching how I was going to run my tests against older bash versions, I discovered the Official Bash Docker Repository

Local Testing

This made running the tests on a specific bash version very easy to do on my local machine:

$ docker --rm -it -v"${PWD}:/bash-tpl" bash:3.2

bash-3.2# apk add bats diffutils
# ...
OK: 8 MiB in 21 packages

bash-3.2# cd /bash-tpl
bash-3.2# bats test

1..92
ok 1 BASH_TPL_TAG_DELIMS: Should do nothing when unset or empty
ok 2 BASH_TPL_TAG_DELIMS: Should error on invalid input
ok 3 BASH_TPL_TAG_DELIMS: Should process valid input
# OK looking good !
# ...
# DOH!
not ok 42 misc-function: escape_regex
not ok 43 misc-function: normalize_directive
# ...

Hate Compatibility Much?

So it turned out that my script only worked on Bash 5.1 :(

Checking Multiple Bash Versions

Once I identified all of the compatibility issues, I was able to work on fixing them, using the above technique to test against multiple Bash versions. Specifically, I ran tests against:

Bash Version 3.2 (bash:3.2)
Bash Version 4.4 (bash:4.4)
Bash Version 5.0 (bash:5.0)
Bash Version 5.1 (bash:5.1)

Auto-Detecting Future Issues

Having successfully fixed the compatibility issues, and making a new release, I set out to create a github workflow to run these tests for me in the future.

Workflow Research

Bash Docker Images

I had to determine if I could utilize the previously discovered Bash docker images within a github workflow.

Run a Single Bash Image

My first goal was to determine if I could run the workflow within just a single docker image.

Thanks to the well-documented Workflow Syntax For Github Guide, I discovered the jobs.\.container configuration.

jobs:
  my_job:
    container:
      image: "bash:3.2"

This will run your job steps inside the specified container !

Configuring BATS

Next I needed to install BATS into my fresh new Bash container.

I knew I could issue an apk add bats command, but I wanted to have more control over the actual version of BATS being used.

GitHub Actions Marketplace

I figured I probably wasn't the first person who wanted to run BATS in a workflow, so I searched the github actions marketplace.

Sure enough, there was already an action for configuring BATS:

https://github.com/marketplace/actions/setup-bats-testing-framework

Most importantly, it allows you to declare which version of BATS you want to install.

For me, that was the shiny new 1.5.0:

jobs:
  my_job:
    steps:
      - name: Setup BATS
        uses: mig4/setup-bats@v1
        with:
          bats-version: 1.5.0

Multiple Bash Versions

Now that I could successfully run my tests using a single container, it was time to figure out how to use multiple containers.

Enter The Matrix

From the github workflow documentation of the jobs.\.strategy.matrix strategy:

You can define a matrix of different job configurations. A matrix allows you to create multiple jobs by performing variable substitution in a single job definition. For example, you can use a matrix to create jobs for more than one supported version of a programming language, operating system, or tool. A matrix reuses the job's configuration and creates a job for each matrix you configure.

Not Exactly New

I first learned about the github workflow matrix strategy when implementing github-repo-stats to save stat history for all of my repositories.

But What About Containers?

But I couldn't find any examples of a matrix being used for container setup.

... Favors The Bold

I decided to give a try anyway - And It Worked !

jobs:
  my_job:
    strategy:
      matrix:
        bash: ["bash:3.2", "bash:4.4", "bash:5.0", "bash:5.1"]
    container: 
      image: ${{ matrix.bash }}

Putting It All Together

Now that I had all the pieces worked out, I could finally implement my workflow.

Below is the version of the workflow at the time of posting:

bats-multi-bash.yml

name: BATS Tests

on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]

  workflow_dispatch:

jobs:
  bats:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        bash: ["bash:3.2", "bash:4.4", "bash:5.0", "bash:5.1"]
    container: 
      image: ${{ matrix.bash }}
    steps:
      - name: Install packages
        run: apk add diffutils

      - name: Setup BATS
        uses: mig4/setup-bats@v1.2.0
        with:
          bats-version: 1.5.0

      - name: Checkout code
        uses: actions/checkout@v2

      - name: Run bash-tpl tests
        run: bats test/

      - name: Run template tests
        run: bats test/tpl

Results

The BATS tests run successfully on all specified Bash versions, and they're fast!

Conclusion

Thank you for reading - I hope you found this post helpful. Please let me know if you have any comments or questions.

-TekWizely

My-Alternatives: A wrapper for update-alternatives offering user-level customizations. Supports Debian, SUSE, and RedHat

TekWizely — Wed, 10 Nov 2021 21:40:00 GMT

I'm excited to announce release v0.7.0 of my-alternatives

https://github.com/TekWizely/my-alternatives

My-Alternatives is a light-weight wrapper over update-alternatives, offering user-level customizations.

Supports Debian, SUSE, and RedHat

Easy to Get Started

With my-alternatives, configuring custom alternatives is as easy as:

# initialize my-alternatives
# note: place this in your .profile
$ eval "$( my-alternatives init )"

# customize an alternative
$ my-alternatives select

See the Project's Homepage for more information

Release Notes - v0.7.0

Re-Write

This release marks a full re-write of the my-alternatives tool, changing it into wrapper for update-alternatives that manages user-level configurations and uses them when invoking update-alternatives (i.e --admindir & --altdir).

This new approach makes it much easier to support all available commands for managing your custom alternatives.

RedHat

This release also includes support for RedHat.

This is significant because RedHat maintains its own implementation of update-alternatives, which is slightly different than the Debian version.

The RedHat version does not include the uber-useful --query command, making it impossible to integrate it using just the public API.

As such, supporting it requires a hack, exploiting knowledge of the tool's private internals.

SUSE

The SUSE implementation of update-alternatives is really just a re-branded version of the Debian tool.

Multiple Scripts

As part of introducing RedHat support, this release now includes two versions of the my-alternatives script, each named according to the primary OS flavor they support:

Debian / SUSE : my-alternatives-debian
RedHat / CentOS : my-alternatives-redhat

Bash-TPL - A smart, lightweight shell script templating engine

TekWizely — Sat, 18 Sep 2021 16:46:24 GMT

I know the landscape is crowded, but I'd like to offer my submission for an Easy To Use / Easy To Maintain template engine:

Introducing Bash-TPL : A Smart, lightweight shell script templating engine, written in Bash

https://github.com/TekWizely/bash-tpl

FEATURES

Lightweight

Single bash script
Easy to ship with your project or build process

Smart Indentation Tracking

Removes unwanted whitespace from rendered text files
Encourages well-formatted mix of template tags and textual content

Generates Reusable Shell Scripts

Can manage & invoke shell scripts without bash-tpl
Only need bash-tpl when your source templates change

Shell Agnostic

Uses standard printf, variable, and sub-shell constructs
Templates can target any modern shell

Supports Includes

Can organize your templates as smaller, reusable components
Indentation tracking even works across includes !

Flexible Delimiter Support

Can customize all template delimiters
Can modify delimiters mid-template
Can include templates with differing delimiters

Quick Example

As is the way: A quintessential hello world example is of course in order:

hello.tpl

Hello, <% ${NAME:-World} %>

compile + run

$ source <( bash-tpl hello.tpl )

Hello, World

compile + run with NAME

$ NAME=TekWizely source <( bash-tpl hello.tpl )

Hello, TekWizely

view compiled template

$ bash-tpl hello.tpl

printf "%s\n" Hello\,\ "${NAME:-World}"

More Information

This hello example is really just the start of what bash-tpl can do.

There's a full README on the Project Home Page.

If you're looking for an easy solution for creating maintainable templates to generate well-formatted text files, I hope you'll give my project a try.

I'm happy to answer any questions or work through any use-cases you might have for templates and see how bash-tpl might help.

Feel free to comment and thanks for looking!

-TW

Bingo - The missing package manager for golang binaries

TekWizely — Mon, 17 Feb 2020 20:11:48 GMT

Introduction

Do you love the simplicity of being able to download & compile golang applications with 'go get', but wish it were easier to manage the compiled binaries?

Introducing Bingo:

GitHub : https://github.com/TekWizely/bingo

Bingo makes installing and managing golang-compiled binaries a bit easier.

Features

Keeps a link between the installed binary and the source package
Can install binaries to a location of your choice
Can control the name of the installed binary
Can install multiple versions of the same package (using different names for the binary)
Each binary's source package is isolated and managed in its own separate $GOROOT

Examples

Compiling + Installing Binaries

To install a binary with bingo, use the golang application's full package path, same as you would with 'go get'.

hello example

$ bingo install github.com/golang/example/hello

Installing binary hello from package github.com/golang/example/hello
Downloading package (folder: '~/.bingo/pkg/hello')
Compiling package
Installing binary (file: '~/.bingo/bin/hello')
Done

$ ~/.bingo/bin/hello

Hello, Go examples!

Binary Naming

By default, the installed binary will be named after the last folder element in its package path.

As you saw above, installing the github.com/golang/example/hello package installed a binary named hello.

You can override this behavior and specify the binary name at the time of installation:

install hello example as 'foo'

$ bingo install -n foo -q github.com/golang/example/hello

$ ~/.bingo/bin/foo

Hello, Go examples!

Listing Installed Binaries

To see a list of installed binaries, use the installed command:

$ bingo installed -p

Bingo-managed binaries (folder: '~/.bingo/bin')

 - foo github.com/golang/example/hello
 - hello github.com/golang/example/hello

Displaying A Binary's Associated Package

If you need a reminder of which package a binary was compiled/installed from, you can use the package command:

$ bingo package hello

github.com/golang/example/hello

Uninstalling Binaries / Packages

Use the uninstall command to uninstall binaries:

$ bingo uninstall foo

Uninstalling binary foo from package github.com/golang/example/hello
Removing binary (file: '~/.bingo/bin/foo')
Removing package (folder: '~/.bingo/pkg/foo')
Done

$ bingo installed -q

hello

NOTE: Uninstalling a binary also removes the associated package folder.

Requirements

Run

Bingo exists as a Runfile, and requires the Run tool to operate:

https://github.com/TekWizely/run

Bash

Bingo (currently) uses bash for its command scripts.

Readlink

Bingo uses symbolic links to associate binaries to their packages.

The scripts use readlink to resolve symbolic links.

Installing Bingo

Releases

See the Releases page for downloadable archives of versioned releases.

Brew Tap

While brew core support is in the works, I have also created a tap to ensure the latest version is always available:

https://github.com/TekWizely/homebrew-tap

install bingo directly from tap

$ brew install tekwizely/tap/bingo

install tap to track updates

$ brew tap tekwizely/tap

$ brew install bingo

Configuration

See the Work Folders section on the project's main page for details on configuring the various folders needed by bingo, including which folder to install binaries in.

License

The tekwizely/bingo project is released under the MIT License.

Conclusion - A Toy Trying To Take Itself Seriously

Although designed as a set of toy scripts to play around with the idea, bingo is trying to take itself seriously and make a real run at being a useful tool.

If you're looking for better ways to manage binaries installed via 'go get', I hope you will give my project a try.

I am happy to answer any questions you might have.

Thank you for your time,

-TekWizely ( https://github.com/TekWizely )

Run - Easily manage and invoke small scripts and wrappers (Written in Golang)

TekWizely — Fri, 20 Dec 2019 21:12:37 GMT

Introduction

Do you find yourself using tools like make to manage non build-related scripts?

Build tools are great, but they are not optimized for general script management.

Introducing Run:

GitHub : https://github.com/TekWizely/run

Run aims to be better at managing small scripts and wrappers, while incorporating a familiar make-like syntax.

Features

Here are a few of Run's features:

Terse, make-like syntax
Manage multiple script types: Bash, Python, Ruby, etc.
Automatic help text generation
Command scripts are executed in a single sub-shell, not like make's (default) behavior of executing each line of a recipe in a separate sub-shell.
Define cli options and have the values passed to your scripts as environment variables.

Runfile

Where make has the ubiquitous Makefile, run has the cleverly-named "Runfile"

By default, run will look for a file named "Runfile" in the current directory, exiting with error if not found.

See the project README for details on specifying alternative runfiles, as well as other special modes you might find useful.

Hello World Example

The project's README starts with a much simpler example and walks through the various features, but I thought I would share a more fleshed out example to demonstrate several of the features together:

Runfile

##
# Hello world example.
# Prints "Hello, ".
# OPTION NAME -n,--name  Name to say hello to
hello:
  echo "Hello, ${NAME:-World}"

list commands

$ run list

Commands:
  list     (builtin) List available commands
  help     (builtin) Show Help for a command
  hello    Hello world example.
Usage:
       run [-r runfile] help <command>
          (show help for <command>)
  or   run [-r runfile] <command> [option ...]
          (run <command>)

show help for hello command

$ run help hello

hello:
  Hello world example.
  Prints "Hello, ".
Options:
  -h, --help
        Show full help screen
  -n, --name <name>
        Name to say hello to

invoke hello command with no options

$ run hello

Hello, World

invoke hello command with options

$ run hello --name=Newman
$ run hello -n Newman

Hello, Newman

Installing

Currently, installing requires using go get, but there is an active PR for a brew formula and I'm also working on a PKGBUILD file.

Conclusion

If you're at all interested in managing task runners and scripts, I hope you will give my project a try.

I am happy to answer any questions you might have.

Thank you for your time and Happy Holidays!

-TekWizely ( https://github.com/TekWizely )