Setting up a simple device CI/CD pipeline for your project: part IV

In this series of posts, we propose a simple and full open source CI/CD pipeline for your Linux device development on GitLab that takes advantage of the power of Pantahub and Pantavisor.

Table of Contents (4 part series)
Part I: keep devices up to date
Part II: generate flashable images
Part III: template internals
Part IV: pipeline customization

In the previous post, we explained the internals of our CI/CD pipeline in order to understand how it works.

In the fourth part, we are going to show you how to customize the pipeline to fit your needs.

Background

Let’s get back to the device-ci diagram we saw in part III.

As we want to change the behavior of the templates, we are interested in overrriding them from our project in .gitlab-ci.yml, taking advantage of Gitlab CI capability to do so.

For demonstration purposes, we are going to add a new test stage to the pipeline, taking the test results from the device after each update. Some other customizations were considered and discarded for the sake of keeping the post short and simple: build an image for all states of the device instead of just for tags, triggering the update when a change in a dependency is pushed, store images in a different cloud service instead of AWS, add more content to the images metadata, etc.

How to customize the CI

I’m going to link again the great GitLab CI/CD pipeline configuration reference. I recommend you to take a look at it if you want to get into GitLab CI/CD and have it always at hand when doing the changes as it contains almost everything you will need.

In our case, as we mentioned before, we are going to perform a simple change to the pipeline for demonstration purposes: add a test stage to the pipeline that takes a mock test result from the board and makes the pipeline fail if the test failed. This is going to require changes from the device side, to publish the mock test results. Also, from the CI/CD project, to add the new testing stage and check the results uploaded by the device.

Device side changes

Starting from device side. We are going to have to get back to our own forked gpio app project.

Changes are relatively simple. We are only going to simulate the test and direclty push fail or pass to the device metadata at Pantahub. Pantahub metadata is accesible by using pvr, so it is perfect for communication between device and the pipeline script.

To upload the metadata from a container running on the device, we first need to install our pvmeta utility in it. These are the changes that we have to perform over the Dockerfile to install pvmeta:

--- a/Dockerfile.ARM32V6
+++ b/Dockerfile.ARM32V6
@@ -1,5 +1,22 @@
+FROM alpine as qemu
+
+RUN wget -O /qemu-arm-static https://github.com/multiarch/qemu-user-static/releases/download/v4.1.0-1/qemu-arm-static; \
+       chmod a+x /qemu-arm-static
+
+FROM registry.gitlab.com/pantacor/pantavisor-runtime/pvtoolbox:ARM32V6-master as pvtoolbox
+
 FROM arm32v6/bash
 
+COPY --from=qemu /qemu-arm-static /usr/bin/
+
+COPY --chown=0:0 --from=pvtoolbox /usr/local/bin/pvsocket /usr/local/bin/pvsocket
+COPY --chown=0:0 --from=pvtoolbox /usr/local/bin/pvlog /usr/local/bin/pvlog
+COPY --chown=0:0 --from=pvtoolbox /usr/local/bin/pvmeta /usr/local/bin/pvmeta
+
+RUN chmod +x /usr/local/bin/pvsocket; \
+       chmod +x /usr/local/bin/pvlog; \
+       chmod +x /usr/local/bin/pvmeta
+
 ADD gpio.sh /usr/bin/gpio.sh
 
 ENTRYPOINT [ "/usr/bin/gpio.sh" ]

The actual pushing of the metadata can be done in the gpio script. Let’s name the metadata key gpio-test and, for now, we will set its value to fail:

--- a/gpio.sh
+++ b/gpio.sh
@@ -33,6 +33,7 @@ set_gpio_value() {
 
 while [ true ];
 do
+       pvmeta update gpio-test=fail
        for i in `ls $GPIO_META_PATH`;
        do
                set_gpio_value $i `cat $GPIO_META_PATH/$i`

CI/CD project changes

Now, the changes in the device-ci project we created in part I. These are two changes in our .gitlab-ci.yml that are going to override the template jobs: redefine the stage list with a new test stage after validation and specify the test job that will just get the gpio-test key from device metadata and exit 1 if its value is not pass:

--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -3,6 +3,13 @@ include:
   ref: 35f3cf627279aabb2b241ca79bf00f7b16e7d12c
   file: '/yml/update-ci.yml'
 
+stages:
+  - post
+  - push
+  - build
+  - validate
+  - test
+  - deploy
 
 update-gpio-latest-device:
   extends: .update-scheduled
@@ -25,3 +32,12 @@ check-gpio-stable-device:
   extends: .check-status-stable
   variables:
     PH_TARGET_DEVICE: "gpio-stable-device"
+
+test-gpio:
+  stage: test
+  script:
+    - TOKEN=`http --ignore-stdin POST https://api.pantahub.com/auth/login username=$PHUSER password=$PHPASS | jq -r .token`
+    - if [ $(pvr -a $TOKEN dev get anibal/gpio-latest-device | jq '."device-meta"."gpio-test"') != "\"pass\"" ]; then exit 1; fi
+  except:
+    - schedules
+    - web

We haven’t even bothered to make the device name configurable, which would be mandatory if we were talking about a real life example.

Test it

After we have pushed all the changes from previous section to their respective master branches, we can manually trigger the update jobs from GitLab user interface so the device gets the (failing) new update.

Time to make it pass, by commiting this to the gpio app source code:

--- a/gpio.sh
+++ b/gpio.sh
@@ -33,7 +33,7 @@ set_gpio_value() {
 
 while [ true ];
 do
-       pvmeta update gpio-test=fail
+       pvmeta update gpio-test=pass
        for i in `ls $GPIO_META_PATH`;
        do
                set_gpio_value $i `cat $GPIO_META_PATH/$i

After manually spinning a new update job, the test will pass as the device contains now the pass uploading version.

Conclusion

We have finally reached the end of the series. We have gone through setting up the basic CI/CD to keep devices up to date, releasing flashable images from the state of those devices, overviewing how it works and customizing the pipeline.

I hope it was useful to you in some way. I’d really love to hear your feedback if you liked it, hated it, though something was missing or had used a similar solution.