Automated Testing at Crank!

Here at Crank Software, we are continuously updating Storyboard in order to deliver a better experience for our customers every day. In preparation for the upcoming Storyboard 3.2 release, new features are being added all the time and the Storyboard Engine has to be tested against all our supported platforms to make sure that the latest changes don’t regress existing behaviour.

Our current continuous integration system already runs automated tests on self-hosted platforms such as Windows, MacOS and Linux. We wanted to expand the automated tests to the runtime platforms and reduce our manual testing using the same framework. To start things off, we set up two Texas Instruments AM335x Starter Edition boards in our lab which should remain there for the most part. These boards run a Linux distribution and QNX 6.5 respectively. We run our tests on Linux using the OpenGLES 2.0 renderer and the FBDev renderer. On the QNX side of things, we have started testing using the OpenGLES 2.0 renderer for now. These boards are connected to the buildbot which issues the commands for the tests and records all the data received. This is useful because we can see what our data generally looks like over time. For example, we can create a graph that plots performance data over time. It is possible that the tests all pass but performance may take a hit. When we notice something like this, we take the necessary steps to fix it.

By adding our embedded targets to our automated tests we can perform testing more quickly, effectively and there is less room for error once we have it all set up. This should ultimately help us create a better product for everybody.

-- Huzefa