Creating a medical HMI with Storyboard Suite

The following is a post from Nik, our awesomely bearded and multi-talented Field Application Engineer. Nik recently attended the Microchip MASTERs event in Phoenix, Arizona, and describes the making of the medical demo that we brought to the show.

Storyboard Suite medical demo for the Microchip sama5d2

If you were at Microchip MASTERs 2017, you might have seen the Patient Monitor demo at the “Ask the Experts” table. Perhaps you saw the demo at the Crank Software booth and spoke to Jason Clarke or me about it. If you didn’t get the opportunity to see it live, read on to learn more. This medical HMI, created in collaboration with the fine folks at Microchip, showcases how Storyboard Suite can help you quickly prototype your UI, foster collaboration between designers and engineers, and deliver a polished product to market faster.

The goal of this medical demo was to take real data from medical devices and display it in an aesthetically pleasing, feather-rich, and user-friendly UI. The system powering this demo is the Microchip SAMA5D2 Xplained board running Linux. The following Microchip medical breakout boards are connected to the SAMA5D2:

With multiple pieces of hardware communicating to the UI, we needed a couple of design iterations to create the UI we envisioned for this setup. Storyboard allowed us to iterate designs quickly and easily by importing and reimporting content from Photoshop. We did not have to start from scratch each time the design changed. The application logic and event bindings that were previously declared stayed in place and Storyboard took care of managing the new assets. When we had a design that we were happy with, we moved on to integration.

In the integration phase, UI and engineering teams come together and connect the UI to real data. Storyboard IO enables this merger by allowing an IO interface to and from the UI. Defining a structured event and data protocol over Storyboard IO has a number of benefits. It forces a clean separation between low-level business logic and UI logic and it allows the UI to be developed in parallel to any back-end systems. The event protocol was defined early on in the design, which made the integration cycle fairly seamless. I was able to work on the C application which handled all the interfacing to hardware devices while one of our designers built the UI in parallel. When implementing features like live trend data in the UI, we were able to easily inject simulation data via Storyboard IO without needing to connect to actual hardware. With the hardware interface completed, it was simply a matter of dropping it in place. Many bugs had already been ironed out due to the ability to use simulated data, so the final stage of this project was focused on fine-tuning and tweaking the experience.

When I hooked up the UI to real data we were finally able to polish and stress test the system. The final phase was to ensure that the UI was smooth and responsive under heavy load before demo completion. Initially, there was some event flooding when all four breakout boards started sending event data at the same time. Having four active data streams also highlighted some areas for optimization in the UI which led to faster redraw times. While the designer worked on tweaking the UI, I was able to modify how frequently the back end sent data. After a few quick iteration cycles, we were able to finalize the Medical demo that shipped to Microchip MASTERs.

This easy iteration and Storyboard’s collaboration support meant we were able to work in parallel without compromising the design or underlying code in the process. The end product is a user-friendly, aesthetically pleasing, and high-performance medical UI.

To try a Storyboard application on a Microchip platform, download one of our pre-packaged and easy-to-install demo images.

If you aren’t using Storyboard, start your free trial now and see how easy it is to create beautiful embedded UIs from concept to production.


Storyboard Suite accelerates UI performance on the Renesas RZ/A1

The best and most user-friendly user interfaces are those that are responsive and have great performance. One of the biggest hits to performance comes from drawing graphics. In order to boost speed and reduce User Interface (UI) lag, many platforms, including the Renesas RZ/A1, provide a solution to optimize how graphics are drawn.

The Renesas RZ/A1 is built upon an ARM A9 running at 400Mhz, which provides good performance when rendering only with a software framebuffer. However, on the H and M variants of the platform, users can leverage even more impressive 2D acceleration, either through OpenVG or the Renesas Graphics Architecture (RGA) API interface.

The RGA library allows high-speed drawing using 2D hardware acceleration and provides an API to boost graphics performance, such as image and font blending. While integration requires development time to configure and optimize, there are desirable performance advantages that make it worth the effort.

Storyboard Suite 5.1 reduces this effort and allows you to take advantage of the 2D acceleration provided by the RGA library, without having to worry about additional work and configuration.

Enhances existing Storyboard application performance

If you have a Storyboard application already, moving to a Renesas RZ board using RGA is as simple as selecting a different runtime to build with. After that, you’ll notice some significant performance improvements as your application takes advantage of native 2D hardware acceleration.

The video below shows the performance improvements when taking advantage of Storyboard utilizing both software rendering and the RGA. Both provide solid performance, but there are obvious advantages with the RGA.

To compare the differences in performance, we ran some tests on an RZ/A1H board with and without using RGA. Here are the results with the application’s frame rate capped at 60 frames per second (FPS). We saw performance gains of 100-200% using RGA.

Software rendering frame rate (FPS) RGA frame rate (FPS)
Medical Demo 17 34
Oven Demo 24 60
Bubble Mark 20 60

We ran these tests by deploying an RGA and non-RGA runtime to the same board, no additional modifications to the application code or build settings were necessary. These are the same runtimes provided for your use in your development environment.

Reduces effort for new UI applications

If you’re building a new Storyboard application for boards that support RGA, the benefits are even better. Not only do you get the performance improvements stated above, Storyboard reduces the development time and effort.

We’ve done the research, development, and testing using RGA for you, so all you need to do is focus on making your UI applications as awesome as possible. This could save you weeks to months of development and integration effort.

Saving development time and improving performance—there’s no reason why Storyboard isn’t the best choice for developing UIs on Renesas platforms.

To try out a Storyboard application for your RZ/A1 platform, download our latest demo image and experience the performance first-hand.

Download the Storyboard Renesas RZA1H demo image

If you don’t have Storyboard Suite, start a free trial now and see how easy it is to create beautiful embedded UIs from concept to production.


Try it now! Storyboard demo image for the Renesas RZA1

Want to be able to quickly and easily interact with Storyboard Suite applications on your hardware? Our pre-built and packaged demos make it easy for you to test drive multi-market applications designed for specific hardware platforms, letting you experience the powerful controls and smooth animations that create beautiful UIs and rich user experiences.

A washing machine that makes laundry enjoyable?

Intuitive and beautiful user interfaces make us want to interact with the appliances and gadgets that are a part of our everyday lives. The Storyboard Suite smart washing machine application is a shining example of an engaging user experience. Drill down into operational settings, track progress through cycles, and delight in the modern aesthetic expected by today’s technology-savvy user. It might even make you want to do laundry.

This demo image includes everything you need to get up and running on a Renesas RZA1 platform in minutes.

Download the Storyboard Renesas RZA1H demo image

To try Storyboard Suite for yourself, start your free trial now and see how easy it is to create beautiful embedded UIs from concept to production.


Build better embedded UIs. Meet Storyboard Suite 5.1.

Earlier this year we introduced Storyboard Suite 5.0, which gave UI designers more control than ever over the aesthetic and user experience of embedded application projects. The release included a lot of new functionality and features for both designers and engineers, especially in Storyboard Designer.

We’ve recently made enhancements and updates to Storyboard with our latest release, Storyboard Suite 5.1. Don’t worry, these updates won’t trip you up much from a functionality perspective. We haven’t moved around the elements in the UI or introduced anything to the workflow that will require you to invest too much time to get up to speed. We’ve tweaked some things behind the scenes to make UI development work a little smoother and faster. More importantly, we’ve extended the reach of embedded application development by introducing support for new platforms and technologies, making it easier to deliver Storyboard applications across markets and hardware.

Storyboard Designer 5.1 updates and enhancements

Here’s what’s new in Storyboard Designer:

  • Eclipse – Storyboard Designer is now based on the Eclipse Framework NEON 4.6.
  • x64 – Storyboard Designer has been switched from a 32-bit application to 64-bit on all hosted platforms.
  • Cloning – When cloning an object, we now check to see if a control with the new name already exists, so that you don’t have to.
  • Keyboard Shortcuts to group and ungroup – We’ ve introduced the concept of grouping and un-grouping through a new action bound to Control/Command + G and Control/Command + SHIFT + G.
  • SCP – The SCP command execution now has the ability to write output to the console.
  • Lua Debugger – Storyboard Designer now queries the Lua debug port number in use and passes it into the runtime.

Increased platform support

With each release, we continue to increase the reach of Storyboard Suite across platforms, making it easier for our customers to work with a wide range of target hardware. Here’s what’s new in Storyboard 5.1:

  • NXP i.MX8 – Take advantage of the powerful combination of Storyboard Suite and the NXP i.MX8.
  • QNX 7.0 SDP swrender and OpenGL runtime – Develop high-performance, innovative, and secure applications for the new QNX 7.0 SDP.
  • ST Accordo5 Wayland OpenGL runtime – Deliver cutting edge in-car experiences with Storyboard Suite for the ST Accordo5.
  • Wind River VxWorks – Combine Storyboard Suite with the safety and security of the VxWorks real-time OS to bring world-class applications to market.
  • Renesas RZ/A1 RGA– Storyboard 5.1 fully supports the Renesas Graphics Architecture (RGA) library to deliver impressive application performance for the RZ/A1 platform.

Try Storyboard Suite 5.1 for yourself! Start your free trial now and see how easy it is to create beautiful embedded UIs from concept to production, on more platforms than ever before.


See us at Microchip MASTERs in Phoenix

Microchip MASTERs is just a couple of days away (August 16-19, in Phoenix, AZ), and we’re busy finalizing, flashing, and packing some really cool demos for you to see and interact with. This show provides a great opportunity to learn and share the best in embedded design and development.

We’ll be there to showcase the latest in HMI user interface (UI) development, including the innovative Storyboard Suite platform that lets designers and developers work side-by-side to create amazing user experiences on a wide range of Microchip platforms. Drop by our booth for a chat and a Storyboard demo.

Multi-market demos – whether you’re in the medical, smart home, industrial or consumer/white goods industry, there’s an application of interest in the Crank Software booth. In advance of the show, you can download demo images for different Microchip platforms here to get a preview of what we’ll be showing. We will also be showcasing a brand new medical demo on the Microchip SAMA5D2 that includes blood pressure, heart rate, oxygen, and temperature monitoring, as well as ECG display.

Storyboard Suite demos – want to see us build a UI while you drink your coffee? Nik, our awesome field applications ninja, will walk you through the entire UI creation process, from Adobe Photoshop to deploying and debugging on a real target.

We’re expecting Microchip MASTERs to be very busy, so drop an email to Nik to set up a time to chat and see Storyboard Suite in action.

Meet Nik