.png?width=180&height=67&name=Crank-AMETEK-HZ-Rev%20(4).png)
In this guest post, embedded systems expert Jacob Beningo explains that to deliver successful embedded GUI products, developers must have an understanding of core skills in graphics, optimization, design, and more.
Graphical user interfaces (GUI) are becoming more popular among embedded systems. The need for a convenient and connected interface is finding its way into wearable technologies, industrial equipment, commercial kitchens, and even home appliances. Throwing a GUI with a modern and sophisticated user experience (UX) on a device opens the door for many businesses’ use cases, but it can also add unexpected challenges to a design if the team does not have the right skillset.
Today's post will explore the five essential skills that every embedded GUI developer needs to be successful.
Skill #1 – Graphics driver skills
The last skill that any team will think about when designing an embedded GUI, but perhaps one of the most important, is graphics driver skills. A graphics driver is the software that directly interfaces with the microcontroller or microprocessor graphics peripheral and is responsible for drawing the image on the LCD that the user sees and interacts with. The graphics driver is code that takes the image data from the rendering engine and tells the processor what it is drawing.
Do you know how objects are rendered to the screen?
A good embedded GUI tool today will typically handle the graphics driver for developers. Developers need a hardware-agnostic rendering engine where most of the heavy lifting is done for them. There are several examples available in the embedded industry today, but let's use Crank Software's Storyboard for our discussion. Storyboard's rendering engine is 90% agnostic to the hardware and is separate from the hardware board support package (BSP). There is a small 10% that is written for specific hardware to optimize rendering and leverage accelerators in specific hardware targets. The engine itself handles the graphics driver for the developer.
Since Storyboard handles developers' drawing and graphics, why is it important to still have graphic driver skills? The GUI developer should understand what is happening under the hood so that they can resolve any issues that come up during implementation, leverage any additional optimizations, and have a familiarity with how the rendering engine works.
Skill #2 – Tracing and debugging
It's good to be optimistic, but embedded systems today are complex, and the chances that something isn't going to work as expected are relatively high. Therefore, it's crucial to have tracing and debugging skills when something is not working or is perhaps working but not as well as one would like. A good debugger can pull data, observe behaviors, hypothesize, and ultimately improve the system behavior.
To minimize debug time, developers should be skilled in tracing technologies as well. There are several different types of tracing technology that is available to GUI developers. First, general tracing tools are available on the microcontroller, such as the serial wire output (SWO) and serial wire debug (SWD) interfaces available on Arm microcontrollers. These provide high-level user-configurable tracing capabilities. Next, software tracing technologies such as the instrumentation trace macrocell (ITM) and the embedded trace macro (ETM) are available. The ITM allows developers to customize trace data using printf statements, while the ETM provides an instruction trace for every instruction executed by the CPU. So far, these are all general embedded software tracing technologies. The final tracing technology is specifically for tracing and debugging embedded UX applications.
An embedded GUI engine should have logging and trace built into it to aid the developer. Logging and tracing are often added to a GUI engine through a plug-in to configure how trace data is received. Developers usually can log data to a file, a memory address, through the debug tool, and so forth. In general, developers need to access two types of data: performance and metrics successfully. With access to this data, developers can understand important information such as:
- Rendering engine performance
- Image load times
- Frames per second
- CPU usage
- Memory usage
A developer can then use this information to troubleshoot an issue or improve the system's performance through optimizations.
Skill #3 - Optimization
Embedded systems today have become quite powerful, but they are still resource-constrained systems. They may be resource-constrained in processing power, memory, or battery life. Many systems require optimization, and the GUI can play a huge role in ensuring the system is efficient, optimized, and fit for its end purpose.
Storyboard's Application Footprint Preview helps you understand how memory is being used
When it comes to optimizing the UX, there is a lot that can be done. The first optimization is to leverage an event-driven software architecture. An event-driven architecture only does something when it is necessary! For example, I've seen many systems that try to push a specific frame rate to the display, such as 60 or 120 Hz. Why push a frame rate, though, if the display doesn't need an update? If the display doesn't need to change, CPU clock cycles and battery life can be saved by doing nothing! Developers who are trying to optimize their UX should be asking themselves the following questions:
- What needs to change?
- What data needs to change?
- Can I minimize what I update?
Another helpful optimization that can be applied is to draw images directly from flash using uncompressed TGAs. That's right! No compressed PNGs. Developers can draw directly from flash into their rendering buffer, reducing memory costs and saving a ton of clock cycles. PNGs are compressed and require more memory and clock cycles to read the PNG, expand it, and finally draw it. Flash memory is relatively cheap and can be an excellent optimization for an embedded system.
These are just a few examples, but as the reader can ascertain, optimizing the UX and the software / hardware system that drives it can have far-reaching consequences for the system. Furthermore, optimization skills are critical, given how vital energy minimization is for wearable devices or even for home appliances that need to meet specific energy consumption requirements.
Skill #4 – Managing expectations
A simple skill that developers almost always overlook is to manage expectations correctly. My experience has been that developers are optimistic. We see where things are going and promise to build peppy, fantastic user interfaces. But, unfortunately, if we don't manage expectations properly, what would have been considered a successful project can suddenly be viewed as a failure by the users or project stakeholders.
Developers need to be careful to manage their stakeholders' expectations properly! Those stakeholders could be a project manager, upper management, or even the end customer. It's a subtle skill but necessary. The fictional master at managing expectations was none other than Scotty from Star Trek. Captain Kirk would constantly ask the impossible. Scotty wouldn't give in but would damper expectations to something reasonable. Then, once done, he would deliver on the expectation he set or exceed them!
Developers need that ability to manage expectations. Don't be overly optimistic and promise the moon. Set a reasonable expectation that can be met, and then if time allows, wow the stakeholder and leave them with a feeling of excitement!
Skill #5 – Graphic design
The final skillset is undoubtedly the skill that many readers would have expected to be first! GUI developers need to have graphic design skills. They need to understand what colors work well, not just on a retinal display but on a resource-constrained, microcontroller-driven panel! Colors aren't everything, though. A critical point, which I see engineers often fail miserably at is too often an interface that I call "engineering ugly".
Engineers tend to be functional creatures. We are very good at arranging buttons and things that adhere to a logical order. The problem is that while something may be engineering functional, it often is not simultaneously user-friendly! Developers need to be able to design and build graphical interfaces that look good and are user-friendly. Graphic design is a skill that sometimes must be borrowed from a graphics design specialist. The developer, though, will often still need to adjust and tweak the artwork based on how the embedded system is functioning.
Conclusions
To successfully develop an embedded system with a user interface, the developer and or team must have these five essential skills. It can often be challenging to find a single engineer with all these skills but leveraging teammates and outside sources can dramatically help to improve the chances for success.
If you are currently starting or have started a GUI/UX design, take a few moments to evaluate how well your team now matches these skills. Here are a few questions to get your thoughts going:
- Are you missing any critical skills?
- Are there any weaknesses among your skillsets?
- What strengths do you have that can be leveraged to accelerate your development?
To learn how Storyboard supports the skills mentioned here, browse and watch our Advanced User Training webinars, including graphics, optimization, and design concepts: