The future of UI design for embedded systems

It’s time for embedded user interfaces to catch up to the rest of the world.

We know this already, as you can’t read an article or go to a conference without someone saying “our users want their smartphone experience in their car and on their fridge!” The allure of blending powerful features and beautiful user experiences (UX) comes too easily, and embedded systems are found lacking.

So why are we falling behind and what market forces are driving us to catch up?

With less power comes more responsibility

There’s a common belief in the sliding scale of hardware-to-UI-capability ratio, in the sense that the smaller the platform, the less you can do with the UI. While true in a strictly functional sense—if you don’t have 3D hardware acceleration, you can’t use 3D hardware acceleration—it’s often the case that the application architecture, graphics library limitations, and suboptimal performance result in UIs that look like this:

Example of old UI design

Instead of this:

Example of good UI design

We know this has to do with not having enough development time and resources to squeeze as much out of the hardware as possible but users are left scratching their heads as to why its easier to operate a smartphone that does fifty things than a dishwasher that does only one thing.

It certainly doesn’t have to be this way. Not only are CPU and GPU hardware capabilities increasing, right down to the MPU, UI toolkits are getting better. Check out the recently announced ARM Dynamiq processors and Mali-G72 GPU, or the first MCU with an integrated 2D GPU and integrated DDR2 memory from Microchip Technology Inc. We’re being offered more speed and capability than ever before and with less power consumption and heat.

And it will only get better. Embedded processors have traditionally been slow to adopt cutting-edge components, being more concerned with power, heat, and longer in-service times. But with the Internet of Things, smart homes, connected cars, and a host of other trends that have become reality, the demand for faster, more capable devices by both consumers and industry is just getting started.

So, what can embedded UI teams look forward to in 3 to 5 years? A look at the forces transforming the technology world right now provides the answers.

UI design is equal to code

The 2017 Design in Tech Report, presented at SXSW Interactive, offers an analysis of design trends changing the face of tech. Three points stand out as particularly relevant for embedded systems, by the very fact that users are demanding more out of their UIs and that more customers are on their way. The IoT market alone is expected to grow to 75.4 billion devices by 2025.

Perhaps the biggest change is that designers, or at least the unique skillset of UI design, is becoming a priority for development teams. Developers know how to create, test, and optimize apps, but it’s the lack of a designer’s knowledge that results in so many of the “traditional-looking” UIs that are still being churned out by embedded teams.

Only by making the job of the designer equal to that of the developer can products be made beautiful, useful, engaging, and functional—qualities that drive adoption and brand loyalty. The report points out that not only have Facebook, Amazon, and Google grown their art and design personnel by 65 percent in the past year, two out of every five designers are now involved in code development.

We’re seeing this with embedded software teams as well, as bringing a designer’s mindset into the developer’s machine is the surest way to achieve UX and commercial success.

UI design must be inclusive

“Adopting an inclusive design approach expands a tech product’s total addressable market.”
2017 Design in Tech Report

The report observes that, historically, technology products weren’t designed for a broad range of users, rather they were built to be used by the people who created them. This exclusivity has softened over time, of course, with a whole industry devoted to user research and feedback, but again, smartphones continue to move the UX goalpost for embedded. Why can’t every device operate with just one button, a simplified set of graphics, and be understood by everyone around the world?

Users don’t care that your graphics library doesn’t support pixel shaders or hardware acceleration. Embedded systems are highly technical but cannot survive by serving only a technical audience, especially as they sell into newer consumer-dominated markets all over the world. UIs must be thought out, designed, and built for not only maximum usability but inclusivity as well—whether it’s geographic, cultural, or socioeconomic—to satisfy global user expectations.

Although originally intended for the web, the World Wide Web Consortium’s guidelines for accessibility, usability, and inclusion apply equally to embedded systems because, at the end of the day, all our users are human.

“Inclusive design, universal design, and design for all involves designing products, such as websites, to be usable by everyone to the greatest extent possible, without the need for adaptation. Inclusion addresses a broad range of issues including access to and quality of hardware, software, and Internet connectivity; computer literacy and skills; economic situation; education; geographic location; and language — as well as age and disability.”

One major example of a company tackling this head on is Microsoft, check out their inclusive design toolkit.

The key to inclusivity is designing UI elements that are simple and unambiguous while eliciting an emotional connection. In this way, your UI is less frustrating to use and more likely to be used.

To design embedded UIs that include, rather than alienate, as broad an audience as possible, it’s the designer to the rescue. Unlike developers, they have the specific techniques and experience with distilling diverse personas into a usable design, so your product ends up with better UX/UI that also happens to be easy to use by everyone.

The need for supporting this diversity has found its way into the makeup of design teams, as the Design in Tech Report shows:

UI design unlocks value

“Design tool companies and design community platforms occupy new positions of value for tech.”
2017 Design in Tech Report

Software like Adobe Photoshop, GIMP, and Sketch are just a few of the many examples of design tools that have democratized graphic design, lowering the barrier to UI excellence, and reducing time to release (does anyone do back-of-the-napkin sketches anymore?).

The same applies to embedded UI tools. Our developers have mostly moved away from using graphics primitives to use mid-level libraries, such as OpenGL and Direct3D, with higher-level platforms such as Qt gaining popularity. With these pre-packaged, easier to use toolkits, building UIs has also become democratized. Anyone can take a Qt widget and run with it.

The problem is, to unlock the maximum value out of all that design effort, you need to go one step further and not get locked into how pre-packaged libraries look and behave.

“Every dollar invested in UX brings 100 dollars in return. That’s an ROI of a whopping 9900 percent.”
Good UX is Good Business: How to Reap Its Benefits, Forbes

Going straight from the designer’s tool to deployment, with little to no modification of the design, is the only way to ensure your product offers the best UX/UI. Otherwise, development tradeoffs (software limitations, pre-canned widgets, resource constraints, power, time, etc.) will chip away at your optimal experience.

Storyboard Suite is one such tool, with built-in parallel workflows between the designer and developer to reduce the common frustrations and friction between them. By working side-by-side, the designer can focus on creating the best UX/UI possible, the developer can focus on the back-end, and the whole application is created, tested, and deployed within one single environment. The app also happens to run on a highly-optimized engine tuned for a variety of platforms and architectures.

This video from last year’s Apple WWDC event sums up why smart design matters:

“When you put so much hard work and creativity into an app, you really want it to reach people. But ultimately, everyone will experience your app in their own way and if you think about the millions of people who could use your app, you have to consider how diverse their perspectives are.”

The key statement here is that we all want our products to reach people, in terms of both usability and market share. Using collaborative tools that make design equal to code is just the way to do it.

To see how easy it is to put design into action, try the sample applications included in our free 30-day Storyboard Suite trial:

5 capabilities that entrench embedded UI designers in the development process

The following is a post from Fancy Dan, one of our creative and talented in-house designers. Dan uses Storyboard Suite on a daily basis to create awesome user experiences for embedded applications. I asked him to share the top 5 features that make his job easier and his designs awesome.

Storyboard Suite top features

Dan’s top 5 list

  1. Integrate content from Adobe Photoshop » Storyboard brings a .psd file to life
  2. Add and refine motion easily » Easy to use, but do a lot
  3. Layout and position content precisely » Subtle features that go a long way
  4. Enhance user experience with actions » The combination of trigger events, actions, and parameters
  5. Make it real. Touch the design. » From simulator to target

Integrate content from Adobe Photoshop

As a designer, I find working in Storyboard to be a great experience, and a major contributing factor for this is the Photoshop integration. Photoshop has been around for decades, and it plays nicely with Adobe Illustrator, two tools that I use to design and plan a UI. Many of the design-based decisions I make in Photoshop, such as organization, naming, and planning, are brought forward to Storyboard when I import my .psd. When I start working in Storyboard, I’m working on a project that I already have a good handle on because the Application Model in Storyboard is based on the content from my layers view in Photoshop.

With Smart Objects, I’m able to go back into Photoshop and make edits to my Illustrator content or layer effects and bring them back into Storyboard. Because I can import multiple Photoshop files into Storyboard, when I complete one section of the UI design I can bring it into Storyboard and start adding functionality. At the same time, I can continue to work on the UI design for the next section and bring the UI together in Storyboard as each part is ready for import.

What’s extra great?

The Photoshop re-import process. If I change the look and layout of an existing design in Photoshop, I can re-import the artwork into my Storyboard project. The project compares itself to the new Photoshop file and allows me to replace existing content with the new designs and updates the look, size, and positioning from my redesign and retain existing functionality.

Add and refine motion easily

animation_timelineMotion makes a UI come to life, so it’s important to have a workflow that includes working with animations. In Storyboard, working with animations isn’t convoluted. If I want to create, edit, or preview an animation, it’s pretty straightforward. When recording an animation, changes made to the UI are recorded as animation steps and saved. The animation timeline view shows the saved steps. Here I can see the steps in relation to one another and make further edits to the timing, easing rates, and values. Using the animation preview, the animation can be played back, paused, and scrubbed through so that I can see how all of the movements work together.

What’s extra great?

I can quickly create a reversed version of an existing animation, finely tune the ease rate of motion, and apply a single animation to multiple model objects. These capabilities allow me to easily control how things move without having to touch any code.

Layout and position content precisely

When the design is complete, most of the UI layout has been established. Further refinement like adding logic, motion, and additional content is easy with the layout and positioning options in Storyboard.

reference_calculator

Reference point location – The reference point location might appear to be a small detail, but it does a lot for controlling the position and transformation of an object. The reference point location icon is available in the Properties view for dimensions. The top-left reference point shows the x,y location of an object, but selecting another reference point shows and applies transformations based on that relative point. If I want to increase the width of a control by 50 pixels, and have the center point remain the same, I shouldn’t have to move the control 25 pixels to the left and then add 50 pixels to the width. I simply want to add 50 pixels to the width that expands from the middle and let the software figure out the rest.

Property value calculator – An easy way to control the values of a property is to calculate that number in the property field. If I want to move a control 20 pixels left to a third of the way across a screen designed at 720p, I could go to the properties view X: positioning and enter 1280/3-20 (which is the screen width divided by 3, then subtract 20) or enter 1280*.333-20 (which is the screen width multiplied by one-third and then subtract 20) and the X value will automatically calculate 406. If I want a circular progress bar to fill up 43%, I can enter 360*.43 in the Rotation field and the angle will calculate to 154.8º.

Applying some quick math makes it easy to land content in a specific spot. I can apply it to an alpha value, apply it to an offset, or apply it to the timing in an animation. It saves me from going to a separate calculator app, writing it out, or doing it on my phone.

What’s extra great?

Q: Does it use BEDMAS? A: Yes, it does.
Q: What the heck is BEDMAS?

Enhance user experience with actions

It becomes increasingly engaging to work on a UI as it takes on more functionality. Adding actions to the UI improves functionality. There are a lot of combinations and triggers that make actions happen so I’m not limited by what I can do.

Storyboard provides an extensive list of trigger events that goes beyond the basic taps and gestures that are available in most prototyping tools. Many things can trigger an action, such as the completion of an animation, arriving at a particular screen, completed timers, scrolling starts or stops, etc. With a trigger in place, I can select an action. Actions can call an animation, stop an animation, go to a new screen, change some data, start a timer, or, for those who are are scripting savvy, call a Lua event. Depending upon the type, an action’s parameters can vary. An action parameter might cause something to loop, indicate a direction, set a duration, or select an external file. Events can happen any number of ways, to any of the model objects that trigger an event.

What’s extra great?

I can copy an action from one control to other controls. For example, I can copy a screen change action from one button and paste it to other buttons to give them the same functionality.

Make it real. Touch the design.

Working in design, sometimes your work is used only as a representation of the UI. This isn’t limited to UI design. Mock-ups, presenting artboards, and showing prototypes for apps or websites are great, but at the end of the day, seeing the design in the real world and its surrounding context is awesome.

Storyboard keeps the design in the designer’s hands, which enables me to deliver my design vision to the final product. In traditional workflows, this often isn’t the case, and instead, the UI on the embedded device isn’t a reflection of the designer’s work. It’s common for designers to create prototypes that demonstrate how the UI should look and behave, and then provide the development team with the UI assets to create the UI in code. It’s an inefficient workflow to have the development team recreate what the designer has already solved.

Storyboard provides tools that make it easy to quickly export applications directly to hardware for testing. Seeing how an animation runs or how a transition looks on an embedded device gives me insight and confidence that the UI performs as intended. Good design principles value function over form. Using Storyboard, I have everything I need to test how my design will function on an actual target device.

What’s extra great?

I can test my application on an embedded board, make changes to the UI, and then export the app back to the board in under a minute.
*Drops Microphone*

To try Storyboard Suite for yourself (and to feel like a design rockstar), start your free trial now and see how easy it is to create beautiful embedded UIs from concept to production.

StoryboardTrialBtn

See scalable Storyboard demos on the Microchip SAMA5D2

Curious to see Storyboard Suite in action on your hardware? Our pre-built and packaged demos make it easy for you to test drive different types of applications designed for specific hardware platforms, letting you experience the powerful controls and smooth animations that create beautiful UIs and rich user experiences.

Scalable UIs across display sizes: SAMA5D2 demos for medical, white goods, and home automation

Storyboard applications are scalable across a variety of screen sizes and resolutions. Experience this on 480 x 272 and 800 x 480 displays with Storyboard demo images for the Microchip SAMA5D2 platform. These images include everything you need to get up and running in minutes, offering the following demos in both resolutions:

  • Smart washing machine – drill down into operational settings and track progress through cycles
  • Medical – browse through typical patient diagnostics, electronic health records (EHR), and security verification
  • Home automation – play with temperature controls, check out local weather, and customize lighting throughout the floorplan

Also included are instructions for flashing the image to an SD card and running it on your own board.

download

To try Storyboard Suite for yourself, start your free trial now and see how easy it is to create beautiful embedded UIs from concept to production.

StoryboardTrialBtn

Meet the New Storyboard Suite at Embedded World 2017

embedded_world_2017

Join us at Embedded World 2017 in Nuremberg, Germany, March 14-16, to experience the latest and greatest in innovative embedded technology. It’s always an exciting show and we will be there again this year with a booth filled with embedded application UI demos built in Storyboard Suite across a number of platforms. Our passionate UI design professionals will be on hand to showcase the latest features and capabilities in Storyboard Suite, our development framework for building beautiful UIs for embedded systems. If you are at the show, stop by Hall 4/4-270 to say hello and check out demo applications we’ve designed. While there, ask us to show you what’s new in Storyboard Suite for 2D and 3D artists and embedded systems developers.

Interact with Storyboard GUI demos on Embedded Hardware

Hardware. We are bringing so much hardware.
We know that embedded teams want to be able to interact with real demos instead of watching edited videos and listening to us boast about features. Our booth will be brimming with multi-market Storyboard demos on a number of platforms, including:

Our UI demos this year cover a wide range of markets, including automotive, home automation, white goods, medical, and kiosk solutions. All demos have been designed and developed in Storyboard Suite, using the latest features from our upcoming 5.0 release.

In addition, you’ll find Storyboard demos in other booths at the show, including NXP (Hall 4A/4A-220), Embedded Artists (Hall 4/4-544), Toradex (Hall 1/1-639), EBV (Hall 3A/Stand 229), TQ-Systems (Hall 1/1-578), and VeriSilicon (Hall 3/3-510).

Follow us on Twitter to see photos and videos from the exhibition floor throughout the week.

Hear us Speak!

We are fortunate to be speaking at Embedded World this year, so reserve some time on March 14th to learn more about our thoughts on and approaches to UI design and development for embedded systems.

Speaker: Thomas Fletcher, VP Research & Development, Crank Software
Time: March 14th, 5:00 – 5:30 PM
Presentation: Keep Designers in the Driver’s Seat and Build Better User Interfaces

Check out Storyboard Suite 5.0

We are getting set to release Storyboard Suite 5.0  and we are excited to show off the features and capabilities we’ve added for embedded UI design teams. Stop by the booth ask to see it in action.

To learn more about what to expect in the Storyboard Suite 5.0 release, check out our recent webinar, now available on-demand.

If you want to get together with us at Embedded World, please contact us to set up a meeting. Alternatively, just drop by our booth and chat with us. We are friendly Canadians and we’d be happy to talk with you about embedded UI design and maybe hockey and cold winters.

As always, we provide a full-featured 30-day evaluation for you to try out Storyboard Suite first hand.

StoryboardTrialBtnSee you at the show!

Driving Excellence in Automotive HMI Design: Storyboard on the NXP i.MX 8 at CES

Whether it’s commuting to and from work, chauffeuring the kids to hockey*, or running errands, we spend countless hours driving on a daily basis. Fortunately for those of us who feel like we live part-time in our cars, technology is evolving at a rapid rate to enhance the driving experience and to allow us to connect with our vehicles more seamlessly and safely.

Storyboard Suite Automotive HMI NXP imx8 CES 2017

If you were at CES last week, you might have noticed a few automotive HMI demos at the show. Every year we see stunning innovation in automotive technology and new features that inspire consumers to hand over their money. (No flying car yet, however.) This year was no exception. CES 2017 showcased the coolest automotive HMI technology from the biggest automakers to small, low-cost suppliers, and everyone in between.

Keeping up with such a fast-growing and ever-changing technology and meeting consumer expectations isn’t an easy feat for automotive HMI design and development teams. Consumer expectations are high. Take a look at your smartphone. Impressive graphics, right? You have a powerful mobile computer that fits in your pocket. It’s a navigation system, audio/video player, and communication device that’s available to you when you need it. We’ve come to expect these same capabilities to be available to us on demand when needed, even in our cars. And we expect them to look great.

So how do teams deliver graphically stunning, connected, responsive, feature-rich automotive HMI experiences? Development teams need a high-performance and reliable processor that can support rich graphics, display to multiple screens, and handle a variety of inputs, such as the NXP i.MX 8. Making high-performance hardware look great and delivering an intuitive user experience is the job of the UI development framework. What consumers see on their car screens is the graphical representation of all of the information and data available to them, from their digital instrument cluster to their in-vehicle infotainment and entertainment displays.

Making Automotive HMIs that meet consumer expectations

It can be challenging for teams to deliver beautiful, user-friendly, and safety-conscious graphical content to a range of automotive displays. They need tools that support the following:

  • Design-centric – Design and user experience are critical to automotive HMIs. To achieve a high-quality design aesthetic and intuitive interaction, it’s critical to keep the designer in control throughout the development process.
  • Collaborative Workflow – Automotive HMIs are often built by teams, not by individuals. The software processes and workflow should support that.
  • Design ChangeIterative design is a reality. From minor tweaks to major updates in artwork, changes can, and should, happen. If it’s painful to iterate the design, it’s more likely that design compromise will occur.
  • Embedded Focus – Hardware for embedded displays has unique capabilities and constraints. The UI development software should have a focus on embedded.

We built Storyboard Suite with these requirements and others in mind. With our vast experience in the embedded marketplace and our focus on making Storyboard Suite feature-rich, yet user-friendly, many companies choose Storyboard for their exciting HMI projects.

Speaking of exciting HMI projects…

NXP showcased a distributed eCockpit and Infotainment at CES 2017 that was built using Storyboard Suite for the i.MX 8.This impressive multi-screen demo was a collaborative effort by NXP and Crank Software and included the following displays:
  • Digital Instrument Cluster
  • Head-up Display
  • In-Vehicle Infotainment
  • Backseat Entertainment

Thanks to the powerful i.MX 8, this demo was able to run four high-resolution screens at frame rates up to 60fps off one processor.

Integrating a mix of 2D and 3D content, the team was able to work collaboratively on the applications using Storyboard’s multi-file support, and seamlessly merge the project files and iterate the design throughout the lifecycle. The 3D models were created with high-fidelity detail to take advantage of the rich GPU capabilities of the i.MX 8.

 

To try Storyboard for your own automotive HMI projects, download a free 30-day trial.

StoryboardTrialBtn

*So Canadian of me