UI Developers Can Deliver In-Browser HTML5 Content Easily With New WebKit Engine

Ottawa, October 23, 2013

Crank Software, an innovator in embedded user interface (UI) solutions, today announced Crank Storyboard Browser, an extension of Storyboard Suite with a WebKit-based HTML5 renderer that allows users to bring HTML content directly into a Storyboard Application and deploy it on an embedded target.

“We see the benefits of HTML5 and wanted to bring its capabilities to UI engineers without making them outsource WebKit development, which takes months and strains budgets,” said Brian Edmond, Crank Software president. “With Storyboard Browser, developers get exactly what they need – a version of WebKit that integrates HTML5 content into an easy-to-use Storyboard application.”

To get a UI development tool and a WebKit-based application that renders HTML5 content, product teams traditionally must piece together a multiple-vendor solution, but with Crank Software, teams can simply use the proven Storyboard Suite with the Storyboard Browser extension for a single-vendor, seamless solution to creating the best UI possible.

Read More>>

Crank Software WebKit Chosen to Power Nintendo Web Framework

Ottawa, On. – October 2, 2013 – Crank Software, an innovator in embedded user interface (UI) solutions, announced today that Crank Software has been selected by Nintendo to provide the WebKit technology that powers the Nintendo Web Framework. With Crank Software’s WebKit, web developers, in addition to traditional developers, can now use the Nintendo Web Framework to develop their newest games and applications on the Wii U system with the best technologies possible – including HTML5, JavaScript, and CSS. In a recent analyst briefing, Nintendo stated that there has been tremendous interest in the new Nintendo Web Framework development environment. Read More >>

Loading JavaScriptCore extensions dynamically

When doing a WebKit port many people are looking to extend JavaScriptCore with custom functions which expose features of their particular hardware platform. Usually, they are also looking to load JavaScript extensions dynamically at runtime rather than being forced to recompile the WebKit libraries each time they want to add or change a function.

This blog posting looks at two ways in which WebKit can support dynamically loading JavaScript. JavaScriptCore can also be extended by compiling in extensions and writing IDL files, but that is a subject for a future post.

JavaScriptCore provides a C public interface, which can be found under Source/JavaScriptCore/API. There are JavaDoc style comments describing the individual functions, and some documentation is available on Apple’s site. Another blog posting provides a nice, although somewhat OS X specific, overview of the API.

A good strategy when creating your dynamic library is to have a single entry point which will create and return a JavaScript object containing your extension functions and objects. This function should have a signature something like this:

JSObject jsextension_load(JSContextRef ctx);

This function will use the supplied JSContextRef to create a class and make and return an object of that class. The class will contain static functions and possibly values that you want the extension to export and is defined through a JSClassDefinition structure instance.

The class definition is passed in as an argument to a call to JSClassCreate. You only need a single class instance for the lifetime of your  dynamic library. Once the class instance exists, an object can be made using the JSObjectMake call, and this object is the return value of your jsextension_load function.

The next step to to build a dynamic library which exports jsextension_load. Once this is complete, you then need a way of loading the extensions. This could involve specifying the name of the dynamic library to open, or simply scanning a specified directory for dynamic libraries, opening them and seeing if they export a jsextension_load symbol.

This post will look at two ways of loading extensions. The first is by using something similar to the CommonJS “require” function. The second is adding a method to the WebView to load an extension.

WebKit does not support the CommonJS require function, so this will have to be added to the JavaScript interpreter. An easy way of doing this is to add it to the set of static functions associated with the JavaScriptCore global object. These functions are defined in JSGlobalObjectFunctions.h and JSGlobalObjectFunctions.cpp which are found in JavaScriptCore/runtime, and the require function can be added to these files, along with a reference to it in the global object table found in JSGlobalObject.cpp.

The require function can be given a string as an argument and then attempt to dlopen a dynamic library corresponding to that string. The advantage of this approach as opposed to modifying WebView is that the extensions are available to the standalone JavaScript interpreter and the extensions to be loaded are controlled through JavaScript rather than through the WebView.

A second way of loading extensions is to modify the WebView API to support an extension loading function. The problem that occurs with this approach is to ensure you have a valid JavaScript context that can be used when loading the dynamic libraries. The current JavaScript context can be retrieved by calling

toGlobalRef(frame->script()->globalObject(mainThreadNormalWorld())->globalExec()

where frame is the current “main” frame in the WebView.

This is fine, unless you want to load an extension once and have it persist across page loads, as each page load will cause a new JavaScript context to be  created. Fortunately, there is a callback in the FrameLoaderClient which is called after a new global object is created, called dispatchDidClearWindowObject. This callback can be used to trigger the WebView to reload any extension into the new JavaScript context.

The advantage of managing extensions through the WebView rather than through JavaScriptCore is that is minimizes the amount of change to core WebKit code and makes the full interface provided by the WebView available to extensions.

Using WebKit’s Remote Web Inspector for Embedded Webkit ports

I am sure every JavaScript developer is familiar with the “Inspect Element” feature in Chrome and Safari.  It is a very useful tool for debugging web applications.  On an embedded target, pulling up the web inspector is technically possible, but may not be practical with limited screen real estate or limited inputs. Luckily, WebKit.org has recently added support to make all this easier with the Remote Web Inspector.

The Remote Web Inspector allows you to open up the Inspector in your desktop browser (Chrome or Safari) and connect to your target platform.  Now you can use the luxuries of a desktop environment to debug your web application on the embedded target.

If you want to try it out launch Chrome Canary with the following Option:

chrome --remote-debugging-port=9222

then launch a second browser and navigate to http://localhost:9222

HOW IT WORKS

Here is how it works under the hood.  The inspector has 2 main components, the front-end and the backend client. The Front-end is simply a JavaScript webpage, which is the UI for this tool.

To view Front-end Source Code:
http://trac.webkit.org/browser/trunk/Source/WebCore/inspector/front-end

The inspector backend is the agent that interacts with WebKits engine.  The inspector frontend and backend communicate with each other via a Json formatted debug protocol.

In the case of the local inspector the backend calls for receiving and sending message to/from the  frontend are as follows:
Receive Message from Frontend:

InspectorController::dispatchMessageFromFrontend()

Send Message to Frontend:

InspectorClient:: doDispatchMessageOnFrontendPage(..)

For the Remote Web Inspector the JSON debug protocol messages are sent over a Web Socket.   To enable this, a new component was added called WebInspectorServer.  The WebInspectorServer is a mini webserver that is able to serve up the inspector frontend, establish a websocket connection for the debug JSON message communication and tie in to the Inspector backend.

The Web Inspector Server listens on a specified port (in the chrome example above port 9222 is used).  When you initiate the connection via the remote browser by navigating to landing page (ex. http://localhost:9222), the inspector backend returns a list of webviews (note of you have a single view implementation you could bypass this and load the inspector frontend directly).   When you load the Inspector Frontend, the front-end will establish a Web Socket connection. The Web Inspector server will notify the Inspector backend of the remote Frontend.

The send and receive functions for the Remote communication are as follows:

Receive Message from Frontend

WebInspectorProxy::dispatchMessageFromRemoteFrontend(const String& message)

Send Message to frontend.

 WebInspectorProxy::sendMessageToRemoteFrontend(const String& message)

 PORTING REMOTE INSPECTOR TO WEBKIT:

The only downside to the Remote Inspector is webkit.org only supports it on WebKit2.  So if your running WebKit, you will need to take the Inspector Server from WebKit2:

http://trac.webkit.org/browser/trunk/Source/WebKit2/UIProcess/InspectorServer

and put it in your WebKit directory.

The WebInpectorServer will need to be modified to replace WebInspectorProxy calls with InspectorClient calls.

 

Depending on the vintage of the port you may need to grab the changes to WebSockets to support the changes needed to support the WebInspectorServer.

FURTHER READING:

For more information on the Remote Inspector check out webkit.org blog post:

(http://www.webkit.org/blog/1875/announcing-remote-debugging-protocol-v1-0/)

Further information on web sockets:

http://www.websocket.org/

Coming soon: WebKit for QNX

As I’ve posted before on the QNX development blog Send Receive Reply Crank has been working away for a couple of months now getting WebKit up and running in the QNX Neutrino 6.4.0 environment using the Advanced Graphics (AG) framework. 

It has been running well on x86 for several weeks now, but as always no source code that is supposed to be platform agnostic is ever as clean as it should be, so it took a few days of tinkering to get it up and running on all of the important architectures that QNX supports.  As much as WebKit is touted as being more “embedded friendly”, we still ended up spending a week doing some light tuning to improve its performance on lower end hardware (A note to QNX developers out there repetitive file open’s are not your friend … but more on that later).

We’ve got lots of ideas on how we can Crank up the performance of these browsers even further but for now, we’re excited about what we’ve brought to QNX Neutrino 6.4.0: A modern web browser framework (WebKit) running in a high performance graphics environment (AG) that runs on the four major processor families (x86, PowerPC (PPC), SuperH (SH), ARM) supported by QNX.

We’re working in conjunction with QNX to get a new WebBrowser project seeded into Foundry27, the QNX developer portal and will be making binaries available for users to try (in addition to our original Photon port).  However, in the meantime, if you are interested in integrating web browsing technology into your product and want to know more about how WebKit might work for you, give us a shout.

In case you were wondering what the platforms were were running on, here’s the list of which evaluation boards and platforms that we used for testing:

  • Jade (ARM)
  • Media 5200 (PPC)
  • Solution Engine 7751 (SH)
  • Intel Atom (x86)

Thomas