When considering the various best practices surrounding the design of Mobile Web Experiences and Architectures, such works as the W3C’s Mobile Web Application Best Practices guide, or the excellent Mobile Web Best Practices site, and of course, the seminal text, Mobile First, are likely to come to mind. The concepts and strategies presented in these works are a staple in the design of many modern Mobile Web Experiences and are without question an invaluable resource. In addition to these and other similarly related works, another new and valuable resource has been made available from a very important player in the Mobile Space indeed – an actual Wireless Carrier, AT&T.
Recently, I was contacted by a representative of the AT&T Developer Program informing me of the research conducted by the AT&T Research Labs and, the subsequent resources made available by AT&T as a result of their findings. Since I was unaware of this work, I was very interesting in learning more and, after reading the introductory statements, I was quite eager to apply AT&T’s recommendations as well; to quote specifically:
We quickly saw that a few, simple design approaches could significantly improve application responsiveness.
Having read through the material in it’s entirety (provided below) I must say I am rather impressed. The information provided has very real and practical implications on the design of Mobile Web Applications. Specifically, I found the clear and concise explanation of the underlying implementation of the Radio Resource Control (RRC) protocol to be particularly relevant and useful. RRC is by far one of the most important design factors to consider in terms of battery life and Application responsiveness and, as the research suggests, this may not have been common knowledge.
By far, the most interesting and notable aspect of the AT&T Research Lab’s work in this area is the fact that all of the information provided is applicable in the context of all Wireless Carriers, not just AT&T. That is, the recommendations given, such as those regarding the RRC State Machine, for example, are all based on carrier-independent standards and protocols implemented by all Wireless Carriers. As such, understanding the implementation specifics and recommendations provided is certain to prove valuable for all users of your Application, regardless of their Carrier.
If you haven’t all ready, I highly recommend reading and applying the principles provided by AT&T’s research to your current and future Mobile Web Application Designs.
Last week, shortly after I blogged about the release of jQuery Mobile 1.0, I received an email informing me of the release of another Mobile Web Framework: DHTMLX Touch 1.0.
Being that I was unfamiliar with DHTMLX Touch (as I have been using jQuery Mobile almost exclusively), I was quite interested to learn more; and, having tried the Examples and reviewed the Documentation, I was rather impressed by DHTMLX Touch.
I recently read a preview of a column which is to be published in the next addition of ACM CHI magazine, Interactions. This particular article is a rather interesting read in that it touches upon what the authors argue are the many short-comings in current Gestural Interfaces; stating that they pose a huge step backwards in terms of Usability.
This may not have raised many eyebrows if it were not for the expertise of the articles authors, Donald A. Norman and Jakob Nielsen; both of whom know quite a bit about HCI.
Experimentation in new technology and the process of learning what works and what does not can be challenging. This article raises some important, yet mostly overlooked, concerns surrounding new technologies which are built upon Gestural Interfaces; i.e. current touch screen devices such as iOS and Android. Certainly a good read for anyone interested in Touch Screen development. Gestural Interfaces: A Step Backwards In Usability
The article is particularly interesting in that it touches on (pun intended) the human factors involved in how we physically interact with devices. The Activity Zones outlined in the article equate to the areas which provide the greatest level of physical comfort when interacting with a touchscreen device.
The general physicality of natural, symbolic and sequential gestures associated with designing touchscreen experiences as well as the environmental distractions and engagement models of mobile experiences is a topic which I find quite interesting. This is a significant leap forward from the traditional WIMP interaction model. All of these considerations allow for a more Human centered design focus, just as it should be; after all, this is the purpose of UI Engineering.
“We are in the midst of a revolution across a variety of screens”
You may recall first hearing the notion of a “Multiscreen Revolution” during the keynote at Max 2010. If you take a moment and think about it that’s a rather profound statement, and by all apparent indications a very true one.
This is also how Kevin Lynch, CTO at Adobe, begins his recent article, aptly titled “The Multiscreen Revolution” in which Kevin provides a succinct breakdown covering the driving forces behind this revolution and how it is guiding the future of the Flash Platform.
Allow me to digress for a moment as, in a way, for me at least, the “Multiscreen Revolution” tends to conjure up the hypothetical notion of a Multiverse. This may be a suitable analogy I suppose as we have been focused in a predominantly Personal Computer based reality for many years, and while this remains relevant, it is no longer exclusive.
I have been giving a lot of thought lately regarding designing software in a multi form factor paradigm and felt I would share some initial thoughts on the subject. Keep in mind much of this is still quite new and subject to change; however, I have made an attempt to isolate what I feel will remain constant moving forward, particularly in the context of developing native applications with the Flash Platform across a variety of screens.
First, User Experience Design
My initial thoughts on the implications of what a Multiscreen paradigm will have on the way we think about the design of software are primarily concerned with User Experience Design (UX Design). While simply using CSS3 media queries to facilitate dynamic runtime layouts will be needed for most HTML based web applications, I do not believe these types of solutions alone will allow for the kinds of compelling experiences users have come to expect. Sure they are useful, and for some HTML based websites they may suffice. However, in the context of more complex applications, RIAs in particular, but also just about every application developed specifically for a PC, too, I believe UX Design will need to encompass the best of what a particular form factor, “screen” or whatever you prefer to call it, has to offer, be it a PC, smartphone, tablet or TV.
For example, it is extremely rare that a UX Design intended for users of a PC will translate directly to a Mobile or Tablet User Experience. The interactions of a traditional physical keyboard and mouse do not equate to those of soft keys, virtual keyboards and touch gesture interactions. Moreover, the navigation and transitions between different views and even certain concepts and metaphors are completely different. In simplest terms; it’s not “Apples to Apples”, as the expression goes.
With this in mind, UX Design should remain at the forefront of Software Design, and in order for that to happen UX Design will not only need to continue to reflect the needs of users, but also by leverage the capabilities of the particular devices and screens on which an application will run. This is necessary as it better serves the needs of users in new and useful ways. Likewise, UX Design will need to account for the limitations (resources in particular) of those same form factors.
Second, Architecture
Mulitiscreen design obviously poses some new challenges considering the growing number of form factors which will need to be taken into account. The good news is most existing well designed software architectures have been designed with this in mind all along. That is, the key factor in managing this complexity I believe will be code reuse. A common theme amongst many of my posts, code reuse has many obvious benefits, and in the context of Multiscreen concerns it will allow for different screen specific applications to leverage general, well defined and well tested APIs. Code reuse will certainly be of tremendous value when considering the complexities encountered with Multiscreen design. Such shared APIs can be reused across applications which are designed for particular screens or extended to provide screen / device specific implementations.
For example, the Architectures I have worked on are designed such that each is broken out into specific modules (libraries) which, on a high level, are typically as follows:
unittesting-support: Provides convenience extensions of unit testing and mock frameworks.
commons: Provides all generic, reusable APIs which have no dependencies outside of those of the Flex Framework and Flash Player API.
framework-support: Provides reusable extensions of a particular framework. There can be multiple framework-support libraries, each of which would be specific to an individual framework; e.g. parsley-support, swiz-support, robotlegs-support, cairngorm-support, springactionscript-support etc.
business: Provides domain dependent, business specific reusable APIs which are common amongst all projects within the business domain. This includes domain models, shared services, Presentation Models, UI components and anything else which is specific to the business domain. While all of the previously listed libraries could be used with any AS3 / Flex project, the business library is intentionally coupled to the domain.
All of the above projects are used by the different business applications, allowing for significantly reduced complexity of each individual application. Moreover, each library provides isolated test coverage which allows for a greater sense of confidence when building applications which are dependent on them. This type of structure also lends itself well with common SCM and build conventions, allowing for simplified branching and versioning.
Architectures designed similar to what I have described above I would consider to be “multiscreen ready” (provided they are optimized and efficient) in that much of the underlying implementation has already been completed, tested and proven. What’s left is the mobile, tablet or TV application designs which should be mainly concerned with UX, particularly interactions, navigation and device capabilities. From these screen specific UX Designs the application architecture is mainly focused around view concerns specific to those devices; leveraging the existing APIs as needed. This is where the Presentation Model Pattern (which I have been recommending for years) or similar patterns will be of great value.
I also anticipate additional libraries being abstracted in addition to those I have listed above as a result of these device specific projects being developed. For example, I could easily imagine “device-support”, “mobile-support” and “tablet-support” projects which would provide reusable APIs specific for those screens so as to be leveraged across applications. In fact, I am working on libraries such as these at the moment.
Reusable libraries are nothing new; however, their role now is perhaps even more important as it is quite likely that multiple implementations of the same application will be needed for the various form factors and contexts. Existing reusable libraries may also need to be further optimized to provide the best performance against the slowest devices in order to be considered suitable for devices with the most limited resources. A natural side effect of such optimizations is that implementations of an application targeting the fastest form factors (typically, PCs) will benefit greatly.
Some Concluding Thoughts
In short, I believe both users and developers alike will be best served by providing unique User Experiences for specific screens as opposed to attempting to adapt the same application across multiple screens. One of the easiest ways of managing the complexity of multiscreen design and development will inevitably be code reuse.
I also believe the main point of focus should be on the medium and small form factors; i.e. Tablets and Smart phones. Not only for the more common reasons but, also because I believe PCs and Laptops will eventually be used almost exclusively for developing the applications which run on the other form factors. In fact, I can say this from my own experiences already.
While there is still much to learn in the area of Multiscreen Design, I feel the ideas I’ve expressed here will remain relevant. Over the course of the coming months I plan to dedicate much of my time towards further exploration of this topic and will certainly continue to share my findings.
The Web Timing Specification (draft) aims at providing a standard set of APIs which allow for true end-to-end instrumentation of page load times across browsers.
To quote the w3 spec: “This specification (Web Timing Specification) defines an interface for web applications to access timing information related to navigation and elements.” The API is based on the Navigation Timing and Resource Timing interfaces, respectively.
While I haven’t seen this specification mentioned as part of the HTML5 Family before, in many ways I would consider it to be a worthy candidate for membership as it provides a standards based API through which web applications can be tested for load efficiency. This is obviously something quite useful for any web application as, the ability to precisely measure page load times – and implement optimizations as needed – affords developers the opportunity to provide an improved user experience.
Historically, the ability to accurately measure page load times of web applications has been quite challenging for a number reasons. Just knowing when and where to begin is debatable and, determining the best means of doing so can be a challenge in of itself. Regardless of any current strategies being used, the result is never entirely accurate. With Web Timing developers need not be concerned with these specifics as the API provides the ability to truly measure page load times by encompassing the full scope of loading and parsing a page. This includes the time involved to request, receive and render an HTML document.
For more information, try out the examples in the current supported browsers; IE9, Chrome 6.
“If everyone is moving forward together, then success takes care of itself.” – Henry Ford
The HTML5 Family has been receiving considerable coverage lately; and, rightfully so, as, many next generation browsers – specifically those in the Mobile space based on WebKit: Android, iPhone, iPad, Blackberry etc. are now beginning to implement it’s specification, or parts thereof. On the Desktop more HTML5 support is also being seen in the latest versions of Chrome, Firefox, Safari, IE9 and Opera.
The HTML5 Family of technologies will without question play a vital role in the future of the web; and currently, in the mobile Web space, that future is now.
A Brief Overview of the HTML5 Family
For anyone who is unfamiliar with what has been termed “The HTML5 Family“, allow me to provide succinct overview of the technologies which I feel encompass what has already become a rather overloaded term. In general, on a very high level, I would summarize the HTML5 Family simply as follows:
HTML5
CSS3
JavaScript
While the above could be considered the umbrella Technologies upon which The HTML5 Family is based, there are certainly more associated technologies which themselves further augment what could be considered the HTML5 Family, some of which are (based on current specification status at the time of this writing ):
Microdata
Geolocation API
Device APIs
Web Storage (localStorage, sessionStorage) APIs
Web SQL Database API
Web Workers API
Web Sockets API
HTML5
First, HTML5. HTML5 is the next major revision of HTML which aims to advance the open Web through web standards and semantically rich content. HTML5 defines an emphasis on semantic structure and meaning.
In general, HTML5 provides a content model which can be broadly defined into the following categories: Metadata content, Flow content, Sectioning content, Heading content, Phrasing content, Embedded content and Interactive content as well as form-associated elements etc.. HTML5 defines new tags such as canvas, audio, video, keygen, header, footer, nav, article, aside, datalist and more.
Explaining what JavaScript is may seem like a moot point as it is the language of the browser and therefore, the language of the web. However, it is important to outline some key underlying specifics of the language. In particular, JavaScript is a dynamic, prototypal, object-oriented scripting language. Its prototypal nature is quite different from the classical concepts of traditional object oriented languages. In order to get the most out of the language one needs to understand and embrace prototypalism and dynamism. Many of JavaScript’s true potential can be mistakenly overshadowed by it’s assumed design flaws; however, this needn’t be the case. As long as one understands the fundamental concepts of the language, it’s true potential can be realized to enrich development and allow for a level of expressiveness unmatched in type-safe languages.
Microdata
HTML5 Microdata provides a mechanism which allows machine-readable data to be embedded in HTML documents in the form of annotations, with an unambiguous parsing model. Microdata is compatible with numerous data formats such as JSON. Micro-data is intended to provide a standard to replace other similar concepts such as RDFa, from which browsers and other applications can discover relevant content based on the context of an applications markup. Such examples include markup for contact information, calendar events and more. This markup is understood by HTML5 compatible browsers which can then automatically offer to add the relevant content to the appropriate application. At an implementation level, microdata simply consists of a group of name-value pairs; with the groups being called items, and each name-value pair is a property.
Geolocation API
The HTML5 Geolocation API is rather straightforward; it simply provides a means by which the location of a device can be determined via a native API (as opposed to say, determining the clients IP address). The Geolocation spec is currently in last call status in the W3C.
Device APIs
Device APIs are client-side APIs which allow for direct interaction with native device services such as a device Camera, Calendar, Contacts etc.
Web Storage API
The Web Storage API allows for the persistence of local (permanent) and session based (browser session) data on the client. The API for Web Storage is extremely simple as it is based upon simple Key / Value pairs; with which Keys are simply Strings. Each site contains its own separate storage area.
Web SQL Database
While not a part of the actual HTML5 specification, the Web SQL Database presents some extremely interesting possibilities within Web Applications. The Web SQL Database provides a set of APIs which allow for the manipulation of client-side databases using SQL. The Web SQL Database is based upon SQLite (3.1.19) thus supporting the features as specified therein.
Web Workers API
Web Workers provide a mechanism by which web content can execute scripts in background threads. Web Workers allow for a much needed multi-threaded implementation for web based applications executing in a browser. While somewhat similar, Web Workers are different from threads in that they are primarily intended for executing long running, expensive computations and algorithms so as to facilitate non-blocking UI background processes. One specific aspect of Web Workers which has considerable positive implications for the web moving forward is that they run in native threads as opposed to Green Threads; as is the case in VM architectures. This is quite significant as it essentially means Web Workers can scale vertically. Considering the inevitable proliferation of multi-core desktop and mobile devices, this is certainly something that will prove advantageous.
Web Sockets API
Web Sockets provide native, full-duplex communications channels which operate over a single socket that enables HTML5 compliant browsers to use the WebSocket protocol (exposed via a JavaScript API) for two-way communication with a remote host.
If you are interested in learning more about each of these technologies I recommend the following resources:
Moving forward, I plan to go into further detail for each of these associated HTML5 Family technologies, providing working examples and detailed information as to how each can be utilized to create some very unique and interesting possibilities on the Web.
Although still in it’s early stages, the core platform is quite stable and support from the AIR engineering team has been very good while the pre-release forums have also been quite active with lots of useful information being shared daily. In just a little under an hour I was able to have two POCs demonstrating the Accelerometer and MultiTouch Gesture capabilities running flawlessly on my Droid. Additionally, I was also able to develop a very basic Geolocation prototype in next to no time at all which accurately conveyed latitude / longitude, altitude and even speed. In the time since I have been focusing on real world applications and the results have been excellent for such early stages of the platform.
Some notable features I have been working with are: GPS, Accelerometer, Multitouch / Gestures, SMS/TEL URI Schemes, IME, S/W Keyboard, Screen Orientation, Screen Dimming, Menu/Back keys and more.
As the pre-release and my applications built on AIR for Android progress I will share my findings as well as provide open source APIs, code examples, videos and / or screen shots of the apps I am working on, so stay tuned for more information.
Rather than going into any detail regarding my thoughts surrounding Apple’s updated iPhone developer license clause last week, I instead prefer to focus on the more exciting and positive developments the future has to hold for the Flash Platform in the mobile space; and at the moment, it’s Android
As you may be aware, beginning with Adobe Flash Player 10.1, the AIR 2.5 for Android SDK and Android, the Flash Platform will now begin to close the gap in terms of developing and deploying Web, Desktop and Mobile applications. Thus it appears this could open up some very exciting possibilities in the RIA space as, a write-once, deploy-anywhere solution for Mobile, Web and Desktop applications is obviously highly desirable.
For those of you unfamiliar with Android, it is a premiere software stack for mobile devices which provides an Operating System built on the Linux kernal, a very well designed middleware layer and core applications including an E-mail client, SMS program, calendar, maps, browser, contacts and more. Android also provides an Application framework, a Dalvik virtual machine which is optimized for Mobile Devices, an integrated web browser based on the widely known WebKit engine, SQLite storage, common Media support, hardware dependent Bluetooth, EDGE, 3G, WiFi, Camera, GPS, Compass, and Accelerometer support as well as many other features.
Originally developed by Android Inc., which was later acquired by Google, Android is now governed by the Open Handset Alliance; a consortium devoted to advancing open standards for mobile devices. Currently, over 50 mobile phones are expected to come shipped with Android in 2010. Moreover, Google and their hardware partners are now shipping 60,000 Android handset units each day! If this trend continues (which it certainly appears will be the case) this equates to over 21.9 million devices shipping with Android per year.
Traditionally, getting started with Android has been quite simple for developers who have experience with Java as one need only download and unpack the Android SDK distribution and install the Android Development Tools (ADT) Eclipse plugin. Managing different Android platforms as well as other SDK components is accomplished via AVD Manager which come with the SDK. As expected, the Android SDK also comes with a very high quality device emulator which feels similar to the BlackBerry JDEs device Simulators.
While developing applications for Android with ADT is certainly convenient (and quite fun), from a Flash Platform development perspective it is much more desirable (as well as economical) for developers to leverage their existing skill-set and APIs to develop a single application targeting Flash Player or the AIR runtime that will work with any device shipped with Android. And with Flash Player 10.1 and the current private beta of Android AIR 2.0, the Flash Platforms reach will now include the Android platform. The most significant of these new possibilities is the ability to develop a single application which supports both Web and Mobile devices alike. Thus considerably simplifying the development and deployment process. Of particular interest is the ability to leverage Mobile Device specific features such as Accelerometer, GPS, multi-touch, gestures screen orientation etc. from an AIR application.
Flash Player 10.1 will support devices running on Android that meet the minimum software and hardware requirements, which at the moment appear to be devices with an ARM v7 (Cortex) processor. Both Droid and Nexus One carry ARM v7. Architecturally, I am quite interested in seeing how this all comes together in terms of memory and cpu optimization.
Working in conjunction with Adobe, as part of the Open Screen Project, Motorola is helping to develop Flash Player 10.1 so it works on Android. Motorola will also be deploying the Flash Player broadly across its Android product portfolio going forward; releasing Flash Player updates for existing devices such as the Droid (which I happen be actively developing for).
Adobe is targeting the end of July 2010 to have the Android AIR 2.0 Beta and Flash 10.1 for Android available. For updates sign up for: