Rich Internet Applications vs HTML
Forrester Research has done a usability comparison between RIA (Rich Internet Apps) and HTML apps, in a report entitled Smackdown: Rich Internet Applications Versus HTML. The result:
"We looked for examples of rich Internet and HTML applications in each of four categories and compared how well they supported relevant user goals. We found that, on average, RIAs outperform HTML interfaces; at the same time, RIA usability can fall prey to basic design mistakes."
(emphasis ours)
I checked out the report, which reviewed 22 Web applications across four categories: hotel search and reservation engines, mapping tools, PC configurators, and product finders. Note that by RIA, Forrester bundles together Flex, Ajax and other interactive Web technologies.
In Forrester's analysis, only 5 of the 22 apps passed their usability test - and all 5 were RIAs. The evaluation criteria was a set of 15 questions such as "Is essential content available where needed?" and "Is the task flow efficient?".
Here is one example of the usability results, in the mapping apps:
The RIA apps are the ones colored dark brown - Google Maps, an Ajax application, scored the highest (the pass mark was 15 btw, so Microsoft and Yahoo Maps only just passed!). I wonder if these kinds of results say more about the design resources present in Microsoft, Yahoo and Google - compared to Maps.com, Rand McNally and MapQuest? It's probably a bit of both - RIAs do achieve better usability for maps, and the big companies have more resources to spend on achieving high interactivity (and thus better usability).
The report also notes some usability flaws in RIAs, but these are basic design problems rather than issues with the RIA technologies. For example using unintelligible icons and symbols, and problems with text legibility.
Overall, I don't think the results are overly surprising - or anything new. The question, circa 2007, is more about which type of RIA is best: Ajax (what I've called 'browser-based apps' in the past), or ones using browser plug-ins or downloads such as Flex and WPF (Windows Presentation Foundation).
Share
Comments
Subscribe to comments for this post OR Subscribe to comments for all ReadWriteWeb posts
The question is what expense are paying to moving wholesale to Rich Internet Applications (RIA). Most of these apps are not accessible. While they may be flashy and usable for some, there is a good chunk of the population who aren't going to be able to even get in the front door. I hope that 2007 is not just the year of RIA but smart and accessible RIA.
It has long been known, from the early days of the web, that plugins are a loser.
Could you elaborate more on the 15 RIA evaluation questions for us budding designers? Thanks.
Different RIA have specific features with them to suit every designers need,however they might be marked with prices accordingly.
IMO Local.Live.Com has a better mapping solution than Google Maps (both browser based and as a mobile app) but that is a personal view.
I'd be really interested to see the criteria that were used that give Google the "winning" score and put the Y! and MS solutions on the borderline... just presenting the figures without any justification isn't that helpful (eg is there a factor "Is product produced by Google? If Yes add 25% to score"!)
Ajax solutions at the moment, though fairly well understood are a bit haphazard in support for accessibility (but then again even Flash has really poor keyboard and voice control). Flex (and Apollo) and WFE are still early days to see how well they take off as something users are willing to have invading their PC. WPF/E as a browser hosted plug-in technology is in its very early days so it's hard to say how it's going to address the accesability / usability issues. Hopefully MS will have something more fleshed out than the current (though kinda cool) CTP available in time for Mix07
The technologies behind RIAs provide an increased flexibility to application development, and that can be either a good or bad thing, depending on who is wielding it. My feeling is that as you move from HTML to AJAX to Flash, you increase flexibility, but have to be more actively involved in making sure you address issues of usability.
I think what turns people off about a technology like Flash is the ease with which one can create an application that completely violates existing standards of usability. With HTML interfaces, there is a more restrictive range of possibilities, and you're less likely to use a paradigm that is unfamiliar to a user. Maybe the adoption of Flex 2 will help standardize some of these Flash RIAs so that interface designers will have a firmer base to work from?
Great stuff Richard. I requested a copy so hopefully I'll have some comments for you (though they'll be on ZDNet).
"It has long been known, from the early days of the web, that plugins are a loser."
Bob, what? Without plugins the web would be a bunch of static text pages. Plugins have shown what the web is REALLY capable of; video, audio, interactivity. The problem with the web is that it's still stuck in a browser.
Do you think it would be pushing fair use of the paper to tell us the 22 apps? Maybe we could decide for ourselves. :)
Do you think Mozilla's XUL has a place in this classification? If WPF has, XUL also may have - at least it's cross platform.
Interesting. I associate RIA with flex-based apps, not necessarily ajax. I do agree that a more compelling study would be to compare the effectiveness of a flex-based app to an ajax-based app. I will be facing this exact decision in 2007 (do we transition from ajax to flex)?
Having spent the past year managing a team of developers on an ajax-based site (which is still not released, so I can't give details on it quite yet, unfortunately), there's a lot to deal with on the technical side of things -- especially as you test in different browsers. It really reminds me of the earlier web days where you would truly have to have a different set of HTML for your site to look "good" in Netscape vs. IE. It has been a fun, sometimes bumpy ride, but ajax is certainly a viable solution and when done right, can get very high marks (i.e. Google Maps).
Where the "flex" play is intriguing is that you get to bypass all of the ugly browser incompatibilities. There is value in knowing that because you're using the adobe/macromedia flash plug-in, it will behave the same from browser-to-browser and OS-to-OS. If your solution requires a richer user interface, where animated transitions are important, flex/flash is also a nice-to-have. The site we are building is for the female demographic, so I'm very interested in creating an online experience that is both functional but also inspirational. The clunkiness of CSS hacks, javascript, and general website behavior presents challenges and occupies a considerable amount of time simply because the application just behaves differently from IE to Firefox to Opera, etc.
Bypassing this compatibility struggle and truly focusing on improving the application would be superb. At this point however, will a flex-based app generate more revenue than a flex-based app? If done right, consumers probably don't care, as long as it gets the job done for your target audience. However, there may be a total cost of ownership question too -- time spent debugging an ajax-based solution across browsers > or
I would be really interested to know (without buying the report) if accessibility was taken into account at all when gaging the value of user experience in this research. Can anyone shed light here?
Apologies for not being able to provide more details from the report - it's a shame it's a paid-for one, but I guess the Forrester analysts have to eat.
Tracy, I didn't see any specific accessibility questions. One question was "6. Is text legible?" - but I presume that doesn't take into account accessibility for e.g. visually impaired people.
Well put E. Long. Our startup company, Convos,is building an innovative personal / group information management service (launching in 2 months) in Flex 2 for many of the reasons you pointed out. The major ones being the cross-browser compatibility and upcoming Apollo. Flex is an important asset to Adobe and I'm sure they'll be making sure Apollo can play well with it.
On the interaction side, Flex 2 ships with an extensive library of controls that have already been proven in user existing experiences in desktop apps. Custom controls can also be built for innovative ways of managing content / interaction.
As for design, Flex gives you an open canvas. It works well with everything from basic CSS to fireworks graphics. The obvious key is appropriate usage. Flash based sites are often so over-designed that it impacts the user experience.
Hello,
Quick note on some of the comments. Some seem to think thatthe use of Flex implies lack of accesibility. It is important to correct that. While designing for accessibility takes thought with any technology, Flash and Flex are fully capable of accessible apps, and in fact there is a lot built into the Flex framework that makes it quite straightforward to be accessed by screenreaders vs most (any?) AJAX frameworks. For more info:
http://www.adobe.com/macromedia/accessibility/features/flex/
Regards,
David
Adobe