Digg Demos Optimized Datastreams
Any good webmaster knows the cardinal rules of website optimization. Yahoo! wrote them all up years ago on its Developer Network site. And the more of these rules you can adhere to, the faster your site will load for your visitors. But, more and more often as sites turn to using asynchronous technologies like AJAX to make their sites more responsive and act more like applications, the old rules lose their effectiveness.
Today, the website wizards behind Digg have revealed a new technology called MXHR, or Multi-Part XML HTTP Requests, as a method for optimizing delivery of Digg's complex AJAX-enhanced site. The implementation of MXHR is an addition to Digg's User Interface Library, called DUI.Stream. While still in a fairly rough early stage, Digg believes that MXHR will eventually give it a huge boost in un-cached page rendering efficiency.
There's a couple of problems that Digg has with using traditional optimization techniques. First, the site violates the first rule in a big way, since it uses multiple HTTP connection requests in order to render all parts of the page. Dialog boxes, comments, user icons and more all have to be delivered quickly and with equal priority, and multiple requests was the only way to go until recently.
Second, it can't rely on too much caching to help speed page load times. Digg is a discussion board as much as it is a news site, so caching is a huge problem. Active stories can sometimes have hundreds of new threaded comments being created every minute, and caching would cause huge discrepancies on how much of the conversation each client would actually see from moment to moment.
These issues are what MXHR is designed to optimize. Utilizing DUI.Stream to open only one HTTP connection between server and client, the separate 'page objects' such as dialog boxes, CSS and more can be bundled and sent through the same open connection. By eliminating, in some cases, hundreds of separate HTTP connection requests for a page render, it can approach the rendering speed of a fully-cached page. Plus, this allows the server to control the order in which each page element will be built. This will result in the page appearing almost instantaneously for the user with the the hidden framework being backfilled as needed.
Digg provides two examples of the DUI.Stream library in action, the first being an un-cached text demo, and the second an un-cached image demo. While the first example isn't too amazing, from what we saw the new library performs on par with traditional rendering, the second example with icons is stupidly faster. It's absolutely no contest that MXHR is the clear winner there.
Code is provided in the post for you developer-types out there to play around with along with a GitHub repository of examples. Digg is looking for feedback and suggestions on how to improve on this so please go take a look! We can't wait to see this tech get deployed so that we can have all the bells and whistles of Web 2.0 architecture plus all the speed of a well-optimized site.
Share
Facebook
Comments
Subscribe to comments for this post OR Subscribe to comments for all ReadWriteWeb posts
You've spelt AJAX wrong (twice)
Yikes! Good catch, Richard. I was focusing so much on getting 'MXHR' in the right order, I totally missed that. Corrected now.
I ran both the demos in firefox. The normal is faster than MXHR by tiny amount almost everytime, iff I open in a different tab(and not jump to that tab). It takes more time (like you report), if I open it in the same page/(or diff tab and if I jump to it). Why is that?
Wow man, right when you thought Digg couldnt get no better! LOL
RT
www.privacy.pro.tc
This is cool...the demos for me the normal was faster....is that weird?
This is pretty awesome. I ran both demos 5 times each and the MXHR Stream was faster every single time. Great work.
Very clever - but maybe when you have to invent ways to make AJAX faster, you should stand back and think "am I using too much AJAX?".
Sorry, but how repeating the same image 300 times could count as an example of new technology? Any 10 years old javascript one-liner could do the same.
that's pretty interesting, but shouldn't they be working on crunching down on code and minimizing requests or to load everything internally and then display it as necessary.
yep works like described.
Some of you guys cannot see it because you have bandwidth in mB/s.Out here where the bandwidth is just 30-40kB/s, the difference is profoundly visible.
awesome technique digg crew !
@Alex
Technically it's not repeating the same image. Each one displayed is a different resource with a different URL. They've used the same image for the simplicity of the demo but it is as if there are 300 different images being loaded.
Those images provide a test that's somewhat analogous to the hundreds of digg user images served up in a commment thread.
check http://yourtvonline.com
this sounds very similar to COMET, which has been around quite a while now :-
http://en.wikipedia.org/wiki/Comet_(programming)
and an example of a COMET server :-
http://www.freeliberator.com/
I'm all for crazy new tech solutions, I really am, but this feels wrong to me.
Point 1, graceful degradation:
Up until this point, even with crazy, crazy ajax applications and absurd CSS, if you're good enough at it, you've been able to create applications that "gracefully degrade", meaining, without javascript or css, the applications still work. This method just completely throws that out the window, if javascript doesn't work (if you make a typo somewhere that stops execution!) or if the first javascript file doesn't transfer, or times out, then the entire page isn't going to work.
Point 2, server side optimization
Almost all of Yahoo!s guidelines for optimizing page load times can be significantly improved by performing back end tweaks, the largest ones are:
minifying and combining js and css files
spriting css images (not really back end, but you get the picture)
Point 3, no gzip
If they are using a custom stream to read content from the server and then spit it back up onto the page, there is a huge problem with large javascript, css or html fragments transferred via this method: they can't be zipped server side unless they plan on writing unzipping functionality into the client side javascript, which I imagine would be ridiculously slow.
Point 4, semantics, search engine indexing
This ties in with one of the reasons that Point 1 is so important, web pages without css or javascript are the way that search engines interpret the internet. Adding methods like this to a webpage make it impossible for search engines to apply meaning or understand what your content is about.
Anyways, this is just my two cents, I can see how MAYBE in extremely high traffic scenarios such as digg has on their hands, this tech would be useful, but as a whole I think this is a step backwards in the web development world.
Not to rag on you digg developers! I can appreciate the work you do, and the effort put into this, and I'll be learning how to use your libraries anyways just because I do think they are cool. I just can't speak favorably about how practical I think they are.
~Mike