This post was originally published on Techyard, an internal Yahoo! site. I’m republishing it here (lightly edited) since it’s relevant to this blog’s audience, and since it explains what I’ve been working on for the last few months that’s been keeping me from blogging.
One of the primary goals behind the design of Yahoo! Search Pad was to create a simple structured document editor that was fast enough and intuitive enough that it would be a viable replacement for the desktop apps (primarily text editors and, to a limited extent, Microsoft Word) that many people use to keep notes while doing research online.
To that end, it was important that Search Pad be as responsive as a desktop app while coexisting seamlessly with the web-based Search experience. It was also a strict design requirement that the user not have to install any browser addons or external apps in order to use Search Pad; it had to work perfectly and effortlessly in any A-grade browser.
Since a Search Pad document is really just a collection of notes, our original approach was to store the document in server-based session storage while it was being edited, with changes (new notes, deleted notes, updated notes, etc.) persisted to the server instantly via Ajax. Unfortunately, there were several major issues with this Ajax-heavy approach:
- Note changes occur very frequently in typical Search Pad usage, which meant we were firing off a lot of Ajax requests. This put a substantial load on our frontend servers.
- The frequent Ajax requests made the app feel very…well…Web-ish. It wasn’t slow, but it certainly wasn’t as responsive as a desktop app.
- Users had a tendency to work with their Search Pad documents in several tabs or browser windows at once, which created consistency issues since the document state was stored on the server and the multiple tabs/windows couldn’t be reliably kept in sync without significant complexity.
- Due to the stateless nature of HTTP and the fact that two nearly simultaneous Ajax requests might be dispatched to two completely different frontend servers and, indeed, might even be handled out of order, we were faced with the possibility of having to avoid asynchronous requests entirely in order to prevent race conditions and data loss
It quickly became apparent that these technical issues and the user frustration that resulted were simply not going to be acceptable, so we began working on an alternative: using client-side browser storage to store the active document entirely on the user’s machine during editing. This would eliminate the need for frequent Ajax requests and drastically simplify the server-side architecture while also making the app feel more responsive.
There was only one problem: not all A-grade browsers support HTML5 Storage yet. And of those that do support it, not all of them support all of it; since it’s not yet a finalized specification, there are implementation differences. Many web apps make use of Flash storage, Yahoo! BrowserPlus, or Google Gears to smooth out these differences, but since Search Pad could not require any browser plugins, those were all off the table.
Fortunately, while not all A-grade browsers support HTML5 Storage, all of them except Opera do at least support some form of usable client-side storage better than cookies. Since Opera accounts for only a tiny fraction of Yahoo! Search pageviews, we made the decision to go ahead and create a simple JavaScript storage abstraction library that would allow us to support all the major browsers including IE6 and 7. We’re hoping Opera will add support for HTML5 storage soon.
Here are the different storage layers our library uses in order to provide HTML5-like key/value storage on all supported browsers:
- Firefox 3.5, Safari 4, IE8: HTML5 localStorage; these modern browsers all support the core localStorage functionality defined in the HTML5 draft.
- Firefox 2.x and 3.0: Gecko globalStorage, a very early implementation similar to HTML5’s localStorage.
- Safari 3.1 & 3.2: HTML5 Database Storage, because Safari 3.1 and 3.2 don’t support HTML5 localStorage.
- IE6, IE7: userData persistence, a rarely used IE feature for associating string data with an element on a web page and persisting it between pageviews.
- Google Chrome: Gears Database API, which is built into Chrome and thus doesn’t require a separate install. Surprisingly, Chrome doesn’t yet natively support any form of HTML5 Storage.
We’ve found that using client-side storage rather than making frequent Ajax requests makes Search Pad feel incredibly responsive, even on ancient browsers like IE6, while also decreasing the load on our frontend servers. In addition, being able to persist relatively complex data on the client between pageviews has opened up exciting new possibilities.
The YUI team and Mint.com’s Matt Snider are currently working on a client-side storage utility similar to (but more powerful than) the one we created for Search Pad. It’s slated for inclusion in YUI 2.8.0, which is currently scheduled for a Fall ’09 release.