I wasn’t at Yahoo! when Paul Graham was. He was there a long time ago. I can’t speak to whether his blog post accurately reflects what Yahoo! was like then. I can tell you that it doesn’t mesh at all with the experiences I’ve had at Yahoo! since I joined the company in early 2007.

The main point of Graham’s article is that Yahoo! didn’t have a hacker-centric culture. If there was a time when that was true, it must have been before I joined.

A company without a hacker-centric culture doesn’t encourage the kind of risk-taking and experimentation I saw when I was at Yahoo! Search. As an engineer, I had direct input into product features at every level, from ideation to design to implementation to launch. If I had a crazy idea, I was encouraged not just to tell people about it (up to and including executives), but to implement it and see if it tested well with users. I was able to add my own personal touch to parts of the product (sometimes big parts) without needing to ask permission or wade through excessive red tape.

This may not sound impressive to someone who’s used to the way things work at startups or small companies. But this was at one of the largest Internet companies in the world, on one of the most visited websites in the world. For Yahoo! to give me and other engineers the kind of freedom and power we had is not normal for a company or a product that operates at this scale.

Earlier this year, I transferred to the YUI team, where I get to work with some of the smartest frontend engineers on the planet on an open source JavaScript library that we develop not just for use by Yahoo!, but also by thousands of other developers around the world. The ideas and the work that I see coming from this team, and from the other teams we work with throughout Yahoo!, are amazing and often groundbreaking.

I have my gripes about Yahoo!, sure. It hasn’t been all kittens and rainbows. But the hacker-centric culture and the brilliant people Paul Graham seems to think don’t exist here are the reason I’ve been here for 3.5 years and counting, and they’re the reason I don’t plan on leaving anytime soon.

I originally wrote this as an answer to a question on Quora. I thought it was worth reposting here. My opinions, as always, are my own, and don’t necessarily reflect the views of my employer.

A little over three years ago, I opened a purple box containing a job offer and some boring forms to sign. It yodeled at me when I opened it. That’s when I knew I’d made the right choice in deciding to join Yahoo!.

Working on Yahoo! Search has been an incredible experience. I can now say that I’ve written code that’s used by hundreds of millions of people around the world. I can also say that I’ve broken code that’s used by hundreds of millions of people around the world (fortunately that only happened once). During my time there I did a little bit of everything, from writing build tools and pushing pixels around for bucket tests to leading frontend development on the September 2009 Search redesign—the biggest in the history of Yahoo! Search—which the team managed to pull off in only a matter of months.

At its best, working on Search was exciting and fulfilling in a way no other job has been for me; at its worst, it was stressful and relentlessly demanding. But whether it was exciting or stressful, fulfilling or demanding, I always learned something new every day, and I got to work with some of the smartest people in the business. As far as I’m concerned, that’s what makes a thing worth doing.

It’s in this spirit that I’m launching myself on a new journey. Not an altogether different one—I’ll still be at Yahoo!—but certainly one that I expect will be both incredibly challenging and a lot of fun. As a member of the YUI team, I’ll get to work on an awesome project that I love, with awesome people from whom I will doubtless learn truly epic amounts of stuff on a daily basis.

A little over three years ago, if someone had told me I’d be this lucky, I wouldn’t have believed them.

This post was originally published on Techyard, an internal Yahoo! site. I’m republishing it here (lightly edited) since it’s relevant to this blog’s audience, and since it explains what I’ve been working on for the last few months that’s been keeping me from blogging.

One of the primary goals behind the design of Yahoo! Search Pad was to create a simple structured document editor that was fast enough and intuitive enough that it would be a viable replacement for the desktop apps (primarily text editors and, to a limited extent, Microsoft Word) that many people use to keep notes while doing research online.

To that end, it was important that Search Pad be as responsive as a desktop app while coexisting seamlessly with the web-based Search experience. It was also a strict design requirement that the user not have to install any browser addons or external apps in order to use Search Pad; it had to work perfectly and effortlessly in any A-grade browser.

Since a Search Pad document is really just a collection of notes, our original approach was to store the document in server-based session storage while it was being edited, with changes (new notes, deleted notes, updated notes, etc.) persisted to the server instantly via Ajax. Unfortunately, there were several major issues with this Ajax-heavy approach:

1. Note changes occur very frequently in typical Search Pad usage, which meant we were firing off a lot of Ajax requests. This put a substantial load on our frontend servers.
2. The frequent Ajax requests made the app feel very…well…Web-ish. It wasn’t slow, but it certainly wasn’t as responsive as a desktop app.
3. Users had a tendency to work with their Search Pad documents in several tabs or browser windows at once, which created consistency issues since the document state was stored on the server and the multiple tabs/windows couldn’t be reliably kept in sync without significant complexity.
4. Due to the stateless nature of HTTP and the fact that two nearly simultaneous Ajax requests might be dispatched to two completely different frontend servers and, indeed, might even be handled out of order, we were faced with the possibility of having to avoid asynchronous requests entirely in order to prevent race conditions and data loss

It quickly became apparent that these technical issues and the user frustration that resulted were simply not going to be acceptable, so we began working on an alternative: using client-side browser storage to store the active document entirely on the user’s machine during editing. This would eliminate the need for frequent Ajax requests and drastically simplify the server-side architecture while also making the app feel more responsive.

There was only one problem: not all A-grade browsers support HTML5 Storage yet. And of those that do support it, not all of them support all of it; since it’s not yet a finalized specification, there are implementation differences. Many web apps make use of Flash storage, Yahoo! BrowserPlus, or Google Gears to smooth out these differences, but since Search Pad could not require any browser plugins, those were all off the table.

Fortunately, while not all A-grade browsers support HTML5 Storage, all of them except Opera do at least support some form of usable client-side storage better than cookies. Since Opera accounts for only a tiny fraction of Yahoo! Search pageviews, we made the decision to go ahead and create a simple JavaScript storage abstraction library that would allow us to support all the major browsers including IE6 and 7. We’re hoping Opera will add support for HTML5 storage soon.

Here are the different storage layers our library uses in order to provide HTML5-like key/value storage on all supported browsers:

• Firefox 3.5, Safari 4, IE8: HTML5 localStorage; these modern browsers all support the core localStorage functionality defined in the HTML5 draft.
• Firefox 2.x and 3.0: Gecko globalStorage, a very early implementation similar to HTML5’s localStorage.
• Safari 3.1 & 3.2: HTML5 Database Storage, because Safari 3.1 and 3.2 don’t support HTML5 localStorage.
• IE6, IE7: userData persistence, a rarely used IE feature for associating string data with an element on a web page and persisting it between pageviews.
• Google Chrome: Gears Database API, which is built into Chrome and thus doesn’t require a separate install. Surprisingly, Chrome doesn’t yet natively support any form of HTML5 Storage.

We’ve found that using client-side storage rather than making frequent Ajax requests makes Search Pad feel incredibly responsive, even on ancient browsers like IE6, while also decreasing the load on our frontend servers. In addition, being able to persist relatively complex data on the client between pageviews has opened up exciting new possibilities.

The YUI team and Mint.com’s Matt Snider are currently working on a client-side storage utility similar to (but more powerful than) the one we created for Search Pad. It’s slated for inclusion in YUI 2.8.0, which is currently scheduled for a Fall ’09 release.

Unbeknownst to me, Bill Scott captured this brief screencast of a really old Search Assist bucket test I implemented, complete with ugly Yahoo!-only internal messaging. I had forgotten how rough the design was at that early stage. It’s so much more refined now.

How many things can you spot in the video that are different from the current iteration of Yahoo! Search?

Everyone and their dog seems to have written an example of how to use the Yahoo! Search BOSS API to build a simple search tool. I wanted to take that one step further and build something that would serve both as an example and as a usable service, and that could be extended and enhanced by other developers. Since my day job involves constant tradeoffs between making Yahoo! Search slower (by adding features) and making it faster (by optimizing those features), my primary goal here was to make something as fast as technically possible.

To do this, I wrote a very simple JavaScript module called Trogdor that uses dynamic script nodes to make cross-domain JSONP requests to the BOSS API as you type your query. Search results are returned and rendered almost instantly on each keystroke, and you can use the up and down arrow keys (or tab) and enter to quickly select the result you want—no mouse necessary.

Trogdor doesn’t require a JavaScript framework and works great in all modern browsers (and even some ancient, crappy browsers like IE6). The entire package (HTML, CSS and JS) weighs just a smidge under 2KB after minification and gzip, and it’s wonderfully fast.

Try it out for yourself at pieisgood.org/search and be sure to grab the heavily-commented source code on GitHub. If you’ve got ideas for features and improvements, fork the repo and go nuts (and be sure to let me know what you come up with). You’re also welcome to use Trogdor (modified or unmodified) in your own projects, although I do ask that you please use your own BOSS API key rather than the one included in the example.

Update, 11/26: Changed the name of the library from FastSearch to Trogdor, since dragons are awesome (and apparently there’s a Microsoft search product called FastSearch).

I’ve written an article for the YUI Blog describing some of the techniques I used to optimize Yahoo! Search for the iPhone. You can use these techniques to improve the performance of any JavaScript-heavy iPhone web app, even if you don’t use YUI. Check it out!

In June, my boss came to me with a challenge: bring the full Yahoo! Search experience—including SearchMonkey, Search Assist, shortcuts, and other awesome Yahoo! Search features—to the iPhone with as few compromises as possible. He wanted an iPhone search experience that matched the desktop experience and took full advantage of Mobile Safari’s excellent featureset. And he wanted it in a month.

I grabbed Tom Chi and Jeremy Hubert from Search UED and we hashed out a plan over lunch. We used Jeremy’s existing iPhone Search prototype as inspiration. Tom had to fly to Paris to judge the Imagine Cup 2008 interface design competition, but in his spare time he churned out interface mockups and perfected the design while watching “young ladies in debutante dresses drinking and getting on a boat” on the River Seine. Meanwhile, I wrote the code.

Two weeks later, we quietly launched Yahoo! Search for iPhone. Here are just a few of the awesome features you can now enjoy on your iPhone:

• Search Assist saves you time by completing your queries before you’ve finished typing them
• All your favorite SearchMonkey modules will follow you from your desktop browser to your iPhone (just make sure you’re logged into your Yahoo! account on your iPhone)
• Movie showtimes, weather, local results, breaking news, Flickr photos, and other useful Yahoo! Search shortcuts are now at your fingertips
• Other helpful features like Quick Links, same-host indent, and more

Try it out and let us know what you think. And if you don’t have an iPhone handy, head over to Flickr to see some sexy screenshots of the iPhone SRP.

Next Thursday, May 15th, Yahoo! Search will host a developer launch party for SearchMonkey, our awesome new open developer platform. The festivities will take place from 5:30 to 8:30pm at Yahoo! Headquarters in Sunnyvale, California.

If you’re a developer and you’d like to learn more about SearchMonkey, meet the folks behind it (including yours truly), see live demos, eat free food, and drink free beer, register for the event at upcoming.org and send an email containing your full name and the name of your company (if any) to searchmonkeyevent@yahoo-inc.com.

Space is limited and the friendly security guards at the front gate won’t let you in if you’re not on the list, so be sure to RSVP soon.

If you’re an awesome web developer or QA engineer and want to work on awesome things with me and lots of other awesome people at Yahoo! Search, send me your résumé.

You know how, when something really incredible happens to you that you had always sort of dreamed about but never thought would actually happen, there’s this adjustment period where it just doesn’t feel real? And you feel pretty much the same as you always did, but then occasionally the full realization of what’s happened hits you, just for a minute or two, and leaves you kind of stunned? That’s how the last few weeks have been for me.

I’ve accepted a job at Yahoo! as a Senior Web Developer on the Yahoo! Search team. In the coming weeks, Felicity and I will be leaving the drizzly Pacific Northwest and moving down to sunny Santa Clara, California. We’re both pretty excited.

I remember a conversation Felicity and I had a few years ago when I was feeling depressed about my job at the time, which was pretty boring and unfulfilling. She asked me what company I would want to work for if I could work for any company in the world, and I told her I’d want to work for Yahoo! because it seemed like a really fun place to work.

When I tore open the purple box containing my official offer letter and it actually yodeled at me, I knew I had made the right choice.