wonko.com

The original Inquisitor for Safari is one of my favorite browser plugins ever. Since switching from Safari to Firefox 3, I’ve missed it terribly. Now, thanks to Yahoo! and Inquisitor creator David Watanabe, Firefox and IE users can experience the joys of Inquisitor as well. Hooray!

About a year and a half ago, Felicity and I left the Silicon Forest and moved to the Silicon Valley so I could take a job at Yahoo!. In a little over a month, we’ll trade Valley for Forest and return once more to drizzly Portland.

It’s strange: until Portland, I had never lived in one place long enough to think of it as home. I grew up in a military family, moving every few years, from Texas to Illinois to New Mexico to Japan to Texas again and, finally, to Oregon. When I decided to move to California I knew I would miss Portland, but I thought I’d get over it. I was wrong. Once you’ve lived in Portland you belong to it; you can leave, but only physically.

Silicon Valley, by contrast, is not the kind of place I can imagine ever calling home. People don’t so much live in the Valley as exist in it. Apart from the nearby oasis of San Francisco, there’s nothing particularly special about this place. Half a century ago, fate decided this would be the site of a world-changing technological revolution, and so a gravity well was formed that attracted innovation, which attracted money, which attracted more innovation, and so on. The result is a place people go to not because they want to be there, but because they need to be there.

Thankfully, I don’t need to be here anymore. I’m lucky to work for a company that actually cares about my happiness and is progressive enough that they’re willing to let me work remotely if that’s what will make me happy. I can’t even begin to tell you how awesome that is.

Portland, here we come!

I’ve just released version 0.2.1 of Thoth, the simple Ruby blog engine that powers wonko.com. Notable changes in this release include:

• Requires Sequel 2.5.x (this fixes an issue with SQLite and DateTime objects).
• Requires RedCloth 4.0.x. Markdown syntax is no longer supported as a result.
• Names of existing tags are now suggested as you type in the “Tags” field when creating or editing a blog post.
• Blog post URL names can now be specified just like pages, and names are automatically validated and checked for uniqueness via Ajax. If you don’t manually enter a name, one will be created automatically based on the title.
• Comments are now deleted via Ajax when JavaScript is enabled, making it easier to delete multiple comments without having to reload the page.
• A rel="nofollow" attribute is now automatically added to links in user comments.
• When minification is enabled, filenames in the public /css and /js directories that have names ending in -min.css or -min.js will be served as is since they are presumably already minified.
• Fixed a bug that prevented client-side caching of CSS and JS when server-side caching and minification were enabled.
• Fixed internal server error when viewing a tag that isn’t attached to any posts.

To install Thoth for the first time, run:

sudo gem install thoth

Or, to upgrade an existing Thoth blog:

sudo gem update thoth
cd /path/to/blog
thoth -d stop
thoth --migrate
thoth -d start

Note the database migration step, which is necessary if you’re upgrading from a version of Thoth older than 0.2.0. Your existing data will be preserved, but you may want to make a backup first if you’re paranoid.

Now that I’ve been using my Dash Express on a daily basis for several months, it’s time to sum up my thoughts on the device and the service. In a sentence: The Dash Express is good—and getting better with each software update—but not great.

Is it worth $300 plus $10 a month? Maybe. It depends heavily on where you live and what kind of driving you do. I commute about 34 miles a day through San Jose, California on Highway 101, so the Dash’s live traffic info is both very accurate (due to the high density of Dash users in the area) and very useful. On the other hand, if my commute were shorter, or if I lived in an area where traffic was less of a problem, the Dash would probably be a waste of money.

It’s possible to use the Dash as a plain old GPS device without the live traffic info or Yahoo! Local and avoid paying the monthly fee, but without the always-on connectivity the Dash is nothing special. Its routing functionality, mapping capabilities, and turn-by-turn directions are respectable, but as you would expect from a first-generation device, the Dash isn’t as refined as competing devices from companies like Garmin and TomTom, who’ve had years to perfect their products.

Case in point: the Dash’s directions, while usually accurate, occasionally fail in inexplicable ways. On the final leg of my morning commute, there’s about a 50% chance that the Dash will instruct me to keep left on the Great America exit ramp when I really need to keep right. Some days it knows I should keep right, some days it doesn’t; there doesn’t seem to be any pattern.

Routing oddities aside, the live traffic information and the ability to choose from multiple routes (and route around unexpected traffic) are the Dash’s killer features. For me at least, they make up for its shortcomings. I’ve found the traffic info and drive-time estimates to be absolutely spot on. When Google Maps or Yahoo! Maps tell me my commute will take 35 minutes and the Dash says 23, I always trust the Dash. So far it hasn’t let me down.

The ability to search Yahoo! Local from the device is nice, but sometimes backfires. A month or two ago I ran over a bolt and needed to find a tire shop. The first three tire shops Y! Local directed me to didn’t actually exist (one was a house). This is more Yahoo!‘s fault than the Dash’s, but it was frustrating all the same. In the end, I used my iPhone to find a shop and then entered the address into the Dash.

Third-party hacks and POI feeds are touted as major selling points of the Dash. While there are certainly some cool hacks (like Twitter and Fire Eagle integration), they don’t mesh well at all with the Dash UI. This makes them feel flimsy and bolted on. There’s a lot of room for improvement here, but the good news is that the Dash’s over-the-air software updates make it very likely that, if we’re patient enough, we actually will see improvements.

It’s important to note that while many device makers claim they’ll offer fancy new features and bug fixes in software updates but fail to follow through, the Dash software has already received several minor updates and one major one, and many of my early gripes have been fixed.

This is really the key: if these software updates continue to be as frequent as they have been so far, especially if they keep fixing common complaints and adding useful features, then the Dash could easily become one of the best GPS devices on the market. At the moment, though, it’s still a 1.1 device with room for improvement.

On Friday, Stephen Colbert asked the Internets to help make John McCain seem younger and more vital. In my opinion there’s nothing younger or more vital than Brunslo dancing like a drunken fool. Here’s my contribution to Colbert’s green screen challenge.

I’ve written an article for the YUI Blog describing some of the techniques I used to optimize Yahoo! Search for the iPhone. You can use these techniques to improve the performance of any JavaScript-heavy iPhone web app, even if you don’t use YUI. Check it out!

In June, my boss came to me with a challenge: bring the full Yahoo! Search experience—including SearchMonkey, Search Assist, shortcuts, and other awesome Yahoo! Search features—to the iPhone with as few compromises as possible. He wanted an iPhone search experience that matched the desktop experience and took full advantage of Mobile Safari’s excellent featureset. And he wanted it in a month.

I grabbed Tom Chi and Jeremy Hubert from Search UED and we hashed out a plan over lunch. We used Jeremy’s existing iPhone Search prototype as inspiration. Tom had to fly to Paris to judge the Imagine Cup 2008 interface design competition, but in his spare time he churned out interface mockups and perfected the design while watching “young ladies in debutante dresses drinking and getting on a boat” on the River Seine. Meanwhile, I wrote the code.

Two weeks later, we quietly launched Yahoo! Search for iPhone. Here are just a few of the awesome features you can now enjoy on your iPhone:

• Search Assist saves you time by completing your queries before you’ve finished typing them
• All your favorite SearchMonkey modules will follow you from your desktop browser to your iPhone (just make sure you’re logged into your Yahoo! account on your iPhone)
• Movie showtimes, weather, local results, breaking news, Flickr photos, and other useful Yahoo! Search shortcuts are now at your fingertips
• Other helpful features like Quick Links, same-host indent, and more

Try it out and let us know what you think. And if you don’t have an iPhone handy, head over to Flickr to see some sexy screenshots of the iPhone SRP.

Dear startups: I know you think your new company is important enough that everyone wants to hear about it all the time, and I know the marketing folks tell you that email newsletters are a great way to keep new customers from forgetting you exist after they sign up, but you’re wrong and they’re lying.

When I sign up to try out your new web app or download your software, giving you my email address is an act of trust. Every signup form these days goes out of its way to make it clear that email addresses won’t be shared with or sold to unsavory characters; everyone knows that’s evil. But when I give you my email address, I’m also trusting that you won’t send me email I don’t want to receive.

There’s nothing inherently evil about email newsletters, but if you don’t allow me, during the signup process, to opt out of receiving your newsletter, you’ve betrayed my trust.

It doesn’t matter if the newsletter contains an easy one-click unsubscribe link; by assuming that I want to receive it and by not explicitly offering me an opportunity to opt out, you’ve guaranteed that I will instantly regret having signed up when I see your newsletter in my inbox. You’ve also guaranteed that I’ll mark your newsletter as spam in Gmail.

Browser cookies have a very specific purpose: they are meant to store tiny bits of data and to pass that data to the web server on every request. Because they must be uploaded to the server along with every single request the browser makes, browsers limit cookies to a maximum size of 4 kilobytes per domain.

Given this storage limitation and the fact that even a 4KB cookie is absolutely guaranteed to increase client/server latency, you’d think web developers would avoid packing cookies full of data, right? Sadly, web developers are a silly bunch and are prone to sheeplike behavior.

Standard practice up until about 2007 was to store session data on the server, typically in a file, a database, or a distributed memory cache. Cookies were used only to store the session keys—short strings that uniquely identified a user and allowed the server to retrieve session data from the session store as necessary. This had little effect on client/server latency since the cookies were small, and it was reasonably secure since users couldn’t tamper with the actual contents of their session data; the best a malicious user could hope to do was to sniff out another user’s session id and impersonate them, which could be easily prevented using SSL.

Then, in 2007, Ruby on Rails added support for using cookies themselves as session stores. Rails wasn’t the first web framework to try to shove a bunch of crap into cookies, but it was possibly the most popular one to do so. Naturally this feature was enabled by default, which meant that web developers around the world instantly began to abuse the hell out of it.

The justification for the feature—and for all the added complexity involved in verifying that users haven’t tampered with the data inside their cookies—was that “cookie-based sessions are dramatically faster than the alternatives”¹. This is undoubtedly true if you invite your users to come to your data center and plug their computers into the local network before using your website. But if, as I suspect is often the case, your users are using your website via DSL, Cable, or—God forbid—dialup, this is almost certainly a naughty naughty lie.

Performance problems

According to research carried out by the Yahoo! Exceptional Performance team, a 1000-byte cookie adds 16ms to the response time of a single request made over a broadband DSL connection². Now imagine a typical web page that includes one external CSS file, two external JavaScript files, and five images, all hosted on the same domain. That’s a total of nine requests, which means that little 1000-byte cookie has added 144ms to the response time of the page (and that’s for broadband users, a best-case scenario).

“Pish posh!” you say. “144ms is nothing! The snap of a finger! The blink of an eye!” Nevertheless, Amazon found that a 100ms increase in response time resulted in a 1% decrease in sales, and Google found that a 500ms increase resulted in a 20% decrease in traffic³. In my day job at Yahoo! Search I’ve seen these findings verified again and again. The tiniest change in response time can have significant effects on abandonment, user satisfaction, and overall traffic.

Security problems

When you store session data in cookies, you must be absolutely certain that users can’t tamper with the data in any way. There’s no way to keep users from altering the data in a cookie; it’s ridiculously easy. So, in order to ensure that your website doesn’t accept cookies containing altered data, you need to either encrypt the cookie values or sign them with a hash that allows you to verify their integrity. This introduces more complexity into your application and increases the number of possible attack vectors that malicious users can exploit.

Most web frameworks that support cookie-based session stores try to hide this complexity from the developer by performing the cookie validation themselves. However, in order to encrypt or sign cookies securely, you need a secret key (like a password) known only to the server; otherwise, an attacker who knows the hash inputs could easily reproduce the hash and forge a valid cookie signature.

Obviously the web framework can’t choose this secret key for you, so it’s up to the developer to do this. Many web apps that use cookie-based sessions come with pre-configured secret keys. As you might have guessed, this results in epic failure of the worst kind since web developers are lazy and often don’t bother changing the default key. Some folks on the Rails Core mailing list recently discovered this and are currently flipping out about it as if it were some kind of surprise or something.

Do the right thing

Server-based session stores are the industry standard for a very good reason: they work well and they’re relatively foolproof compared to the alternatives. They do put some extra load on the server, and sharing session data among multiple frontend servers can be a tricky problem (although there are plenty of solutions), but it’s almost always better than resorting to a cookie-based session store.

As with most techniques—even techniques with as many downsides as cookie-based session storage—there are some rare cases when it actually makes more sense than the alternative. But these cases are very rare, and cookie-based session storage certainly should not be the default in any web framework.

¹ Rails Trac – Changeset 6184

² YUI Blog – When the Cookie Crumbles

³ Make Data Useful – presentation by Greg Linden at Stanford

JavaScript’s var statement declares and optionally initializes one or more variables in the scope of the current function (or as global variables when used outside a function). Since var accepts multiple declarations, separated by commas, there’s usually no reason to use it more than once per function; it’s just a waste of bytes and—especially if you redeclare variables inside a loop—CPU cycles.

Overuse of var statements is one of the most common problems I see in JavaScript code. I was guilty of it myself for quite a while and it took me a long time to break the habit.

Bad:

function getElementsByClassName(className, tagName, root) {
  var elements = [];
  var root     = root || document;
  var tagName  = tagName || '*';
  var haystack = root.getElementsByTagName(tagName);
  var regex    = new RegExp('(?:^|\\s+)' + className + '(?:\\s+|$)');

  for (var i = 0, length = haystack.length; i < length; ++i) {
    var el = haystack[i];

    if (el.className && regex.test(el.className)) {
      elements.push(el);
    }
  }

  return elements;
}

There are several things wrong with the example above.

The most obvious problem is that I’ve used the var statement no less than seven times. Somewhat less obvious, but far worse: I’ve used it inside a loop, which means that I’m unnecessarily redeclaring a variable on each iteration. I’ve also unnecessarily redeclared two variables that were passed in as function arguments.

Naturally, there’s a much better way to do this.

Good:

function getElementsByClassName(className, tagName, root) {
  root    = root || document;
  tagName = tagName || '*';

  var elements = [],
      haystack = root.getElementsByTagName(tagName),
      length   = haystack.length,
      regex    = new RegExp('(?:^|\\s+)' + className + '(?:\\s+|$)'),
      el, i;

  for (i = 0; i < length; ++i) {
    el = haystack[i];

    if (el.className && regex.test(el.className)) {
      elements.push(el);
    }
  }

  return elements;
}

There are circumstances in which it is actually necessary to redeclare a variable within a single scope, but they’re very rare, and are more often than not a warning sign that you need to rethink the code you’re writing.