On working remotely

2013-02-24

Yesterday, Yahoo! internally announced a new policy: no more remote workers.

[…]

Beginning in June, we’re asking all employees with work-from-home arrangements to work in Yahoo! offices. If this impacts you, your management has already been in touch with next steps. And, for the rest of us who occasionally have to stay home for the cable guy, please use your best judgment in the spirit of collaboration. Being a Yahoo isn’t just about your day-to-day job, it is about the interactions and experiences that are only possible in our offices.

[…]

— Leaked Yahoo! email published by AllThingsD

This new policy is shockingly shortsighted, and is a significant step backwards for Yahoo!.

I worked remotely for Yahoo! for over three years, and I currently work remotely for SmugMug. During my time at Yahoo!, many of my most brilliant, passionate, and productive coworkers also worked remotely. Some of them are still there, and they’ll be affected by this policy.

The shocking thing about this policy is not that Yahoo!’s management thinks employees will be more productive, communicative, or creative if they work together in an office. In many cases, this is true. What’s shocking is that they seem to think this will be true in all cases.

It won’t, and I’d like to explain why.

Some people can’t work remotely

First, let’s acknowledge the obvious: not all jobs can be done remotely, and not all people are good remote workers.

My experience working remotely has been as an engineer. At times my job has involved close collaboration with many people — sometimes as a team lead — and at other times my role has been as a self-directed individual contributor. The latter is much easier to do remotely, while the former required a great deal of attention and care both on my part and on the part of my coworkers.

In general, I would not recommend trying to lead a team or manage a highly collaborative project as a remote worker. It’s possible, but it’s hard, not just for you but for the team.

Working remotely as an individual contributor has its own set of challenges, and still requires significant discipline, self direction, and communication.

I’ve known brilliant engineers who were terrible and unreliable remote workers because they were undisciplined or didn’t communicate effectively when not in an office setting. It’s virtually impossible to know whether someone will be a good remote worker until you try them.

Some people, when left to their own devices, lack the discipline to stay on task. Absent supervision and the physical presence of coworkers, they relax — consciously or unconsciously — and get less done.

They may justify it to themselves (“I got a lot done yesterday! I deserve a day off.”), or they may encounter an obstacle and use their remoteness as an excuse to procrastinate (“Crap, Sara’s the only one who knows how this code works and she’s not online right now…”), or they may simply be lazy.

I suspect this is one of the reasons behind Yahoo!’s policy shift. It’s notoriously difficult to fire someone at Yahoo!, and for that matter at many large companies. If you hire a remote worker and it later turns out they’re not cutting it, the best you can hope for is to require them to work from the office or try to get them transferred somewhere else in the company so they’ll be someone else’s problem.

Remote workers are more of a risk than non-remote workers, and you’ve got to be willing to let them go if things don’t work out.

Some people are much more effective remotely

Some people, when left to their own devices, become unstoppable forces of productivity.

I’m one of those people. Dav Glass is one of those people. So are Luke Smith, Eric Ferraiuolo, and others I’ve had the pleasure of working with over the years. The four of us all worked remotely for Yahoo! (Dav and Eric still do), and would occasionally take trips to the office.

We all found it virtually impossible to get anything done during our time on site. The difference in productivity was staggering, to the point where I ended up doing my best to finish as much work as possible before a trip so I wouldn’t fall behind while spending a week at the office dealing with constant interruptions and distractions.

Not everyone is like this. Some people thrive on face to face interactions and random hallway chats, and can handle frequent interruptions. The four of us, and many other remote workers like us, thrive on long stretches of quiet time alone, with limited interruptions.

But, most importantly, we have the discipline to stay on task, get our work done, and communicate frequently and effectively with our coworkers even though they aren’t physically present.

It’s about people, not locations

People are different. Jobs are different. Remote doesn’t work for everyone, but the office doesn’t work for everyone either. Some companies understand this and some don’t.

There was a time when Yahoo! seemed to understand this. They didn’t let just anyone work remotely, but they didn’t prohibit everyone from working remotely either. They understood that on a case by case basis, some people, in some jobs, could be excellent remote workers.

Telling effective and responsible remote workers that they can no longer work remotely — not because they’re bad at it, but because some other people might be bad at it — will only make those people less effective at their jobs.

Or it’ll make them leave and find better jobs at companies that see them as people with unique strengths and weaknesses rather than as homogeneous cogs in a corporate machine.

rawgithub.com

2013-01-29

GitHub has always exposed raw source files in public repos via the raw.github.com domain, but — presumably for security reasons — they serve most files with a text/plain content type. This prevents all browsers from rendering HTML, and prevents some browsers from executing JavaScript and CSS.

I've always thought this was unfortunate. Being able to point directly to raw files in a GitHub repo can be useful for testing and for quick demos. I especially want it when I'm writing jsPerf tests and want to compare the performance of code in different branches.

So I threw together rawgithub.com.

It's essentially a proxy for raw.github.com that serves things with the correct content type based on the file extension. Just replace raw.github.com in any GitHub URL with rawgithub.com and you can render HTML, execute JavaScript, and apply CSS right from your GitHub repos.

Have fun with it, but don't use it for anything important! It's slow and it might break occasionally.

Oh, and the source is on GitHub, naturally.

How to run Continuum (SubSpace) on a Mac

2013-01-12

Continuum (also known by its original name, SubSpace) is a ridiculously fun massively multiplayer online space shooter that I've played off and on since its public beta in 1996. The original game wasn't commercially successful, but in the years since it was abandoned, community servers have sprung up and the game client has been rewritten and released as freeware.

For an online game that's been around for over 16 years, it's withstood the test of time well, and is just as fun as it ever was. I recently had a hankering to play it again. Only problem: Continuum is a Windows game, and I use OS X these days.

In the past I've run Continuum in a VirtualBox VM, but that's a pain and it's not as fast as I'd like. This time I decided to see if I could get it running in Wine, which would make it easier to play, and faster as well.

It turns out a fellow calling himself spiffyguy on the SubSpace forums had already gotten this working!

Using WineSkin, spiffyguy bundled Continuum and the necessary Wine runtime and config into a standalone OS X app. Here's how to get it running.

Download Continuum.app. I've hosted the file on my server so you don't need to dig through the SubSpace forums to find it.
Extract the zip file, copy Continuum.app to your Applications folder, and run it.
If you're running Mountain Lion, you may be prompted to install XQuartz as described here. Install it, then run Continuum.app again.

That's it! You should now have a working Continuum client on your Mac. I've been playing happily for the last few days with no problems. It runs smooth as silk on my mid-2011 MacBook Air.

If this is your first time playing Continuum, this quick start guide should help get you going.

And if you'd like a few pointers on how to rack up kills in Trench Wars, the most popular Continuum game type, you might enjoy this Succinct Guide to Trench Wars Badassery I wrote back in 2004.

Pie Trip Down Under

2012-12-25

A few years ago my dear friend Loren, who you may remember from such films as The Pie Trip, Pie Trip II: Pie Harder, and Pie Trip 2.5, met a wonderful Australian girl named Caitlin. She was so wonderful that Loren moved to Australia and, earlier this year, they were married.

Pie Trip II was an epic cross-country road trip to reach a wedding in Wisconsin. What better way to top it than with an epic international journey to reach a wedding in Australia? This trip had sequel written all over it.

The result is Pie Trip Down Under, starring Kyle Kingsbury, Tyler Schuett, Andy Howe, Felicity Shoulders, Loren Bruns, and my creepifying disembodied voice. I hope you enjoy it.

The Video

Behind the Scenes

Technology has advanced somewhat since 2004, and so have production values. This was both a blessing and a curse, in that it enabled us to shoot so much footage with so many different cameras in so many different time zones that it took me months (admittedly, lazy months, but still months) just to wade through it all and even longer to make something interesting out of it. And yet, in keeping with Pie Trip tradition, we managed not to get any usable footage of anyone actually eating a pie (although I assure you that many, many delicious meat pies were et).

It's amazing to think that on Pie Trip II we had a single camera—a bulky, clunky thing that recorded crappy SD video onto actual magnetic tape, which then had to be captured to a hard drive in real time. Tape, hard drive space, and actual time were serious limiting factors in the making of that video.

On this trip we had more HD cameras than we knew what to do with (everyone literally had one in their pocket), shot hundreds and hundreds of gigabytes of footage, and never gave a single thought to storage space. We live in the future.

The majority of the video was shot on a Canon XA10. This was my first major project with this camera, and in hindsight there are a lot of things I would now do differently. For one, I shot at 24fps, which turned out to be a bad idea because the XA10 has major autofocus lag at that framerate. I now shoot at 60i instead, which produces much better results and still allows me the freedom to adjust the framerate in editing.

Certain scenic shots and many shots in the animal montage were shot on a Canon EOS Rebel T3i, which was also new for this trip. This turned out to be pretty good for static shots, but I was disappointed by the obvious compression artifacts and noise in the T3i's videos at wider focal lengths. In spite of the much better lens flexibility and control over depth of field available on the T3i, footage from the XA10 often ended up looking better simply because it suffered from fewer compression artifacts.

The opening motorcycle scene and the Sky Tower jump POV footage at the end of the video were shot with a GoPro HD Hero. Kyle actually shot hours and hours of motorcycle and ATV footage with the Hero, much of which is gorgeous, but sadly I wasn't able to fit it into the narrative of the video.

And of course, several scenes were shot on a humble iPhone 4S (or rather, on one of several humble iPhone 4Ses), which shoots passable HD footage and has the distinct benefit of being easy to carry and quick to use, especially in airports and other places where larger cameras are less convenient to carry or just harder to get to at a moment's notice. The iPhone was also easy to smuggle into a footie match in Melbourne where cameras were forbidden, although I didn't end up using that footage.

The video was edited in Final Cut Pro X, which I absolutely love and have been using happily since it was released. It was mastered in 1080p (a massive jump in quality compared to the grainy SD of the previous Pie Trip videos). I did quite a bit of audio sweetening and color correction for this project, all of which was painless and intuitive. It's clear why many professional editors are upset over the changes in FCPX, but I apparently fall precisely in the prosumer sweet spot that Apple was aiming for with their rewrite.

YUI from the inside

2012-12-11

Remember when I complained about YUI's opaque governance model and lack of external committers? They've fixed it!

These changes have been a long time coming and most of them were already in the works when I wrote that post, but I want to thank the YUI team for listening to me and to others in the community and making things better.

New contributor model

At YUIConf last month, Dav Glass announced YUI's new contributor model in his keynote. He also announced the first two non-Yahoo! committers in YUI's history: me and fellow SmugMugger Luke Smith.

The contributor model is based heavily on the Meritocratic Governance Model and outlines the distinct roles (users, contributors, committers, and reviewers) within the project, as well as their rights and responsibilities. It also defines a decision-making process based on lazy consensus, which is designed to prevent roadblocks.

But perhaps the most important thing Dav announced—and certainly the most surprising—is that most of the YUI team are committers, not reviewers. This means that even though they're Yahoo! employees paid to work on YUI, they have exactly the same rights and access as external committers, and any significant changes they want to make must be approved by a reviewer. It also means that their votes carry no more weight than the votes of external committers.

This is a bold move designed to ensure that the YUI project is truly in the hands of the community and can't be too heavily influenced by Yahoo!. While there's currently only one reviewer—Dav—I'm told the team hopes to add non-Yahoo! reviewers in the future. This fills me with joy.

More transparency

In addition to posting more development details on YUI's GitHub wiki, the team at Yahoo! has opened up their weekly meetings to external contributors and users via Google+ Hangouts On Air.

Every Thursday at 2pm Pacific Time, YUI users can join the hangout (or just watch it live on YouTube) and discuss project decisions, pull requests, and more with members of the YUI team at Yahoo!. This is the same weekly meeting I attended while I was on the team, and the same discussions take place, only now they're open to the world and recorded for folks who can't catch the live hangout.

The weekly hangout isn't the only forum for discussion, though. Ad hoc hangouts occur frequently and are announced on IRC, Twitter, and the YUI Blog. Pull request activity has increased, and the new contributor mailing list is spinning up.

Eric Ferraiuolo held a great community town hall at YUIConf in which he laid out his vision of how the YUI project can embrace its community and be more open.

Gallery 2.0

YUIConf also served as the backdrop to Dav's introduction of a vastly improved process and tools for contributing and maintaining YUI Gallery modules. Everything I complained about was addressed, and because he's Dav, he also added a significant helping of awesomeness that I hadn't anticipated.

The Future

I'm excited about YUI's future. Yahoo! has shown a willingness to listen to criticism and address the project's problems. The community asked for more transparancy and more control, and Yahoo! gave it to us. Now it's up to us to make YUI what we want it to be.

Like any long-lived project, there are parts of YUI that are new and awesome, and there are parts that are old and crufty. Those old and crufty parts have often been frustrating to YUI's users, because it hasn't always been clear how we can improve them.

In my YUIConf talk, I advocated not using certain parts of YUI and thinking carefully before using others. I'm eager to tackle YUI's "bad parts" and improve them, especially now that I feel like I really do have the power to influence the course of the project despite no longer working for Yahoo!.

As I said during Eric's town hall at YUIConf, "the YUI team" used to mean "the Yahoo! employees who work on YUI". Now, that's no longer the case. There are still Yahoo! employees who work on YUI, but referring to them as the YUI team feels wrong, since that "team" now includes anyone who contributes a pull request or takes part in a hangout or offers a suggestion on the mailing list.

It sounds corny, but we're all the YUI team now, and that's exactly how it should be.

Video: When Not to Use YUI

2012-11-30

The video of my YUIConf 2012 talk, When Not to Use YUI, is now available.

The slides were apparently captured with some kind of crufty VGA capture hardware that made them look muddy and dark, so you may want to follow along with the slide deck if you care about that sort of thing.

Respawn

2012-11-29

I finally got around to rewriting my ancient Ruby blog engine in Node. You're looking at it. Say hello to Lectroid.

It's not that I've stopped liking Ruby or anything like that. But the old blog was a crufty and complex beast built on an old and defunct Ruby web framework and backed by an actual honest to god database (remember when blogs used databases?) and bogged down by features like comments (remember when blogs had comments?).

I dreaded writing blog posts because it meant typing them into that old complex beast I had created and no longer had any desire to maintain, and that meant seeing all the things about it that I hated, and that meant wanting to change them, and that led to depression because, seriously, who the fuck wants to maintain a goddamn blog engine? I sure don't.

But I want even less to use someone else's goddamn blog engine, so here we are.

The only thing special about Lectroid is that I don't yet hate it, and I've tried to build it in such a way that I'll hopefully be able to completely ignore it for years and years without it breaking.

It was also designed to allow me to host my blog on Heroku, which I'm now doing, as part of a gradual effort to stop maintaining my own server. We'll see how that goes.

Oh, and naturally I've taken the opportunity to redesign everything. There are these things called smartphones now, and also tablets, and these things have tiny little screens. This blog looks less shitty on them now.

Also I've increased the font size to, like, a million, because I'm getting old and my eyes don't work anymore.

When Not to Use YUI

2012-11-02

YUIConf 2012 is coming up in Santa Clara on November 14th and 15th. I usually avoid public speaking like the plague, but this time there’s something I really want to talk about: When Not to Use YUI.

I’ll provide code samples and real-world anecdotes to illustrate how to decide when to use YUI, when to use vanilla JavaScript, when to consider other libraries, and what the tradeoffs are in terms of performance and maintainability. Advice will range from simple rules of thumb to more nuanced discussion of complex architectural decisions, with examples drawn from my time working on YUI at Yahoo! and using YUI at SmugMug.

Come see the fireworks in the main room at 9AM on Thursday, November 15th!

Update, 2012-11-15: Here are the slides from my talk. This was a talk in which the slides served mainly as background to me saying stuff, so keep an eye out for the video as well, which I’ll post as soon as it’s available.

Update, 2012-11-30: The video of my talk is now available. The slides in the video were apparently captured with some kind of crufty VGA capture hardware that made them look muddy and dark, so you may want to follow along with the slide deck if you care about that sort of thing.

Why I believe in YUI

2012-09-11

Some people mistook my blog post last Friday as a condemnation of YUI, or of the YUI team. I’ve heard that the team is feeling some heat as a result, and this is unfair. I want to set the record straight, and I also want to apologize for my part in the negative tone this discussion has taken in some quarters, since that’s not helping anyone.

Credit where credit is due

First, I want to make it clear that the YUI team is fully aware of every problem I mentioned. Nothing I wrote in that post was news to the team, and they’ve been working on some of those problems for a while now. If the YUI team were to fix every single one of those problems tomorrow, it would be because they’ve been busting their asses for months, in some cases years, to solve them. It would not be because they read about them on my blog.

If I thought the YUI team were unaware, unwilling, or incapable of fixing these problems, I would fork the project and be done with it. I’m not doing that, because I believe in the team and its leadership, and because I know how resilient they can be in the face of seemingly unsurmountable obstacles. If I thought YUI couldn’t or wouldn’t change, I wouldn’t have wasted my time writing that blog post.

That said, YUI can’t succeed in overcoming these obstacles without the community’s help. And the community can’t begin to help until the YUI project becomes more transparent, less secretive; more of an open source project and less of a Yahoo! product.

The team is not entirely to blame for this. I’ve been in their shoes. I know exactly how easy it is to slip into the mindset of working on a product instead of on a community project, and how hard it is to remember what it means to work on an open source project when you’re being paid to spend eight hours a day in an office (even if, as in my case, it happens to be a virtual office most days).

It’s not the team’s fault that Yahoo! itself tends to see YUI as an internal product meant to serve the needs of internal customers. Nor is it really Yahoo!‘s fault. A publicly traded company can’t be expected to fund projects that don’t serve its bottom line, so YUI must serve Yahoo!‘s bottom line. YUI exists because Yahoo! needed a shared JavaScript and CSS library. It would be a waste of time and money for every Yahoo! product to build its own single-use framework. YUI is only public because the developers working on it, and the engineering managers who’ve led the team, wanted to give something back to the web and convinced the company to allow them this indulgence.

Yahoo! and YUI should be commended for this.

But what Yahoo! and the YUI team need to understand — and this is the source of my recent frustration and the reason for my blog post on Friday — is that, if they’ll let us, there’s a whole lot the web can give back to YUI, and to Yahoo! as well. In order for that to happen, Yahoo! and the YUI team need to change some of their ways of thinking and challenge some of the ways they’ve historically done things.

My blog post on Friday stemmed from my frustration at how it feels to be an outsider wanting to contribute to YUI — frustration that I probably wouldn’t feel so acutely if I hadn’t had the experience of contributing to YUI from the inside. I wanted to be frank, but I also wanted my criticism to be constructive, which is why I suggested solutions instead of just complaining.

It wasn’t my intent to be discouraging, and I think I probably should have waited a while and written that post from a place of less frustration. I stand by what I wrote, but I regret that it was couched in negative emotions. I owe the YUI team an apology for that.

Good news

Since Friday, I’ve heard a few encouraging things from individual members of the YUI team that I’d like to share.

I’m told that the team is planning to be more proactive about managing and responding to pull requests. I’m already seeing increased activity and transparency on pull requests, which is awesome.

YUI hopes to be able to provide commit access to trusted non-Yahoo! committers soon. This was already in the works, unbeknownst to me.

Dav Glass has big plans for Gallery 2.0 — some of which are already in the works — and my complaints certainly weren’t news to him. In fact, the solution I proposed in Friday’s post was directly inspired by much of the work he’s already done on making the rest of YUI more npm-friendly and easier to build and maintain, so my proposal isn’t news to him either.

I haven’t heard much else yet, but I do get the sense that the team generally agrees with the need for more transparency and more community involvement.

I want to stress that these changes (and any other changes you see from the YUI team in the near future) aren’t my doing. All I’ve done, if anything, is provide a little bit of a window into the sausage factory.

On the awesomeness of YUI’s engineering managers

I’d like to take a moment here to kiss some asses, because frankly, these asses deserve to be kissed and don’t get nearly enough credit.

The YUI team currently has, and has always had, fantastic leadership. From Thomas Sha (who founded the team) to Eric Miraglia (who managed the team for over five years) to the team’s current manager Jenny Donnelly (herself a former YUI engineer), YUI has been led by the best engineering managers I’ve ever worked with.

I want to single out Jenny in particular, both because she’s the team’s current manager and because she keeps a pretty low profile and lets the kudos go to the engineers she manages, which prevents her from getting the credit she deserves. If you’ve been to YUIConf in the past few years, Jenny’s the person who made it happen, and she’s the one you saw running around making sure everything went smoothly — often (as luck would have it) while also extremely pregnant.

Jenny, like Eric and Thomas before her, is an engineering manager who gets it. Any engineer can tell you how truly rare and valuable this is.

If there’s one thing I regret about my blog post last week (and, let’s be honest, there are several things), it’s that Jenny will probably take heat for it, and she doesn’t deserve that. Jenny is as passionate about YUI and YUI’s community as anyone, and more so than most. She’s one of the best things YUI — and Yahoo! — has going for it.

I have full confidence that Jenny and the rest of the YUI team will, with the community’s help, overcome the challenges facing the project and that YUI will continue to be one of the best JavaScript libraries in the world. You should too.

On the awesomeness of YUI

Like any project worth caring about, YUI has problems. All software projects have problems. Particularly open source projects. The difference between an open source project with problems and any other project with problems is that an open source project’s problems ideally get discussed and dissected in the open.

Please don’t confuse my attempt to discuss YUI’s problems in the open for an attempt to convince people they shouldn’t use YUI. That’s the last thing I want, especially since I personally use YUI and want to see it thrive.

My personal decision to stop contributing to YUI until I feel my time is better respected doesn’t mean I’m going to stop using YUI, and it doesn’t even mean I’m going to stop working on YUI itself — it just means I don’t plan to contribute that work upstream until the process has been smoothed out.

If you currently use YUI, there’s no reason to stop. If you don’t yet use YUI, you should consider it. It’s a fantastic library, and it’s still my first choice.

YUI from the outside

2012-09-07

Update, 2012-09-10: I’ve posted a followup to this post entitled Why I believe in YUI. Please read it.

When I quit my job at Yahoo! a few months ago, it didn’t feel like I was really quitting. I was leaving the company, sure, but I still planned to continue working on YUI, even though Yahoo! would no longer be paying me to do it. Getting paid to work on an open source project has many benefits, and one of them is that if you love your job, you can keep right on doing it even if you decide to leave the company.

I’m extra lucky that my new employer loves YUI, uses it extensively, and is more than happy to let me contribute the work I do back to YUI whenever it makes sense. Since I left Yahoo! I’ve sent ten pull requests to YUI. Some for minor bug fixes, enhancements, or performance improvements; others for brand new components like LazyModelList, Template.Micro, and node-scroll-info.

This has been one of the most frustrating experiences I’ve ever had contributing to an open source project.

Pull request graveyard

Even though I’m still in touch with the core team and talk to many of them on a daily basis, several of my pull requests sat in limbo alongside over 40 other open pull requests on the YUI 3 project (some dating back nearly a year). Compare this to jQuery, which — despite being more popular than YUI — currently has only 22 open pull requests, none older than two months.

Not all my pull requests were large. One was a minor change that fixed a critical performance bug in Y.Attribute#getAttrs(). Another was a minor change that improved the performance of Y.merge(), a core utility method used throughout the library. In both cases, the pull requests sat unreviewed as release windows passed them by, even though reviewing them would have taken minutes.

No external committers

Why send pull requests? Why not just commit these minor changes myself? Because there are no YUI committers who don’t work for Yahoo!. Despite having been a trusted senior member of the core team for over two years, I lost commit access to YUI the instant I stepped off the Yahoo! campus for the last time.

What many people don’t know about YUI is that the YUI GitHub repository is just a read-only mirror. YUI’s actual source-of-truth repo lives on a server inside the Yahoo! corporate firewall, and can only be accessed by Yahoo! employees. The reasons for this have to do primarily with typical Yahoo! bureaucracy and not with any desire on the part of the YUI team to exclude external developers, but nevertheless, that’s exactly the effect it has. YUI has many external contributors, but the only way to get commit access is to draw a Yahoo! salary.

The YUI team also has no actual set procedure or policy for dealing with pull requests. In theory, when budgeting time and making development estimates for each release cycle, team members are expected to budget time for handling pull requests, responding to community questions, etc. In practice, the team members who actually care about that stuff make time for it, and the ones who don’t care as much don’t make time for it. So some team members respond to pull requests, hang out on IRC, and drive YUI’s community engagement efforts, while others are completely invisible and unreachable to the YUI community.

Unfortunately, some of the team members who don’t care very much about engaging with the community outside Yahoo!‘s walls are also very senior members of the team who “own” many important parts of the library. Because they avoid spending time interacting with the community, they’re often out of touch with what YUI’s users need and care about, and tend to say “no” far more often than they say “yes”. They also tend to either ignore pull requests or excuse themselves from dealing with them by citing how busy they are.

As a result, some of the most important parts of YUI — Y.Node, Y.Event, Y.Base, Y.Attribute, and more — are also the most ignored, and the hardest for external developers to contribute to.

Long story short? Important changes and improvements to YUI are frequently blocked by people who rarely interact with the community and cannot be held to task by the community. These team members mean well, and they do what they do because they genuinely think it’s good for YUI. But it’s often not, and I’ve learned that fighting with them from the outside is far more difficult than fighting with them from the inside was.

The Template.Micro fiasco

A few weeks ago I opened a pull request for a new YUI component called Template.Micro. This is something I created for my own use, but I thought it would be a great addition to YUI. Some fantastic community discussion occurred on that pull request — lots of developers, including members of the core team, had excellent feedback, and we worked together to make Template.Micro awesome. Eric Ferraiuolo and I even chatted about Template.Micro in a public Google Hangouts on Air session and posted the recording on YUI Theater.

Once everyone who participated was happy with the pull request, Eric (who is a member of the core team) arranged to discuss Template.Micro at one of YUI’s weekly engineering meetings. These meetings are normally private, but the team occasionally invites external contributors to join in, and they invited me to talk about Template.Micro.

Every YUI team member in that room who had an opinion had already participated in the discussion on the pull request, except for — guess who? — the ones who rarely engage with the community. They hadn’t looked at it. Not even a glance. Naturally, they were the only ones who objected to merging it in. They objected strenuously and continuously for nearly two hours as I summarized and reiterated every discussion that had already taken place on the pull request or on IRC. Eventually, with the help of others on the team who had participated from day one, I managed to convince them it should be merged.

Then nothing happened, and nothing has continued to happen for over a week. Rumor has it that someone changed their mind (or just “doesn’t have time” to review the fiftyish lines of code, not counting comments) and has decided Template.Micro shouldn’t be merged after all. Code freeze for the next release approaches. I can’t actually get an answer on the pull request itself, so it sits unmerged, and I have no idea whether I or anyone else who wants to see this in the library will be able to use it in the next release.

I’m left feeling that I’ve wasted my time, and I no longer have any desire to contribute to YUI. It’s not worth it. This is just one of the battles I’ve had to fight recently, and fighting battles is not how I enjoy spending my free time, nor is it a good way for me to spend my employer’s time.

What needs to change

Problem #1: There’s no incentive for YUI core team members to review or merge pull requests.

Everyone on the core team is busy (usually incredibly busy) with their own tasks. No explicit time is ever set aside for managing external contributions, so the only time this happens is when someone takes the initiative to do it. There are people on the team who rarely take this initiative, and others who are often too busy to take this initiative. As a result, even small pull requests tend to sit around for a long time. This provides a significant disincentive for external developers to contribute minor fixes, and an incentive for them to simply bake monkeypatches into their own code, which YUI will never benefit from.

My recommendation: YUI should make it part of everyone’s job to triage and manage pull requests. They should strive for zero open pull requests at the end of every sprint — any pull request that shouldn’t go into the current sprint should either be denied or scheduled for a future sprint (and then should not be forgotten when that sprint rolls around).

Problem #2: There are no external contributors with commit access to YUI.

Obviously only highly trusted contributors should have commit access to YUI. The core team is highly trusted. Luke Smith (another recent former YUI team member who still contributes) and I were highly trusted when we were on the core team. As soon as we turned in our Yahoo! badges, we became just as untrusted as everyone else on the Internet. This is strange.

The weirdest thing about this? I’m now dogfooding YUI far more than I ever did when I was paid to work on it, so I’m naturally finding a lot of things I want to improve. I feel like I have a much better idea of YUI’s pain points and opportunities for improvement than I ever did before — but I also feel like there’s so much of a process barrier that the amount of effort required to get my changes back into YUI begins to approach the amount of effort required to simply fork YUI. That’s really scary to me, because it makes me wonder how many other YUI users and potential contributors have the same feeling.

My recommendation: Make GitHub YUI’s source of truth repo. Grant commit access to highly trusted external contributors, on the condition that non-trivial changes must be reviewed by the core team first. If an external contributor screws up, it’s not a big deal — revert the commit. If they screw up often, revoke their commit access.

Problem #3: There’s not much external visibility into YUI’s release planning and scheduling process.

I didn’t realize how frustrating this could be until I was on the other side. I want to contribute lots of stuff to YUI. Not just new stuff, but improvements and fixes. Naturally, I want to do this in a way that aligns with YUI’s own plans and schedules, but I also want to do it in such a way that I don’t have to babysit my contributions for months until they can be integrated into core.

Knowing YUI’s release plans and what the team is focusing on for an upcoming release would be a huge help in budgeting my time. I could look at the team’s plans and say, for example, “Hey, it looks like YUI is focusing on performance improvements for this release. I’ve got some of those on the back burner, so this would be a great time to contribute them.” Or “Well, I’ve cooked up this new widget, but it looks like the next YUI release will be a bug fix release, so I should roll this into a gallery module for now instead of letting it rot in my fork for three months.”

Efforts like YUI Open Hours and This Week in YUI help a little, but they don’t provide the nitty gritty insight into release planning and scheduling that are really needed, and Open Hours in particular requires a level of time commitment that people just can’t always make — it’s a lot harder to watch an hour-long screencast and try to extract meaningful info than it would be to read a blog post that simply says “here’s what we’ve got planned for the next release and here’s how you can help”. Even just knowing the dates for preview release windows would help contributors immensely in prioritizing our contributions.

My recommendation: Post the results of planning discussions and other non-confidential internal discussions on the blog (or on GitHub, with an announcement on the blog or Twitter) soon after they happen. A public document of some kind should be the end result of any non-confidential planning meeting or discussion. Consider inviting trusted external contributors to participate (via G+ Hangouts) in discussions that won’t involve confidential Yahoo! information.

Problem #4: Creating and maintaining gallery modules is far too much work.

The usual response to “it’s too hard to contribute to YUI core” or “YUI core doesn’t want my awesome new widget” is “well, just put your stuff in the gallery”. But it’s so painful to create, publish, and maintain a simple gallery module that I’d just rather not do it at all. It eats up far too much time that could be better spent writing code.

The main problem is how much manual labor there is, and how difficult it is to organize and maintain gallery modules as a result. Dav and others on the team are aware of my specific complaints in this regard, but until something is done, I think the team needs to stop using the gallery as a crutch to justify not solving problems 1 through 3 above.

My recommendation: Change the gallery to rely on the exact same build tools and procedures that YUI core uses. Remove the requirement for all CDN-hosted gallery modules to be maintained in a single yui3-gallery repo, so that gallery modules can use their own GitHub repos as their source of truth. Remove the manual form-filling required to create a gallery module on the website, and instead rely on the module’s own metadata and a simple tool that publishes gallery modules similar to how npm publishes npm modules.

If it were as easy to create a gallery module as it is to create an npm module, I’d have so many gallery modules people’s heads would explode. In a good way.

Problem #5: YUI can be a community project or a Yahoo! project. It can’t be both.

The primary mission of any good internal Yahoo! project is to further Yahoo!‘s corporate goals. The primary mission of any good open source project is to further the goals of its community. Since its inception, YUI has tried to do both. This isn’t working.

I can tell you from my time working at Yahoo! on both YUI and other projects that YUI is unlike any other project at Yahoo!. It sometimes doesn’t feel like it’s even part of the same company. And I can tell you from my time working on open source projects that YUI is unlike any other open source project, and in many ways doesn’t feel like an open source project. YUI tries to be a hybrid of both, and as such it tries to serve two masters — the corporation and the community.

This is why YUI is subject to Yahoo!‘s bureaucracy, it’s why some team members feel justified never engaging with the community, and it’s why contributing to YUI is a major pain in the ass. When I worked for Yahoo!, it was easy to assume that the external users (and non-users) who complained about this were just being too sensitive, too selfish, or too shortsighted. Now that I’m one of them, the shoe’s on the other foot and I’ve achieved enlightenment.

Despite the efforts and dedication of many members of the core team — seriously, YUI has an incredible group of core developers — YUI is not really an open source project. It’s a Yahoo! project whose source is public. The open source community has zero involvement in the management of the project. They have zero transparency into day-to-day happenings except for the mirrored source tree and whatever information leaks out via the blog, IRC, and Open Hours. There is no true sense of community ownership or responsibility for the project, and community members often feel frustrated at their inability to influence YUI’s direction.

My recommendation: Create a YUI foundation, separate from Yahoo!, and entrust the governance of the YUI project to it. Encourage Yahoo! and other companies that benefit from YUI to provide funding. This isn’t a new idea — it’s something that’s been discussed both inside and outside of Yahoo! for years, and is a structure embraced by many other open source projects — but it’s more and more apparent now that it’s crucial to YUI’s future.

I still love you, YUI

YUI has many flaws, whether we’re talking about the code itself or how the project is governed. But it’s also an amazing library with a vibrant, intelligent, devoted community. I plan to continue using it, and I hope to be able to contribute to it again at some point. But until some things improve, contributing to YUI just isn’t worth my time.