Wednesday, August 27, 2008

Wanted: Bugs. $100. Cash money.

I need my beta releases of Delicious Library 2 to be tested, for serious; otherwise I might miss things I've broken when I was fixing other things, and put out a general release that has new bugs in it in addition to the fixes. Nobody likes that.

I've never had much luck with full-time paid testers. I'd happily go this route if I thought it'd work, but, honestly, in 25 years of writing software, I've NEVER had a paid beta-tester that found half as many bugs as volunteer beta testers. It's a mind-numbingly boring job to test software, and one naturally tends to fall into patterns and test the same things over and over. It takes a variety of users, who actually _use_ the software because they want it, to discover things like, "Shelves without names can't be published to the iPhone." (This was a real bug, fixed in 2.0.3.) I would never think to create a shelf and not name it, then publish it.

There are a million issues like this.

Here's the problem: volunteer beta testers generally like beta testing because they get to play with new software before anyone else. In the point releases I'm doing now (eg, 2.0.1, 2.0.2), I'm not really adding features (well, one or two), I'm just making Delicious Library 2 more robust. That's, like, not very exciting to test.

So, I'm going to try an experiment. I'm going to offer EVERYONE the possibility of making $100 by finding and reporting a regression bug in a current beta release, which I think is important enough to fix. Note that it has to be a "regression" bug, which I define as "this worked in a previous 2.x release, but doesn't work in the current beta release."

Note that I'm also ONLY offering this bounty on the current beta, not on production releases. The logic is, if you find a bug on the product AFTER I've released it, well, the horse has already left the barn — my customers are already affected by the bug. You have to find the bugs BEFORE I release them to the public.

The latest beta is always at: http://delicious-monster.com/downloads/Delicious%20Library%202%20Beta/DeliciousLibrary2.zip - I rebuild that every few days, and the build number in the about panel will go up. (Note that if the beta build number is less than the current release build number, there's nothing to test right at the moment.)

The latest release notes will always be here: http://delicious-monster.com/downloads/Delicious%20Library%202%20Beta/DeliciousLibrary2.html.

Here's the rules, to hopefully make some lawyers happy:
  • Bugs should contain the word "regression" in their title in order to be found for this hunt.
  • Bugs have to be filed by either sending mail to <bugs@delicious-monster.com> or by using the in-application bug reporter. Don't send 'em to me directly. Don't post 'em.
  • You can report multiple bugs and get multiple prizes. However, if you report a bunch of crap, we'll start ignoring you.
  • I have to think the bug is important enough that I fix it before I do the real release in order for you to get the bounty.
  • If multiple people report a bug, I'll only give money to the first person who reported it. (Well, I might give money to multiple people, but I'm not promising anything.)
  • I'll use a reasonable effort to find the earliest person who reported any particular regression bug, but I might not find your report. Use good keywords to make sure that I'll be able to look up your bug - all awards are final.
  • I've initially set aside a $4,000 for this. Beyond that, I may increase the budget or may discontinue the bounties.
  • If it's not legal for you to get money for this (like, your country doesn't allow it, or something), then I can't pay you.
  • If this whole thing ends up being illegal, then I'll cancel it.
  • I can cancel this at any time by posting in this blog. (I'll do this if I'm just not getting good reports, for example, and this turns out to be not worth the hassle.)
Obviously, if you want to only use stable software, stick to the latest release builds, which are always at: http://delicious-monster.com/downloads/DeliciousLibrary.dmg. If you're a current version two user, note that by default your application will auto-update itself whenever a new release build comes out, but NOT to beta versions — you have to go get the first one of those yourself, using the link above.

So, remember: Only regressions, in beta versions, from 2.0, that I fix. $100. Cash money. (Well, check, unless you insist.)

We'll see if this works, and if people like it. If not, meh, nothing lost. At least we're trying new things!

Happy hunting!
-Wil