When Cabel and I first started Panic, we tracked bugs in a FileMaker database. This didn't last long.
Next, we went to a web-based offering that we self-hosted. This worked generally pretty well, except for two main problems:
When there were problems with the tracker, we either had to fix them ourselves, or hope an update would come out soon.
Having each operation bracketed by an HTTP request/response really slows down the rate at which you can batch process things.
It was mainly the latter problem that led us to develop a custom bug tracker, known as Hive. Hive is a Cocoa front-end to a MySQL database. We've been using it for years, and it works pretty well. People have even asked us to release it publicly (trust me, it's not ready). However, it too has its drawbacks:
When something goes wrong, we have to fix it ourselves. Maintaining Hive requires a similar effort to maintaining one of our public-facing applications. It's almost like an entire "hidden app" that requires developer resources which we don't have enough of.
There is virtually never time for anyone to add new features or improve it beyond bug fixes. This means it is lacking critical features, such as being able to attach files to bugs.
It has a "last save wins" policy if multiple developers make changes that collide, which has bitten us.
We cache bugs to reduce network traffic, but there have been problems lately with stale caches leading to confusion.
It's getting to the point where we need the extra functionality that we don't have time to add, and are willing to forego a little bit of speed by going back to web-based, if we can find a web-based tracker that does what we want.
Simplicity and speed, first and foremost. Our bug records are extremely simple. Bugs have an ID number, a one-line summary, a description, a creation/modification/close date, a status (open or closed), a severity (only 4 levels), an owner, the version filed against, the version targeted for resolution, and that's pretty much it.
A lot of trackers go way, waaay too overboard with the fields. We need to work on bugs, not placate a needy database. The process has to be as fast and streamlined as possible. A light, clean design wouldn't hurt either.
Bugzilla.
Some factors to consider:
Custom vs. off-the-shelf: It's tempting to roll our own internal web-based tracker. That way we could tailor it specifically to our needs, and I suspect it would be easier to maintain than our existing Cocoa app. However this is at odds with our need reduce developer time spent on internal tools.
Hosted vs. open source: Hosted solutions are nice, in that you never have to deal with updates. But being able to install something on our own server would give us extra speed and hackability. This is a tough one to call.
Importing: We have plenty of existing bugs, and we would need a way to get them into the new system without any re-typing.
Exporting: For when we need to move again in the future.
If you have first-hand experience with a bug tracking service (either hosted or open source) and you like it, feel free to send an email my way.
I'm not sure which way we're going to go with this yet, so it would be good to hear some experiences while weighing our options.
Thanks to everyone who has emailed with their thoughts and suggestions!