Gentoo Bug Bounties

Some may have noticed that the Gentoo Foundation has funded a bug bounty. This is something fairly new for the Foundation, and I wanted to offer some comments on the practice. Please note that while I’d love to see some of these make their way into policy some day, these are nothing more than my own opinion, and I reserve the right to change my opinion as we gain experience.

The recent bug bounty was for bug #418431, which was to address a problem with git-svn which was holding up stabilization of the latest version of git, which is a blocker for the migration of the Portage tree to git.

What follows are some principles for the use of bug bounties and how I think we fared in this particular case. I’d like to see the use of bounties expand, as right now I believe we under-utilize our donations. However, it is important that bounties be used with care as they have the potential to cause harm or be wasteful.

One more upfront note – I supported the git-svn bounty as it was ultimately worded, as did the other Trustees. Looking back I think we could have done things a little differently, but hindsight is always 20/20, and no doubt we’ll continue to learn as we experiment with this further.

1. Bounties Should Be Used Strategically
While the Foundation has money to spend, we aren’t swimming in it, so we can’t use bounties for any little bug that annoys us. Bounties should be reserved for matters where spending a little money has a large impact.

I think we did well here – the git-svn issue was going nowhere either within Gentoo or upstream, but the number of other blockers to the git migration are fairly small and within Gentoo’s control. Getting rid of this issue should open the way towards the git migration, which is of course of strategic importance to Gentoo.

2. The Solution Must Be Sustainable
This might also be stated as “consider the total cost.” Before agreeing to fund this bug there was some due diligence to ensure that upstream would carry forward any patches we generated. The problem was the result of changes on the SVN side, and the solution included some general cleanup and refactoring of code to make git-svn more maintainable upstream. Upstream also expressed an interest in accepting the fix, and it was the opinion of the package maintainer that this would be a one-time fix as a result.

When considering whether a solution is sustainable, we need to think about how we got where we are, and consider whether we’re just going to end up back in the same place again. If the solution won’t be maintainable, then any money spent is wasted unless it truly is a one-time event.

3. Gentoo Can’t Fix It With Volunteer Effort
Gentoo is a community distribution. We have some very talented developers. We can usually fix our own problems, and doing so as a volunteer community effort is usually the healthiest solution.

The sense for git-svn was that this was an upstream problem in a language our maintainers were not comfortable with. The bug languished despite attention by several developers and discussions in other forums. It was felt that offering a bounty would allow targeted expertise to tackle the problem, which otherwise was not of great interest to our community.

A policy to not offer bounties unless a bug has been open for some period of time except in unusual circumstances would be appropriate.

4. Be Ready To Capitalize On the Work
If the work is strategic (see #1), then we ought to have a plan ready for when the bug is closed. Otherwise there really should be no urgency to pay somebody to close the bug and it is basically a pig in a snake (clear the jam, and the problem just moves one step down the chain).

I think the jury is still out on how we’re doing here. I think there is a lot of enthusiasm about git but we could have a bit more organization here. None of this is intended as a slight to those who have been laboring hard to make this work – I hope getting this blocker cleared will inspire more to step up and resolve the other issues. (I won’t say more here as I don’t want to make this about the Git migration.)

5. Define the Problem and Success
A bounty is a contract. At the very least misunderstandings can lead to hurt feelings, and at worst they can lead to HIGHLY contentious, expensive, and distracting legal action. While a 10 page document shouldn’t be necessary for a token expense, any bounty should be very up-front about what exactly is to be done, and how success will be evaluated.

I think we could have done a little better in this regard, but there was some iteration on the wording of the bounty to clarify the “victory conditions.” I think it is important to focus on outcomes – in this case we wanted code that upstream was likely to accept. I’d actually have been happier making upstream acceptance a condition of payment, but the sense was that this would be inevitable but might delay payment unduly. I think the jury is still out on this one. What is important is that we don’t just achieve technical resolution of the bug, but that we fully realize the benefits we had in mind when we funded the bounty.

6. Cover Code Ownership and Licensing
This is a work for hire – we can dictate ownership of the code (yes, I realize that the legalities of this vary internationally, but the US is the only nation that legally recognizes the Gentoo Foundation at the moment, and the US will enforce this insofar as its jurisdiction allows). Per the Gentoo Social Contract, if we’re funding the creation of code, it ought to be free (generally GPL).

This was covered in the git-svn case. We didn’t insist on ownership of copyright, but we did ensure the code was licensed using the upstream license (GPLv2). My feeling is that if the bounty really represents payment for a majority of the work Gentoo should just own the code outright. If the bounty is really a token gesture for what is mostly a volunteer effort I think the author should retain copyright as long as the code is FOSS. In practice it doesn’t matter too much, so I think we should use discretion here.

7. Offers of Bounties Should Be Fair
This topic led to some internal debate, and I think that we can probably do a little better in terms of transparency in the future. The bounty was posted publicly on the bug, and anybody already interested in the bug and on the CC list would of course have gotten notification. In retrospect I think that bounties are a significant enough occasion that perhaps a proposal should be offered for comment on -dev or -nfp and the final version announced on -dev-announce. I think that the way we handled the git-svn case met all legal obligations, but I really want to make sure that the whole community has an opportunity to participate when they come up.

Another potential issue with bounties is that you can only pay one person (unless there is some side agreement to share it), and there can be resentment if work gets done but isn’t reimbursed. This was addressed in the present case by asking anybody working on the bug to state their intent. If a bounty is very large it probably would make sense to go through a more formal bidding process and just award a contract more conventionally.

I think that this last point of fairness is actually the most critical. While messing up on any of the others could cause us to waste a few hundred dollars, getting the fairness bit wrong could literally destroy the community. When you start paying people to do what used to be volunteer work the result can be demoralizing to the community. I think the key is to only do this when the community lacks the ability/desire to do the work itself, and especially when the work lies outside of our core expertise. Paying an accounting firm a reasonable fee to ensure our taxes are filed correctly isn’t viewed with much controversy. We should try to keep bug bounties limited to similar sorts of situations.

Trustees of course have duties both under the bylaws and under US law to properly manage conflicts of interest. These certainly apply to any kind of expenditure of money.

---

So, what do you think? I’m very open to criticism about how we handled our first bug bounty, and how the community feels about this use of money. As is evident from the Treasurer’s Report at today’s Annual General Meeting, Gentoo is currently receiving more than it spends in donations, so I think making a little more use of this approach will allow our supporters to benefit Gentoo. Seeing donations in action probably will help encourage an increase in donation as well. However, I think we also need to tread carefully here, as the community matters far more than squashing a few bugs.

Finally, while I’d like to see policy around bounties formalized, I think doing so right away would be a mistake. I think we should try to consciously apply principles like these but wait until we see how they work in practice before trying to codify them.