The Balance of Power in Gentoo
The recent concerns with the request to re-populate QA have re-opened a debate that is a few years old now. I’ve already made some specific recommendations on the lists, but I wanted to step back and explain why I feel the way I do.
Gentoo’s system of governance has some internal ironies – ones which occasionally even lead to calls to establish a benevolent dictator position. I think the mistake that Gentoo makes is that the problem is perceived as being democracy, when in reality the problem is with competing governance bodies with differing constituencies…
The greatest irony is that the same organization that contemplates appointing a supreme dictator often recoils from centralizing power in committees.
Please note that nothing I write here is really directed at any particular project/org/etc within Gentoo, or at any individuals who have served in governing roles in the past/present. My intent is to focus entirely on the system of governance that Gentoo uses, and to explain how the kinds of conflicts we deal with are basically designed into the organizational structure.
Degrees of Centralism in Governance
Before talking about Gentoo I want to contrast a few forms of democratic governance and the role of centralism in avoiding or fostering stalemate. Democratic governments vary greatly in the number of centers of power with different constituencies.
The most extreme centralization of power lies in states that use a unicameral unitary parliamentary system, such as Norway (and most corporations/organizations). In such a state the people vote for parties to form a parliament, and that parliament in turn exercises nearly complete control over all aspects of the government. They appoint ministers, and any governance bodies in the state derive their power from the parliament. Such organizations tend to have little internal division, as the legislature has the power to overcome almost any stalemate (the courts are usually an exception which I’ll set aside for now). Some definitions:
- Unicameral – power is invested in a single legislative body.
- Unitary – a single layer of government exists.
- Parliamentary – the executive power is appointed by the legislature.
An example of a very decentralized system of government is in a bicameral federal presidential system, such as in the United States. In such a state there are many somewhat independent governance bodies which have differing scopes of power and constituencies. No single legislative body has the power to create laws, the executive power is independent, and even acting together these do not have absolute sovereignty over all aspects of the operation of the state. For example, in the current US government different parties hold sway over each of the legislative bodies and the executive branch, and a recent court case upheld the power of states to choose to not enact some elements of the Affordable Care Act. Individual US voters have differing levels of power in electing officials to each governing body based on geography and state population. Such organizations tend to have quite a bit of internal division. Some definitions:
- Bicameral – power is invested in two legislative bodies which usually have to collaborate to pass a law.
- Federal – multiple layers of semi-independent government exist.
- Presidential – the executive power is directly elected.
Governance Bodies in Gentoo
Now let’s examine the governance bodies that exist within Gentoo.
There are 5 special projects and a large number of general projects. Here are the scope and constituencies of each of the special projects:
- The Council is tasked with decisions that affect the technical design/operation of the distribution. It is elected by all developers annually. It is the court of appeals for QA and Comrel.
- The Trustees are tasked with the legal operation of the Gentoo Foundation, which owns the IP of Gentoo and manages the money. It is elected by all foundation members biannually.
- QA is tasked with the quality of the distribution and has enforcement powers. Its lead is elected annually by its members, and members are appointed by the lead.
- Comrel is tasked with dispute resolution and discipline and has enforcement powers. Its lead is elected annually by its members, and members are appointed by the lead.
- Infra is tasked with keeping the lights on, both for Gentoo-owned hardware and coordinating/managing resources donated to our use. Its lead is elected annually by its members, and members are appointed by the lead.
Then there are a multitude of normal projects. Anybody can join a normal project, and they all elect their leads annually. In addition to anybody being able to join a normal project, anybody else can also start a competing normal project.
I’ll set aside the normal projects, because for the most part I don’t think they are the cause of many governance issues. Their power is very limited, and they are very open for any developer to join. Instead I’ll focus on the special projects.
If you look at the 5 special projects, every single one of them has a different constituency. Three have a constituency consisting solely of their own members, and the two elected bodies have largely overlapping but not identical constituencies. Since they have different constituencies, it is entirely possible for them to be in long-term conflict. Since no one body has power over the others, there is also the potential for stalemate.
Where We Should Be, and How to Get There
I would advocate that Gentoo would be better served if it had fewer independent governance bodies, and if there were clear hierarchy between them. Less independence means that there is less potential for conflict, and hierarchy means that when there is conflict there is less potential for stalemate.
That said, I still think it makes sense to preserve these organizations – they really do have different roles in sustaining Gentoo and different skill-sets are needed to make them work. I also don’t think that we need to change everything in the span of a day.
Here is my outline of how I think Gentoo should operate, eventually:
- The Trustees are the legal board of directors for the distribution. The Foundation owns everything about Gentoo that can be legally owned. Legally the Foundation IS Gentoo. In practice I would not suggest changing the scope of their operations, but legally they would have the power to resolve any matter as long as they are consistent with the Charter/Bylaws.
- The Council is tasked with leading the technical operation of the distribution. It would be legally subordinate to the Trustees in the event that the Foundation believes that the Council is causing Gentoo to violate the law or its Charter, but it would be otherwise self-governing and elected by the developers. The Council would be the court of appeal for any enforcement action, and would additionally have the power to step in and resolve disputes among projects. All projects are sub-ordinate to either the Council, or directly to the Trustees.
- QA, Devrel, and Infra would continue to appoint their own members and elect a lead, and would maintain the same scope/powers. However, the leads of each would need to be confirmed by the Council, and could be replaced by a decision of the Council. The leads would also be required to report status periodically to the Council.
- Individual projects are all subservient to the Council/Trustees in principle, but would be largely hands-off as long as they allow for open membership, do not interfere with each other, and elect their leads annually.
I could see debate over whether Devrel or Infra should report to the Trustees directly vs to the Council. I really don’t have a strong opinion there and you could make a good case for either.
In the short term I see QA as the best place to start as it is an organization that is going to be reconstituted in any case, and the new membership could help set a model for the other special projects to follow. Infra is functioning very well currently and would probably be the last thing I’d advocate touching, with the goal of minimal disruption. Devrel falls somewhere in-between, but in my proposal all special projects still select their leads and it is the role of the Council to accept them – the Council stepping in and interfering with the constitution of a special project should be used only to resolve serious disputes.
So, that was long-winded as usual. I’d be very interested in reactions/thoughts. I’d also be interested in examples of how models like the one I propose have been successful or have failed.
Rich0's Gentoo Blog 
Your ideas are good. In fact, except for confirming the team leads, this was my understanding of the structure when we started the foundation. Of course, that is a while ago, and things have changed. But the original intend was that day-to-day management fell under the council (possibly moving a little faster than it has done at some times) and that the foundation would be back-stop, but not actively involved in day-to-day running beyond approving expenditure.
Paul de Vrieze
November 18, 2013 at 4:53 am
I think the reality is that the Foundation basically has priority since legally that is the only organization that actually exists. That isn’t really written down anywhere, but if any dispute actually made it into a court of law then the Foundation would be recognized as the entity that can speak for Gentoo.
The reality is that any kind of “civil war” would destroy the distro in any case. We’re more dependent on volunteers than bank accounts, and many Gentoo services are provided using hardware that the Foundation doesn’t actually own. That’s one of the reasons why I think it would be wise to keep the Foundation and Developer populations nearly identical – that reduces the risk of conflict.
Anytime you have governing bodies with different constituencies you create a potential for internal conflict. Anytime you do not define subordinate relationships between governing bodies you create the potential for stalemate. Our organization design creates a real opportunity for both.
But if you like the current situation in the US Government, then Gentoo is just fine today. 🙂
rich0
November 18, 2013 at 7:36 am
Comrel should answer to Trustees, as QA answers to the Council.
EU Law provides some nice parallels; you have the Council (of Ministers), subordinate to the European Council (heads of State), along with the Commission which is effectively the civil service, charged with initiating legislation to carry out Treaty-mandated tasks.
Interestingly, the EU Code of Conduct is seen as applying to the organs of the EU more than anyone else. In the Gentoo case that would be developers, staff and Trustees.
Comrel as currently constituted has no mandate: only Proctors were agreed on by the whole Community, over an intense discussion lasting more than six months, and their basis was Forum Moderators, who have always moderated the entire community (remember, “devs are users too” so they all started out via that route.)
Comrel is entirely a developer construct, and has no basis at all in democratic negotiation. The way it is managed is entirely as a developer herd, and it only answers to the developer Council, so it has no claim to any mandate over, nor representation of, the entire Community.
The Trustees are the only group that speaks for the entire Community, both legally and objectively. The Council’s remit has always been strictly the technical aspects of Gentoo, which is usually what we care about. However when discussing community relations, that is clearly outside their scope, just as QA is clearly within it.
Steve Long
December 1, 2013 at 5:59 am
Honestly, I’m not entirely convinced we should be using the EU as a model – it is constituted much like the US Congress was constituted prior to the 17th amendment and has multiple constituencies. I’m not really a fan of the Council and Trustees having separate constituencies for the same reason, but in practice that hasn’t much of an issue since I can probably count the number of non-dev Foundation members on one hand. If that ever changes I think it could have a negative impact on the distro.
I don’t have a problem with Comrel reporting to the Trustees though.
The reason I’m not a big fan of having non-devs as Foundation members is that it creates another situation where you have multiple constituencies. If the Foundation ever entered into conflict with the developer community it could tear the distro apart, so what is the point of giving the Foundation the power to move in a direction other than what the developers want? If we weren’t primarily a volunteer-based organization that wouldn’t be a problem – if a bunch of Mozilla devs got fed up with some policy they could simply be fired and replaced as in any corporation, and when you’re getting paid to do something you tend to care less about how it gets done (a double-edged sword). The reason that Gentoo tends to be fairly hands-off with our developers is that we don’t have much choice – you can’t really tell a volunteer what to do unless you have a very long waiting list.
I’m not suggesting that Gentoo shouldn’t try to care for the community outside of the developer pool. I just don’t think it is a good idea in practice to make developers accountable to non-developers.
rich0
December 1, 2013 at 7:09 am
I’m not saying we should use the EU as a model, by any means: merely that it provides interesting parallels, especially when one sees how the institutions operate wrt legislative consequences: the very balance of power you mention. Let’s leave it to one side though, as it’s not the actual topic, although the telelogical approach (or “purposive construction”) inherent in Continental legal traditions, and implicit in the EU Treaties, is of relevance, IMO. It’s much closer to what techies seem to prefer (ie not a long, detailed list of provisions; that is left to “implementation” effectively.)
You restrict your comments to the “developer community” but I think it is important to acknowledge that that is a strict subset of the user Community. As far as I know, it is impossible to be a Gentoo developer without also using it in some form or another, such that one can test ebuilds. When it comes to social, legal, and political aspects of software development, the wider Community, as represented by the Trustees, is whom everyone is accountable to under the Code of Conduct, as is Gentoo itself: it is accountable to its users, by Charter.
Further, if all the developers resigned en-masse, or switched to exherbo, the Trustees could still resurrect the distro, in collaboration with its user-base, should that be the consensus decision. The Foundation owns the IP, and is the only entity with any legal basis, and the Trustees are the only people whose names are truly on the line when it comes to consequents of Gentoo staff behaviour, should there ever be any.
No-one is suggesting telling the developers what to do when it comes to their work; merely that they, like everyone else, are accountable to the wider Community for their behaviour, not just to each other.
WRT Foundation members, I think it’s important to remember that the Gentoo user-base is not the same as most Linux distros: by definition they are more clued-up on Linux and the system software, or they cannot maintain their install. Typically they are on the older end of the bell-curve, and have several years experience of the working IT world. The younger ones soon will do, or they wouldn’t be into Gentoo.
This is important in the context of the way the dev ML has been setup from the beginning to allow posts from anyone in the Community: firstly for the aforementioned wider experience, eg in the GLEP process, and secondly because drobbins clearly wanted to avoid Gentoo becoming a clique of developers, as his previous distro had fallen into. This is relevant to the discussion of behaviour, and responsibility: developers are supposed to be held to a higher standard of conduct, and any bad behaviour should be called out on the medium where it happens, the same as it does for users, not kept to back-channels. Or one is left with the impression that developers can beat up on users at will, and no-one says a word.
Forum-mods used to be good at that sort of thing. Perhaps that’s why the developers didn’t want them moderating their behaviour more widely. A lost opportunity, IMO. The Forums are what has kept Gentoo consistently excellent, and have nurtured the Community and its progressive development. The credit for that belongs with its moderators.
Steve Long
December 1, 2013 at 9:49 am
> Further, if all the developers resigned en-masse, or switched to exherbo, the
> Trustees could still resurrect the distro, in collaboration with its user-base,
> should that be the consensus decision. The Foundation owns the IP, and is the
> only entity with any legal basis, and the Trustees are the only people whose
> names are truly on the line when it comes to consequents of Gentoo staff
> behaviour, should there ever be any.
Legally, perhaps, but in practice not really. The Trustees wouldn’t have to
resurrect the distro – it would still exist as it always existed, legally. The
problem is that unless somebody actually does work to move the distro along, it
is dead even if it has a board of directors and a bank account. The Foundation
doesn’t have the kind of money it would take to actually operate the distro
without a substantial volunteer effort, and those volunteers would effectively
be the developers. The Foundation might own the rights to the domain name/etc,
but that’s about as useful as owning the name “OpenOffice” or “XFree86.”
So, in practice Gentoo is almost entirely dependendent on its developers, which
is why I’m not really a fan of having Foundation members who aren’t developers,
or for keeping those who are no longer active on the member roles. It doesn’t
cause problems only because in practice the list of active Foundation members
and active Devs is almost identical – if that ever changed we could have
problems.
> This is important in the context of the way the dev ML has been setup from the
> beginning to allow posts from anyone in the Community: firstly for the
> aforementioned wider experience, eg in the GLEP process, and secondly because
> drobbins clearly wanted to avoid Gentoo becoming a clique of developers, as his
> previous distro had fallen into. This is relevant to the discussion of
> behaviour, and responsibility: developers are supposed to be held to a higher
> standard of conduct, and any bad behaviour should be called out on the medium
> where it happens, the same as it does for users, not kept to back-channels. Or
> one is left with the impression that developers can beat up on users at will,
> and no-one says a word.
I guess where I get concerned is where to draw the line. I’m all for having
more control over misbehavior on the lists/etc. Hopefully you realize by now
that I’m always eager to see non-developers contribute and I try to be as
supportive as I can when they participate on the lists or get into conflicts
with developers. The thing is that I really don’t want to give non-developers
authority over developers unless they’ve really done something to earn that
authority. In my experience giving somebody the power to interfere with
something without making them earn it with hard work tends to result in a lot of
nitpicking and serious demotivation for anybody actually doing the work.
However, I agree wholeheartedly that I don’t want to see Gentoo be a distro that
is merely by the devs for the devs. Ultimately, though, FOSS does have a
scratch-an-itch slant to it, and that is going to translate into how it is
governed. As a member of the Concil I can vote for all the policies I want to,
but if they antagonize the people doing the work then the Council will end up
governing itself with nobody else remaining. The same applies to the
Foundation.
rich0
December 1, 2013 at 1:59 pm
> Legally, perhaps, but in practice not really. The Trustees wouldn’t have to
> resurrect the distro – it would still exist as it always existed, legally. The
> problem is that unless somebody actually does work to move the distro along, it
> is dead even if it has a board of directors and a bank account.
Well, firstly the infra, and thus the distro, would continue to operate, so as you say there’d be nothing to resurrect. It’s also very unlikely that *every* developer would resign all at once; but I agree that Gentoo is a volunteer effort, and the Trustees are merely there to shield those volunteers from the real world, essentially.
However the social aspects of development are both important, and not under the purview of Council, nor does anyone want that afaict; the Council has always been about technical matters, which is what usually interests us.
> The Foundation doesn’t have the kind of money it would take to actually operate the
> distro without a substantial volunteer effort, and those volunteers would effectively
> be the developers.
indeed, and they would have come from the user-base, exactly as they do now. Again, Gentoo’s
userbase constituency is very relevant: since it’s from-source and requires user-involvement, bug-reports and patches are much closer to development work than in bindists. My point is that the “devs are users too” applies much more in Gentoo, since a Gentoo user is far more likely to get involved with the software compilation process.
And that feeds through to the whole Comrel/Proctors debate; Forum moderators are “non-developers” who have _earnt_ that authority, in exactly the sphere under discussion. Further every user has accepted their authority on the Forums, including developers. So why bother with anything else?
The only case for an exception is an IRC-channel; email lists should definitely be moderated in the same manner, just more strictly focussed to staying on-topic, since there is no capability to move or split sub-threads. And again, the most pressing need is for developers to be picked-up on their behaviour on the medium where it happens, exactly the same as users are, or the whole thing falls apart and unnecessary and counter-productive ill-feeling arises. I hope to see you supporting users in that regard in the future, should the occasion arise.
I don’t think that Gentoo ever could be a distro “by devs for devs” however some might like to think otherwise. Ultimately, without users you’re not a distro: you’re a clique of people no-one listens to. The same applies to all software: without users, it might as well not exist, since it won’t be doing anything. All that can be hoped for is that someone rediscovers it down the line, which is unlikely if no-one found it
useful. It’s *always* about the results.
Steve Long
December 4, 2013 at 6:49 am
So, personally I think that getting rid of the proctors was a mistake, especially since they had only had a few days to do their job in the first place. I think forum mods is a perfectly acceptable place to look for them.
However, I still think that they should be accountable to the Council or Trustees, and that those bodies should be accountable to the developer community. I certainly agree that users are essential for the operation of the distro, but I just don’t think that it makes sense to give them the same voice as developers when for the most part they haven’t invested as much.
Maybe the current Trustee approach is the right one – non-devs can apply, but they need to demonstrate significant investment in Gentoo. I think a bigger issue is retaining past devs on the rolls, but for the most part that is dealt with by the requirement to vote in one of last two elections.
My main concern is avoiding situations where those who are actually doing work decide it isn’t worth doing, because of conditions imposed by others who aren’t contributing as much. I do agree with Donnie’s whole “Assholes are Killing Your Project” thing, and sometimes you have to lose somebody who contributes heavily. However, that needs to be done with care because we can’t afford to just alienate large swaths of our developer community.
rich0
December 4, 2013 at 7:39 am