Gentoo Bug Bounties

Some may have noticed that the Gentoo Foundation has funded a bug bounty. This is something fairly new for the Foundation, and I wanted to offer some comments on the practice. Please note that while I’d love to see some of these make their way into policy some day, these are nothing more than my own opinion, and I reserve the right to change my opinion as we gain experience.

The recent bug bounty was for bug #418431, which was to address a problem with git-svn which was holding up stabilization of the latest version of git, which is a blocker for the migration of the Portage tree to git.

What follows are some principles for the use of bug bounties and how I think we fared in this particular case. I’d like to see the use of bounties expand, as right now I believe we under-utilize our donations. However, it is important that bounties be used with care as they have the potential to cause harm or be wasteful.

One more upfront note – I supported the git-svn bounty as it was ultimately worded, as did the other Trustees. Looking back I think we could have done things a little differently, but hindsight is always 20/20, and no doubt we’ll continue to learn as we experiment with this further.

A Quick Dracut Module

Since the general trend on many linux distros is towards requiring /usr to be mounted at boot time, I figured I’d see what it would take to get it working using dracut.

I’ve been messing with dracut for a while, and for some reason it stubbornly refuses to detect my raid devices. The kernel autodetection works fine, but this is disabled when booting from an initramfs. Dracut would timeout and drop me to a dash shell, and if I just typed mdadm -As followed by exit it would boot just fine.

Dracut is using udev to set up raid devices, and obviously that is not working.

Beyond this, I’d like to get my /usr mounted pre-boot, and there is a module called usrmount that purports to do just this. However, it isn’t working in my case because /usr is a bind mount to a subdir on an lvm volume, and it just isn’t figuring that out (it doesn’t even run lvm in the first place despite having the module installed, let alone figuring out what to mount in what order – I suspect the lvm module only works if root is on lvm).

My solution to both problems is to build my own simple dracut module. If you want to try it out:

1. cd /usr/lib/dracut/modules.d/
2. mkdir 91local
3. cat > 91local/module-setup.sh
   #!/bin/bash
   # -*- mode: shell-script; indent-tabs-mode: nil; sh-basic-offset: 4; -*-
   # ex: ts=8 sw=4 sts=4 et filetype=sh

   check() {
   return 0
   }

   depends() {
   return 0
   }

   install() {
inst_hook pre-trigger 91 "$moddir/mount-local.sh"
}


4. cat > 91local/mount-local.sh
   #!/bin/sh
   # -*- mode: shell-script; indent-tabs-mode: nil; sh-basic-offset: 4; -*-
   # ex: ts=8 sw=4 sts=4 et filetype=sh

   mount_local()
   {
   mdadm -As
   lvm pvscan
   lvm vgscan
   lvm lvscan
   lvm vgchange -ay
   }

   mount_local

Then run dracut to build your initramfs, and it should let mdadm and lvm auto-detect everything before it gets to mounting stuff. You can then use the fstab-sys to mount whatever you need to mount user. However, in your fstab.sys if you’re configuring a bindmount be sure to prepend /sysroot/ before the source directory.
Example fstab.sys:
/dev/vg1/data /data ext4 noatime,user_xattr,barrier=1 0 0
/sysroot/data/usr /usr none bind 0 0
/sysroot/data/var /var none bind 0 0

Hopefully this helps somebody out – the dracut documentation is pretty sparse. In fact, if somebody connected to dracut stumbles upon this I’d be open to a better way of hooking my script – pre-trigger just doesn’t seem right – I’d rather let udev try to do everything first. However, I couldn’t find any way to hook after udev runs but before it bombs out not finding my root device. Suggestions welcome.

Another MythTV Update

Agreeing with some advice on gentoo-dev, I’m going to post this as a blog entry instead of a Gentoo news item. The quick version of this update is expect to see 0.24.1 in portage in a few days. The long version follows…

KDump on Gentoo

I’ve been trying to learn a little more about kernel debugging (part of my quest to learn a little more about btrfs), and I figured I’d post a quick howto on getting kernel crash dumps captured on Gentoo.

The Foundation Activity Tracker

If you haven’t noticed on gentoo-nfp, I’ve created an activity tracking page (with the help of David Abbot) to track periodic compliance activities for the Foundation.

Some have asked why do this, and why not simply use the Gentoo Calendar/etc to accomplish this?

What’s Up With MythTV on Gentoo?

I’ve gotten a few fair questions over the last few weeks about the status of MythTV on Gentoo. Here is a quick synopsis of where things stand…

Gentoo and Security Updates

While it pains me a bit to say this, and I don’t have a great deal of time to type this, I wanted to at least try to get some kind of word out to our user community that the high-profile kernel regression announced a few weeks ago (and fixed a few weeks ago in almost every other distro), remains a vulnerability in Gentoo with no clear timeline for resolution.

Gentoo bug 337654 is tracking this issue.

Users can emerge a more recent version of gentoo-sources to get the patch, and I’d recommend doing so if local root exploits are something that concern you.

I’d like to dwell a bit longer on solutions, but I don’t really have time to do so right now. Clearly the kernel team could use help with security issues. The security team probably could use help as well in staying on top of these kinds of issues. I don’t want to kick people when they are down – Gentoo is an all-volunteer effort. However, situations like this really don’t do much to improve the reputation of the distro, and at the very least we need to inform users when problems like this arise.

An Appeal to Devs – Please Use News

Well, I spent half of today rebuilding my system, and upgrading mysql.

I figured that I might use the opportunity of my newly-found spare time while running revdep-rebuild to perhaps put out a general plea for developers to make use of the news feature in portage.

Upgrading to mysql 5.1 requires doing a full dump of your databases, some manual cleanup, an upgrade, and then some manual restore steps. Oh, and that dump has to be done BEFORE the upgrade or you end up having to revert back to 5.0 (which I ended up doing). Usually mysql upgrades are relatively painless, but jumps between major versions (0.1 level) are often not.

The upgrade also breaks anything that links to libmysql, which is quite a bit on a system that runs any number of services (mail, mythtv, ulog, etc).

It might have been nice if a news item were published a day or two before stabilizing mysql 5.1 so that users might have some advance warning and could plan accordingly.

Now, this upgrade didn’t rise to the level of some of the past breakages that actually were very painful to recover from and could result in unbootable systems/etc. Still, it never hurts to give users notice. The beauty of news items is that they only pester users who will actually be impacted by them. I don’t think anybody running mysql would mind a reminder that an upcoming upgrade requires careful planning – this is far more relevant to users than half the stuff we put in elogs/etc.

On the other hand, I do appreciate the mysql upgrade guide on the gentoo website (might not hurt to update it a tiny bit), and Peter Davies’s blog entry from 1.5 years ago was very helpful. If these had been pointed out before stabilizing the build the stable experience would have been a little smoother.

EC2 Custom Kernels

One minor issue with EC2 is that they supply the kernel, and that already caused difficulties with my first EC2 tutorial – the image I created doesn’t let you create a new snapshot from a running image since the EC2 kernel lacks loopback support, and I didn’t supply a matching kernel module.

Amazon has a nice guide on how to do it – here is a gentoo-specific one.

A Google Rant

I love what Google has been doing, and they’ve made huge contributions to FOSS. However, I have to join the chorus of those who are concerned with their lack of distro-friendliness.

The start of my saga was Gentoo bug 320407. Apparently Google re-bundled swt in their android SDK, and the version they re-bundled breaks sometimes.

So, the solution is to not install swt, and patch their android script so that it uses the system library. I still have to figure out which version of swt they re-bundled so that I can try to match it. Maybe that won’t be much work.

Then I need to look at all those other libraries and see which of those can go. I’ll need to patch in their paths, and I’ll need to figure out which upstream versions they re-bundled so that I can set the correct dependencies. Maybe each of those won’t be much work. Maybe some poor user will get burned when it turns out that they modified one of them and I miss it in testing.

Oh, and every time they do a new release they’re not going to tell me if they upgraded one of those bundled libs to a newer API/etc, so maybe if I’m lucky I’ll spot problems during testing and not burn users. Maybe that isn’t too much work either.

Maybe each of these things won’t be much work, but this is already sounding like a royal pain to me. It is also a recipe for end-user problems.

Let me pick my next favorite Google package – chromium. I have a chromium upgrade pending that I’ve been postponing. Building and installing chromium takes hours on my system (an old Athlon 64 3200+). Actually, building chromium probably isn’t the problem – it is building the other half-gigabyte of re-bundled dependencies that get rebuilt every time I upgrade chromium, even though I already have most of them on my system (and if I didn’t the package manager would take care of that for me – ONCE).

My hat is off to the chromium maintainers because they’ve done a good job managing it, and I understand that they’re trying to strip out the embedded libs. However, the project facing them makes my android headaches seem like a trifle.

Google – just use and list dependencies! If you want to have an alternate all-in-one package for those without package managers, feel free – other projects do it. However, if Mozilla can play nice with distros, you can do it too.

All that said, I have no objections to embedding contributed libraries in the sdk itself – the part used to build and test android apps. In this case app developers need to build and test their apps against the libraries that will be on target devices and not their development workstation. Since no code will run natively (except perhaps on an emulator) there aren’t really the usual compatibility and security issues associated with this.