EC2 Custom Kernels
One minor issue with EC2 is that they supply the kernel, and that already caused difficulties with my first EC2 tutorial – the image I created doesn’t let you create a new snapshot from a running image since the EC2 kernel lacks loopback support, and I didn’t supply a matching kernel module.
Amazon has a nice guide on how to do it – here is a gentoo-specific one.
These instructions start from a working ami image created using my previous instructions. Go ahead and mount it and chroot into the image.
- Install a kernel using emerge xen-sources
- Be sure to enable xen support in the kernel config file. Here is a working config.
- Build the kernel with make && make modules_install as usual
- Copy the kernel image to /boot as usual.
- Create /boot/grub/menu.lst, containing:
default 0
timeout 3
title EC2
root (hd0)
kernel /boot/kernel root=/dev/sda1
- While optional you might want to clean up the kernel source directory, or even remove it – you’re paying by the gb and the sources and object files are large.
- Bundle it up with the instructions in the first guide, but on the ec2-bundle-vol command line add –kernel followed by a suitable kernel ID from the pdf linked above. For a 64-bit ami in the US-east region use aki‐427d952b
That’s it – upload, register, and boot it up!
I am currently maintaining two images at the moment:
64-bit: ami-4047b929
32-bit: ami-ae49b7c7
Both are running stable kernel and udev, and I’ll update them periodically for a while.
On both images I have an entry in root’s ssh authorized_keys so that I can more conveniently access these instances when I’m using them. If you use this for anything other than experimentation you should remove this before creating your image. I’d host a version with and without this, but that does cost money on S3, so consider yourself warned…
Did you left your public key at root’s .ssh/authorized_keys intentionally?
zed_0xff
July 27, 2010 at 5:42 am
Yes – sorry, no security hole intended. It was mainly for convenience so that I can just use my existing ssh identity. If using this in production I would create a new image and clean out the authorized key file. I’ll add a note to the article as a disclaimer. Thanks for catching this.
rich0
July 27, 2010 at 9:45 am
I’m no gentoo or ec2 expert, but simply attempting the ec2-bundle-vol command the loopback problem error suggested simply: modprobe loop
After doing so the ec2-bundle-vol worked and the AMI I built from it seems to be working fine.
So are the instructions in the post at all necessary?
Joel
July 29, 2010 at 5:16 pm
Are what instructions necessary? What did you start with, and what did you do, to end up with something that worked?
The post is about how to build your own custom kernel and get it running. If you don’t want a custom kernel you don’t need to follow this post. If you do want a custom kernel, then this is how you’d go about it (or some similar variation on this).
The instructions I gave in my first post resulted in a kernel that didn’t have a working loopback module, so a modprobe loop would not have worked. Now, if you subsequently booted one of the ami’s listed above that were created using THIS post then it would have a working loopback module, and so a modprobe loop should work fine.
Indeed, if you check the latest ami you’ll find from the bash history that I rebundled it from another ami via ec2, confirming that loopback support now works.
Sorry if it is all a bit confusing – I no longer host the ami’s with the defective loopback module in favor of the ones created using the procedure above.
rich0
July 29, 2010 at 6:59 pm
Thanks. Like I said, not an expert. I used one of your AMIs sort-of from this post. More like what I found under the owner ID on the public AMIs (the latest 32bit AMI you put out there I believe).
That explains it then. Appreciate the clarification.
Joel
July 29, 2010 at 7:04 pm
Yup – those ami’s were created using this procedure, so they should “just work.”
In fact, I’d say that other than being xen guest’s there are about as genuine gentoo as they get. These ami’s are basically a vanilla gentoo install comparable to what you’d get on a local install. You can of course also emerge away and then bundle a new ami from the running image to tailor it.
rich0
July 29, 2010 at 9:41 pm
I am confused. Is this a fully running gentoo ec2 machine or is this the a stage 3 to start from? (the two images that you are maintaining)
Thanks,
Nadin
September 23, 2010 at 10:13 pm
They’re fully-running ec2 machines, that are basically just stage3s with just a few extra packages. They’re not fully-configured servers or anything like that – they aren’t intended for any particular use/etc.
Feel free to make use of the images as a starting point, or follow the instructions to create your own…
rich0
September 24, 2010 at 6:06 am
I am having trouble getting grub to work.
My machine boots to the grub menu and then stops.
Grub.conf
default 0
timeout 1
title EC2
root (hd0)
kernel /boot/kernel root=/dev/sda1
EC2 System Log
Failed to read /local/domain/0/backend/vbd/250/2051/feature-barrier.
Failed to read /local/domain/0/backend/vbd/250/2051/feature-flush-cache.
1835008 sectors of 0 bytes
**************************
[H
[J
GNU GRUB version 0.97 (1740800K lower / 0K upper memory)
[ Minimal BASH-like line editing is supported. For
the first word, TAB lists possible command
completions. Anywhere else TAB lists the possible
completions of a device/filename. ]
grubdom>
[9;10H
Nadin
September 29, 2010 at 5:46 pm
Hi Nadin
I’m having the exact same problem, did you find the reason why you get this ?
My setup is basically a debian squeeze, using the xen latest xen kernel found in repository (2.6.32-5-xen-686).
Am I missing some ec2/xendomU patches here ?
Also, following instructions from amazon pdf on running custom kernel, I chosen option 2 at start to configure GRUB, and made up my own menu.lst without providing grub binaries (which are said “not necessary”).
My next steps are :
– try it with grub binaries
– compile my own kernel applying domU patchset from gentoo (I highly doubt I’ll be successful there)
– find a guru which solved it.
romain
November 2, 2010 at 1:16 pm
Any success?
Nadin
November 23, 2010 at 9:14 pm
I was having the same trouble various grub configs all producing the same error – I remove /boot entirely, same error.
I am assuming then, that pvgrub can’t see the grub install on the snapshot. Not entirely sure why, but i thought i read somewhere that the partition that /boot is on needs to be ‘bootable’.
This led me to create a new empty volume, which I this time partitioned (just as 1 partitition across the whole block), and set the partition to bootable then synced the filesystem onto that partition and then snapshot this volume for us in the ami. This didn’t fix it initially, but then I also added the line:
(hd0) /dev/sda
to /boot/grub/device.map
Now it gets past grub, but I’m hitting a kernel panic, cannot mount VFS blah blah.
I’m not sure that /dev/sda exists once grub tries to boot to stage2. Maybe its because the scsi stuff is compiled as a module and I think it might need to be in the kernel to be able to boot.
Simon Bettison
December 23, 2010 at 6:51 pm
“Maybe its because the scsi stuff is compiled as a module”
Quite the opposite, in fact.
CONFIG_SCSI=n
in kernel config.
Then create your AMI with
–root-device-name /dev/sda
(not the default sda1)
So to recap your volume/snapshot is a block device, that contains a bootable partition (so that pvgrub can boot it)
Your AMI can boot using one of the pvgrub kernels (no intird necessary) and you must specify the root device as sda not sda1.
phew.
Simon Bettison
December 28, 2010 at 5:39 pm
I also had the same issue. In my case I was using EBS backed AMI which was failing to boot with EBS PVGRUB kernel (aki-4e7d9527), I decided to use non-EBS based PVGRUB kernel (aki-427d952b) and it worked like a charm. No need to switch /dev/sda1 to /dev/sda.
HTH
Ashish
March 19, 2011 at 10:02 am
Thanks for this post! It made it easy for me to change my own AMI to one with an included kernel.
Jeremy
January 7, 2011 at 9:44 pm
We have been successful in booting a custom patched kernel (cluster filesystem patches)for centos 5.5. This blog entry was very useful !! Thanks!!
Here is the link talking about the steps we followed: http://clogeny.com/index.php/clogeny-blog/2011/01/21/12-booting-a-custom-kernel-ec2
Chirag
January 27, 2011 at 6:40 am
Dear Friends,
I have follow-up the above url and pdf file, Please help me, How to configure in slackware13.1 linux with chain-boot kernel.
I am using this aki (aki-407d9529), But i am unable to login the server getting error.
ec2-get-console-output i-ajht0c9
i-ajht0c9
2011-01-29T06:41:40+0000
Please help me.
Ramesh
Ramesh
January 29, 2011 at 4:02 am
I’ve noticed that your AMI’s can only be launched as large instances or higher – the same with my AMI’s. Is there any particular reason why we can’t use small instances, and is there any way to enable them?
Jim T
February 13, 2011 at 6:15 pm
The 32-bit AMIis can run as small instances. The 64-bit ones have to run as large ones (or higher).
This is an EC2 limitation – the different instance sizes run different architectures.
To run the micro instances you need an EBS-based image, and I haven’t gotten around to figuring out how to make one of those…
rich0
February 13, 2011 at 6:37 pm
For who is interested in getting up a micro istance of ec2 this link explain how to migrate an s3 image to ebs.
http://coderslike.us/2009/12/07/amazon-ec2-boot-from-ebs-and-ami-conversion/
I personally skip some steps (basically download and unbundle image), just copied to ebs volume.
from this script:
rich0, what do you think about to write down this article to wiki of gentoo? It will great, that this instructions get more visibility and a place to collaborate and tune settings.
m47730
February 18, 2011 at 12:33 pm
thanks so much for you guide!
how did you manage to have it show up as “Gentoo” instead of “Other Linux”?
does it have to do with the –productcodes variable? which code would Gentoo be? 🙂
thanks
nutz
nutz
July 25, 2011 at 12:12 pm
I actually have no idea. Either it somehow parsed it out of my image filename, or maybe somebody at AWS spotted the image and tagged it or something.
rich0
July 25, 2011 at 10:22 pm
huh – weird .. okay – i’ll let you know when i find out 🙂
nutz
July 26, 2011 at 8:58 am
okay. i figured it out and it’s no witch craft:
if there is a “gentoo” in the –name (from ec2-register) it’ll show you the neat Gentoo symbol 🙂
nutz
July 27, 2011 at 10:27 am
Great work! Unfortunately, the images can’t be used it the free tier because “launching a t1.micro instance requires that you select an AMI with an EBS-backed root device.”
John
September 5, 2012 at 9:46 am
Yup. See my most recent post, based on the work of edowd on bootstrapping gentoo on EBS. Those will work fine on a free tier.
rich0
September 24, 2012 at 10:29 am
[…] previously written on building a Gentoo EC2 image from scratch, but those instructions do not work on EBS instances without adjustment, and they’re fairly […]
Gentoo EC2 Tutorial / Bootstrapping « Rich0's Gentoo Blog
September 24, 2012 at 10:20 am
Hi,
1. I have installed slackware14.0 (64bit) in my local machine.
2. I have created the 10 gb image space in slackware machine using below command. (dd if=/dev/zero of=slack14.img bs=1M count=10075). and mount the image in slack14.img. mount -o loop slack14.img /mnt/slack1464.
3. I have format the image (slack14.img)
4. I have installed the custom package through ruby script. The custom package for installed without any error.
After that While login the mounting image (/mnt/slack1464). and I have installed the package.
and bundle the image with this aki (aki-427d952b) and upload to amazon.
While start the instance.
I am unable to login the server, I am getting below error
6535502.145187] ip_tables: (C) 2000-2006 Netfilter Core Team
[6535502.145235] TCP cubic registered
[6535502.145244] NET: Registered protocol family 17
[6535502.245110] XENBUS: Device with no driver: device/console/0
[6535502.247428] EXT3-fs: barriers not enabled
[6535502.257460] EXT3-fs (xvda1): mounted filesystem with writeback data mode
[6535502.257484] VFS: Mounted root (ext3 filesystem) readonly on device 202:1.
[6535502.257779] Freeing unused kernel memory: 484k freed
[6535502.257953] kjournald starting. Commit interval 5 seconds
[6535502.471724] mount used greatest stack depth: 4296 bytes left
[6535512.662690] touch used greatest stack depth: 4120 bytes left
[6535576.762574] xenbus_dev_shutdown: device/console/0: Initialising != Connected, skipping
[6535577.114477] Restarting system.
earlier I have created the .img image in (slackware13.1 & 13.37) without any error.
But I am getting the error in Slackware14.0 64bit and 32bit only.
Thanks for advance.
BY
DAVID
david
November 21, 2012 at 8:11 am