Sunday, June 1, 2008

A Noob Guide to Creating a Debian Installer ISO


It is hard to find an easy to understand set of instructions for creating a custom debian installer the real way. And when I say the 'real way' I mean using the same tool that the debian folks use to create their ISOs. That is why I am going to take some time to write out a very basic set of instructions. While this set won't tell you how to do that neat-o custom thing you are trying to do that has brought you here in the first place, it will get you from 0 to custom ISO in a hurry, orienting you to the process and giving you enough knowledge to tackle some of the more interesting customizations you want to try out.

Before we get to the nuts and bolts, here is the high level of what we are going to do. We are creating a bootable debian-installer ISO. To do this we use a tool called debian-cd, which is basically a build file and lots of perl scripts that automate everything. debian-cd requires that we have a local debian repository, and that is where most of the work is. Then you configure debian-cd to point to your local repository, and you tell it what you want it to make, there are tons of options here, but I am going to give instructions for the i386 netinstaller CD. Once it is configured you just run a few make tasks and you are done. Now without further blabbing I'll cut to the chase.

Create Your Local Repository

The steps enumerated here are just taking the package set from an existing debian net installer cd and putting them into a local repository on your machine. Be aware there are many different ways to do this including using apt-move, apt-mirror, reprepro, etc. Don't be confused if you see other people doing this different ways in different places.
  1. Download a debian netinst ISO
  2. Mount it like this: mount -o loop -t iso9660 filename.iso /mnt/place_to_mount_it
  3. recursively copy the pool directory into an empty directory called debian (we will refer to this directory as our local repository from now on).
  4. To get 2 other packages (debootstrap and installation-guide) we will need in there do the following. Download this and this. Move them to where they belong in the local repository. mv /path/to/debootstrap_blah_blah.deb /path/to/local/repo/debian/pool/main/d/debootstrap/ and mv /path/to/installation-guide_blah.deb /path/to/local/repo/debian/pool/main/i/installation-guide/
  5. We also need the override files for our repository. I just grab these from the remote repository here and copy them into the local repository in a new directory called indices. Make sure you get all the ones for the dist you are trying to build (etch).
  6. Now in order to make this a repository that apt will recognize we need to create the Packages.gz file, which is basically an index of everything in our repo. We use the apt-ftparchive command to do this in the next step.
  7. create a directory called config in your local repository
  8. create a file called apt.conf like this but with your paths substituted.
  9. create a file called etch-packages.conf like this but with your paths substituted.
  10. put both those files in the local repository config directory you just created.
  11. run the command like this: apt-ftparchive -c /path/to/apt.conf generate /path/to/etch-packages.conf
  12. You should get some errors like this: E: Could not open file /path/to/debian/dists/etch/main/binary-i386/Packages.gz.new - open (2 No such file or directory). Basically apt-ftparchive needs you to make the directory structure for it. Use the mkdir -p with the path the errors give you to create all the necessary directories.
Congratulations, when you get through all that you have a valid local repository with all the packages a debian netinstall needs! On to the next bit.

Put debian-installer in The Local Repository

Now debian-cd needs us to have our debian-installer (which is basically a mini Debian image in its own right) in a certain place in our local repository. These steps accomplish that. Note that there are multiple ways to do this and you can even build your own from source for further customization. For our demonstration we will just use the stock installer from the remote repo.
  1. Download the installer-i386 directory from a remote repository (it MUST be the from the same release as the ISO you grabbed so don't go mixing different repositories in here unless you know what you are doing). use wget to grab it, like this: wget -r ftp://ftp.us.debian.org/debian/dists/etch/main/installer-i386/ (make sure you run against the ftp site. the http site gives all kinds of junk html to you).
  2. Move the debian/dists directory that you just downloaded into the local repository like this: mv ftp.us.debian.org/debian/dists /path/to/debian
  3. Now we have to create a packages.gz for the installer packages too. So create a config file like this called etch-di.conf and put it into the same directory as the other repository config files from earlier.
  4. Now run this to generate: apt-ftparchive -c /path/to/apt.conf generate /path/to/etch-di.conf
Finishing Touches on the Local Repository
  1. create a directory called tools in the root of your local repo: mkdir /path/to/repo/debian/tools
  2. create a directory called doc in the root of your local repo: mkdir -p /path/to/repo/debian/doc/FAQ/html
  3. execute: touch /path/to/repo/debian/README
  4. execute: copy the release file from the ISO: cp /media/cdrom0/debian/dists/etch/Release /path/to/repo/debian/dists/etch/Release
  5. make sure you have a directory called current under /path/to/repo/debian/dists/etch/main/installer-i386/ (I didn't have that but had a dir called 20070308etch2 and I made a simlink from current to that with this command: ln -s /path/to/repo/debian/dists/etch/main/installer-i386/20070308etch2 /path/to/repo/debian/dists/etch/main/installer-i386/current)
Great! Now our local repository is complete and we are ready to generate an installer ISO from it!

Configure debian-cd

Here is where we set about configuring debian-cd to use our local repo.

  1. Edit CONF.sh and change the export MIRROR line to 'export MIRROR=/path/to/repo/debian'
  2. Edit CONF.sh and change the export TDIR line to 'export TDIR=/path/to/a/tmp/dir'
  3. Edit CONF.sh and change the output directory line to 'export OUT=/path/to/some/debian-cd-test
  4. Edit CONF.sh and change the apt tmp dir line to 'export APTTMP=/path/to/a/tmp/dir/apt'
  5. Uncomment the line that says #export INSTALLER_CD=2
  6. Login as root: sudo su - (don't forget the minus symbol at the end)
  7. Source the CONF.sh file: source CONF.sh
  8. run the following commands in order (if you get errors from them make sure to run make distclean before you try to re-run any commands otherwise you will have a dirty build). You can also consult the README in the /usr/share/debian-cd directory for more info about what is happening.
  9. make status
  10. make packagelists TASK=tasks/debian-installer+kernel-etch
  11. make image-trees
  12. make images
That is everything!!! Check the output directory you specified in CONF.sh for the image. You will find a bootable ISO that will install Debian on your system. Please note I have tried to be as specific as possible with these instructions, but frankly it was lots of typing and remembering. If you see or better yet experience problems with these let me know and I'll be happy to fix them.

10 comments:

Rul said...

Excellent guide! Is absolutely the most complete that I could find. Thank you :)

I was able to follow all the steps, and I managed to get the ISO in the right path. The problem comes when I boot with the ISO. This is the error: "Could not find kernel image: vesamenu.c32"

Have you got any idea of where the problem can be?

Thanks in advance.

Shlomo said...

Thanks rul. I was afraid this was too much of a brain dump to be useful, but with the scattered and scanty level of documentation out there I thought this might be helpful too. Too bad it didn't work perfectly for you. :) I'd say 80% of the problems boil down to incorrect of incomplete repositories. One thing you might look at is being sure that the kernel the debian-installer is built for is the same kernel that you have in your repository. Debian installer is picky like that and only works for the specific kernel that it was created for. The vesamenu.c32 part of the error message is confusing to me and I'm not sure what it means. Hope that helps. Let me know what happens I'm happy to help in any way I can.

Rul said...

Hello!
I've tried what you told me, but the error persist. I don't know what can be, but I was able to fix it by not executing the steps of the Make.The only thing different I've done was to run the build.sh script. That was enough to generate the ISO with the entire pool.

This time, my problem is different. I've downloaded a debian mirror, and I configured it as a repository. Following the step of this guide, if I try to generate an ISO, what my logic say is that the entire mirror will be in the pool, and that is obviusly impossible.
The wiki of Debian CD has an item saying "Create List Of Files To Include On CD", but the process is not explained very well. Do you know where this list should be?

BTW, I've been reading other post of you, and they're excellent! Good job! :)

Shlomo said...

Thanks Rul, have you looked in the tasks directory? The files in there are how debian-cd decides what to put on the iso. I'm not sure the right way to generate these for debian... in ubuntu they use an application called germinate and some scripts that translate the results of germinate into the task files. Hope that helps.

Rul said...

Hi there!
Thanks for your fast answer. Today I was able to create all kind of CD using debian-cd.
And yes, you're right. I can generate my own package list by using the germinate tool.

Thanks for everything! :)

Kousik Maiti said...

I build an ISO image from the instructions. But I got the following error:-
Just after the installer identified an ISO image, I get this error:
--------------------------------------------------------------
Load installer components from an installer ISO

No kernel modules were found. This probably is due to a mismatch
between the kernel used by this version of the installer and the kernel
version available in the archive.
Can you help me?

Shlomo said...

Sounds like the debian installer you grabbed requires a different version of the kernel than you have in your repository. I think if you dig around in the debian-installer files you can find out which version of the kernel it is expecting. That would confirm it. If that is the case you could try and find a version of the debian-installer that is built for that kernel, or you could try grabbing the kernel and adding it to your repository, or you could try building your own debian-installer matched to your kernel. Those options are in order of difficulty (according to me). Wish I could be more specific, but it has been quite some time since I thought about this stuff. If nothing works out go ask some folks on the debian-installer irc channel, they could help. Good luck.

Zloy_dekster said...

# apt-ftparchive -c /root/Debian/config/apt.conf generate /root/Debian/config/lenny-di.conf
/root/Debian/pool/installer-i386: 0 files 0B 0s
Done Packages, Starting contents.
Done. 0B in 0 archives. Took 0s
# ls /root/Debian/pool/installer-i386/20090123lenny1/images/netboot/
debian-installer/ mini.iso xen/
gtk/ netboot.tar.gz
i have trouble to create packages.gz files for the installer packages((( part "create your local repository" i finished successful, my Yahoo Alexander_dhvs, thx 4 help...

Mac said...

This is amazing. I am going to work on this tomorrow and I'll post a follow up.

This tutorial is a lot easier to follow than the guides on the debian wiki : http://wiki.debian.org/DebianCustomCDand http://wiki.debian.org/DebianInstaller/Modify/CD.

seth said...

I don't get what i need to fill in in the apt.conf and etch.conf, is that an ftp site or a folder with the packages i want to be added in the install disk?