Showing posts with label ubuntu. Show all posts
Showing posts with label ubuntu. Show all posts

Thursday, January 1, 2009

Creating a Launchpad PPA For a Simple Python Project


For some background information on Launchpad and PPA's see my previous post. For more information about information given here check out the list of links at the bottom of this post. Now on to the bullet points:

Getting Launchpad Ready
  1. Create a new Launchpad project.
  2. Have imported your PGP key to your Launchpad account (apt uses these to verify the authenticity and integrity of packages it installs).
  3. become an Ubuntero (i.e. you must sign the Ubuntu Community Code of Conduct)
  4. activate your PPA on your profile page or your team's overview page.
  5. Build a source package for your project.
Building A Source Package For Your Python Project
  1. I followed these instructions but since we are trying to build a source package, not a binary package I made the following changes.
  2. Added a buildsrc target to the Makefile. See my makefile for specifics.
  3. Changed rules file to have a full path to the destination directory. See my rules file. I did this because Launchpad's process will be calling the rules file directly (probably via debuild) as opposed to using ghantoos's makefile which sets up the DESTDIR environment variable.
Upload Your Source Package
  1. install dput - sudo apt-get install dput
  2. edit ~/.dput.cf so it looks like
    [my-ppa]
    fqdn = ppa.launchpad.net
    method = ftp
    incoming = ~your-launchpad-id/ubuntu/
    login = anonymous
    allow_unsigned_uploads = 0
  3. Run dput my-ppa P_V_source.changes Replace P with the package name and V with the version number.
  4. Now you have to wait 5 or 10 minutes. Launchpad is spinning up a whole virtual server that will grab all the dependencies you listed using pbuilder or some such and will do a clean build of your package. An email will get sent out when your build kicks off.
  5. If you go look at your PPA your build will have a little spinny icon next to it. Here is my PPA site for example.
  6. If something goes wrong you will want to read the build logs to diagnose the problem. Click on one of the various binary builds to get to the link. Here are some sample ones from my build.
That is pretty much it. It is alot of stuff, but I hope my code serves as an example for you to get your own PPA setup. Questions are probably best answered on IRC. #launchpad on irc.freenode.net is loaded with knowledgeable types. Or you can drop your questions in the comments of this post if you have em. I'm no expert, but I might be able to help.

Those links I promised:
  • https://wiki.ubuntu.com/PackagingGuide/Complete
  • http://ghantoos.org/2008/10/19/creating-a-deb-package-from-a-python-setuppy/
  • https://help.launchpad.net/Packaging/PPA

Distribute Your Software Just Like Ubuntu With Launchpad


Launchpad is project hosting provided by Ubuntu. It is free for projects with an open source license. The service consists of distributed version control, bug tracking, team collaboration and other features too numerous to list. The most unique feature and the one under consideration here is the PPA, "Personal Package Archive". This feature allows you to publish your software to users using the standard ubuntu software installation and update tools (apt/synaptic). Imagine finishing a new release of your software, and at the click of a button Launchpad builds your software for multiple architectures, and notifies your users that updates are available. Brilliant! Before PPA's, maintaining a standalone package repository was out of the realm of what mere mortals could accomplish. Now launchpad has removed nearly all the barriers to entry. In my opinion this makes Launchpad the best project hosting out there.

It isn't all beer and skittles though. There is much to know about building packages for ubuntu. I had to spend the better part of a day learning about how debian packages work in order to get my PPA up and running. (It was worth the effort). And launchpad does have its warts. The user interface isn't very nice when compared to git-hub or lighthouse, and with the large number of features packed in there, it takes too long to figure out how to do something for the first time. Also, Launchpad can seem sluggish, especially the bug tracker, which can be infuriating if you are updating lots of bugs.

To sum it all up Launchpad isn't perfect, but it is the only solution for all-in-one project hosting: distributed version control, issue tracking, collaboration and software distribution. In my mind git-hub, lighthouse and google code are the only real contenders for the title. Git-hub has a great UI but doesn't offer issue tracking. Lighthouse has another great UI, but doesn't have version control. Google code is wonderfully simple, but doesn't offer distributed version control. If only I could combine the best parts of Git-hub, Launchpad, Lighthouse and Google code...

Coming up... Based on my own experiences of creating a PPA on launchpad for a project of my own I am going to write a walkthough of setting up your own PPA. However, I am going to make it a separate entry. If you are interested just read my next post.

Wednesday, June 11, 2008

Do it Yourself Ubuntu Installer ISO's

This picture has a lot to say, but it says nothing whatsoever about Ubuntu installer ISO's. Who cares. This picture is so awesome I am considering using it to illustrate every blog entry I do from now on. "On the latest humanitarian crises" illustration: Ike's middle finger. "On alternative energy" illustration: Ike flying the bird and Tina. "Google's newest web framework" illustration: Ike's studded leather belt and his middle finger. Tell you what. In the spirit of the illustration i'll try to keep this post as bad ass as possible.

Listen up: A while back I blogged about debian-cd and creating a Debian net installer ISO. Imagine creating an entire operating system from scratch! Ok, not quite from scratch since you just download binaries, but hey, when you bake bread you don't mill your own flour do you? I didn't think so. So shut up. That previous debian-cd trip was just one stop along the road to creating a Xubuntu installer ISO from scratch and that is what this post will be about.

The most important thing to know about creating a Xubuntu installer is that there is a guy named Colin Watson who makes it all happen, ok so there is a whole army of people, but Colin's name is on everything. He is like the Ike Turner of the debian-installer set. Here is a link to the back room of his fun house where all the image making magic happens. He has bazaar branches of the following projects: cdimage, debian-cd, germinate, britney. You should check out branches of all those things with bzr at the link I gave you. The next thing to do is put them together. cdimage is the set of scripts that drives the whole thing, it handles synchronization of your local repository, generating the debian-cd task lists from seeds, calling debian-cd to burn the ISO's, publishing the ISO's and some other good stuff. So to "put them together" just drop debian-cd, britney and germinate into the cdimage directory.

Unless you think you are some kinda hotshot it would do you good to read the README that comes with each of these four tools, especially the cdimage one. If you don't, you better not come crying to anyone on the mailing list, because they will rip your arms off and beat you with them. It happens. I wish I could try to give exact directions for making this work, but there are too many steps and I forget things, so my recommendation is to immediately start trying to run cdimage, and resolve all the problems and complaints it has one at a time. Mostly you will just be figuring out what environment variables need to be set. (There are about 6 or 8 I set. I created a script with an export for each one and I source that before I run cdimage.) Here is the command for running cdimage: ./for_project xubuntu ./cron.daily

Great! You scrolled this far down the page so you must still have arms, or you are working your mouse with your mouth, either way, good job. Once you get through the first few errors (for not having the right variables defined) you will notice that the command is going to take forever to run. This is because it tries to rsync your local mirror with a remote mirror (Mine is about 100 GB for all the i386 stuff). Since you probably don't have a local mirror you probably can't skip this step, just rest assured that once you have it, you can re-use it and things will happen much faster, say 5 minutes or less. If you already have a local mirror you can just point the script to use that one . (I just put a symlink in the cdimage directory called ftp and it points to my local mirror at /opt/mirror.) Try not to get frustrated if you get stuck. There are lots of moving parts in here! LOTS! You are probably just missing some environment variable or you have one set to point at the wrong directory. The best way to work through things it is to read the source. Most error messages are pretty descriptive, but RESIST the temptation to hack the code. Colin's stuff really works. I got it all working with only one minor change to build_all.sh in debian-cd. For posterity, here is a list of the changes, tips and tricks that might help:

  • when debian-cd kicks off via the build_all.sh script it always complains that it can't find CONF.sh even though it is clearly in the right place. I change the ". CONF.sh" line at the beginning of the file to ". `pwd`/CONF.sh" and that fixes it.
  • If you already have a local mirror you can skip all the rsync business. To do this edit anonftpsynch in the cdimage bin directory and comment out the two rsync lines. I also comment out the lines that synch the indicies and overrides.
  • Britney is partially C code and you have to build it. Make sure you have the python-dev-all (I think that is the name) package as well as the one listed in the README. There is a step in there with some perl that I didn't do, but it didn't seem to matter.
  • Getting a local copy of the seeds really speeds up your build time a lot, so do it! You can set the LOCAL_SEEDS environment variable to the path of your seeds. If you make changes to the seeds be sure to commit them or germinate ignores your changes.
  • You can disable the creation of the Jigdo (stands for jigsaw domino, which is awesome) files if you don't care about them and save a bit of time and disk. Just edit debian-cd's CONF.sh and set the JIGDOsomething-something environment variable to 0.
  • Obviously you don't need 100GB mirror to create a 700 MB CD image. There are techniques for creating small local mirrors. None of them very handy. I'm using extensive --exclude parameters in my rsync command to make sure I only get i386 binaries. Check out this wiki page for info on other techniques.
  • It is imperative to use debian-cd and germinate from colin's repository. You can't mix and match the packaged versions with cdimage because colin has modified things quite a bit, so stick to his sources in bazaar for everything.
  • Fri. June 13th Edit: Be sure you have your gpg keys in order and cdimage is properly configured to use them or your APT repo on the CD won't be trusted, so nothing will get installed, which is kind of a bummer.
  • Make sure your GPG key you are using doesn't have a password, because GPG is running in batch mode when the CD is signed and thus won't be able to prompt you for a password.
Whatever you do don't get discouraged. I recommend asking questions on the ubuntu-installer IRC channel and mailing lists. People don't go out of their way to help you, mostly they ignore you, but sometimes you get lucky. Also, take time to pull back and understand the big picture about how the installer works and what all the files going onto the installer are for. That sort of context can help you avoid mistakes and direct your research. Good luck. And remember what Ike Turner said: "In China, when you get to the airport everyone be talking in American slang."

Wednesday, January 2, 2008

Disk Usage Analyzer


Have you ever wondered where that other 140GB of your Hard Disk went to? For ages I have relied on the venerable "du -sh *" command or the "df -h" command to answer me that question. Today after a conversation with a colleague who uses GrandPerspective on his Mac I felt obligated to find the Linux answer to GrandPerspective. I quickly stumbled upon Baobab which is Gnome's answer. It is bundled with Ubuntu and can be found under applications > accessories > disk usage analyzer. The Baobab idiom for displaying your disk usage is, in my mind, nicer than the Treemap used by GrandPerspective. As the picture indicates it displays your folders in this cylindrical graph. What the picture doesn't show is that it is interactive, so you can hover over a particular part of the graph and get the folder name and size. I also like the listing of directories down the left side which are ordered by size. The killer feature is that it will let you scan remote drives via samba, ssh, ftp or whatever.

Now that I am done writing this up I find myself wondering why I even bothered with this. My command line judo clearly solved the problem at hand. It was only a moment of insecurity that obliged me to prove once again that Gnu/Linux is the best platform out there, by finding a solution to the problem as flashy and whiz-bangy as the one on the Mac. How embarrassing, and I still prefer the wonderful obscurity of the command line. But for those of you who don't, know this, Baobab works fine and it looks prettier than GrandPerspective nyah, nyah, nyah.

Thursday, August 2, 2007

Annoying Weirdness in Mailcap File


Ever since i upgraded to Ubuntu 7.04 I have been unable to successfully run all the unit tests for our project. One particular test kept failing in a part of the system that handles email. Here was the stack trace I kept seeing.

java.lang.NullPointerException
[junit] at java.lang.String.concat(Ljava.lang.String;)Ljava.lang.String;(Unknown Source)
[junit] at com.sun.activation.registries.MailcapFile.parseLine(MailcapFile.java:234)
[junit] at com.sun.activation.registries.MailcapFile.parse(MailcapFile.java:198)
[junit] at com.sun.activation.registries.MailcapFile.createMailcapHash(MailcapFile.java:158)
[junit] at com.sun.activation.registries.MailcapFile.(MailcapFile.java:47)
[junit] at javax.activation.MailcapCommandMap.loadFile(MailcapCommandMap.java:171)
[junit] at javax.activation.MailcapCommandMap.(MailcapCommandMap.java:125)
[junit] at javax.activation.CommandMap.getDefaultCommandMap(CommandMap.java:54)
[junit] at javax.activation.DataHandler.initDefaultCommandMap(DataHandler.java:149)
[junit] at javax.activation.DataHandler.getDataContentHandler(DataHandler.java:615)
[junit] at javax.activation.DataHandler.getContent(DataHandler.java:540)
[junit] at javax.mail.internet.MimeMessage.getContent(MimeMessage.java:1068)


I figured out that there is a file called .mailcap in my home directory. In my case, for whatever reason, the first line of the file had some junk in it. There was a -e there. By deleting that line of the .mailcap the test passes. Totally lame. I don't know if this is an Ubuntu thing or PEBKAC, but at least it is easy to fix (if you know what is wrong).

Tuesday, June 26, 2007

How to add search domains in Ubuntu


I have lived with the following annoyance for the past year. At work I can't resolve various server names without editing the /etc/resolv.conf file and adding the following
search www.example.com
No big deal, except every time you connect to a new network this file is automatically overwritten with information received from DHCP. This is really annoying and after adding that line by hand for a year I finally got a solution!

It is really simple. Edit /etc/dhcp3/dhclient.conf and add the following line in there:
prepend domain-name "thoughtworks.com ";
duh! Many thanks to my friendly neighborhood sysadmin and this webpage