vcs-packaging

I've had a discussion on the way to LUG-Camp 2007 with Sebastian Harl about how to properly manage debian package in a VCS. We identified several ways of doing this.

Why manage a package in a VCS at all? A good rationale for this was already given on https://wiki.ubuntu.com/NoMoreSourcePackages. There are several reasons for managing a package with a VCS.

First of all, it makes contributions from non-maintainers easy. They can easily access the history of the package, and create and test and record their modifications easily. The VCS makes it then easy to examine and integrate (merge) the contribution in the tree of the maintainer (or distribution) easily.

Moreover, modern VCS offer extensive merging capabilities. Strictly speaking, every package upload to a distribution creates a “fork” of the software you want to package. To use the nomenclature of version control systems, we will call them branches. In VCS managed packages, we want to exploit these capabilities to faciliate updating to new upstreams or merging changes from other distributions.

In this mode, the VCS doesn't manage the upstream source. Only the debian/ directory of the source package is versioned.

This mode can be implemented with virtually every VCS, including RCS, CVS and SVN.

If no changes to the upstream source is done, you can upgrade to a new upstream version by just editing debian/changelog.

Changes to upstream source can be managed by using a “patch system” like dpatch, quilt or cdbs. Upgrading to new upstream version needs you to “port” every single patch. This can become tedious if larger changes need to be maintained.

In this mode you adopt upstream's VCS. If upstream happens to use the same VCS as you, you just branch at the point where upstream bases the release you want to package. If upstream uses another system, you have to import/convert it to your VCS. How this can be done highly depends on the VCS you use. In general, tailor can help you here. You can directly access any upstream revision easily to track upstream's development.

This mode can be used with any decentralised VCS. With SVN, you would need to have access to upstream's repository.

Changes to upstream can be handeled inline. When upgrading to the new upstream version, you identify the revision where upstream bases the new release, create a new branch from there and merge your packaging branch in.

It doesn't make too much sense to use a “patch system” like quilt or dpatch, because your VCS will support you with “porting” your changes to the new upstream version.

There are several problems to this approach: If the upstream history is large, it can become unfeasible to have every revision in your branch. Many VCS become very slow when managing big trees with many revisions. Also, if you don't track upstream very closely, you probably aren't interested in every upstream commit anyway.

In projects that use autotools (like xine), there is another problem: Upstream in general doesn't version generated files like Makefile.in or configure that however show up in the release tarball. In these cases you need to manually import the upstream tarball over the revision which was used as base for the release. This step is doable, but error prone.

This is a variant to the method described in the last section. Here, you don't import the complete upstream VCS, but only the contents of the release tarballs. Start with the first release, and import every upstream tarball in a new commit. This way you get an Upstream Branch, where every revision represents a new upstream release.

This upstream branch serves as base for packaging branches. A working branch is the main branch you are working on. You can do as many commits as you seem fit here. As packager, you'll mainly do changes below the debian/ directory, but you are free to patch any upstream source file. The VCS will help you maintaining your patches. If the VCS supports tags, you should tag every revision which you uploaded appropriately.

If you already have done several uploads of the current upstream version, you might want to unpack and import every upload to this packaging branch. This is not scrictly necessary, but useful to track the changes introduced by package uploads (can be interesting while triaging bugs).

Here a recipe how to convert a series of source packages to this branch concept. First, identify your package uploads on either http://snapshot.debian.net and/or http://launchpad.net. Download all foo-XX.orig.tar.gz you can find. Import each tarball to an empty branch in ascending order. You have created your Upstream Branch. Now start a packaging branch for this upstream version by importing your first upload for this upstream version. Identify and download all uploads for this upstream version. Unpack your first upload and import it. Repeat this for all your uploads for this upstream version.

Changes to the upstream source are handled inline. This is similar to Method 2: In both cases, your changes will show up 'plain' in the diff.gz.

To upgrade to a new upstream version, first you continue the upstream branch. Then merge the changes from the packaging branch for the previous version. Alternatively, you can also “replay” the changes on the revision representing the new upstream release. The difference here is that there is no “merge”, but a new series of revisions. git does offer the command git-rebase for this; whereas bzr doesn't seem to offer this functionalty.

The following picture explains how the upstream branch and packaging branch are related to each other:

bzr makes this process easy. The bzrtools package provide a command called bzr import for this, which work on both a tarball, as well as on a directory. To make a new revision in the upstream branch, you can use the following command:

bzr import package_1.2-3.orig.tar.gz

Decentralised version control systems can be of great help with maintaining packages. Their merging capabilities can be helpful when merging upstream releases. Moreover, they make contributions easy.

However, packages using patch systems do not profit from this. Here, patches to the source are not in the control of the VCS. However, those packages don't suffer either, they just don't profit. In order to make those packages profit from this, a tighter integration of patch systems is needed. One of the next articles will analyse the problem in more detail.

~~DISCUSSION~~