Colors of Noise

I've started uploading snapshots of git-buildpackage to experimental recently. Here's a short list of what changed over the version in wheezy and sid:

• git-buildpackage now has a --git-pristine-tar-commit option that commits a tarball right back to the pristine-tar branch if it's not already there. This makes tracking upstream git and dfsg-clean source handling easier.
• git-buildpackage's --git-upstream-tree now accepts a tree directly without the need to use --git-upstream-branch.
• git-buildpackage's --git-export now parses the exported changelog instead of the working copy. This way we automatically pick up the correct tarballs, branches, etc. reducing the number of options needed.
• Jan Čapek added a --git-postexport hook to allow to mangle the source after exporting it.
• gbp-pull and gbp-fetch now support shallow clones thanks to Markus Lehtonen.
• gbp-pull got a speed up by skipping the checkout for fast-forward. This makes updating large source trees less time consuming.
• git-import-orig and git-import-dsc can now import into bare repositories.
• Russ Allbery updated git-pbuilder to 1.27.
• gbp-pq should be easier to use thanks to the --force and --switch options. It also handles patches without descriptions more robustly.
• The test coverage of the internal test suite increased quiet a bit. We're now generating coverage information during the build in build/cover.
• We're now generating apidocs which also serves as usage examples.

This blog is flattr enabled.

In order to ease creation of remote repositories to publish your Debian packaging work git-buildpackage ships gbp-create-remote-repo: If you run

gbp-create-remote-repo --remote-url-pattern='ssh://git.debian.org/home/users/agx/public_git/%(pkg)s.git'

from inside a git repository it will create a remote repository on git.debian.org in the given directory, push your debian, upstream and pristine-tar branches into it and set up branch tracking so you can easily use gbp-pull or git pull to update your local repository from there. '%(pkg)s' will be replaced by the source packages name. If you want to be more explicit you can use:

gbp-create-remote-repo --remote-url-pattern='ssh://git.debian.org/home/users/agx/public_git/foo.git'

so no substitution will take place. In case you only want to push to that repository but not pull from there you can use the --no-track option so gbp-create-repo won't set up any branch tracking.

If you specify the remote-url-pattern via gbp.conf calling:

gbp-create-remote-repo

is enough to create a repo on your favorite git server. The default is to create new repositories in the collab-maint repository on alioth.debian.org.

This feature is available in your $PATH since 0.5.16 (it was shipped in examples/ before) and was inspired by the Debian OCaml Maintainers dom-new-git-repo work.

This blog is flattr enabled.

Since waiting for long running builds is boring and having to check if the build already finished distracts one from doing other stuff, git-buildpackage 0.5.16 reports the build status of a package via desktop notifications if python-notify is installed:

To disable this feature use:

[git-buildpackage]
notify = off

in your ~/.gbp.conf.

This blog is flattr enabled.

To make xul-ext-nostalgy work with icedove 3.1 I finally wanted to update to a new upstream version. The git repository is tracking upstream's svn via git-svn which works great but when I typed

git svn rebase

I got a

Connection refused: Can't connect to host 'frisch.fr': Connection refused at /usr/lib/git-core/git-svn line 2310

The svn upstream repo moved to google code. It took me a moment to figure out that this can be fixed easily with:

 git config svn-remote.svn.url http://nostalgy.googlecode.com/svn/
 git config svn-remote.svn.rewriteRoot svn://frisch.fr/nostalgy
 git svn rebase

Just replace the url with the new svn repository url and set rewriteRoot to the old svn repository url and you're done.

This blog is flattr enabled.

When reviewing packages from mentors.debian.net I like to have them under version control so I can easily compare different versions, write patches and clean up autogenerated files. So when looking at a package I do a

git-import-dsc --download http://mentors.debian.net/debian/pool/main/x/xul-ext-firetray/xul-ext-firetray_0.2.8-1.dsc

This command first fetches the files via dget from devscripts and then imports the upstream sources onto the upstream branch (upstream by default) and the debian modifications onto the debian branch (master by default). I can then invoke git-buildpackage as usual. Since this works incrementally I can import newer versions into the same repository and use git diff to see the changes.

If I'd like to hack on a random Debian package (say xul-ext-nostalgy) I usually use:

git-import-dsc --download xul-ext-nostalgy

This uses apt-get source to fetch the source package and imports it into git using the same branch layout as described above so I can use all the familiar git commands to generate patches, etc..

This blog is flattr enabled.

Let's say you want to clone a git repository that has the debian packaging on master and the upstream source is kept on upstream/master (since it tracks upstream's master branch). You can simply use git clone or you can do a

gbp-clone --upstream-branch=upstream/master --pristine-tar git://git.debian.org/git/pkg-lvm/multipath-tools.git

This will additionally set up master, upstream/master and pristine-tar to track the corresponding remote branches. There's no --debian-branch option in the above example since master is the default.

If you want to safely update these branches at a later point invoke

gbp-pull --upstream-branch=upstream/master --pristine-tar

from inside the repository. This will only update the branches if they can be fast-forwarded. If the repository has a debian/gbp.conf describing the branch layout a call to

gbp-pull

is enough.

Many team maintained packages in Debian are maintained in Subversion which makes a lot of sense if people agreed on a common workflow. If you want to use git to work on these packages you can still do so with a bit of configuration without changing anything for the other team members. Let's take system-config-printer as example. git-svn can be used to checkout the svn repository:

git svn clone svn+ssh://svn.debian.org/svn/pkg-gnome/ -T packages/unstable/system-config-printer -t tags/system-config-printer system-config-printer

Since the pkg-gnome team keeps all the packages in one huge SVN repository and splits the directories by release first and then by package the above command uses -T to point git-svn to system-config-printer's trunk and -t to point it to the tags directory. In simpler repositories where trunk, branches and tags are subdirectories of the same base directory using --stdlayout instead is enough.

Since in SVN based packaging the upstream sources usually aren't kept in the repository git-buildpackage needs to know where to find them and how to combine things in the build-directory:

cd system-config-printer
cat <<EOF >.git/gbp.conf
[git-buildpackage]
# Build in a separate build-area
export-dir = ../build-area/
# Look for tarballs in ../tarballs
tarball-dir = ../tarballs/
# Overlay the upstream sources on top of it
overlay = True
# Don't use pristine-tar
pristine-tar = False
# Compression type of the tarball
compression = bzip2

Like with svn-buildpackage the orig-tarball is expected to be found in ../tarballs where you should put it now e.g. using the get-orig-source target of debian/rules (yes, git-buildpackage should support this implicitly). You can then work on the package as usual using all the nifty git commands like git-rebase, etc. and build the package (in this case using git-pbuilder):

git-buildpackage --git-pbuilder

or if you don't want to export the current branch's HEAD of your git repository but your working copy (which might contain uncommitted changes) to the build-area instead use:

git-buildpackage --git-pbuilder --git-export=WC

The built packages can then be found in ../build-area. Once the package is ready pushing the changes into the SVN remote repository is as simple as:

git svn dcommit
git svn tag -m"Tagging system-config-printer (1.2.3-0.2)" 1.2.3-0.2

The last command appropriately tags the new version. This saves a bit of typing. In order to fetch changes others have made use:

git svn rebase

Git-buildpackage 0.5.6 in experimental got rid of it's pbuilder/cowbuilder examples and added Russ's nice git-pbuilder script. I'm very happy how this turned out, now building with cowbuilder is as simple as:

# Initially create a cowbuilder environment once
git-pbuilder create

# Build the package
git-buildpackage --git-pbuilder

# At a later point update the cowbuilder environment with new versions
git-pbuilder update

Or for another distribution:

DIST=lenny git-pbuilder create 
git-buildpackage --git-pbuilder --git-dist=lenny

The new version also better interacts with dch from devscripts. So things like:

git-dch --bpo|--qa|--nmu

are now also supported when adding new changelog sections. Git-dch now detects new upstream version numbers automatically and bumps the version accordingly. In order to get this working I finally fixed the tag format.

Since the current version in unstable should get a chance to move to testing first, I've uploaded 0.5.6 to experimental.

Git-buildpackage 0.5.0 in experimental fixes quiete some bugs related to the import of upstream sources and Debian source packages. By getting rid of a complete source tree copy it also got about 50% faster when doing so. Since the code changed quiete a bit in that area it'd be great if some of you could give it some testing before I'll push it to unstable.

Here's a short overview of 3.0 source format support in git-buildpackage 0.4.64:

• git-import-dsc & git-import-dscs import 3.0 (quilt and native) format packages.
• git-import-orig: imports gzip and bz2 compressed archives. Other formats will be enabled once pristine-tar supports them (#499489).
• git-buildpackage handles creating gzip, bzip2, lzma and xz compressed tarballs with the --git-compression option although it could get a bit smarter and autodetect the compression type.

One loose end is the handling of multiple upstream tarballs (#561071, #561072). I'd be great to hear from people already using this feature how this can best be added into the gbp workflow.

The newer kids in town gbp-clone, gbp-pull (for keeping the repo up to date when using team maintenance) and gbp-pq (for managing the quilt patch-queue) don't care about the underlying source package format. These three are a bit underdocumented in the manual but information about the workflow is in the wiki.

As of git-buildpackage 0.4.60 gbp-pull supports --redo-pq which additionally recreates the patch-queue branch after a pull. Now you can keep your clone fresh with one command.

Inspired by tools developed by Debian's OCaml maintainers git-buildpackage now ships gbp-pull that can be used to keep your debian-, upstream- and pistine-tar branch up to date when doing team maintenance.

I added a short explanation on howto handle backports with git-buildpackage to the GBP workflow page.

git-buildpackage 0.4.57 adds gbp-clone to /usr/share/doc/examples/. If you use it instead of git pull it will automatically set up your upstream and pristine-tar branches. This was inspired by the helper scripts the ocaml team uses.

git-import-orig now calls a postimport hook which can be used to call git-dch after import so I could remove the hardcoded dch call.

git-buildpackage 0.4.55 now ships gbp-pq for easier serializing of git patch queue branches into quilt patch series and vice versa. It's currently in /usr/share/doc/git-buildpackage/examples/. I'll move it to /usr/bin/ once I feel a bit more confident that this is the right interface.

As of version 0.4.53 git-buildpackage's git-import-orig supports --fast-import. This uses git fast-import to import the new upstream version and speeds up imports by about a factor of two.

It's not as well tested as importing without --fast-import so please be careful and file bugreports if it fails.

As of version 0.4.47 git-buildpackage exports these environment variables to a called posttag hook:

• GBP_TAG: the tags name
• GBP_BRANCH: the branch you were on when creating the tag
• GBP_SHA1: the sha1 of the commit the tag was created at

This allows you to write posttag hooks that push out only the changes that correspond to the created tag. For a simple example see the GBP Manual. Using such a hook makes team maintenance a lot easier since you can't forget to push out the commits and tags for a newly uploaded package while making sure you don't push out any other changes.

When following Debian changelogs (e.g. via apt-listchanges or update-manager) one often wonders what an actual commit messages actually means or how a bug got actually fixed. Currently finding this information is cumbersome:

• Find out the version control system of the package, either via the PTS or via debian/control.
• Browse it's VCSs revision history either by using the web interface or by cloning the repository.
• Try to locate the correct commit from it's commit message. Hopefully it's the same as in the changelog and hopefully it mentions the bug number - otherwise: tough luck!

However this process can be automated if a package uses the following format for changelog entries:

* [commitd] changelog message

e.g.:

* [fed3f3d] fixed segfault during daemon startup (Closes: #7005180)

then you can use Cl2vcs to locate the exact commit that corressponds to that changelog entry. Cl2vcs uses the PTS to parse the packages Vcs-Browser URL to create a link back to the commit.

If you maintian your package in Git, creating this type of changelog entry is simple: You can tell git-dch to include N digits of the commit's SHA1 (the commit id) using the --id-length option, e.g.:

git-dch --release --auto --id-length=7

This way git-dch will include the first 7 digits of the commit's SHA1 (this options can of course also be specified in gbp.conf). Packages already using this are e.g. libvirt and calendarserver.

Note: a possibly useful extension of the above format would be:

 * [commitid1,commitid2,commitid3] changelog message 

so several commits can make up a changelog entry. For Subversion this would then look like:

 * [r123,r125.r127] cleanup whitespace errors

Nothing earth shaking in this release:

• [bf8738f] add --git-tag-only (Closes: #485114)
• [b06bcf0] document git-import-dscs
• [c1deeb2] add epoch to dch call (Closes: #483718)
• [6640ac7] Fix typo (Closes: #484243)

''--git-tag-only'' allows you to create a tag without having to build the whole project again - useful for large packages. You can either clone from:

git clone http://honk.sigxcpu.org/git/git-buildpackage.git/

or fetch the package from the Debian Archive once it went through the new queue. The updated documentation is available here