Debian's libvirt in unstable finally passes the Technology Compatibility
Kit (libvirt-tck) for qemu:///system:
Files=59, Tests=1579, 315 wallclock secs ( 0.77 usr 1.02 sys + 31.40 cusr 10.14 csys = 43.33 CPU)
We're running libvirt's internal test suite since 0.9.0 but this doesn't launch
any real virtual machines to check things like suspend, resume, snapshotting,
migration and it doesn't create storage pools and volumes or networks. This
means a lot of testing was done manually with each release.
libvirt-tck provides a framework to perform these kinds of integration testing
between libvirt and it's drivers, comes with hundreds of testcases already and
it's easy to set up since it's available in experimental:
apt-get install -t experimental libvirt-tck
Edit /etc/libvirt-tck/default.cfg if you don't want to use the default
connection URI. Then just run libvirt-tck:
Since this is supposed to stress libvirtd and the hypervisor and since it
creates and deletes storage pools, networks and virtual machines it's
recommended to run this on a dedicated system to make sure the tests don't blow
away any precious config.
If a test fails one can debug that single test using:
LIBVIRT_TCK_CONFIG=/etc/libvirt-tck/default.cfg prove --verbose /usr/share/libvirt-tck/tests/hooks/051-daemon-hook.t
This already revealed errors in our package like a missing parted build
dependency breaking the creation of disk based storage pools, bugs in
libvirt, crashes in QEMU and minor errors in the test suite itself.
We're currently cheating a bit since the tests for hooks and nwfilter are
currently disabled due to license problems with the used Perl modules.
libvirt-tck can easily be integrated into Jenkins since it can now
use libtap-formatter-junit-perl to generate JUnit XML. To do so simply
configure a free-style software project to execute these commands:
rm -f libvirt-tck.xml
sudo libvirt-tck --timer --format junit --force > libvirt-tck.xml
allow Jenkins to run libvirt-tck as root
jenkins ALL=(ALL) NOPASSWD: /usr/bin/libvirt-tck --timer --format junit --force
and configure it to publish a JUnit test result report.
Currently I'm only running the tests for qemu:///system so any help running
and debugging this for LXC, VirtualBox or XEN is very welcome.
Many thanks go to Salvatore Bonaccorso for packaging the missing Perl modules
needed by libvirt-tck.