autopkgtest - test an installed binary package using the source package's tests
...] -- virt-server
autopkgtest runs tests on binary Debian or Click packages, as installed on a
system (called "testbed"). The tests are supplied in the source
autopkgtest runs each test supplied by a particular package and reports the
results. It drives the specified virtualisation regime as appropriate, parses
the test description metadata, and arranges for data to be copied to and from
the testbed as required.
See /usr/share/doc/autopkgtest/README.running-tests.rst.gz for an introduction
about how to use autopkgtest.
Positional (non-option) arguments specify exactly one source package (containing
the test code) and optionally some binary packages to test.
testsrc can be one of:
- .dsc file
- Run tests from Debian .dsc source package. By
default the package will also be built and the resulting binaries will be
used to satisfy test dependencies; to disable that, specify the
- source package directory
- Run tests from a Debian source tree directory. If that is
an unbuilt tree, this is very similar to specifying a .dsc. If that
is a built tree, all test dependencies get satisfied by archive packages,
unless you explicitly specify locally built .debs as well.
Attention: If you just specify a bare directory name which is a legal
Debian source package name, it will be interpreted as the latter (see
below). In this case, prefix the directory name with ./.
- current directory
- If no source package is specified on the command line and
the current directory is a Debian source package, this will be tested.
- source package name
- Downloads the given source package name with apt-get
source in the testbed and run its tests. This is similar to specifying
a .dsc but avoids copying the source from the host to the testbed.
Possibly built binaries (if the test specifies needs-build) will
not be used to satisfy dependencies, as usually in this mode you
want to test binaries from a real archive.
- git URL or URL#branch
- Git-clones the given URL (which must contain an unbuilt
Debian source tree) and runs the tests from that. If branch is
given, this branch will be checked out instead of the default (usually
"master"). This can also be a more general refspec such as
#refs/pull/123/head" for a GitHub pull request.
This is very similar to cloning manually and specifying the checkout
directory as test; i. e. this is commonly used with
--no-built-binaries. The git package will be installed if
- .changes file
- Run tests from the .dsc source package in the given
.changes file. If the .changes contains .deb packages, they will be used
for the test. Acts as if you had specified the .debs and .dsc from the
.changes file as explicit arguments. Note that if the .changes contains
only debs, the corresponding .dsc still needs to be specified
alongside, or the current directory must be the source package.
All other positional arguments must be .deb binary packages. They will be used
for both build and test dependencies of the source package. If any binary
package is given, then --no-built-binaries
This needs exactly one .click package. If its manifest specifies an
URL, the corresponding source package will be downloaded and
tests run from there; then a source directory does not need to be given.
Otherwise specifying a source directory as positional argument is mandatory.
For testing a local .click package, give its path to autopkgtest.
testing a preinstalled click, use --installed-click
(this cannot be given as positional argument as this cannot be disambiguated
from Debian source package names).
Examples with explicit click source directory:
com.example.myclick src/myclick/ -- [...]
autopkgtest --installed-click com.example.myclick src/myclick/ -- [...]
Examples without click source directory; click package needs to have
autopkgtest src/myclick_0.1_all.click -- [...]
autopkgtest --installed-click com.example.myclick -- [...]
- -B | --no-built-binaries
- Binaries from unbuilt source packages (see above) will not
be built or ignored, and dependencies are satisfied with packages from the
archive. Note that the source package still gets built if a test requires
- Read the test metadata from PATH instead of
debian/tests/control (for Debian sources) or the Click manifest.
- Run only the given test name (from test control file). This
replaces --testname which is deprecated.
If you don't specify any option, autopkgtest
only writes its
output/results to stderr.
- -o dir | --output-dir=dir
- Specifies that test artifacts (stderr and stdout from the
tests, the log file, built binary packages etc.) should be placed in the
given directory. dir must not exist yet or be empty, otherwise
autopkgtest will refuse to use it.
- -l logfile |
- Specifies that the trace log should be written to
logfile instead of to output-dir.
- Specifies that a summary of the outcome should be written
to summary. The events in the summary are written to the log in any
- -q | --quiet
- Do not send a copy of autopkgtest's trace logstream
to stderr. This option does not affect the copy sent to logfile or
output-dir. Note that without the trace logstream it can be very
hard to diagnose problems.
- Run commands after opening the testbed. This can be
used e. g. to enable additional apt sources, run apt-get update or
similar. If commands is an existing file name, the commands are
read from that; otherwise it is a string with the actual commands that
gets run as-is. File names without directory will be searched in both the
current directory and in /usr/share/autopkgtest/setup-commands/ so
you do not need to give the full path for setup scripts shipped with
Normally, if the setup commands fail, autopkgtest will consider this a
transient testbed error (exit code 16). However, if the setup commands
exit with code 100, autopkgtest will consider this an "erroneous
package" (exit code 12) instead, so this can be used to e. g. detect
upgrade errors to a new version. Note that apt exits with exit code 100 in
This option can be specified multiple times.
If --user is given or the test bed provides a
suggested-normal-user capability, the
$AUTOPKGTEST_NORMAL_USER environment variable will be set to that
If the setup commands affect anything in boot directories (like /boot or
/lib/systemd/system) and the testbed supports rebooting, the testbed will
be rebooted after the setup commands. This can be suppressed by creating a
- Run commands after the --setup-commands, and
after every reboot. For example, these commands could be used to add files
in a tmpfs.
These commands never cause the testbed to be rebooted (because that could
lead to an infinite loop). Otherwise, they are just like the
This option can be specified multiple times.
- --apt-upgrade | -U
- Run apt-get update and apt-get dist-upgrade
-y in the testbed before running the tests.
- Add apt sources for release-pocket. This
finds the first deb line in /etc/apt/sources.list which does
not already specify a pocket and adds a deb and deb-src line with that
pocket to /etc/apt/sources.list.d/pocket.list. This
also calls apt-get update for the new pocket (but not for anything
If a package list is given after =, set up apt pinning to use only those
packages from pocket. An entry "src:srcname" expands to
all binary packages built by that source. This can be used for minimizing
dependencies taken from pocket so that package updates in that
pocket can be tested independently from each other for better isolation.
Attention: This does not currently resolve some situations where
dependencies of the given packages can only be resolved in the given
pocket. In that case the apt pinning will be removed and package
installation will be retried with the entirety of pocket.
- Copy file or directory from host into testbed after
opening. This happens before --setup-commands thus you can use
these files in the setup commands.
- Set arbitrary environment variable in the build and test.
Can be specified multiple times.
- -u user | --user=user
- Run builds and tests as user on the testbed. This
needs root on the testbed; if root on the testbed is not available then
builds and tests run as whatever user is provided.
- Prefixes debian/rules binary with gain-root.
The default is not to use anything, except that if --user is
supplied or root on the testbed is not available the default is
- Include additional debugging information in the trace log.
Each additional -d increases the debugging level; the current
maximum is -ddd. If you like to see what's going on, -d or
-dd is recommended.
- Run an interactive shell in the testbed after a failed
build, test, or dependency installation.
- Run an interactive shell in the testbed after every test.
- Use a different timeout for operations on or with the
testbed. There are five timeouts affected by five values of which:
short: supposedly short operations like setting up the testbed's
apt and checking the state (default: 100s); install: installation
of packages including dependencies (default: 3,000s); test: test
runs (default: 10,000s); copy: copy files/directories between host
and testbed (default: 300s); and build: builds (default: 100,000s).
The value must be specified as an integer number of seconds.
- Multiply all of the default timeouts by the specified
factor (see --timeout-which above). Only the defaults are
affected; explicit timeout settings are used exactly as specified.
- When running commands on the testbed, sets the LANG
environment variable to langval. The default in autopkgtest
is to set it to C.UTF-8.
- Disable automatic test generation with autodep8, even if it
is installed. In that case, packages without tests will exit with code 8
("No tests in this package") just like without autodep8.
- Set parallel=N DEB_BUILD_OPTION for building
packages. By default this is the number of available processors. This is
mostly useful in containers where you can restrict the available RAM, but
not restrict the number of CPUs.
- Show command line help and exit.
- -- virt-server virt-server-arg...
- Specifies the virtualisation regime server, as a command
and arguments to invoke. virt-server must be an existing
autopkgtest virtualization server such as schroot or qemu.
All the remaining arguments and options after -- are passed to the
virtualisation server program. See the manpages of the individual servers
for how to use them.
During a normal test run, one line is printed for each test. This consists of a
short string identifying the test, some horizontal whitespace, and either
reason or SKIP
reason where the pass/fail
indication is separated by any reason by some horizontal whitespace.
The string to identify the test consists of a short alphanumeric string invented
to distinguish different command-line arguments, the
, followed by a hyphen and the test name.
Sometimes a SKIP
will be reported when the name of the test is not known
or not applicable: for example, when there are no tests in the package, or a
there is a test stanza which contains features not understood by this version
. In this case *
will appear where the name of the
test should be.
detects that erroneous package(s) are involved, it will
print the two lines blame: blamed-thing...
. Here each whitespace-separated blamed-thing
(representing a pathname found in a command line
(a source package name),
(a binary package name) or possibly other strings to
be determined. This indicates which arguments and/or packages might have
contributed to the problem; the ones which were processed most recently and
which are therefore most likely to be the cause of a problem are listed last.
If you use lots of options or nontrivial virt server arguments, you can put any
part of the command line into a text file, with one line per option. E. g. you
can create a file sid.cfg
with contents like
and then run
autopkgtest foo_1_amd64.changes @sid.cfg
The contents of the configuration file will be expanded in-place as if you would
have given its contents on the command line. Please ensure that you don't
between short options and their values, they would become a
part of the argument value.
0 all tests passed
2 at least one test skipped
4 at least one test failed
6 at least one test failed and at least one test skipped
8 no tests in this package
12 erroneous package
16 testbed failure
20 other unexpected failures including bad usage
This manpage is part of autopkgtest, a tool for testing Debian binary packages.
autopkgtest is Copyright (C) 2006-2014 Canonical Ltd.
for the list of contributors and
full copying conditions.