[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Enabling/disabling tests by command line parameters


Hi all,

As far as I know from the reference manual and code, cmocka doesn't
come with a build-in command line runner. What do you think about
implementing one?

It would be nice to be able to run only specific test, or prevent some
tests from running. It may be valuable for, say, debugging or when
some tests require additional hardware or software resources to be
actually run (for example, testing OpenCL code is useless without
having CL-capable hardware).

Basically, I propose something like the following syntax:

  $ ./test                        # run all registered tests/groups;
  $ ./test snafu tarfu            # run only 'snafu' and 'tarfu'; if
                                  # any of them is a group, run all
				  # tests it contains;
  $ ./test --without snafu tarfu  # run the rest of tests.

Specifying both whitelist and blacklist doesn't make much sense,
unless you should run all test from a group without some specific
tests:

  $ ./test snafu tarfu --without test_foobar_is_fubar

The exact switch name may vary: '--without', '--except', '--exclude',
or their variants with a single dash. There may be even more command
lines, for example, for choosing report output format.

It seems to be easy to implement, and I'm going to do it for one
of my projects. But isn't it better to have one "standardized" way for
running specific tests built-in?

This runner may or may not rely on getopt for arguments parsing, it
may even reside in a separate code and header files from the rest of
library.

P.S.:
BTW, the issue tracker have a similar feature request open [1].
The idea is nice, but the implementation may require having an
additional dependency to a regular expression or wildcard matching
library.

[1]: https://open.cryptomilk.org/issues/42
     


Follow-Ups:
Re: Enabling/disabling tests by command line parametersAndreas Schneider <asn@xxxxxxxxxxxxxx>