.\" Copyright 2012 Google Inc.
.\" All rights reserved.
.\"
.\" Redistribution and use in source and binary forms, with or without
.\" modification, are permitted provided that the following conditions are
.\" met:
.\"
.\" * Redistributions of source code must retain the above copyright
.\"   notice, this list of conditions and the following disclaimer.
.\" * Redistributions in binary form must reproduce the above copyright
.\"   notice, this list of conditions and the following disclaimer in the
.\"   documentation and/or other materials provided with the distribution.
.\" * Neither the name of Google Inc. nor the names of its contributors
.\"   may be used to endorse or promote products derived from this software
.\"   without specific prior written permission.
.\"
.\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
.\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
.\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
.\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
.\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
.\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
.\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
.\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
.\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
.\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
.\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
.Dd December 26, 2012
.Dt KYUA-ATF-TESTER 1
.Os
.Sh NAME
.Nm kyua-atf-tester
.Nd Scriptable interface to interact with ATF test programs
.Sh SYNOPSIS
.Nm
.Op Ar generic_flags
.Ar subcommand
.Op Ar subcommand_flags
.Op Ar subcommand_args
.Sh DESCRIPTION
.Nm
provides a scriptable interface to interact with test programs that implement
the
.Xr kyua-atf-interface 7
test interface.
This manual page describes the specific features that apply to this specific
tester.  For the generic manual page, which describes the command-line interface
in detail, please see
.Xr kyua-tester 1 .
.Ss Results conversion
The result format outputted by
.Nm
(described in
.Xr kyua-tester-result 5 )
is more restricted than what the ATF test case itself can output.  The following
list details maps ATF test results to their generic results:
.Bl -tag -width expectedXfailureXX
.It expected_death
Maps to
.Sq expected_failure
if the death happens;
.Sq failed
otherwise.
.It expected_exit
Maps to
.Sq expected_failure
if the test exits cleanly;
.Sq failed
otherwise.
.It expected_failure
Maps to
.Sq expected_failure
if there is any failure during the execution;
.Sq failed
otherwise.
.It expected_signal
Maps to
.Sq expected_failure
if the test terminates due to a signal;
.Sq failed
otherwise.
.It expected_timeout
Maps to
.Sq expected_failure
if the test times out;
.Sq failed
otherwise.
.It failed
Maps to
.Sq failed .
.It passed
Maps to
.Sq passed .
.It skipped
Maps to
.Sq skipped .
.El
.Pp
Any of the previous test results
.Em can also map
to
.Sq broken
in those cases where there is an unexpected problem in the communcation between
the test and
.Nm .
For example, this will happen if result file created by the ATF test case does
not comply to what
.Xr kyua-atf-interface 7
describes.
.Ss Cleanup routines
Because
.Nm
cannot keep any contextual information between the listing of test cases and
their execution, the tester cannot easily (aka cheaply) know whether a test has
or has not a cleanup routine when it is run.  For this reason, the
.Sq test
subcommand of
.Nm
will assume, by default, that a test case
.Em does have
a cleanup routine.  This postive assumption ensures that, if the tester is
incorrectly executed, the test will be properly cleaned up.
.Pp
However, as an optimization, the caller can pass
.Sq -v has_cleanup=false
to the
.Sq test
subcommand to tell
.Nm
that the test being run does not have a cleanup routine.
.Sh SEE ALSO
.Xr kyua-tester 1 ,
.Xr kyua-atf-interface 7