@c Copyright (C) 2005, 2007 David Bateman
@c Copyright (C) 2002, 2003, 2004, 2005 Paul Kienzle
@c
@c This file is part of Octave.
@c
@c Octave is free software; you can redistribute it and/or modify it
@c under the terms of the GNU General Public License as published by the
@c Free Software Foundation; either version 3 of the License, or (at
@c your option) any later version.
@c 
@c Octave is distributed in the hope that it will be useful, but WITHOUT
@c ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
@c FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
@c for more details.
@c 
@c You should have received a copy of the GNU General Public License
@c along with Octave; see the file COPYING.  If not, see
@c <http://www.gnu.org/licenses/>.

@node Test and Demo Functions
@appendix Test and Demo Functions
@cindex test functions

Octave includes a number of functions to allow the integration of testing
and demonstration code in the source code of the functions themselves.

@menu
* Test Functions::
* Demonstration Functions::
@end menu

@node Test Functions
@section Test Functions

@DOCSTRING(test)

@code{test} scans the named script file looking for lines which
start with @code{%!}. The prefix is stripped off and the rest of the
line is processed through the Octave interpreter. If the code
generates an error, then the test is said to fail.

Since @code{eval()} will stop at the first error it encounters, you must
divide your tests up into blocks, with anything in a separate
block evaluated separately.  Blocks are introduced by the keyword
@code{test} immediately following the @code{%!}.  For example,

@example
@group
   %!test error ("this test fails!");
   %!test "test doesn't fail. it doesn't generate an error";
@end group
@end example

When a test fails, you will see something like:

@example
@group
     ***** test error ('this test fails!')
   !!!!! test failed
   this test fails!
@end group
@end example

Generally, to test if something works, you want to assert that it
produces a correct value.  A real test might look something like

@example
@group
   %!test
   %! @var{a} = [1, 2, 3; 4, 5, 6]; B = [1; 2];
   %! expect = [ @var{a} ; 2*@var{a} ];
   %! get = kron (@var{b}, @var{a});
   %! if (any(size(expect) != size(get)))
   %!    error ("wrong size: expected %d,%d but got %d,%d",
   %!           size(expect), size(get));
   %! elseif (any(any(expect!=get)))
   %!    error ("didn't get what was expected.");
   %! endif
@end group
@end example

To make the process easier, use the @code{assert} function.  For example,
with @code{assert} the previous test is reduced to:

@example
@group
   %!test
   %! @var{a} = [1, 2, 3; 4, 5, 6]; @var{b} = [1; 2];
   %! assert (kron (@var{b}, @var{a}), [ @var{a}; 2*@var{a} ]);
@end group
@end example

@code{assert} can accept a tolerance so that you can compare results
absolutely or relatively. For example, the following all succeed:

@example
@group
   %!test assert (1+eps, 1, 2*eps)          # absolute error
   %!test assert (100+100*eps, 100, -2*eps) # relative error
@end group
@end example

You can also do the comparison yourself, but still have assert
generate the error:

@example
@group
   %!test assert (isempty([]))
   %!test assert ([ 1,2; 3,4 ] > 0)
@end group
@end example

Because @code{assert} is so frequently used alone in a test block, there
is a shorthand form:

@example
   %!assert (@dots{})
@end example

which is equivalent to:

@example
   %!test assert (@dots{})
@end example

Sometimes during development there is a test that should work but is
known to fail.  You still want to leave the test in because when the
final code is ready the test should pass, but you may not be able to
fix it immediately.  To avoid unnecessary bug reports for these known
failures, mark the block with @code{xtest} rather than @code{test}:

@example
   %!xtest assert (1==0)
   %!xtest fail ('success=1','error'))
@end example

Another use of @code{xtest} is for statistical tests which should
pass most of the time but are known to fail occasionally.

Each block is evaluated in its own function environment, which means
that variables defined in one block are not automatically shared
with other blocks.  If you do want to share variables, then you
must declare them as @code{shared} before you use them.  For example, the
following declares the variable @var{a}, gives it an initial value (default
is empty), then uses it in several subsequent tests.

@example
@group
   %!shared @var{a}
   %! @var{a} = [1, 2, 3; 4, 5, 6];
   %!assert (kron ([1; 2], @var{a}), [ @var{a}; 2*@var{a} ]);
   %!assert (kron ([1, 2], @var{a}), [ @var{a}, 2*@var{a} ]);
   %!assert (kron ([1,2; 3,4], @var{a}), [ @var{a},2*@var{a}; 3*@var{a},4*@var{a} ]);
@end group
@end example

You can share several variables at the same time:

@example
   %!shared @var{a}, @var{b}
@end example

You can also share test functions:

@example
@group
   %!function @var{a} = fn(@var{b})
   %!  @var{a} = 2*@var{b};
   %!assert (@var{a}(2),4);
@end group
@end example

Note that all previous variables and values are lost when a new 
shared block is declared.

Error and warning blocks are like test blocks, but they only succeed 
if the code generates an error.  You can check the text of the error
is correct using an optional regular expression @code{<pattern>}.  
For example:

@example
   %!error <passes!> error('this test passes!');
@end example

If the code doesn't generate an error, the test fails. For example,

@example
   %!error "this is an error because it succeeds.";
@end example

produces

@example
@group
   ***** error "this is an error because it succeeds.";
   !!!!! test failed: no error
@end group
@end example

It is important to automate the tests as much as possible, however
some tests require user interaction.  These can be isolated into
demo blocks, which if you are in batch mode, are only run when 
called with @code{demo} or @code{verbose}. The code is displayed before
it is executed. For example,

@example
@group
   %!demo
   %! @var{t}=[0:0.01:2*pi]; @var{x}=sin(@var{t});
   %! plot(@var{t},@var{x});
   %! you should now see a sine wave in your figure window
@end group
@end example

produces

@example
@group
   > @var{t}=[0:0.01:2*pi]; @var{x}=sin(@var{t});
   > plot(@var{t},@var{x});
   > you should now see a sine wave in your figure window
   Press <enter> to continue: 
@end group
@end example

Note that demo blocks cannot use any shared variables.  This is so
that they can be executed by themselves, ignoring all other tests.

If you want to temporarily disable a test block, put @code{#} in place
of the block type.  This creates a comment block which is echoed
in the log file, but is not executed.  For example:

@example
@group
   %!#demo
   %! @var{t}=[0:0.01:2*pi]; @var{x}=sin(@var{t});
   %! plot(@var{t},@var{x});
   %! you should now see a sine wave in your figure window
@end group
@end example

Block type summary:

@table @code
@item %!test
check that entire block is correct
@item %!error
check for correct error message
@item %!warning
check for correct warning message
@item %!demo
demo only executes in interactive mode
@item %!#
comment: ignore everything within the block
@item %!shared x,y,z
declares variables for use in multiple tests
@item %!function
defines a function value for a shared variable
@item %!assert (x, y, tol)
shorthand for %!test assert (x, y, tol)
@end table

You can also create test scripts for builtins and your own C++
functions. Just put a file of the function name on your path without
any extension and it will be picked up by the test procedure.  You
can even embed tests directly in your C++ code:

@example
@group
   #if 0
   %!test disp('this is a test')
   #endif
@end group
@end example

or

@example
@group
   /*
   %!test disp('this is a test')
   */
@end group
@end example

but then the code will have to be on the load path and the user 
will have to remember to type test('name.cc').  Conversely, you
can separate the tests from normal Octave script files by putting
them in plain files with no extension rather than in script files.
@c DO I WANT TO INCLUDE THE EDITOR SPECIFIC STATEMENT BELOW???
@c Don't forget to tell emacs that the plain text file you are using
@c is actually octave code, using something like:
@c   -*-octave-*-

@DOCSTRING(assert)

@DOCSTRING(fail)

@node Demonstration Functions
@section Demonstration Functions

@DOCSTRING(demo)

@DOCSTRING(rundemos)

@DOCSTRING(example)

@DOCSTRING(speed)