Mercurial > octave-nkf

changeset 19082:ec28b7216501

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
test.m: Overhaul function (also fixes bug #39703). * test.m: Rewrite docstring. Use more meaningful variable names. Run in batch mode if an output argument is requested (bug #39703). Change verbose to -1,0,1 three-level indicator to match "quiet", "normal", "verbose". Change all boolean type variables to true/false rather than double(1)/double(0). Return a boolean pass/fail variable if only one output is requested rather than a double. Remove duplicate subfunction bundle, replace with var2struct. Add many more built-in tests. * aspell-octave.en.pws: Add new words to Octave custom spellcheck dictionary.
author Rik <rik@octave.org>
date Sun, 24 Aug 2014 15:25:21 -0700
parents 1288a2f27769
children c573d9c70ae5
files doc/interpreter/doccheck/aspell-octave.en.pws scripts/testfun/test.m
diffstat 2 files changed, 330 insertions(+), 267 deletions(-) [+]
line wrap: on
line diff
--- a/doc/interpreter/doccheck/aspell-octave.en.pws	Sun May 11 22:24:50 2014 +0200
+++ b/doc/interpreter/doccheck/aspell-octave.en.pws	Sun Aug 24 15:25:21 2014 -0700
@@ -526,6 +526,7 @@
 lm
 loadpath
 Lobatto
+logfile
 logit
 logncdf
 logninv
@@ -952,6 +953,7 @@
 symamd
 symbfact
 symrcm
+Syntaxes
 tcdf
 Tcv
 terminal's
@@ -1113,6 +1115,7 @@
 xPOTRF
 xPTSV
 xtest
+xtests
 xTRTRS
 xu
 xwd
--- a/scripts/testfun/test.m	Sun May 11 22:24:50 2014 +0200
+++ b/scripts/testfun/test.m	Sun Aug 24 15:25:21 2014 -0700
@@ -20,125 +20,173 @@
 ## @deftypefn  {Command} {} test @var{name}
 ## @deftypefnx {Command} {} test @var{name} quiet|normal|verbose
 ## @deftypefnx {Function File} {} test ("@var{name}", "quiet|normal|verbose", @var{fid})
-## @deftypefnx {Function File} {} test ([], "explain", @var{fid})
+## @deftypefnx {Function File} {} test ("@var{name}", "quiet|normal|verbose", @var{fname})
 ## @deftypefnx {Function File} {@var{success} =} test (@dots{})
-## @deftypefnx {Function File} {[@var{n}, @var{max}] =} test (@dots{})
+## @deftypefnx {Function File} {[@var{n}, @var{nmax}, @var{nxfail}, @var{nskip}] =} test (@dots{})
 ## @deftypefnx {Function File} {[@var{code}, @var{idx}] =} test ("@var{name}", "grabdemo")
+## @deftypefnx {Function File} {} test ([], "explain", @var{fid})
+## @deftypefnx {Function File} {} test ([], "explain", @var{fname})
+##
+## Perform built-in self-tests from the first file in the loadpath matching
+## @var{name}.
+##
+## @code{test} can be called in either command or functional form.  The exact
+## operation of test is determined by a combination of mode (interactive or
+## batch), reporting level ("quiet", "normal", "verbose"), and whether a
+## logfile or summary output variable is used.
 ##
-## Perform tests from the first file in the loadpath matching @var{name}.
-## @code{test} can be called as a command or as a function.  Called with
-## a single argument @var{name}, the tests are run interactively and stop
-## after the first error is encountered.
+## The default mode when @code{test} is called from the command line is
+## interactive.  In this mode, tests will be run until the first error is
+## encountered, or all tests complete successfully.  In batch mode, all tests
+## are run regardless of any failures, and the results are collected for
+## reporting.  Tests which require user interaction, i.e., demo blocks,
+## are never run in batch mode.
 ##
-## With a second argument the tests which are performed and the amount of
-## output is selected.
+## Batch mode is enabled by either 1) specifying a logfile using the third
+## argument @var{fname} or @var{fid}, or 2) requesting an output argument
+## such as @var{success}, @var{n}, etc.
+##
+## The optional second argument determines the amount of output to generate and
+## which types of tests to run.  The default value is @qcode{"normal"}. 
+## Requesting an output argument will suppress printing the final summary
+## message and any intermediate warnings, unless verbose reporting is
+## enabled.
 ##
 ## @table @asis
 ## @item @qcode{"quiet"}
-##  Don't report all the tests as they happen, just the errors.
+## Print a summary message when all tests pass, or print an error with the
+## results of the first bad test when a failure occurs.  Don't run tests which
+## require user interaction.
 ##
 ## @item @qcode{"normal"}
-## Report all tests as they happen, but don't do tests which require
-## user interaction.
+## Display warning messages about skipped tests or failing xtests during test
+## execution.
+## Print a summary message when all tests pass, or print an error with the
+## results of the first bad test when a failure occurs.  Don't run tests which
+## require user interaction.
 ##
 ## @item @qcode{"verbose"}
-## Do tests which require user interaction.
+## Display tests before execution.  Print all warning messages.  In interactive
+## mode, run all tests including those which require user interaction.
 ## @end table
 ##
-## The argument @var{fid} can be used to allow batch processing.  Errors
-## can be written to the already open file defined by @var{fid}, and
-## hopefully when Octave crashes this file will tell you what was happening
-## when it did.  You can use @code{stdout} if you want to see the results as
-## they happen.  You can also give a file name rather than an @var{fid}, in
-## which case the contents of the file will be replaced with the log from
-## the current test.
+## The optional third input argument specifies a logfile where results of the
+## tests should be written.  The logfile may be a character string
+## (@var{fname}) or an open file descriptor ID (@var{fid}).  To enable batch
+## processing, but still print the results to the screen, use @code{stdout} for
+## @var{fid}.
+##
+## When called with just a single output argument @var{success}, @code{test}
+## returns true if all of the tests were successful.  If called with more
+## than one output argument then the number of successful tests (@var{n}),
+## the total number of tests in the file (@var{nmax}), the number of xtest
+## failures (@var{nxfail}), and the number of skipped tests (@var{nskip} are
+## returned.
+##
+## Example
 ##
-## Called with a single output argument @var{success}, @code{test} returns
-## true if all of the tests were successful.  Called with two output arguments
-## @var{n} and @var{max}, the number of successful tests and the total number
-## of tests in the file @var{name} are returned.
+## @example
+## @group
+## test sind
+## @result{}
+## PASSES 5 out of 5 tests
+##
+## [n, nmax] = test ("sind")
+## @result{}
+## n =  5
+## nmax =  5
+## @end group
+## @end example
+## 
+## Additional Calling Syntaxes
 ##
 ## If the second argument is the string @qcode{"grabdemo"}, the contents of
-## the demo blocks are extracted but not executed.  Code for all code blocks
-## is concatenated and returned as @var{code} with @var{idx} being a vector
-## of positions of the ends of the demo blocks.
+## any built-in demo blocks are extracted but not executed.  The text for all
+## code blocks is concatenated and returned as @var{code} with @var{idx} being
+## a vector of positions of the ends of each demo block.  For an easier way to
+## extract demo blocks from files, @xref{XREFexample,,example}.
 ##
-## If the second argument is @qcode{"explain"}, then @var{name} is ignored
-## and an explanation of the line markers used is written to the file
-## @var{fid}.
-## @seealso{assert, fail, error, demo, example}
+## If the second argument is @qcode{"explain"} then @var{name} is ignored and
+## an explanation of the line markers used in @code{test} output reports is
+## written to the file specified by @var{fname} or @var{fid}.
+##
+## @seealso{assert, fail, demo, example, error}
 ## @end deftypefn
 
-## FIXME: * Consider using keyword fail rather then error?  This allows us
-## to make a functional form of error blocks, which means we
-## can include them in test sections which means that we can use
-## octave flow control for both kinds of tests.
+## Programming Note: All variables for test() must use the internal prefix "__".
+## Shared variables are eval'ed into the current workspace and therefore might
+## collide with the names used in the test.m function itself.
 
-function [__ret1, __ret2, __ret3, __ret4] = test (__name, __flag, __fid)
-  ## Information from test will be introduced by "key".
-  persistent __signal_fail =  "!!!!! ";
+function [__n, __nmax, __nxfail, __nskip] = test (__name, __flag = "normal", __fid = [])
+
+  ## Output from test is prefixed by a "key" to quickly understand the issue.
+  persistent __signal_fail  = "!!!!! ";
   persistent __signal_empty = "????? ";
-  persistent __signal_block = "  ***** ";
-  persistent __signal_file =  ">>>>> ";
-  persistent __signal_skip = "----- ";
-
-  __xfail = 0;
-  __xskip = 0;
+  persistent __signal_block = "***** ";
+  persistent __signal_file  = ">>>>> ";
+  persistent __signal_skip  = "----- ";
 
-  if (nargin < 2 || isempty (__flag))
-    __flag = "quiet";
-  endif
-  if (nargin < 3)
-    __fid = [];
-  endif
-  if (nargin < 1 || nargin > 3
-      || (! ischar (__name) && ! isempty (__name)) || ! ischar (__flag))
+  if (nargin < 1 || nargin > 3)
+    print_usage ();
+  elseif (! isempty (__name) && ! ischar (__name))
+    error ("test: NAME must be a string");
+  elseif (! ischar (__flag))
+    error ("test: second argument must be a string");
+  elseif (isempty (__name) && (nargin != 3 || ! strcmp (__flag, "explain")))
     print_usage ();
   endif
-  if (isempty (__name) && (nargin != 3 || ! strcmp (__flag, "explain")))
-    print_usage ();
-  endif
-  __batch = (! isempty (__fid));
 
   ## Decide if error messages should be collected.
-  __close_fid = 0;
-  if (__batch)
+  __logfile = ! isempty (__fid);
+  __batch = __logfile || nargout > 0;
+  __close_fid = false;
+  if (__logfile)
     if (ischar (__fid))
-      __fid = fopen (__fid, "wt");
+      __fname = __fid;
+      __fid = fopen (__fname, "wt");
       if (__fid < 0)
-        error ("test: could not open log file");
+        error ("test: could not open log file %s", __fname);
       endif
-      __close_fid = 1;
+      __close_fid = true;
     endif
-    fprintf (__fid, "%sprocessing %s\n", __signal_file, __name);
-    fflush (__fid);
+    if (! strcmp (__flag, "explain"))
+      fprintf (__fid, "%sprocessing %s\n", __signal_file, __name);
+      fflush (__fid);
+    endif
   else
     __fid = stdout;
   endif
 
   if (strcmp (__flag, "normal"))
-    __grabdemo = 0;
-    __rundemo = 0;
-    __verbose = __batch;
+    __grabdemo = false;
+    __rundemo  = false;
+    if (__logfile)
+      __verbose = 1;
+    elseif (__batch)
+      __verbose = -1;
+    else
+      __verbose = 0;
+    endif
   elseif (strcmp (__flag, "quiet"))
-    __grabdemo = 0;
-    __rundemo = 0;
-    __verbose = 0;
+    __grabdemo = false;
+    __rundemo  = false;
+    __verbose  = -1;
   elseif (strcmp (__flag, "verbose"))
-    __grabdemo = 0;
-    __rundemo = 1;
-    __verbose = 1;
+    __grabdemo = false;
+    __rundemo  = true;
+    __verbose  = 1;
   elseif (strcmp (__flag, "grabdemo"))
-    __grabdemo = 1;
-    __rundemo = 0;
-    __verbose = 0;
+    __grabdemo = true;
+    __rundemo  = false;
+    __verbose  = -1;
     __demo_code = "";
     __demo_idx = [];
   elseif (strcmp (__flag, "explain"))
     fprintf (__fid, "# %s new test file\n", __signal_file);
     fprintf (__fid, "# %s no tests in file\n", __signal_empty);
     fprintf (__fid, "# %s test had an unexpected result\n", __signal_fail);
-    fprintf (__fid, "# %s code for the test\n", __signal_block);
+    fprintf (__fid, "# %s test was skipped\n", __signal_skip);
+    fprintf (__fid, "# %s code for the test\n\n", __signal_block);
     fprintf (__fid, "# Search for the unexpected results in the file\n");
     fprintf (__fid, "# then page back to find the file name which caused it.\n");
     fprintf (__fid, "# The result may be an unexpected failure (in which\n");
@@ -162,17 +210,16 @@
     __file = file_in_loadpath ([__name ".cc"], "all");
   endif
   if (iscell (__file))
-      ## If repeats, return first in path.
     if (isempty (__file))
       __file = "";
     else
-      __file = __file{1};
+      __file = __file{1};  # If repeats, return first in path.
     endif
   endif
   if (isempty (__file))
     if (__grabdemo)
-      __ret1 = "";
-      __ret2 = -1;
+      __n = "";
+      __nmax = -1;
     else
       ftype = exist (__name);
       if (ftype == 3)
@@ -188,7 +235,11 @@
       endif
       fflush (__fid);
       if (nargout > 0)
-        __ret1 = __ret2 = 0;
+        if (nargout == 1)
+          __n = false;
+        else
+          __n = __nmax = 0;
+        endif
       endif
     endif
     if (__close_fid)
@@ -202,13 +253,17 @@
 
   if (isempty (__body))
     if (__grabdemo)
-      __ret1 = "";
-      __ret2 = [];
+      __n = "";
+      __nmax = [];
     else
       fprintf (__fid, "%s%s has no tests available\n", __signal_empty, __file);
       fflush (__fid);
       if (nargout > 0)
-        __ret1 = __ret2 = 0;
+        if (nargout == 1)
+          __n = false;
+        else
+          __n = __nmax = 0;
+        endif
       endif
     endif
     if (__close_fid)
@@ -217,10 +272,10 @@
     return;
   else
     ## Add a dummy comment block to the end for ease of indexing.
-    if (__body (length (__body)) == "\n")
-      __body = sprintf ("\n%s#", __body);
+    if (__body(end) == "\n")
+      __body = ["\n" __body "#"];
     else
-      __body = sprintf ("\n%s\n#", __body);
+      __body = ["\n" __body "\n#"];
     endif
   endif
 
@@ -228,26 +283,28 @@
   __lineidx = find (__body == "\n");
   __blockidx = __lineidx(find (! isspace (__body(__lineidx+1))))+1;
 
-  ## Ready to start tests ... if in batch mode, tell us what is happening.
-  if (__verbose)
+  ## Ready to start tests.
+  ## If in batch mode, with a logfile, report what is happening.
+  if (__verbose > 0)
     disp ([__signal_file, __file]);
   endif
 
   ## Assume all tests will pass.
-  __all_success = 1;
+  __all_success = true;
 
   ## Process each block separately, initially with no shared variables.
   __tests = __successes = 0;
+  __xfail = __xskip = 0;
   __shared = " ";
   __shared_r = " ";
-  __clear = "";
-  for __i = 1:length (__blockidx)-1
+  __clearfcn = "";
+  for __i = 1:numel (__blockidx)-1
 
     ## Extract the block.
     __block = __body(__blockidx(__i):__blockidx(__i+1)-2);
 
-    ## Let the user/logfile know what is happening.
-    if (__verbose)
+    ## Print the code block before execution if in verbose mode.
+    if (__verbose > 0)
       fprintf (__fid, "%s%s\n", __signal_block, __block);
       fflush (__fid);
     endif
@@ -263,11 +320,11 @@
     endif
 
     ## Assume the block will succeed.
-    __success = 1;
+    __success = true;
     __msg = [];
-    __isxtest = 0;
+    __isxtest = false;
 
-### DEMO
+    ### DEMO
 
     ## If in __grabdemo mode, then don't process any other block type.
     ## So that the other block types don't have to worry about
@@ -276,7 +333,7 @@
 
     __isdemo = strcmp (__type, "demo");
     if (__grabdemo || __isdemo)
-      __istest = 0;
+      __istest = false;
 
       if (__grabdemo && __isdemo)
         if (isempty (__demo_code))
@@ -294,8 +351,8 @@
           __test__;
           input ("Press <enter> to continue: ", "s");
         catch
-          __success = 0;
-          __msg = sprintf ("%sdemo failed\n%s",  __signal_fail, lasterr ());
+          __success = false;
+          __msg = [__signal_fail "demo failed\n" lasterr()];
         end_try_catch
         clear __test__;
 
@@ -303,10 +360,10 @@
       ## Code already processed.
       __code = "";
 
-### SHARED
+    ### SHARED
 
     elseif (strcmp (__type, "shared"))
-      __istest = 0;
+      __istest = false;
 
       ## Separate initialization code from variables.
       __idx = find (__code == "\n");
@@ -330,7 +387,7 @@
         if (! isempty (__vars))
           eval ([strrep(__vars, ",", "=[];"), "=[];"]);
           __shared = __vars;
-          __shared_r = ["[ ", __vars, "] = "];
+          __shared_r = ["[ " __vars "] = "];
         else
           __shared = " ";
           __shared_r = " ";
@@ -338,79 +395,75 @@
       catch
         ## Couldn't declare, so don't initialize.
         __code = "";
-        __success = 0;
-        __msg = sprintf ("%sshared variable initialization failed\n",
-                         __signal_fail);
+        __success = false;
+        __msg = [__signal_fail "shared variable initialization failed\n"];
       end_try_catch
 
       ## Initialization code will be evaluated below.
 
-### FUNCTION
+    ### FUNCTION
 
     elseif (strcmp (__type, "function"))
-      __istest = 0;
+      __istest = false;
       persistent __fn = 0;
       __name_position = function_name (__block);
       if (isempty (__name_position))
-        __success = 0;
-        __msg = sprintf ("%stest failed: missing function name\n",
-                         __signal_fail);
+        __success = false;
+        __msg = [__signal_fail "test failed: missing function name\n"];
       else
         __name = __block(__name_position(1):__name_position(2));
         __code = __block;
         try
           eval (__code);  # Define the function
-          __clear = sprintf ("%sclear %s;\n", __clear, __name);
+          __clearfcn = sprintf ("%sclear %s;\n", __clearfcn, __name);
         catch
-          __success = 0;
-          __msg = sprintf ("%stest failed: syntax error\n%s",
-                           __signal_fail, lasterr ());
+          __success = false;
+          __msg = [__signal_fail "test failed: syntax error\n" lasterr()];
         end_try_catch
       endif
       __code = "";
 
-### ENDFUNCTION
+    ### ENDFUNCTION
 
     elseif (strcmp (__type, "endfunction"))
       ## endfunction simply declares the end of a previous function block.
       ## There is no processing to be done here, just skip to next block.
-      __istest = 0;
+      __istest = false;
       __code = "";
 
-### ASSERT/FAIL
+    ### ASSERT/FAIL
 
     elseif (strcmp (__type, "assert") || strcmp (__type, "fail"))
-      __istest = 1;
+      __istest = true;
       ## Put the keyword back on the code.
       __code = __block;
       ## The code will be evaluated below as a test block.
 
-### ERROR/WARNING
+    ### ERROR/WARNING
 
     elseif (strcmp (__type, "error") || strcmp (__type, "warning"))
-      __istest = 1;
-      __warning = strcmp (__type, "warning");
+      __istest = true;
+      __iswarning = strcmp (__type, "warning");
       [__pattern, __id, __code] = getpattern (__code);
       if (__id)
-        __patstr = ["id=",__id];
+        __patstr = ["id=" __id];
       else
         if (! strcmp (__pattern, '.'))
-          __patstr = ["<",__pattern,">"];
+          __patstr = ["<" __pattern ">"];
         else
-          __patstr = ifelse (__warning, "a warning", "an error");
+          __patstr = ifelse (__iswarning, "a warning", "an error");
         endif
       endif
       try
         eval (sprintf ("function __test__(%s)\n%s\nendfunction",
                        __shared, __code));
       catch
-        __success = 0;
-        __msg = sprintf ("%stest failed: syntax error\n%s",
-                         __signal_fail, lasterr ());
+        __success = false;
+        __msg = [__signal_fails "test failed: syntax error\n" lasterr()];
       end_try_catch
 
       if (__success)
-        __success = 0;
+        __success = false;
         __warnstate = warning ("query", "quiet");
         warning ("on", "quiet");
         ## Clear error and warning strings before starting
@@ -423,9 +476,9 @@
           ##        the try block which is disabling normal errors.
           lastwarn ();
           eval (sprintf ("__test__(%s);", __shared));
-          if (! __warning)
-            __msg = sprintf ("%serror failed.\nExpected %s but got no error\n",
-                             __signal_fail, __patstr);
+          if (! __iswarning)
+            __msg = [__signal_fail "error failed.\n" ...
+                     "Expected " __patstr "but got no error\n"];
           else
             if (! isempty (__id))
               [~, __err] = lastwarn ();
@@ -436,14 +489,13 @@
             endif
             warning (__warnstate.state, "quiet");
             if (isempty (__err))
-              __msg = sprintf (["%swarning failed.\n" ...
-                                "Expected %s but got no warning\n"],
-                               __signal_fail, __patstr);
+              __msg = [__signal_fail "warning failed.\n" ...
+                       "Expected " __patstr " but got no warning\n"];
             elseif (__mismatch)
-              __msg = sprintf ("%swarning failed.\nExpected %s but got <%s>\n",
-                               __signal_fail, __patstr, __err);
+              __msg = [__signal_fail "warning failed.\n" ...
+                       "Expected " __patstr " but got <" __err ">\n"];
             else
-              __success = 1;
+              __success = true;
             endif
           endif
 
@@ -456,15 +508,15 @@
             __mismatch = isempty (regexp (__err, __pattern, "once"));
           endif
           warning (__warnstate.state, "quiet");
-          if (__warning)
-            __msg = sprintf (["%swarning failed.\n" ...
-                              "Expected warning %s but got error <%s>\n"],
-                             __signal_fail, __patstr, __err);
+          if (__iswarning)
+            __msg = [__signal_fail "warning failed.\n" ...
+                     "Expected warning " __patstr ...
+                     " but got error <" __err ">\n"];
           elseif (__mismatch)
-            __msg = sprintf ("%serror failed.\nExpected %s but got <%s>\n",
-                             __signal_fail, __patstr, __err);
+            __msg = [__signal_fail "error failed.\n" ...
+                     "Expected " __patstr " but got <" __err ">\n"];
           else
-            __success = 1;
+            __success = true;
           endif
         end_try_catch
         clear __test__;
@@ -472,62 +524,62 @@
       ## Code already processed.
       __code = "";
 
-### TESTIF
+    ### TESTIF
 
     elseif (strcmp (__type, "testif"))
       __e = regexp (__code, '.$', 'lineanchors', 'once');
-      ## Strip comment any comment from testif line before looking for features
+      ## Strip any comment from testif line before looking for features
       __feat_line = strtok (__code(1:__e), '#%'); 
       __feat = regexp (__feat_line, '\w+', 'match');
       __feat = strrep (__feat, "HAVE_", "");
       __have_feat = __have_feature__ (__feat);
       if (__have_feat)
-        __istest = 1;
+        __istest = true;
         __code = __code(__e + 1 : end);
       else
         __xskip++;
-        __istest = 0;
+        __istest = false;
         __code = ""; # Skip the code.
-        __msg = sprintf ("%sskipped test\n", __signal_skip);
+        __msg = [__signal_skip "skipped test\n"];
       endif
 
-### TEST
+    ### TEST
 
     elseif (strcmp (__type, "test"))
-      __istest = 1;
+      __istest = true;
       ## Code will be evaluated below.
 
-### XTEST
+    ### XTEST
 
     elseif (strcmp (__type, "xtest"))
-      __istest = 0;
-      __isxtest = 1;
+      __istest = false;
+      __isxtest = true;
       ## Code will be evaluated below.
 
-### Comment block.
+    ### Comment block.
 
     elseif (strcmp (__block(1:1), "#"))
-      __istest = 0;
+      __istest = false;
       __code = ""; # skip the code
 
-### Unknown block.
+    ### Unknown block.
 
     else
-      __istest = 1;
-      __success = 0;
-      __msg = sprintf ("%sunknown test type!\n", __signal_fail);
+      __istest = true;
+      __success = false;
+      __msg = [__signal_fail "unknown test type!\n"];
       __code = ""; # skip the code
     endif
 
     ## evaluate code for test, shared, and assert.
     if (! isempty(__code))
       try
-        ## FIXME: need to check for embedded test functions, which cause
+        ## FIXME: Must check for embedded test functions, which cause
         ## segfaults, until issues with subfunctions in functions are resolved.
         embed_func = regexp (__code, '^\s*function ', 'once', 'lineanchors');
         if (isempty (embed_func))
           eval (sprintf ("function %s__test__(%s)\n%s\nendfunction",
-                         __shared_r,__shared, __code));
+                         __shared_r, __shared, __code));
           eval (sprintf ("%s__test__(%s);", __shared_r, __shared));
         else
           error (["Functions embedded in %!test blocks are not allowed.\n", ...
@@ -535,12 +587,12 @@
         endif
       catch
         if (strcmp (__type, "xtest"))
-           __msg = sprintf ("%sknown failure\n%s", __signal_fail, lasterr ());
+           __msg = [__signal_fail "known failure\n" lasterr()];
            __xfail++;
-           __success = 0;
+           __success = false;
         else
-           __msg = sprintf ("%stest failed\n%s", __signal_fail, lasterr ());
-           __success = 0;
+           __msg = [__signal_fail "test failed\n" lasterr()];
+           __success = false;
         endif
         if (isempty (lasterr ()))
           error ("empty error text, probably Ctrl-C --- aborting");
@@ -550,29 +602,32 @@
     endif
 
     ## All done.  Remember if we were successful and print any messages.
-    if (! isempty (__msg))
+    if (! isempty (__msg) && (__verbose >= 0 || __logfile))
       ## Make sure the user knows what caused the error.
-      if (! __verbose)
+      if (__verbose < 1)
         fprintf (__fid, "%s%s\n", __signal_block, __block);
         fflush (__fid);
       endif
-      fputs (__fid, __msg);
-      fputs (__fid, "\n");
+      fprintf (__fid, "%s\n", __msg);
       fflush (__fid);
       ## Show the variable context.
       if (! strcmp (__type, "error") && ! strcmp (__type, "testif")
           && ! all (__shared == " "))
         fputs (__fid, "shared variables ");
-        eval (sprintf ("fdisp(__fid,bundle(%s));", __shared));
+        eval (sprintf ("fdisp(__fid,var2struct(%s));", __shared));
         fflush (__fid);
       endif
     endif
-    if (__success == 0 && !__isxtest)
-      __all_success = 0;
-      ## Stop after one error if not in batch mode.
-      if (! __batch)
+    if (! __success && ! __isxtest)
+      __all_success = false;
+      ## Stop after 1 error if not in batch mode or only pass/fail requested.
+      if (! __batch || nargout == 1)
         if (nargout > 0)
-          __ret1 = __ret2 = 0;
+          if (nargout == 1)
+            __n = false;
+          else
+            __n = __nmax = 0;
+          endif
         endif
         if (__close_fid)
           fclose (__fid);
@@ -581,10 +636,11 @@
       endif
     endif
     __tests += (__istest || __isxtest);
-    __successes += __success * (__istest || __isxtest);
+    __successes += __success && (__istest || __isxtest);
   endfor
-  ## Clear any test functions created
-  eval (__clear, "");
+
+  ## Clear any functions created during test run 
+  eval (__clearfcn, "");
 
   if (nargout == 0)
     if (__tests || __xfail || __xskip)
@@ -604,20 +660,22 @@
       printf ("%s%s has no tests available\n", __signal_empty, __file);
     endif
   elseif (__grabdemo)
-    __ret1 = __demo_code;
-    __ret2 = __demo_idx;
+    __n    = __demo_code;
+    __nmax = __demo_idx;
   elseif (nargout == 1)
-    __ret1 = __all_success;
+    __n = __all_success;
   else
-    __ret1 = __successes;
-    __ret2 = __tests;
-    __ret3 = __xfail;
-    __ret4 = __xskip;
+    __n      = __successes;
+    __nmax   = __tests;
+    __nxfail = __xfail;
+    __nskip  = __xskip;
   endif
+
 endfunction
 
+
 ## Create structure with fieldnames the name of the input variables.
-function s = varstruct (varargin)
+function s = var2struct (varargin)
   for i = 1:nargin
     s.(deblank (argn(i,:))) = varargin{i};
   endfor
@@ -679,14 +737,6 @@
   str = str(idx:end);
 endfunction
 
-## Make a structure out of the named variables
-## (based on Etienne Grossmann's tar function).
-function s = bundle (varargin)
-  for i = 1:nargin
-    s.(deblank (argn(i,:))) = varargin{i};
-  endfor
-endfunction
-
 function body = __extract_test_code (nm)
   fid = fopen (nm, "rt");
   body = [];
@@ -705,23 +755,24 @@
 endfunction
 
 
-### example from toeplitz
+## example from toeplitz
 %!shared msg1,msg2
-%! msg1="C must be a vector";
-%! msg2="C and R must be vectors";
-%!fail ('toeplitz ([])', msg1);
-%!fail ('toeplitz ([1,2;3,4])', msg1);
-%!fail ('toeplitz ([1,2],[])', msg2);
-%!fail ('toeplitz ([1,2],[1,2;3,4])', msg2);
-%!fail ('toeplitz ([1,2;3,4],[1,2])', msg2);
-% !fail ('toeplitz','usage: toeplitz'); # usage doesn't generate an error
-% !fail ('toeplitz (1, 2, 3)', 'usage: toeplitz');
-%!test  assert (toeplitz ([1,2,3], [1,4]), [1,4; 2,1; 3,2]);
-%!demo  toeplitz ([1,2,3,4],[1,5,6])
+%! msg1 = "C must be a vector";
+%! msg2 = "C and R must be vectors";
+%!fail ("toeplitz ([])", msg1)
+%!fail ("toeplitz ([1,2;3,4])", msg1)
+%!fail ("toeplitz ([1,2],[])", msg2)
+%!fail ("toeplitz ([1,2],[1,2;3,4])", msg2)
+%!fail ("toeplitz ([1,2;3,4],[1,2])", msg2)
+%!test fail ("toeplitz", "Invalid call to toeplitz")
+%!fail ("toeplitz (1, 2, 3)", "Invalid call to toeplitz")
+%!test assert (toeplitz ([1,2,3], [1,4]), [1,4; 2,1; 3,2])
+%!assert (toeplitz ([1,2,3], [1,4]), [1,4; 2,1; 3,2])
+%!demo toeplitz ([1,2,3,4],[1,5,6])
 
-### example from kron
-%!#error kron  # FIXME: suppress these until we can handle output
-%!#error kron(1,2,3)
+## example from kron
+%!error <Invalid call to kron> kron ()
+%!error <Invalid call to kron> kron (1)
 %!test assert (isempty (kron ([], rand (3, 4))))
 %!test assert (isempty (kron (rand (3, 4), [])))
 %!test assert (isempty (kron ([], [])))
@@ -741,101 +792,110 @@
 %!assert (kron ([1, 2; 3, 4], A), [ A, 2*A; 3*A, 4*A ])
 %!test
 %! res = [1,-1,2,-2,3,-3; 2,-2,4,-4,6,-6; 4,-4,5,-5,6,-6; 8,-8,10,-10,12,-12];
-%! assert (kron (A, B), res)
+%! assert (kron (A, B), res);
+%!shared  # clear out shared variables
+
+## Now verify test() itself
 
-### an extended demo from specgram
-%!#demo
-%! ## Speech spectrogram
-%! [x, Fs] = auload (file_in_loadpath ("sample.wav")); # audio file
-%! step = fix (5*Fs/1000);     # one spectral slice every 5 ms
-%! window = fix (40*Fs/1000);  # 40 ms data window
-%! fftn = 2^nextpow2 (window); # next highest power of 2
-%! [S, f, t] = specgram (x, fftn, Fs, window, window-step);
-%! S = abs (S (2:fftn*4000/Fs,:)); # magnitude in range 0<f<=4000 Hz.
-%! S = S/max(max(S));          # normalize magnitude so that max is 0 dB.
-%! S = max (S, 10^(-40/10));   # clip below -40 dB.
-%! S = min (S, 10^(-3/10));    # clip above -3 dB.
-%! imagesc (flipud (20*log10 (S)), 1);
-%! % you should now see a spectrogram in the image window
-
-
-## now test 'test' itself
-
-## usage and error testing
-% !fail ('test','usage.*test')           # no args, generates usage()
-% !fail ('test (1,2,3,4)','usage.*test') # too many args, generates usage()
-%!fail ('test ("test", "bogus")','unknown flag')  # incorrect args
+## Test 'fail' keyword
+%!fail ("test", "Invalid call to test")  # no args, generates usage()
+%!fail ("test (1,2,3,4)", "usage.*test") # too many args, generates usage()
+%!fail ('test ("test", "bogus")', "unknown flag")  # incorrect args
 %!fail ('garbage','garbage.*undefined')  # usage on nonexistent function should be
 
-%!error test                     # no args, generates usage()
-%!error test (1,2,3,4)           # too many args, generates usage()
-%!error <unknown flag> test ("test", 'bogus'); # incorrect args, generates error()
-%!error <garbage' undefined> garbage           # usage on nonexistent function should be
-
-%!error test ("test", 'bogus');  # test without pattern
+## Test 'error' keyword 
+%!error test              # no args, generates usage()
+%!error test (1,2,3,4)    # too many args, generates usage()
+%!error <unknown flag> test ("test", "bogus"); # incorrect args
+%!error test ("test", "bogus");  # test without pattern
+%!error <'garbage' undefined> garbage; # usage on nonexistent function is error
 
-%!test
-%! lastwarn();            # clear last warning just in case
-
-%!warning <warning message> warning ('warning message');
+## Test 'warning' keyword 
+%!warning warning ("warning message");   # no pattern
+%!warning <warning message> warning ("warning message");   # with pattern
 
-## test of shared variables
+## Test 'shared' keyword
 %!shared a                # create a shared variable
-%!test   a=3;             # assign to a shared variable
-%!test   assert (a,3)     # variable should equal 3
+%!test a = 3;             # assign to a shared variable
+%!test assert (a, 3)      # variable should equal 3
 %!shared b,c              # replace shared variables
 %!test assert (!exist ("a", "var"));  # a no longer exists
 %!test assert (isempty (b));   # variables start off empty
 %!shared a,b,c            # recreate a shared variable
 %!test assert (isempty (a));   # value is empty even if it had a previous value
 %!test a=1; b=2; c=3;   # give values to all variables
-%!test assert ([a,b,c],[1,2,3]); # test all of them together
-%!test c=6;             # update a value
-%!test assert ([a, b, c],[1, 2, 6]); # show that the update sticks
-%!shared                     # clear all shared variables
+%!test assert ([a,b,c], [1,2,3]); # test all of them together
+%!test c=6;               # update a value
+%!test assert ([a,b,c], [1,2,6]); # show that the update sticks
+%!shared                  # clear all shared variables
 %!test assert (!exist ("a", "var")) # show that they are cleared
-%!shared a,b,c               # support for initializer shorthand
+%!shared a,b,c            # support for initializer shorthand
 %! a=1; b=2; c=4;
+%!shared                  # clear all shared variables for remainder of tests
 
+## Test 'function' keyword 
 %!function x = __test_a (y)
 %! x = 2*y;
 %!endfunction
-%!assert (__test_a (2),4);       # Test a test function
+%!assert (__test_a (2), 4)  # Test a test function
 
 %!function __test_a (y)
 %! x = 2*y;
 %!endfunction
 %!test
-%! __test_a (2);                # Test a test function with no return value
+%! __test_a (2);            # Test a test function with no return value
 
 %!function [x,z] = __test_a (y)
 %! x = 2*y;
 %! z = 3*y;
 %!endfunction
-%!test                   # Test a test function with multiple returns
-%! [x,z] = __test_a (3);
+%!test                      
+%! [x,z] = __test_a (3);    # Test a test function with multiple returns
 %! assert (x,6);
 %! assert (z,9);
 
-## test of assert block
-%!assert (isempty ([]))      # support for test assert shorthand
+## Test 'assert' keyword 
+%!assert (isempty ([]))     # support for test assert shorthand
+%!assert (size (ones (1,2,3)), [1 2 3])
 
-## demo blocks
-%!demo                   # multiline demo block
+## Test 'demo' keyword 
+%!demo                      # multiline demo block
 %! t = [0:0.01:2*pi]; x = sin (t);
 %! plot (t,x);
 %! % you should now see a sine wave in your figure window
-%!demo a=3               # single line demo blocks work too
+
+%!demo a=3                  # single line demo blocks work too
+
+%!test
+%! [code, idx] = test ("test", "grabdemo");
+%! assert (numel (idx), 4);
+%! assert (code(idx(3):end),
+%!         " a=3                  # single line demo blocks work too");
 
-## this is a comment block. it can contain anything.
+## Test 'testif' keyword 
+%!testif HAVE_BOGUS_FEATURE
+%! error ("testif executed code despite not having feature");
+
+## Test 'xtest' keyword 
+%!xtest
+%! assert (1, 1);      # Test passes
+%!xtest
+%! assert (0, 1);      # Test fails
+
+## Test comment block. it can contain anything.
 %!##
 %! it is the "#" as the block type that makes it a comment
 %! and it stays as a comment even through continuation lines
 %! which means that it works well with commenting out whole tests
 
-% !# failure tests.  All the following should fail. These tests should
-% !# be disabled unless you are developing test() since users don't
-% !# like to be presented with expected failures.  I use % ! to disable.
+%% Test test() input validation
+%!error <NAME must be a string> test (1)
+%!error <second argument must be a string> test ("ls", 1)
+%!error test ([], "normal")
+
+## All of the following tests should fail.  These tests should
+## be disabled unless you are developing test() since users don't
+## like to be presented with expected failures.  I use '% !' to disable.
 % !test   error("---------Failure tests.  Use test('test','verbose',1)");
 % !test   assert([a,b,c],[1,3,6]);   # variables have wrong values
 % !bogus                     # unknown block type
@@ -848,10 +908,10 @@
 % !demo   with syntax error  # syntax errors in demo fail properly
 % !shared a,b,c
 % !demo                      # shared variables not available in demo
-% ! assert(exist("a", "var"))
+% ! assert (exist ("a", "var"))
 % !error
-% ! test('/etc/passwd');
-% ! test("nonexistent file");
+% ! test ('/etc/passwd');
+% ! test ("nonexistent file");
 % ! ## These don't signal an error, so the test for an error fails. Note
 % ! ## that the call doesn't reference the current fid (it is unavailable),
 % ! ## so of course the informational message is not printed in the log.