Mercurial > octave-nkf

annotate scripts/testfun/__run_test_suite__.m @ 19630:0e1f5a750d00

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
maint: Periodic merge of gui-release to default.
author John W. Eaton <jwe@octave.org>
date Tue, 20 Jan 2015 10:24:46 -0500
parents 3f29b433bd5d 446c46af4b42
children db92e7e28e1f
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
17744
d63878346099 maint: Update copyright notices for release.
John W. Eaton <jwe@octave.org>
parents: 17386
diff changeset
1 ## Copyright (C) 2005-2013 David Bateman
16019
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
2 ##
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
3 ## This file is part of Octave.
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
4 ##
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
5 ## Octave is free software; you can redistribute it and/or modify it
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
6 ## under the terms of the GNU General Public License as published by
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
7 ## the Free Software Foundation; either version 3 of the License, or (at
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
8 ## your option) any later version.
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
9 ##
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
10 ## Octave is distributed in the hope that it will be useful, but
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
11 ## WITHOUT ANY WARRANTY; without even the implied warranty of
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
12 ## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
13 ## General Public License for more details.
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
14 ##
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
15 ## You should have received a copy of the GNU General Public License
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
16 ## along with Octave; see the file COPYING. If not, see
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
17 ## <http://www.gnu.org/licenses/>.
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
18
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
19 ## -*- texinfo -*-
16816
12005245b645 doc: Periodic grammarcheck of documentation.
Rik <rik@octave.org>
parents: 16030
diff changeset
20 ## @deftypefn {Function File} {} __run_test_suite__ (@var{fcndirs}, @var{fixedtestdirs})
16019
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
21 ## Undocumented internal function.
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
22 ## @end deftypefn
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
23
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
24 function __run_test_suite__ (fcndirs, fixedtestdirs)
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
25 testsdir = octave_config_info ("octtestsdir");
16030
1af8d21608b7 rename all test files in the test directory from test_X.m to X.tst
John W. Eaton <jwe@octave.org>
parents: 16019
diff changeset
26 libinterptestdir = fullfile (testsdir, "libinterp");
1af8d21608b7 rename all test files in the test directory from test_X.m to X.tst
John W. Eaton <jwe@octave.org>
parents: 16019
diff changeset
27 liboctavetestdir = fullfile (testsdir, "liboctave");
16019
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
28 fixedtestdir = fullfile (testsdir, "fixed");
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
29 fcnfiledir = octave_config_info ("fcnfiledir");
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
30 if (nargin == 0)
18028
9f59f4f74986 test: Run test suite tests in compilation order (liboctave, libinterp, scripts).
Rik <rik@octave.org>
parents: 17744
diff changeset
31 fcndirs = { liboctavetestdir, libinterptestdir, fcnfiledir };
16019
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
32 fixedtestdirs = { fixedtestdir };
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
33 endif
19403
29228fa3edba improve reporting for __run_test_suite__ (bug #43732)
John W. Eaton <jwe@octave.org>
parents: 18115
diff changeset
34 global files_with_no_tests;
29228fa3edba improve reporting for __run_test_suite__ (bug #43732)
John W. Eaton <jwe@octave.org>
parents: 18115
diff changeset
35 global files_with_tests;
29228fa3edba improve reporting for __run_test_suite__ (bug #43732)
John W. Eaton <jwe@octave.org>
parents: 18115
diff changeset
36 files_with_no_tests = {};
29228fa3edba improve reporting for __run_test_suite__ (bug #43732)
John W. Eaton <jwe@octave.org>
parents: 18115
diff changeset
37 files_with_tests = {};
17182
fa724bdd52d0 __run_test_suite__.m: Recode regexps for performance. Don't search private '.' dirs.
Rik <rik@octave.org>
parents: 16994
diff changeset
38 ## FIXME: These names don't really make sense if we are running
fa724bdd52d0 __run_test_suite__.m: Recode regexps for performance. Don't search private '.' dirs.
Rik <rik@octave.org>
parents: 16994
diff changeset
39 ## tests for an installed copy of Octave.
16019
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
40 global topsrcdir = fcnfiledir;
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
41 global topbuilddir = testsdir;
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
42 pso = page_screen_output ();
19478
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
43 orig_wstate = warning ();
19403
29228fa3edba improve reporting for __run_test_suite__ (bug #43732)
John W. Eaton <jwe@octave.org>
parents: 18115
diff changeset
44 logfile = make_absolute_filename ("fntests.log");
19478
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
45 unwind_protect
17182
fa724bdd52d0 __run_test_suite__.m: Recode regexps for performance. Don't search private '.' dirs.
Rik <rik@octave.org>
parents: 16994
diff changeset
46 page_screen_output (false);
19478
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
47 warning ("on", "quiet");
16019
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
48 warning ("off", "Octave:deprecated-function");
19478
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
49 try
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
50 fid = fopen (logfile, "wt");
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
51 if (fid < 0)
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
52 error ("could not open %s for writing", logfile);
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
53 endif
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
54 test ("", "explain", fid);
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
55 dp = dn = dxf = dsk = 0;
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
56 puts ("\nIntegrated test scripts:\n\n");
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
57 for i = 1:length (fcndirs)
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
58 [p, n, xf, sk] = run_test_script (fid, fcndirs{i});
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
59 dp += p;
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
60 dn += n;
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
61 dxf += xf;
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
62 dsk += sk;
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
63 endfor
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
64 puts ("\nFixed test scripts:\n\n");
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
65 for i = 1:length (fixedtestdirs)
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
66 [p, n, xf, sk] = run_test_dir (fid, fixedtestdirs{i});
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
67 dp += p;
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
68 dn += n;
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
69 dxf += xf;
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
70 dsk += sk;
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
71 endfor
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
72 puts ("\nSummary:\n\n");
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
73 nfail = dn - dp - dxf;
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
74 printf (" PASS %6d\n", dp);
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
75 printf (" FAIL %6d\n", nfail);
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
76 if (dxf > 0)
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
77 printf (" XFAIL %6d\n", dxf);
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
78 endif
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
79 if (dsk > 0)
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
80 printf (" SKIPPED %6d\n", dsk);
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
81 endif
16019
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
82 puts ("\n");
19478
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
83 printf ("See the file %s for additional details.\n", logfile);
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
84 if (dxf > 0)
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
85 puts ("\n");
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
86 puts ("Expected failures (listed as XFAIL above) are known bugs.\n");
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
87 puts ("Please help improve Octave by contributing fixes for them.\n");
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
88 endif
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
89 if (dsk > 0)
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
90 puts ("\n");
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
91 puts ("Tests are most often skipped because the features they require\n");
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
92 puts ("have been disabled. Features are most often disabled because\n");
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
93 puts ("they require dependencies that were not present when Octave\n");
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
94 puts ("was built. The configure script should have printed a summary\n");
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
95 puts ("at the end of its run indicating which dependencies were not found.\n");
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
96 endif
16019
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
97
19478
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
98 ## Weed out deprecated and private functions
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
99 weed_idx = cellfun (@isempty, regexp (files_with_tests, '\<deprecated\>|\<private\>', 'once'));
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
100 files_with_tests = files_with_tests(weed_idx);
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
101 weed_idx = cellfun (@isempty, regexp (files_with_no_tests, '\<deprecated\>|\<private\>', 'once'));
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
102 files_with_no_tests = files_with_no_tests(weed_idx);
16019
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
103
19478
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
104 report_files_with_no_tests (files_with_tests, files_with_no_tests, ".m");
16019
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
105
19478
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
106 puts ("\nPlease help improve Octave by contributing tests for these files\n");
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
107 printf ("(see the list in the file %s).\n\n", logfile);
16019
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
108
19478
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
109 fprintf (fid, "\nFiles with no tests:\n\n%s",
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
110 list_in_columns (files_with_no_tests, 80));
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
111 fclose (fid);
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
112 catch
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
113 disp (lasterr ());
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
114 end_try_catch
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
115 unwind_protect_cleanup
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
116 warning ("off", "all");
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
117 warning (orig_wstate);
16019
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
118 page_screen_output (pso);
19478
3f29b433bd5d do a better job of preserving warning state when running tests
John W. Eaton <jwe@octave.org>
parents: 19411
diff changeset
119 end_unwind_protect
16019
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
120 endfunction
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
121
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
122 function print_test_file_name (nm)
18167
441121728230 tweak format of per-file info lines in __run_test_suite__ output
John W. Eaton <jwe@octave.org>
parents: 18166
diff changeset
123 filler = repmat (".", 1, 60-length (nm));
16019
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
124 printf (" %s %s", nm, filler);
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
125 endfunction
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
126
18166
c9b4d3177de3 Don't count expected failures as passed tests.
Carlo de Falco <cdf@users.sourceforge.net>
parents: 18115
diff changeset
127 function print_pass_fail (p, n, xf, sk)
c9b4d3177de3 Don't count expected failures as passed tests.
Carlo de Falco <cdf@users.sourceforge.net>
parents: 18115
diff changeset
128 if ((n + sk) > 0)
18167
441121728230 tweak format of per-file info lines in __run_test_suite__ output
John W. Eaton <jwe@octave.org>
parents: 18166
diff changeset
129 printf (" PASS %4d/%-4d", p, n);
18166
c9b4d3177de3 Don't count expected failures as passed tests.
Carlo de Falco <cdf@users.sourceforge.net>
parents: 18115
diff changeset
130 nfail = n - p - xf;
16019
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
131 if (nfail > 0)
18167
441121728230 tweak format of per-file info lines in __run_test_suite__ output
John W. Eaton <jwe@octave.org>
parents: 18166
diff changeset
132 printf ("\n%71s %3d", "FAIL ", nfail);
18166
c9b4d3177de3 Don't count expected failures as passed tests.
Carlo de Falco <cdf@users.sourceforge.net>
parents: 18115
diff changeset
133 endif
c9b4d3177de3 Don't count expected failures as passed tests.
Carlo de Falco <cdf@users.sourceforge.net>
parents: 18115
diff changeset
134 if (sk > 0)
18167
441121728230 tweak format of per-file info lines in __run_test_suite__ output
John W. Eaton <jwe@octave.org>
parents: 18166
diff changeset
135 printf ("\n%71s %3d", "SKIP ", sk);
18166
c9b4d3177de3 Don't count expected failures as passed tests.
Carlo de Falco <cdf@users.sourceforge.net>
parents: 18115
diff changeset
136 endif
c9b4d3177de3 Don't count expected failures as passed tests.
Carlo de Falco <cdf@users.sourceforge.net>
parents: 18115
diff changeset
137 if (xf > 0)
18167
441121728230 tweak format of per-file info lines in __run_test_suite__ output
John W. Eaton <jwe@octave.org>
parents: 18166
diff changeset
138 printf ("\n%71s %3d", "XFAIL", xf);
16019
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
139 endif
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
140 endif
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
141 puts ("\n");
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
142 endfunction
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
143
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
144 function retval = has_functions (f)
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
145 n = length (f);
17182
fa724bdd52d0 __run_test_suite__.m: Recode regexps for performance. Don't search private '.' dirs.
Rik <rik@octave.org>
parents: 16994
diff changeset
146 if (n > 3 && strcmpi (f((end-2):end), ".cc"))
16019
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
147 fid = fopen (f);
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
148 if (fid >= 0)
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
149 str = fread (fid, "*char")';
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
150 fclose (fid);
17182
fa724bdd52d0 __run_test_suite__.m: Recode regexps for performance. Don't search private '.' dirs.
Rik <rik@octave.org>
parents: 16994
diff changeset
151 retval = ! isempty (regexp (str,'^(DEFUN|DEFUN_DLD)\>',
fa724bdd52d0 __run_test_suite__.m: Recode regexps for performance. Don't search private '.' dirs.
Rik <rik@octave.org>
parents: 16994
diff changeset
152 'lineanchors', 'once'));
16019
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
153 else
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
154 error ("fopen failed: %s", f);
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
155 endif
17182
fa724bdd52d0 __run_test_suite__.m: Recode regexps for performance. Don't search private '.' dirs.
Rik <rik@octave.org>
parents: 16994
diff changeset
156 elseif (n > 2 && strcmpi (f((end-1):end), ".m"))
16019
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
157 retval = true;
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
158 else
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
159 retval = false;
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
160 endif
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
161 endfunction
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
162
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
163 function retval = has_tests (f)
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
164 fid = fopen (f);
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
165 if (fid >= 0)
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
166 str = fread (fid, "*char")';
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
167 fclose (fid);
18115
523dd79ee726 also check for xtest in __run_test_suite__
John W. Eaton <jwe@octave.org>
parents: 18028
diff changeset
168 retval = ! isempty (regexp (str,
523dd79ee726 also check for xtest in __run_test_suite__
John W. Eaton <jwe@octave.org>
parents: 18028
diff changeset
169 '^%!(assert|error|fail|test|xtest|warning)',
523dd79ee726 also check for xtest in __run_test_suite__
John W. Eaton <jwe@octave.org>
parents: 18028
diff changeset
170 'lineanchors', 'once'));
16019
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
171 else
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
172 error ("fopen failed: %s", f);
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
173 endif
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
174 endfunction
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
175
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
176 function [dp, dn, dxf, dsk] = run_test_dir (fid, d);
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
177 global files_with_tests;
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
178 global files_with_no_tests;
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
179 lst = dir (d);
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
180 dp = dn = dxf = dsk = 0;
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
181 for i = 1:length (lst)
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
182 nm = lst(i).name;
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
183 if (lst(i).isdir
17182
fa724bdd52d0 __run_test_suite__.m: Recode regexps for performance. Don't search private '.' dirs.
Rik <rik@octave.org>
parents: 16994
diff changeset
184 && nm(1) != "." && ! strcmp (nm, "private") && nm(1) != "@")
16019
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
185 [p, n, xf, sk] = run_test_dir (fid, [d, filesep, nm]);
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
186 dp += p;
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
187 dn += n;
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
188 dxf += xf;
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
189 dsk += sk;
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
190 endif
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
191 endfor
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
192 saved_dir = pwd ();
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
193 unwind_protect
19314
98a18bc3208e Use cd rather than chdir in core Octave code.
Rik <rik@octave.org>
parents: 19205
diff changeset
194 cd (d);
16019
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
195 for i = 1:length (lst)
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
196 nm = lst(i).name;
17182
fa724bdd52d0 __run_test_suite__.m: Recode regexps for performance. Don't search private '.' dirs.
Rik <rik@octave.org>
parents: 16994
diff changeset
197 if (length (nm) > 4 && strcmpi (nm((end-3):end), ".tst"))
16019
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
198 p = n = xf = sk = 0;
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
199 ffnm = fullfile (d, nm);
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
200 if (has_tests (ffnm))
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
201 print_test_file_name (nm);
16030
1af8d21608b7 rename all test files in the test directory from test_X.m to X.tst
John W. Eaton <jwe@octave.org>
parents: 16019
diff changeset
202 [p, n, xf, sk] = test (nm, "quiet", fid);
18166
c9b4d3177de3 Don't count expected failures as passed tests.
Carlo de Falco <cdf@users.sourceforge.net>
parents: 18115
diff changeset
203 print_pass_fail (p, n, xf, sk);
16019
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
204 files_with_tests(end+1) = ffnm;
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
205 else
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
206 files_with_no_tests(end+1) = ffnm;
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
207 endif
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
208 dp += p;
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
209 dn += n;
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
210 dxf += xf;
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
211 dsk += sk;
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
212 endif
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
213 endfor
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
214 unwind_protect_cleanup
19314
98a18bc3208e Use cd rather than chdir in core Octave code.
Rik <rik@octave.org>
parents: 19205
diff changeset
215 cd (saved_dir);
16019
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
216 end_unwind_protect
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
217 endfunction
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
218
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
219 function [dp, dn, dxf, dsk] = run_test_script (fid, d);
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
220 global files_with_tests;
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
221 global files_with_no_tests;
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
222 global topsrcdir;
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
223 global topbuilddir;
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
224 lst = dir (d);
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
225 dp = dn = dxf = dsk = 0;
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
226 for i = 1:length (lst)
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
227 nm = lst(i).name;
17182
fa724bdd52d0 __run_test_suite__.m: Recode regexps for performance. Don't search private '.' dirs.
Rik <rik@octave.org>
parents: 16994
diff changeset
228 if (lst(i).isdir && nm(1) != ".")
16019
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
229 [p, n, xf, sk] = run_test_script (fid, [d, filesep, nm]);
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
230 dp += p;
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
231 dn += n;
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
232 dxf += xf;
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
233 dsk += sk;
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
234 endif
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
235 endfor
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
236 for i = 1:length (lst)
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
237 nm = lst(i).name;
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
238 ## Ignore hidden files
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
239 if (nm(1) == '.')
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
240 continue
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
241 endif
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
242 f = fullfile (d, nm);
17182
fa724bdd52d0 __run_test_suite__.m: Recode regexps for performance. Don't search private '.' dirs.
Rik <rik@octave.org>
parents: 16994
diff changeset
243 if ((length (nm) > 2 && strcmpi (nm((end-1):end), ".m"))
16030
1af8d21608b7 rename all test files in the test directory from test_X.m to X.tst
John W. Eaton <jwe@octave.org>
parents: 16019
diff changeset
244 || (length (nm) > 4
17182
fa724bdd52d0 __run_test_suite__.m: Recode regexps for performance. Don't search private '.' dirs.
Rik <rik@octave.org>
parents: 16994
diff changeset
245 && ( strcmpi (nm((end-3):end), "-tst")
fa724bdd52d0 __run_test_suite__.m: Recode regexps for performance. Don't search private '.' dirs.
Rik <rik@octave.org>
parents: 16994
diff changeset
246 || strcmpi (nm((end-3):end), ".tst"))))
16019
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
247 p = n = xf = 0;
17182
fa724bdd52d0 __run_test_suite__.m: Recode regexps for performance. Don't search private '.' dirs.
Rik <rik@octave.org>
parents: 16994
diff changeset
248 ## Only run if it contains %!test, %!assert, %!error, %!fail, or %!warning
16019
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
249 if (has_tests (f))
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
250 tmp = strrep (f, [topsrcdir, filesep], "");
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
251 tmp = strrep (tmp, [topbuilddir, filesep], "");
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
252 print_test_file_name (tmp);
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
253 [p, n, xf, sk] = test (f, "quiet", fid);
18166
c9b4d3177de3 Don't count expected failures as passed tests.
Carlo de Falco <cdf@users.sourceforge.net>
parents: 18115
diff changeset
254 print_pass_fail (p, n, xf, sk);
16019
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
255 dp += p;
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
256 dn += n;
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
257 dxf += xf;
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
258 dsk += sk;
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
259 files_with_tests(end+1) = f;
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
260 else
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
261 ## To reduce the list length, only mark .cc files that contain
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
262 ## DEFUN definitions.
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
263 files_with_no_tests(end+1) = f;
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
264 endif
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
265 endif
19627
446c46af4b42 strip trailing whitespace from most source files
John W. Eaton <jwe@octave.org>
parents: 19403
diff changeset
266 endfor
16019
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
267 ## printf("%s%s -> passes %d of %d tests\n", ident, d, dp, dn);
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
268 endfunction
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
269
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
270 function n = num_elts_matching_pattern (lst, pat)
17386
6dbc866379e2 Replace cellfun() occurrences with faster code where possible.
Rik <rik@octave.org>
parents: 17338
diff changeset
271 n = sum (! cellfun ("isempty", regexp (lst, pat, 'once')));
16019
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
272 endfunction
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
273
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
274 function report_files_with_no_tests (with, without, typ)
16994
333243133364 Use matrix concatenation for strings, rather than cstrcat(), for clarity and performance.
Rik <rik@octave.org>
parents: 16816
diff changeset
275 pat = ['\' typ "$"];
16019
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
276 n_with = num_elts_matching_pattern (with, pat);
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
277 n_without = num_elts_matching_pattern (without, pat);
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
278 n_tot = n_with + n_without;
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
279 printf ("\n%d (of %d) %s files have no tests.\n", n_without, n_tot, typ);
5b2126a8c84f install tests and functions to run them
John W. Eaton <jwe@octave.org>
parents:
diff changeset
280 endfunction
17338
1c89599167a6 maint: End m-files with 1 blank line.
Rik <rik@octave.org>
parents: 17182
diff changeset
281
19205
702aa79dc482 Add BIST tests for various quasi-internal m-files.
Rik <rik@octave.org>
parents: 18167
diff changeset
282
702aa79dc482 Add BIST tests for various quasi-internal m-files.
Rik <rik@octave.org>
parents: 18167
diff changeset
283 ## No test coverage for internal function. It is tested through calling fcn.
702aa79dc482 Add BIST tests for various quasi-internal m-files.
Rik <rik@octave.org>
parents: 18167
diff changeset
284 %!assert (1)
702aa79dc482 Add BIST tests for various quasi-internal m-files.
Rik <rik@octave.org>
parents: 18167
diff changeset
285