# HG changeset patch # User Rik # Date 1376594857 25200 # Node ID a28c0d73e2532c0ce1f7de54763bedc9f77e5694 # Parent 7fb4461997aa212b9e44397351c7c0eb7d78f32b assert.m: Added many more %!tests for function. Fixed multiple bugs discovered. * scripts/testfun/assert.m: Adjust call_depth at every exit point of function to fix recursion errors. Fix incorrectly reported errors with row vector input. Add new reason text "expected XXX, but observed YYY" where XXX is "cell" or "struct". Stop reporting multiple errors when condition is both an exceptional value and not meeting tolerance. Improve %!tests to actually catch assert failing in the correct way, rather than just failing. diff -r 7fb4461997aa -r a28c0d73e253 scripts/testfun/assert.m --- a/scripts/testfun/assert.m Wed Aug 07 20:18:04 2013 -0500 +++ b/scripts/testfun/assert.m Thu Aug 15 12:27:37 2013 -0700 @@ -77,6 +77,7 @@ if (nargin == 1 || (nargin > 1 && islogical (cond) && ischar (varargin{1}))) if ((! isnumeric (cond) && ! islogical (cond)) || ! all (cond(:))) + call_depth--; if (nargin == 1) ## Perhaps, say which elements failed? error ("assert %s failed", in); @@ -100,18 +101,15 @@ if (ischar (expected)) if (! ischar (cond)) - err.index{end+1} = "[]"; + err.index{end+1} = "."; err.expected{end+1} = expected; if (isnumeric (cond)) err.observed{end+1} = num2str (cond); err.reason{end+1} = "Expected string, but observed number"; - elseif (iscell (cond)) - err.observed{end+1} = "{}"; - err.reason{end+1} = "Expected string, but observed cell"; else - err.observed{end+1} = "[]"; - err.reason{end+1} = "Expected string, but observed struct"; - end + err.observed{end+1} = "O"; + err.reason{end+1} = ["Expected string, but observed " class(cond)]; + endif elseif (! strcmp (cond, expected)) err.index{end+1} = "[]"; err.observed{end+1} = cond; @@ -120,11 +118,17 @@ endif elseif (iscell (expected)) - if (! iscell (cond) || any (size (cond) != size (expected))) - err.index{end+1} = "{}"; + if (! iscell (cond)) + err.index{end+1} = "."; err.observed{end+1} = "O"; err.expected{end+1} = "E"; - err.reason{end+1} = "Cell sizes don't match"; + err.reason{end+1} = ["Expected cell, but observed " class(cond)]; + elseif (ndims (cond) != ndims (expected) + || any (size (cond) != size (expected))) + err.index{end+1} = "."; + err.observed{end+1} = ["O(" sprintf("%dx", size(cond))(1:end-1) ")"]; + err.expected{end+1} = ["E(" sprintf("%dx", size(expected))(1:end-1) ")"]; + err.reason{end+1} = "Dimensions don't match"; else try ## Recursively compare cell arrays @@ -140,11 +144,18 @@ endif elseif (isstruct (expected)) - if (! isstruct (cond) || any (size (cond) != size (expected)) - || rows (fieldnames (cond)) != rows (fieldnames (expected))) - err.index{end+1} = "{}"; + if (! isstruct (cond)) + err.index{end+1} = "."; err.observed{end+1} = "O"; err.expected{end+1} = "E"; + err.reason{end+1} = ["Expected struct, but observed " class(cond)]; + elseif (ndims (cond) != ndims (expected) + || any (size (cond) != size (expected)) + || rows (fieldnames (cond)) != rows (fieldnames (expected))) + + err.index{end+1} = "."; + err.observed{end+1} = ["O(" sprintf("%dx", size(cond))(1:end-1) ")"]; + err.expected{end+1} = ["E(" sprintf("%dx", size(expected))(1:end-1) ")"]; err.reason{end+1} = "Structure sizes don't match"; else try @@ -219,57 +230,63 @@ B = expected; ## Check exceptional values. - erridx = find ( isna (real (A)) != isna (real (B)) - | isna (imag (A)) != isna (imag (B))); + errvec = ( isna (real (A)) != isna (real (B)) + | isna (imag (A)) != isna (imag (B))); + erridx = find (errvec); if (! isempty (erridx)) - err.index(end+1:end + length (erridx)) = ... - ind2tuple (size (A), erridx); - err.observed(end+1:end + length (erridx)) = ... - strtrim (cellstr (num2str (A(erridx) (:)))); - err.expected(end+1:end + length (erridx)) = ... - strtrim (cellstr (num2str (B(erridx) (:)))); - err.reason(end+1:end + length (erridx)) = ... - cellstr (repmat ("'NA' mismatch", length (erridx), 1)); + err.index(end+1:end+length (erridx)) = ... + ind2tuple (size (A), erridx); + err.observed(end+1:end+length (erridx)) = ... + strtrim (cellstr (num2str (A(erridx) (:)))); + err.expected(end+1:end+length (erridx)) = ... + strtrim (cellstr (num2str (B(erridx) (:)))); + err.reason(end+1:end+length (erridx)) = ... + repmat ({"'NA' mismatch"}, length (erridx), 1); endif + errseen = errvec; - erridx = find ( isnan (real (A)) != isnan (real (B)) - | isnan (imag (A)) != isnan (imag (B))); + errvec = ( isnan (real (A)) != isnan (real (B)) + | isnan (imag (A)) != isnan (imag (B))); + erridx = find (errvec & !errseen); if (! isempty (erridx)) - err.index(end+1:end + length (erridx)) = ... - ind2tuple (size (A), erridx); - err.observed(end+1:end + length (erridx)) = ... - strtrim (cellstr (num2str (A(erridx) (:)))); - err.expected(end+1:end + length (erridx)) = ... - strtrim (cellstr (num2str (B(erridx) (:)))); - err.reason(end+1:end + length (erridx)) = ... - cellstr (repmat ("'NaN' mismatch", length (erridx), 1)); + err.index(end+1:end+length (erridx)) = ... + ind2tuple (size (A), erridx); + err.observed(end+1:end+length (erridx)) = ... + strtrim (cellstr (num2str (A(erridx) (:)))); + err.expected(end+1:end+length (erridx)) = ... + strtrim (cellstr (num2str (B(erridx) (:)))); + err.reason(end+1:end+length (erridx)) = ... + repmat ({"'NaN' mismatch"}, length (erridx), 1); endif + errseen |= errvec; - erridx = find (((isinf (real (A)) | isinf (real (B))) ... - & real (A) != real (B)) ... - | ((isinf (imag (A)) | isinf (imag (B))) - & imag (A) != imag (B))); + errvec = ((isinf (real (A)) | isinf (real (B))) ... + & (real (A) != real (B))) ... + | ((isinf (imag (A)) | isinf (imag (B))) ... + & (imag (A) != imag (B))); + erridx = find (errvec & !errseen); if (! isempty (erridx)) - err.index(end+1:end + length (erridx)) = ... - ind2tuple (size (A), erridx); - err.observed(end+1:end + length (erridx)) = ... - strtrim (cellstr (num2str (A(erridx) (:)))); - err.expected(end+1:end + length (erridx)) = ... - strtrim (cellstr (num2str (B(erridx) (:)))); - err.reason(end+1:end + length (erridx)) = ... - cellstr (repmat ("'Inf' mismatch", length (erridx), 1)); + err.index(end+1:end+length (erridx)) = ... + ind2tuple (size (A), erridx); + err.observed(end+1:end+length (erridx)) = ... + strtrim (cellstr (num2str (A(erridx) (:)))); + err.expected(end+1:end+length (erridx)) = ... + strtrim (cellstr (num2str (B(erridx) (:)))); + err.reason(end+1:end+length (erridx)) = ... + repmat ({"'Inf' mismatch"}, length (erridx), 1); endif + errseen |= errvec; ## Check normal values. ## Replace exceptional values already checked above by zero. A_null_real = real (A); B_null_real = real (B); - exclude = ! isfinite (A_null_real) & ! isfinite (B_null_real); + exclude = errseen | ! isfinite (A_null_real) & ! isfinite (B_null_real); A_null_real(exclude) = 0; B_null_real(exclude) = 0; A_null_imag = imag (A); B_null_imag = imag (B); - exclude = ! isfinite (A_null_imag) & ! isfinite (B_null_imag); + exclude = errseen | ! isfinite (A_null_imag) & ! isfinite (B_null_imag); A_null_imag(exclude) = 0; B_null_imag(exclude) = 0; A_null = complex (A_null_real, A_null_imag); @@ -283,45 +300,45 @@ k = (mtol == 0); erridx = find ((A_null != B_null) & k); if (! isempty (erridx)) - err.index(end+1:end + length (erridx)) = ... - ind2tuple (size (A), erridx); - err.observed(end+1:end + length (erridx)) = ... - strtrim (cellstr (num2str (A(erridx) (:)))); - err.expected(end+1:end + length (erridx)) = ... - strtrim (cellstr (num2str (B(erridx) (:)))); - err.reason(end+1:end + length (erridx)) = ... - ostrsplit (deblank (sprintf ("Abs err %g exceeds tol %g\n", ... - [abs(A_null(erridx) - B_null(erridx)) mtol(erridx)]')), "\n"); + err.index(end+1:end+length (erridx)) = ... + ind2tuple (size (A), erridx); + err.observed(end+1:end+length (erridx)) = ... + strtrim (cellstr (num2str (A(erridx) (:)))); + err.expected(end+1:end+length (erridx)) = ... + strtrim (cellstr (num2str (B(erridx) (:)))); + err.reason(end+1:end+length (erridx)) = ... + ostrsplit (deblank (sprintf ("Abs err %.5g exceeds tol %.5g\n",... + [abs(A_null(erridx) - B_null(erridx))(:) mtol(erridx)(:)]')), "\n"); endif k = (mtol > 0); erridx = find ((abs (A_null - B_null) > mtol) & k); if (! isempty (erridx)) - err.index(end+1:end + length (erridx)) = ... - ind2tuple (size (A), erridx); - err.observed(end+1:end + length (erridx)) = ... - strtrim (cellstr (num2str (A(erridx) (:)))); - err.expected(end+1:end + length (erridx)) = ... - strtrim (cellstr (num2str (B(erridx) (:)))); - err.reason(end+1:end + length (erridx)) = ... - ostrsplit (deblank (sprintf ("Abs err %g exceeds tol %g\n", ... - [abs(A_null(erridx) - B_null(erridx)) mtol(erridx)]')), "\n"); + err.index(end+1:end+length (erridx)) = ... + ind2tuple (size (A), erridx); + err.observed(end+1:end+length (erridx)) = ... + strtrim (cellstr (num2str (A(erridx) (:)))); + err.expected(end+1:end+length (erridx)) = ... + strtrim (cellstr (num2str (B(erridx) (:)))); + err.reason(end+1:end+length (erridx)) = ... + ostrsplit (deblank (sprintf ("Abs err %.5g exceeds tol %.5g\n",... + [abs(A_null(erridx) - B_null(erridx))(:) mtol(erridx)(:)]')), "\n"); endif k = (mtol < 0); - if (any (k)) + if (any (k(:))) ## Test for absolute error where relative error can't be calculated. erridx = find ((B_null == 0) & abs (A_null) > abs (mtol) & k); if (! isempty (erridx)) - err.index(end+1:end + length (erridx)) = ... - ind2tuple (size (A), erridx); - err.observed(end+1:end + length (erridx)) = ... - strtrim (cellstr (num2str (A(erridx) (:)))); - err.expected(end+1:end + length (erridx)) = ... - strtrim (cellstr (num2str (B(erridx) (:)))); - err.reason(end+1:end + length (erridx)) = ... - ostrsplit (deblank (sprintf ("Abs err %g exceeds tol %g\n", - [abs(A_null(erridx) - B_null(erridx)) -mtol(erridx)]')), "\n"); + err.index(end+1:end+length (erridx)) = ... + ind2tuple (size (A), erridx); + err.observed(end+1:end+length (erridx)) = ... + strtrim (cellstr (num2str (A(erridx) (:)))); + err.expected(end+1:end+length (erridx)) = ... + strtrim (cellstr (num2str (B(erridx) (:)))); + err.reason(end+1:end+length (erridx)) = ... + ostrsplit (deblank (sprintf ("Abs err %.5g exceeds tol %.5g\n", + [abs(A_null(erridx) - B_null(erridx)) -mtol(erridx)]')), "\n"); endif ## Test for relative error Bdiv = Inf (size (B_null)); @@ -329,15 +346,15 @@ relerr = abs ((A_null - B_null) ./ abs (Bdiv)); erridx = find ((relerr > abs (mtol)) & k); if (! isempty (erridx)) - err.index(end+1:end + length (erridx)) = ... - ind2tuple (size (A), erridx); - err.observed(end+1:end + length (erridx)) = ... - strtrim (cellstr (num2str (A(erridx) (:)))); - err.expected(end+1:end + length (erridx)) = ... - strtrim (cellstr (num2str (B(erridx) (:)))); - err.reason(end+1:end + length (erridx)) = ... - ostrsplit (deblank (sprintf ("Rel err %g exceeds tol %g\n", - [relerr(erridx) -mtol(erridx)]')), "\n"); + err.index(end+1:end+length (erridx)) = ... + ind2tuple (size (A), erridx); + err.observed(end+1:end+length (erridx)) = ... + strtrim (cellstr (num2str (A(erridx) (:)))); + err.expected(end+1:end+length (erridx)) = ... + strtrim (cellstr (num2str (B(erridx) (:)))); + err.reason(end+1:end+length (erridx)) = ... + ostrsplit (deblank (sprintf ("Rel err %.5g exceeds tol %.5g\n", + [relerr(erridx)(:) -mtol(erridx)(:)]')), "\n"); endif endif endif @@ -369,8 +386,8 @@ ## empty input %!assert ([]) %!assert (zeros (3,0), zeros (3,0)) -%!error assert (zeros (3,0), zeros (0,2)) -%!error assert (zeros (3,0), []) +%!error assert (zeros (3,0), zeros (0,2)) +%!error assert (zeros (3,0), []) %!error assert (zeros (2,0,2), zeros (2,0)) ## conditions @@ -385,85 +402,135 @@ %!error assert ([1,0;1,1]) ## scalars -%!error assert (3, [3,3; 3,3]) -%!error assert ([3,3; 3,3], 3) +%!error assert (3, [3,3]) +%!error assert (3, [3,3; 3,3]) +%!error assert ([3,3; 3,3], 3) %!assert (3, 3) +%!error assert (3, 4) %!assert (3+eps, 3, eps) %!assert (3, 3+eps, eps) -%!error assert (3+2*eps, 3, eps) -%!error assert (3, 3+2*eps, eps) +%!error assert (3+2*eps, 3, eps) +%!error assert (3, 3+2*eps, eps) ## vectors %!assert ([1,2,3],[1,2,3]); %!assert ([1;2;3],[1;2;3]); -%!error assert ([2,2,3,3],[1,2,3,4]); -%!error assert ([6;6;7;7],[5;6;7;8]); -%!error assert ([1,2,3],[1;2;3]); -%!error assert ([1,2],[1,2,3]); -%!error assert ([1;2;3],[1;2]); -%!assert ([1,2;3,4],[1,2;3,4]); -%!error assert ([1,4;3,4],[1,2;3,4]) -%!error assert ([1,3;2,4;3,5],[1,2;3,4]) +%!error assert ([2,2,3,3],[1,2,3,4]); +%!error assert ([2,2,3,3],[1,2,3,4],0.5); +%!error assert ([2,2,3,5],[1,2,3,4],-0.1); +%!error assert ([6;6;7;7],[5;6;7;8]); +%!error assert ([6;6;7;7],[5;6;7;8],0.5); +%!error assert ([6;6;7;7],[5;6;7;8],-0.1); +%!error assert ([1,2,3],[1;2;3]); +%!error assert ([1,2],[1,2,3]); +%!error assert ([1;2;3],[1;2]); ## matrices -%!test +%!assert ([1,2;3,4],[1,2;3,4]); +%!error <\(1,2\)\s+4\s+2> assert ([1,4;3,4],[1,2;3,4]) +%!error assert ([1,3;2,4;3,5],[1,2;3,4]) +%!test # 2-D matrix %! A = [1 2 3]'*[1,2]; -%! assert (A,A); +%! assert (A, A); %! fail ("assert (A.*(A!=2),A)"); +%!test # N-D matrix %! X = zeros (2,2,3); %! Y = X; -%! Y (1,2,3) = 1; -%! fail ("assert (X,Y)"); +%! Y(1,2,3) = 1.5; +%! fail ("assert (X,Y)", "\(1,2,3\).*Abs err 1.5 exceeds tol 0"); ## must give a small tolerance for floating point errors on relative %!assert (100+100*eps, 100, -2*eps) %!assert (100, 100+100*eps, -2*eps) -%!error assert (100+300*eps, 100, -2*eps) -%!error assert (100, 100+300*eps, -2*eps) -%!error assert (3, [3,3]) -%!error assert (3, 4) +%!error assert (100+300*eps, 100, -2*eps) +%!error assert (100, 100+300*eps, -2*eps) ## test relative vs. absolute tolerances -%!test assert (0.1+eps, 0.1, 2*eps); # accept absolute -%!error assert (0.1+eps, 0.1, -2*eps); # fail relative -%!test assert (100+100*eps, 100, -2*eps); # accept relative -%!error assert (100+100*eps, 100, 2*eps); # fail absolute +%!test assert (0.1+eps, 0.1, 2*eps); +%!error assert (0.1+eps, 0.1, -2*eps); +%!test assert (100+100*eps, 100, -2*eps); +%!error assert (100+100*eps, 100, 2*eps); + +## Corner case of relative tolerance with 0 divider +%!error assert (2, 0, -0.1) + +## Extra checking of inputs when tolerance unspecified. +%!error assert (single (1), 1) +%!error assert (uint8 (1), uint16 (1)) +%!error assert (sparse([1]), [1]) +%!error assert ([1], sparse([1])) +%!error assert (1+i, 1) +%!error assert (1, 1+i) ## exceptional values %!assert ([NaN, NA, Inf, -Inf, 1+eps, eps], [NaN, NA, Inf, -Inf, 1, 0], eps) -%!error assert (NaN, 1) -%!error assert ([NaN 1], [1 NaN]) -%!error assert (NA, 1) + +%!error <'NaN' mismatch> assert (NaN, 1) +%!error <'NaN' mismatch> assert ([NaN 1], [1 NaN]) +%!test +%! try +%! assert ([NaN 1], [1 NaN]); +%! catch +%! errmsg = lasterr (); +%! if (sum (errmsg () == "\n") != 4) +%! error ("Too many errors reported for NaN assert"); +%! elseif (strfind (errmsg, "NA")) +%! error ("NA reported for NaN assert"); +%! elseif (strfind (errmsg, "Abs err NaN exceeds tol 0")) +%! error ("Abs err reported for NaN assert"); +%! endif +%! end_try_catch + +%!error <'NA' mismatch> assert (NA, 1) %!error assert ([NA 1]', [1 NA]') +%!test +%! try +%! assert ([NA 1]', [1 NA]'); +%! catch +%! errmsg = lasterr (); +%! if (sum (errmsg () == "\n") != 4) +%! error ("Too many errors reported for NA assert"); +%! elseif (strfind (errmsg, "NaN")) +%! error ("NaN reported for NA assert"); +%! elseif (strfind (errmsg, "Abs err NA exceeds tol 0")) +%! error ("Abs err reported for NA assert"); +%! endif +%! end_try_catch %!error assert ([(complex (NA, 1)) (complex (2, NA))], [(complex (NA, 2)) 2]) -%!error assert (-Inf, Inf) -%!error assert ([-Inf Inf], [Inf -Inf]) -%!error assert (complex (Inf, 0.2), complex (-Inf, 0.2 + 2*eps), eps) + +%!error <'Inf' mismatch> assert (-Inf, Inf) +%!error <'Inf' mismatch> assert ([-Inf Inf], [Inf -Inf]) +%!test +%! try +%! assert (complex (Inf, 0.2), complex (-Inf, 0.2 + 2*eps), eps); +%! catch +%! errmsg = lasterr (); +%! if (sum (errmsg () == "\n") != 3) +%! error ("Too many errors reported for Inf assert"); +%! elseif (strfind (errmsg, "Abs err")) +%! error ("Abs err reported for Inf assert"); +%! endif +%! end_try_catch +%!error assert (complex (Inf, 0.2), complex (Inf, 0.2 + 2*eps), eps) ## strings %!assert ("dog", "dog") -%!error assert ("dog", "cat") -%!error assert ("dog", 3) -%!error assert (3, "dog") -%!error assert (cellstr ("dog"), "dog") -%!error assert (cell2struct ({"dog"; 3}, {"pet", "age"}, 1), "dog"); - -## structures -%!shared x,y -%! x.a = 1; x.b=[2, 2]; -%! y.a = 1; y.b=[2, 2]; -%!assert (x, y) -%!test y.b=3; -%!error assert (x, y) -%!error assert (3, x) -%!error assert (x, 3) -%!test -%! # Empty structures -%! x = resize (x, 0, 1); -%! y = resize (y, 0, 1); -%! assert (x, y); +%!error assert ("dog", "cat") +%!error assert (3, "dog") +%!error assert ("dog", [3 3 3]) +%!error assert ({"dog"}, "dog") +%!error assert (struct ("dog", 3), "dog") ## cell arrays +%!error assert (1, {1}) +%!error assert (cell (1,2,3), cell (3,2,1)) +%!test +%! x = {{{1}}, 2}; # cell with multiple levels +%! y = x; +%! assert (x,y); +%! y{1}{1}{1} = 3; +%! fail ("assert (x,y)", "Abs err 2 exceeds tol 0"); + %!test %! x = {[3], [1,2,3]; 100+100*eps, "dog"}; %! y = x; @@ -479,7 +546,39 @@ %! y = x; y(1,1) = [2]; y(1,2) = [0, 2, 3]; y(2,1) = 101; y(2,2) = "cat"; %! fail ("assert (x, y)"); -## variable tolerance +## structures +%!error assert (1, struct ("a", 1)) +%!error +%! x(1,2,3).a = 1; +%! y(1,2).a = 1; +%! assert (x,y); +%!error +%! x(1,2,3).a = 1; +%! y(3,2,2).a = 1; +%! assert (x,y); +%!error +%! x.a = 1; x.b = 1; +%! y.a = 1; +%! assert (x,y); +%!error <'b' is not an expected field> +%! x.b = 1; +%! y.a = 1; +%! assert (x,y); + +%!test +%! x.a = 1; x.b=[2, 2]; +%! y.a = 1; y.b=[2, 2]; +%! assert (x, y); +%! y.b=3; +%! fail ("assert (x, y)"); +%! fail ("assert (3, x)"); +%! fail ("assert (x, 3)"); +%! ## Empty structures +%! x = resize (x, 0, 1); +%! y = resize (y, 0, 1); +%! assert (x, y); + +## vector of tolerances %!test %! x = [-40:0]; %! y1 = (10.^x).*(10.^x); @@ -487,6 +586,24 @@ %! assert (y1, y2, eps (y1)); %! fail ("assert (y1, y2 + eps*1e-70, eps (y1))"); +## Multiple tolerances +%!test +%! x = [1 2; 3 4]; +%! y = [0 -1; 1 2]; +%! tol = [-0.1 0; -0.2 0.3]; +%! try +%! assert (x, y, tol); +%! catch +%! errmsg = lasterr (); +%! if (sum (errmsg () == "\n") != 6) +%! error ("Incorrect number of errors reported"); +%! endif +%! assert (!isempty (regexp (errmsg, '\(1,2\).*Abs err 3 exceeds tol 0\>'))); +%! assert (!isempty (regexp (errmsg, '\(2,2\).*Abs err 2 exceeds tol 0.3'))); +%! assert (!isempty (regexp (errmsg, '\(1,1\).*Abs err 1 exceeds tol 0.1'))); +%! assert (!isempty (regexp (errmsg, '\(2,1\).*Rel err 2 exceeds tol 0.2'))); +%! end_try_catch + ## test input validation %!error assert () %!error assert (1,2,3,4)