octave: scripts/optimization/fminsearch.m annotate

annotate scripts/optimization/fminsearch.m @ 25828:8b548f2f8086

maint: Strip trailing whitespace from source files.

author	John W. Eaton <jwe@octave.org>
date	Tue, 28 Aug 2018 00:09:59 -0400
parents	8eec0403a370
children	a00379f6f8c4

rev	line source
25054 6652d3823428 maint: Update copyright dates in all source files. John W. Eaton <jwe@octave.org> parents: 24534 diff changeset	1 ## Copyright (C) 2003-2018 Andy Adler
17744 d63878346099 maint: Update copyright notices for release. John W. Eaton <jwe@octave.org> parents: 17281 diff changeset	2 ## Copyright (C) 2002, 2013 N.J.Higham
14895 e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	3 ##
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	4 ## This file is part of Octave.
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	5 ##
24534 194eb4bd202b maint: Update punctuation for GPL v3 license text. Rik <rik@octave.org> parents: 23567 diff changeset	6 ## Octave is free software: you can redistribute it and/or modify it
14895 e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	7 ## under the terms of the GNU General Public License as published by
24534 194eb4bd202b maint: Update punctuation for GPL v3 license text. Rik <rik@octave.org> parents: 23567 diff changeset	8 ## the Free Software Foundation, either version 3 of the License, or
22755 3a2b891d0b33 maint: Standardize Copyright formatting. Rik <rik@octave.org> parents: 22323 diff changeset	9 ## (at your option) any later version.
14895 e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	10 ##
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	11 ## Octave is distributed in the hope that it will be useful, but
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	12 ## WITHOUT ANY WARRANTY; without even the implied warranty of
22755 3a2b891d0b33 maint: Standardize Copyright formatting. Rik <rik@octave.org> parents: 22323 diff changeset	13 ## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
3a2b891d0b33 maint: Standardize Copyright formatting. Rik <rik@octave.org> parents: 22323 diff changeset	14 ## GNU General Public License for more details.
14895 e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	15 ##
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	16 ## You should have received a copy of the GNU General Public License
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	17 ## along with Octave; see the file COPYING. If not, see
24534 194eb4bd202b maint: Update punctuation for GPL v3 license text. Rik <rik@octave.org> parents: 23567 diff changeset	18 ## <https://www.gnu.org/licenses/>.
14895 e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	19
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	20 ## -- texinfo --
20852 516bb87ea72e 2015 Code Sprint: remove class of function from docstring for all m-files. Rik <rik@octave.org> parents: 20735 diff changeset	21 ## @deftypefn {} {@var{x} =} fminsearch (@var{fun}, @var{x0})
516bb87ea72e 2015 Code Sprint: remove class of function from docstring for all m-files. Rik <rik@octave.org> parents: 20735 diff changeset	22 ## @deftypefnx {} {@var{x} =} fminsearch (@var{fun}, @var{x0}, @var{options})
22967 3c896180df16 Pass additional arguments for fminsearch to called function (bug #41888). Markus Muetzel <markus.muetzel@gmx.de> parents: 22963 diff changeset	23 ## @deftypefnx {} {@var{x} =} fminsearch (@var{fun}, @var{x0}, @var{options}, @var{fun_arg1}, @var{fun_arg2}, @dots{})
23113 9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	24 ## @deftypefnx {} {[@var{x}, @var{fval}, @var{exitflag}, @var{output}] =} fminsearch (@dots{})
14895 e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	25 ##
25739 8eec0403a370 fminsearch.m: Update default TolFun to 1e-4 for Matlab compatibility. Rik <rik@octave.org> parents: 25231 diff changeset	26 ## Find a value of @var{x} which minimizes the multi-variable function
8eec0403a370 fminsearch.m: Update default TolFun to 1e-4 for Matlab compatibility. Rik <rik@octave.org> parents: 25231 diff changeset	27 ## @var{fun}.
20165 f1d0f506ee78 doc: Update more docstrings to have one sentence summary as first line. Rik <rik@octave.org> parents: 19596 diff changeset	28 ##
14895 e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	29 ## The search begins at the point @var{x0} and iterates using the
19040 0850b5212619 doc: Add @nospell macro around proper names in documentation. Rik <rik@octave.org> parents: 17744 diff changeset	30 ## @nospell{Nelder & Mead} Simplex algorithm (a derivative-free method). This
0850b5212619 doc: Add @nospell macro around proper names in documentation. Rik <rik@octave.org> parents: 17744 diff changeset	31 ## algorithm is better-suited to functions which have discontinuities or for
0850b5212619 doc: Add @nospell macro around proper names in documentation. Rik <rik@octave.org> parents: 17744 diff changeset	32 ## which a gradient-based search such as @code{fminunc} fails.
14895 e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	33 ##
20165 f1d0f506ee78 doc: Update more docstrings to have one sentence summary as first line. Rik <rik@octave.org> parents: 19596 diff changeset	34 ## Options for the search are provided in the parameter @var{options} using the
f1d0f506ee78 doc: Update more docstrings to have one sentence summary as first line. Rik <rik@octave.org> parents: 19596 diff changeset	35 ## function @code{optimset}. Currently, @code{fminsearch} accepts the options:
25739 8eec0403a370 fminsearch.m: Update default TolFun to 1e-4 for Matlab compatibility. Rik <rik@octave.org> parents: 25231 diff changeset	36 ## @qcode{"Display"}, @qcode{"FunValCheck"},@qcode{"MaxFunEvals"},
25828 8b548f2f8086 maint: Strip trailing whitespace from source files. John W. Eaton <jwe@octave.org> parents: 25739 diff changeset	37 ## @qcode{"MaxIter"}, @qcode{"OutputFcn"}, @qcode{"TolFun"}, @qcode{"TolX"}.
25739 8eec0403a370 fminsearch.m: Update default TolFun to 1e-4 for Matlab compatibility. Rik <rik@octave.org> parents: 25231 diff changeset	38 ##
8eec0403a370 fminsearch.m: Update default TolFun to 1e-4 for Matlab compatibility. Rik <rik@octave.org> parents: 25231 diff changeset	39 ## @qcode{"MaxFunEvals"} proscribes the maximum number of function evaluations
8eec0403a370 fminsearch.m: Update default TolFun to 1e-4 for Matlab compatibility. Rik <rik@octave.org> parents: 25231 diff changeset	40 ## before optimization is halted. The default value is
8eec0403a370 fminsearch.m: Update default TolFun to 1e-4 for Matlab compatibility. Rik <rik@octave.org> parents: 25231 diff changeset	41 ## @code{200 * number_of_variables}, i.e., @code{200 * length (@var{x0})}.
8eec0403a370 fminsearch.m: Update default TolFun to 1e-4 for Matlab compatibility. Rik <rik@octave.org> parents: 25231 diff changeset	42 ## The value must be a positive integer.
8eec0403a370 fminsearch.m: Update default TolFun to 1e-4 for Matlab compatibility. Rik <rik@octave.org> parents: 25231 diff changeset	43 ##
8eec0403a370 fminsearch.m: Update default TolFun to 1e-4 for Matlab compatibility. Rik <rik@octave.org> parents: 25231 diff changeset	44 ## @qcode{"MaxIter"} proscribes the maximum number of algorithm iterations
8eec0403a370 fminsearch.m: Update default TolFun to 1e-4 for Matlab compatibility. Rik <rik@octave.org> parents: 25231 diff changeset	45 ## before optimization is halted. The default value is
8eec0403a370 fminsearch.m: Update default TolFun to 1e-4 for Matlab compatibility. Rik <rik@octave.org> parents: 25231 diff changeset	46 ## @code{200 * number_of_variables}, i.e., @code{200 * length (@var{x0})}.
8eec0403a370 fminsearch.m: Update default TolFun to 1e-4 for Matlab compatibility. Rik <rik@octave.org> parents: 25231 diff changeset	47 ## The value must be a positive integer.
25828 8b548f2f8086 maint: Strip trailing whitespace from source files. John W. Eaton <jwe@octave.org> parents: 25739 diff changeset	48 ##
25739 8eec0403a370 fminsearch.m: Update default TolFun to 1e-4 for Matlab compatibility. Rik <rik@octave.org> parents: 25231 diff changeset	49 ## For a description of the other options, see @code{optimset}. To initialize
8eec0403a370 fminsearch.m: Update default TolFun to 1e-4 for Matlab compatibility. Rik <rik@octave.org> parents: 25231 diff changeset	50 ## an options structure with default values for @code{fminsearch} use
8eec0403a370 fminsearch.m: Update default TolFun to 1e-4 for Matlab compatibility. Rik <rik@octave.org> parents: 25231 diff changeset	51 ## @code{options = optimset ("fminsearch")}.
14895 e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	52 ##
22967 3c896180df16 Pass additional arguments for fminsearch to called function (bug #41888). Markus Muetzel <markus.muetzel@gmx.de> parents: 22963 diff changeset	53 ## Additional inputs for the function @var{fun} can be passed as the fourth
3c896180df16 Pass additional arguments for fminsearch to called function (bug #41888). Markus Muetzel <markus.muetzel@gmx.de> parents: 22963 diff changeset	54 ## and higher arguments. To pass function arguments while using the default
3c896180df16 Pass additional arguments for fminsearch to called function (bug #41888). Markus Muetzel <markus.muetzel@gmx.de> parents: 22963 diff changeset	55 ## @var{options} values, use @code{[]} for @var{options}.
3c896180df16 Pass additional arguments for fminsearch to called function (bug #41888). Markus Muetzel <markus.muetzel@gmx.de> parents: 22963 diff changeset	56 ##
20165 f1d0f506ee78 doc: Update more docstrings to have one sentence summary as first line. Rik <rik@octave.org> parents: 19596 diff changeset	57 ## On exit, the function returns @var{x}, the minimum point, and @var{fval},
22967 3c896180df16 Pass additional arguments for fminsearch to called function (bug #41888). Markus Muetzel <markus.muetzel@gmx.de> parents: 22963 diff changeset	58 ## the function value at the minimum.
14895 e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	59 ##
25739 8eec0403a370 fminsearch.m: Update default TolFun to 1e-4 for Matlab compatibility. Rik <rik@octave.org> parents: 25231 diff changeset	60 ## The third output @var{exitflag} reports whether the algorithm succeeded and
8eec0403a370 fminsearch.m: Update default TolFun to 1e-4 for Matlab compatibility. Rik <rik@octave.org> parents: 25231 diff changeset	61 ## may take one of the following values:
23113 9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	62 ##
23114 19e958974410 fminsearch.m: fix texinfo header (bug #50178) Philip Nienhuis <prnienhuis@users.sf.net> parents: 23113 diff changeset	63 ## @table @asis
23113 9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	64 ## @item 1
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	65 ## if the algorithm converged
25739 8eec0403a370 fminsearch.m: Update default TolFun to 1e-4 for Matlab compatibility. Rik <rik@octave.org> parents: 25231 diff changeset	66 ## (size of the simplex is smaller than @code{TolX} @strong{AND} the step in
8eec0403a370 fminsearch.m: Update default TolFun to 1e-4 for Matlab compatibility. Rik <rik@octave.org> parents: 25231 diff changeset	67 ## function value between iterations is smaller than @code{TolFun}).
14895 e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	68 ##
23113 9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	69 ## @item 0
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	70 ## if the maximum number of iterations or the maximum number of function
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	71 ## evaluations are exceeded.
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	72 ##
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	73 ## @item -1
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	74 ## if the iteration is stopped by the @qcode{"OutputFcn"}.
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	75 ## @end table
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	76 ##
25739 8eec0403a370 fminsearch.m: Update default TolFun to 1e-4 for Matlab compatibility. Rik <rik@octave.org> parents: 25231 diff changeset	77 ## The fourth output is a structure @var{output} containing runtime
8eec0403a370 fminsearch.m: Update default TolFun to 1e-4 for Matlab compatibility. Rik <rik@octave.org> parents: 25231 diff changeset	78 ## about the algorithm. Fields in the structure are @code{funcCount}
8eec0403a370 fminsearch.m: Update default TolFun to 1e-4 for Matlab compatibility. Rik <rik@octave.org> parents: 25231 diff changeset	79 ## containing the number of function calls to @var{fun}, @code{iterations}
8eec0403a370 fminsearch.m: Update default TolFun to 1e-4 for Matlab compatibility. Rik <rik@octave.org> parents: 25231 diff changeset	80 ## containing the number of iteration steps, @code{algorithm} with the name of
25828 8b548f2f8086 maint: Strip trailing whitespace from source files. John W. Eaton <jwe@octave.org> parents: 25739 diff changeset	81 ## the search algorithm (always:
23567 71bfd507663c doc: Periodic spellcheck of documentation. Rik <rik@octave.org> parents: 23220 diff changeset	82 ## @nospell{@qcode{"Nelder-Mead simplex direct search"}}), and @code{message}
71bfd507663c doc: Periodic spellcheck of documentation. Rik <rik@octave.org> parents: 23220 diff changeset	83 ## with the exit message.
23113 9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	84 ##
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	85 ## Example:
14895 e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	86 ##
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	87 ## @example
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	88 ## fminsearch (@@(x) (x(1)-5).^2+(x(2)-8).^4, [0;0])
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	89 ## @end example
25739 8eec0403a370 fminsearch.m: Update default TolFun to 1e-4 for Matlab compatibility. Rik <rik@octave.org> parents: 25231 diff changeset	90 ##
8eec0403a370 fminsearch.m: Update default TolFun to 1e-4 for Matlab compatibility. Rik <rik@octave.org> parents: 25231 diff changeset	91 ## Note: If you need to find the minimum of a single variable function it is
8eec0403a370 fminsearch.m: Update default TolFun to 1e-4 for Matlab compatibility. Rik <rik@octave.org> parents: 25231 diff changeset	92 ## probably better to use @code{fminbnd}.
14895 e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	93 ## @seealso{fminbnd, fminunc, optimset}
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	94 ## @end deftypefn
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	95
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	96 ## PKG_ADD: ## Discard result to avoid polluting workspace with ans at startup.
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	97 ## PKG_ADD: [~] = __all_opts__ ("fminsearch");
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	98
23113 9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	99 ## FIXME: Add support for output function with "state" set to "interrupt".
14895 e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	100
23113 9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	101 function [x, fval, exitflag, output] = fminsearch (fun, x0, options, varargin)
14895 e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	102
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	103 ## Get default options if requested.
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	104 if (nargin == 1 && ischar (fun) && strcmp (fun, "defaults"))
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	105 x = optimset ("Display", "notify", "FunValCheck", "off",
25739 8eec0403a370 fminsearch.m: Update default TolFun to 1e-4 for Matlab compatibility. Rik <rik@octave.org> parents: 25231 diff changeset	106 "MaxFunEvals", [], "MaxIter", [],
14895 e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	107 "OutputFcn", [],
25739 8eec0403a370 fminsearch.m: Update default TolFun to 1e-4 for Matlab compatibility. Rik <rik@octave.org> parents: 25231 diff changeset	108 "TolFun", 1e-4, "TolX", 1e-4);
14895 e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	109 return;
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	110 endif
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	111
22967 3c896180df16 Pass additional arguments for fminsearch to called function (bug #41888). Markus Muetzel <markus.muetzel@gmx.de> parents: 22963 diff changeset	112 if (nargin < 2)
14895 e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	113 print_usage ();
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	114 endif
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	115
22967 3c896180df16 Pass additional arguments for fminsearch to called function (bug #41888). Markus Muetzel <markus.muetzel@gmx.de> parents: 22963 diff changeset	116 if (nargin < 3 \|\| isempty (options))
3c896180df16 Pass additional arguments for fminsearch to called function (bug #41888). Markus Muetzel <markus.muetzel@gmx.de> parents: 22963 diff changeset	117 options = struct ();
3c896180df16 Pass additional arguments for fminsearch to called function (bug #41888). Markus Muetzel <markus.muetzel@gmx.de> parents: 22963 diff changeset	118 endif
14895 e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	119
23113 9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	120 [x, exitflag, output] = nmsmax (fun, x0, options, varargin{:});
14895 e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	121
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	122 if (isargout (2))
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	123 fval = feval (fun, x);
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	124 endif
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	125
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	126 endfunction
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	127
23113 9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	128 ## NMSMAX Nelder-Mead simplex method for direct search optimization.
14895 e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	129 ## [x, fmax, nf] = NMSMAX(FUN, x0, STOPIT, SAVIT) attempts to
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	130 ## maximize the function FUN, using the starting vector x0.
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	131 ## The Nelder-Mead direct search method is used.
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	132 ## Output arguments:
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	133 ## x = vector yielding largest function value found,
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	134 ## fmax = function value at x,
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	135 ## nf = number of function evaluations.
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	136 ## The iteration is terminated when either
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	137 ## - the relative size of the simplex is <= STOPIT(1)
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	138 ## (default 1e-3),
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	139 ## - STOPIT(2) function evaluations have been performed
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	140 ## (default inf, i.e., no limit), or
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	141 ## - a function value equals or exceeds STOPIT(3)
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	142 ## (default inf, i.e., no test on function values).
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	143 ## The form of the initial simplex is determined by STOPIT(4):
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	144 ## STOPIT(4) = 0: regular simplex (sides of equal length, the default)
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	145 ## STOPIT(4) = 1: right-angled simplex.
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	146 ## Progress of the iteration is not shown if STOPIT(5) = 0 (default 1).
21751 b571fc85953f maint: Use two spaces after period to indicate sentence break. Rik <rik@octave.org> parents: 21099 diff changeset	147 ## STOPIT(6) indicates the direction (i.e., minimization or
14895 e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	148 ## maximization.) Default is 1, maximization.
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	149 ## set STOPIT(6)=-1 for minimization
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	150 ## If a non-empty fourth parameter string SAVIT is present, then
15468 6437fa7263dd use ' instead of ` in error messages, warnings and most comments John W. Eaton <jwe@octave.org> parents: 14895 diff changeset	151 ## 'SAVE SAVIT x fmax nf' is executed after each inner iteration.
14895 e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	152 ## NB: x0 can be a matrix. In the output argument, in SAVIT saves,
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	153 ## and in function calls, x has the same shape as x0.
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	154 ## NMSMAX(fun, x0, STOPIT, SAVIT, P1, P2,...) allows additional
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	155 ## arguments to be passed to fun, via feval(fun,x,P1,P2,...).
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	156 ## References:
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	157 ## N. J. Higham, Optimization by direct search in matrix computations,
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	158 ## SIAM J. Matrix Anal. Appl, 14(2): 317-333, 1993.
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	159 ## C. T. Kelley, Iterative Methods for Optimization, Society for Industrial
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	160 ## and Applied Mathematics, Philadelphia, PA, 1999.
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	161
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	162 ## From Matrix Toolbox
17744 d63878346099 maint: Update copyright notices for release. John W. Eaton <jwe@octave.org> parents: 17281 diff changeset	163 ## Copyright (C) 2002, 2013 N.J.Higham
14895 e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	164 ## www.maths.man.ac.uk/~higham/mctoolbox
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	165 ##
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	166 ## Modifications for Octave by A.Adler 2003
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	167
23113 9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	168 function [stopit, savit, dirn, trace, tol, maxiter, tol_f, outfcn] = ...
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	169 parse_options (options, x)
14895 e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	170
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	171 ## Tolerance for cgce test based on relative size of simplex.
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	172 stopit(1) = tol = optimget (options, "TolX", 1e-4);
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	173
23113 9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	174 ## Tolerance for cgce test based on step in function value.
25739 8eec0403a370 fminsearch.m: Update default TolFun to 1e-4 for Matlab compatibility. Rik <rik@octave.org> parents: 25231 diff changeset	175 tol_f = optimget (options, "TolFun", 1e-4);
23113 9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	176
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	177 ## Max number of function evaluations.
25739 8eec0403a370 fminsearch.m: Update default TolFun to 1e-4 for Matlab compatibility. Rik <rik@octave.org> parents: 25231 diff changeset	178 stopit(2) = optimget (options, "MaxFunEvals", 200 * length (x));
14895 e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	179
23113 9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	180 ## Max number of iterations
25739 8eec0403a370 fminsearch.m: Update default TolFun to 1e-4 for Matlab compatibility. Rik <rik@octave.org> parents: 25231 diff changeset	181 maxiter = optimget (options, "MaxIter", 200 * length (x));
14895 e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	182
23113 9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	183 ## Default target for function values.
14895 e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	184 stopit(3) = Inf; # FIXME: expose this parameter to the outside
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	185
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	186 ## Default initial simplex.
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	187 stopit(4) = 0; # FIXME: expose this parameter to the outside
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	188
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	189 ## Default: show progress.
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	190 display = optimget (options, "Display", "notify");
23113 9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	191 switch (display)
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	192 case "iter"
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	193 stopit(5) = 1;
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	194 case "final"
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	195 stopit(5) = 2;
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	196 case "notify"
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	197 stopit(5) = 3;
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	198 otherwise # "none"
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	199 stopit(5) = 0;
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	200 endswitch
14895 e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	201 trace = stopit(5);
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	202
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	203 ## Use function to minimize, not maximize
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	204 stopit(6) = dirn = -1;
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	205
20713 2469d78a1d8b Consistently use 'filename' rather than 'file name' throughout code base. Rik <rik@octave.org> parents: 20231 diff changeset	206 ## Filename for snapshots.
14895 e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	207 savit = []; # FIXME: expose this parameter to the outside
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	208
23113 9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	209 ## OutputFcn
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	210 outfcn = optimget (options, "OutputFcn");
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	211
14895 e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	212 endfunction
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	213
23113 9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	214 function [x, exitflag, output] = nmsmax (fun, x, options, varargin)
14895 e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	215
23113 9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	216 [stopit, savit, dirn, trace, tol, maxiter, tol_f, outfcn] = ...
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	217 parse_options (options, x);
14895 e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	218
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	219 if (strcmpi (optimget (options, "FunValCheck", "off"), "on"))
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	220 ## Replace fcn with a guarded version.
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	221 fun = @(x) guarded_eval (fun, x);
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	222 endif
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	223
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	224 x0 = x(:); # Work with column vector internally.
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	225 n = length (x0);
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	226
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	227 V = [zeros(n,1) eye(n)];
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	228 f = zeros (n+1,1);
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	229 V(:,1) = x0;
23113 9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	230 f(1) = dirn * feval (fun, x, varargin{:});
14895 e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	231 fmax_old = f(1);
23113 9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	232 nf = 1;
14895 e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	233
23113 9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	234 if (trace == 1)
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	235 printf ("f(x0) = %9.4e\n", f(1));
14895 e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	236 endif
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	237
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	238 k = 0; m = 0;
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	239
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	240 ## Set up initial simplex.
23113 9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	241 scale = max (norm (x0, Inf), 1);
14895 e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	242 if (stopit(4) == 0)
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	243 ## Regular simplex - all edges have same length.
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	244 ## Generated from construction given in reference [18, pp. 80-81] of [1].
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	245 alpha = scale / (nsqrt (2)) [sqrt(n+1)-1+n, sqrt(n+1)-1];
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	246 V(:,2:n+1) = (x0 + alpha(2)ones (n,1)) ones (1,n);
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	247 for j = 2:n+1
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	248 V(j-1,j) = x0(j-1) + alpha(1);
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	249 x(:) = V(:,j);
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	250 f(j) = dirn * feval (fun,x,varargin{:});
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	251 endfor
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	252 else
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	253 ## Right-angled simplex based on co-ordinate axes.
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	254 alpha = scale * ones(n+1,1);
23113 9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	255 for j = 2:n+1
14895 e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	256 V(:,j) = x0 + alpha(j)*V(:,j);
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	257 x(:) = V(:,j);
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	258 f(j) = dirn * feval (fun,x,varargin{:});
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	259 endfor
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	260 endif
23113 9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	261 nf += n;
14895 e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	262 how = "initial ";
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	263
23113 9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	264 [~, j] = sort (f);
14895 e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	265 j = j(n+1:-1:1);
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	266 f = f(j);
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	267 V = V(:,j);
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	268
23113 9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	269 exitflag = 0;
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	270
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	271 if (! isempty (outfcn))
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	272 optimvalues.iteration = 0;
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	273 optimvalues.funccount = nf;
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	274 optimvalues.fval = f(1);
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	275 optimvalues.procedure = how;
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	276 state = "init";
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	277 stop = outfcn (x, optimvalues, state);
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	278 if (stop)
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	279 msg = "Stopped by OutputFcn\n";
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	280 exitflag = -1;
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	281 endif
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	282 endif
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	283
14895 e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	284 alpha = 1; beta = 1/2; gamma = 2;
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	285
23113 9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	286 while (exitflag != -1) # Outer (and only) loop.
20735 418ae0cb752f Replace ++,-- with in-place operators for performance. Rik <rik@octave.org> parents: 20713 diff changeset	287 k += 1;
14895 e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	288
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	289 if (k > maxiter)
25739 8eec0403a370 fminsearch.m: Update default TolFun to 1e-4 for Matlab compatibility. Rik <rik@octave.org> parents: 25231 diff changeset	290 msg = "Exceeded maximum iterations\n";
14895 e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	291 break;
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	292 endif
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	293
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	294 fmax = f(1);
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	295 if (fmax > fmax_old)
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	296 if (! isempty (savit))
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	297 x(:) = V(:,1);
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	298 eval (["save " savit " x fmax nf"]);
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	299 endif
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	300 endif
23113 9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	301 if (trace == 1)
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	302 printf ("Iter. %2.0f,", k);
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	303 printf (" how = %-11s", [how ","]);
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	304 printf ("nf = %3.0f, f = %9.4e (%2.1f%%)\n", nf, fmax, ...
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	305 100*(fmax-fmax_old)/(abs(fmax_old)+eps));
14895 e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	306 endif
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	307 fmax_old = fmax;
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	308
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	309 ## Three stopping tests from MDSMAX.M
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	310
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	311 ## Stopping Test 1 - f reached target value?
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	312 if (fmax >= stopit(3))
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	313 msg = "Exceeded target...quitting\n";
25739 8eec0403a370 fminsearch.m: Update default TolFun to 1e-4 for Matlab compatibility. Rik <rik@octave.org> parents: 25231 diff changeset	314 ## FIXME: Add documentation when stopit(3) gets exposed to the outside
23113 9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	315 exitflag = -1;
14895 e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	316 break;
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	317 endif
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	318
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	319 ## Stopping Test 2 - too many f-evals?
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	320 if (nf >= stopit(2))
25739 8eec0403a370 fminsearch.m: Update default TolFun to 1e-4 for Matlab compatibility. Rik <rik@octave.org> parents: 25231 diff changeset	321 msg = "Exceeded maximum number of function evaluations\n";
8eec0403a370 fminsearch.m: Update default TolFun to 1e-4 for Matlab compatibility. Rik <rik@octave.org> parents: 25231 diff changeset	322 exitflag = 0;
14895 e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	323 break;
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	324 endif
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	325
23113 9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	326 ## Stopping Test 3 - converged? The first part is test (4.3) in [1].
14895 e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	327 v1 = V(:,1);
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	328 size_simplex = norm (V(:,2:n+1)-v1(:,ones (1,n)),1) / max (1, norm (v1,1));
23113 9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	329 step_f = max (abs (f(1) - f(2:n+1)));
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	330 if (size_simplex <= tol && step_f <= tol_f )
25739 8eec0403a370 fminsearch.m: Update default TolFun to 1e-4 for Matlab compatibility. Rik <rik@octave.org> parents: 25231 diff changeset	331 msg = sprintf (["Algorithm converged. Simplex size %9.4e <= %9.4e ", ...
8eec0403a370 fminsearch.m: Update default TolFun to 1e-4 for Matlab compatibility. Rik <rik@octave.org> parents: 25231 diff changeset	332 "and step in function value %9.4e <= %9.4e\n"], ...
23113 9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	333 size_simplex, tol, step_f, tol_f);
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	334 exitflag = 1;
14895 e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	335 break;
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	336 endif
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	337
23113 9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	338 ## Call OutputFcn
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	339 if (! isempty (outfcn))
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	340 optimvalues.funccount = nf;
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	341 optimvalues.fval = f(1);
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	342 optimvalues.iteration = k;
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	343 optimvalues.procedure = how;
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	344 state = "iter";
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	345 stop = outfcn (x, optimvalues, state);
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	346 if (stop)
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	347 msg = "Stopped by OutputFcn\n";
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	348 exitflag = -1;
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	349 break;
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	350 endif
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	351 endif
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	352
14895 e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	353 ## One step of the Nelder-Mead simplex algorithm
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	354 ## NJH: Altered function calls and changed CNT to NF.
15468 6437fa7263dd use ' instead of ` in error messages, warnings and most comments John W. Eaton <jwe@octave.org> parents: 14895 diff changeset	355 ## Changed each 'fr < f(1)' type test to '>' for maximization
14895 e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	356 ## and re-ordered function values after sort.
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	357
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	358 vbar = (sum (V(:,1:n)')/n)'; # Mean value
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	359 vr = (1 + alpha)vbar - alphaV(:,n+1);
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	360 x(:) = vr;
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	361 fr = dirn * feval (fun,x,varargin{:});
20231 83792dd9bcc1 Use in-place operators in m-files where possible. Rik <rik@octave.org> parents: 20165 diff changeset	362 nf += 1;
23113 9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	363 vk = vr; fk = fr; how = "reflect";
14895 e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	364 if (fr > f(n))
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	365 if (fr > f(1))
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	366 ve = gammavr + (1-gamma)vbar;
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	367 x(:) = ve;
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	368 fe = dirn * feval (fun,x,varargin{:});
20231 83792dd9bcc1 Use in-place operators in m-files where possible. Rik <rik@octave.org> parents: 20165 diff changeset	369 nf += 1;
14895 e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	370 if (fe > f(1))
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	371 vk = ve;
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	372 fk = fe;
23113 9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	373 how = "expand";
14895 e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	374 endif
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	375 endif
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	376 else
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	377 vt = V(:,n+1);
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	378 ft = f(n+1);
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	379 if (fr > ft)
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	380 vt = vr;
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	381 ft = fr;
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	382 endif
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	383 vc = betavt + (1-beta)vbar;
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	384 x(:) = vc;
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	385 fc = dirn * feval (fun,x,varargin{:});
20231 83792dd9bcc1 Use in-place operators in m-files where possible. Rik <rik@octave.org> parents: 20165 diff changeset	386 nf += 1;
14895 e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	387 if (fc > f(n))
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	388 vk = vc; fk = fc;
23113 9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	389 how = "contract";
14895 e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	390 else
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	391 for j = 2:n
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	392 V(:,j) = (V(:,1) + V(:,j))/2;
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	393 x(:) = V(:,j);
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	394 f(j) = dirn * feval (fun,x,varargin{:});
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	395 endfor
20231 83792dd9bcc1 Use in-place operators in m-files where possible. Rik <rik@octave.org> parents: 20165 diff changeset	396 nf += n-1;
14895 e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	397 vk = (V(:,1) + V(:,n+1))/2;
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	398 x(:) = vk;
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	399 fk = dirn * feval (fun,x,varargin{:});
20231 83792dd9bcc1 Use in-place operators in m-files where possible. Rik <rik@octave.org> parents: 20165 diff changeset	400 nf += 1;
23113 9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	401 how = "shrink";
14895 e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	402 endif
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	403 endif
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	404 V(:,n+1) = vk;
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	405 f(n+1) = fk;
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	406 [~,j] = sort(f);
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	407 j = j(n+1:-1:1);
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	408 f = f(j);
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	409 V = V(:,j);
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	410
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	411 endwhile # End of outer (and only) loop.
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	412
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	413 ## Finished.
23113 9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	414 if ( (trace == 1) \|\| (trace == 2) \|\| (trace == 3 && exitflag != 1) )
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	415 printf (msg);
14895 e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	416 endif
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	417 x(:) = V(:,1);
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	418
23113 9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	419 ## FIXME: Should outputfcn be called only if exitflag != 0,
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	420 ## i.e., only when we have successfully converged?
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	421 if (! isempty (outfcn))
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	422 optimvalues.funccount = nf;
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	423 optimvalues.fval = f(1);
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	424 optimvalues.iteration = k;
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	425 optimvalues.procedure = how;
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	426 state = "done";
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	427 outfcn (x, optimvalues, state);
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	428 endif
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	429
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	430 ## output
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	431 output.iterations = k;
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	432 output.funcCount = nf;
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	433 output.algorithm = "Nelder-Mead simplex direct search";
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	434 output.message = msg;
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	435
14895 e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	436 endfunction
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	437
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	438 ## A helper function that evaluates a function and checks for bad results.
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	439 function y = guarded_eval (fun, x)
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	440
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	441 y = fun (x);
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	442
22962 fba8a6c742c6 fminsearch.m: Fix error in "guarded_eval". Markus Muetzel <markus.muetzel@gmx.de> parents: 22323 diff changeset	443 if (! (isreal (y)))
14895 e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	444 error ("fminsearch:notreal", "fminsearch: non-real value encountered");
22962 fba8a6c742c6 fminsearch.m: Fix error in "guarded_eval". Markus Muetzel <markus.muetzel@gmx.de> parents: 22323 diff changeset	445 elseif (any (isnan (y(:))))
14895 e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	446 error ("fminsearch:isnan", "fminsearch: NaN value encountered");
22962 fba8a6c742c6 fminsearch.m: Fix error in "guarded_eval". Markus Muetzel <markus.muetzel@gmx.de> parents: 22323 diff changeset	447 elseif (any (isinf (y(:))))
14895 e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	448 error ("fminsearch:isinf", "fminsearch: Inf value encountered");
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	449 endif
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	450
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	451 endfunction
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	452
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	453
e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	454 %!demo
23113 9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	455 %! clf;
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	456 %! hold on;
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	457 %! draw_fcn = @(x) (plot (x(1), x(2)) && false);
22962 fba8a6c742c6 fminsearch.m: Fix error in "guarded_eval". Markus Muetzel <markus.muetzel@gmx.de> parents: 22323 diff changeset	458 %! fcn = @(x) (x(1)-5).^2 + (x(2)-8).^4;
14895 e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	459 %! x0 = [0;0];
23113 9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	460 %! [xmin, fval] = fminsearch (fcn, x0, optimset ("OutputFcn", draw_fcn))
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	461 %! hold off;
14895 e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	462
22962 fba8a6c742c6 fminsearch.m: Fix error in "guarded_eval". Markus Muetzel <markus.muetzel@gmx.de> parents: 22323 diff changeset	463 %!assert (fminsearch (@sin, 3, optimset ("MaxIter", 30)), 3*pi/2, 1e-4)
23113 9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	464
25231 e1024058833c test: change remaining %!xtests into %!tests Mike Miller <mtmiller@octave.org> parents: 25054 diff changeset	465 ## FIXME: The following test is for checking that fminsearch stops earlier
e1024058833c test: change remaining %!xtests into %!tests Mike Miller <mtmiller@octave.org> parents: 25054 diff changeset	466 ## with these settings. If the optimizer algorithm is changed, it
e1024058833c test: change remaining %!xtests into %!tests Mike Miller <mtmiller@octave.org> parents: 25054 diff changeset	467 ## may fail. Just adapt the values to make it pass again.
e1024058833c test: change remaining %!xtests into %!tests Mike Miller <mtmiller@octave.org> parents: 25054 diff changeset	468 %!test
23113 9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	469 %! x = fminsearch (@sin, 3, optimset ("MaxIter", 3, "Display", "none"));
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	470 %! assert (x, 4.8750, 1e-4);
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	471 %! x = fminsearch (@sin, 3, optimset ("MaxFunEvals", 18, "Display", "none"));
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	472 %! assert (x, 4.7109, 1e-4);
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	473
22962 fba8a6c742c6 fminsearch.m: Fix error in "guarded_eval". Markus Muetzel <markus.muetzel@gmx.de> parents: 22323 diff changeset	474 %!test
14895 e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	475 %! c = 1.5;
23113 9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	476 %! assert (fminsearch (@(x) x(1).^2 + c*x(2).^2, [1;1]), [0;0], 1e-4);
22962 fba8a6c742c6 fminsearch.m: Fix error in "guarded_eval". Markus Muetzel <markus.muetzel@gmx.de> parents: 22323 diff changeset	477
23113 9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	478 ## additional input argument
22967 3c896180df16 Pass additional arguments for fminsearch to called function (bug #41888). Markus Muetzel <markus.muetzel@gmx.de> parents: 22963 diff changeset	479 %!test
3c896180df16 Pass additional arguments for fminsearch to called function (bug #41888). Markus Muetzel <markus.muetzel@gmx.de> parents: 22963 diff changeset	480 %! x1 = fminsearch (@(x, c) x(1).^2 + c*x(2).^2, [1;1], [], 1.5);
3c896180df16 Pass additional arguments for fminsearch to called function (bug #41888). Markus Muetzel <markus.muetzel@gmx.de> parents: 22963 diff changeset	481 %! assert (x1, [0;0], 1e-4);
23113 9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	482 %! x1 = fminsearch (@(x, c) c(1)x(1).^2 + c(2)x(2).^2, [1;1], ...
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	483 %! optimset ("Display", "none"), [1 1.5]);
22967 3c896180df16 Pass additional arguments for fminsearch to called function (bug #41888). Markus Muetzel <markus.muetzel@gmx.de> parents: 22963 diff changeset	484 %! assert (x1, [0;0], 1e-4);
22962 fba8a6c742c6 fminsearch.m: Fix error in "guarded_eval". Markus Muetzel <markus.muetzel@gmx.de> parents: 22323 diff changeset	485
23113 9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	486 ## all output arguments
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	487 %!test
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	488 %! options = optimset ("Display", "none", "TolX", 1e-4, "TolFun", 1e-7);
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	489 %! [x, fval, exitflag, output] = fminsearch (@sin, 3, options);
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	490 %! assert (x, 3*pi/2, options.TolX);
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	491 %! assert (fval, -1, options.TolFun);
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	492 %! assert (exitflag, 1);
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	493 %! assert (isstruct (output));
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	494 %! assert (isfield (output, "iterations") && isnumeric (output.iterations)
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	495 %! && isscalar (output.iterations) && output.iterations > 0);
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	496 %! assert (isfield (output, "funcCount") && isnumeric (output.funcCount)
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	497 %! && isscalar (output.funcCount) && output.funcCount > 0);
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	498 %! assert (isfield (output, "algorithm") && ischar (output.algorithm));
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	499 %! assert (isfield (output, "message") && ischar (output.message));
14895 e0525ecf156e Add new function fminsearch.m Andy Adler <andy@analyti.ca> parents: diff changeset	500
22962 fba8a6c742c6 fminsearch.m: Fix error in "guarded_eval". Markus Muetzel <markus.muetzel@gmx.de> parents: 22323 diff changeset	501 ## Tests for guarded_eval
fba8a6c742c6 fminsearch.m: Fix error in "guarded_eval". Markus Muetzel <markus.muetzel@gmx.de> parents: 22323 diff changeset	502 %!error <non-real value encountered>
fba8a6c742c6 fminsearch.m: Fix error in "guarded_eval". Markus Muetzel <markus.muetzel@gmx.de> parents: 22323 diff changeset	503 %! fminsearch (@(x) ([0 2i]), 0, optimset ("FunValCheck", "on"));
fba8a6c742c6 fminsearch.m: Fix error in "guarded_eval". Markus Muetzel <markus.muetzel@gmx.de> parents: 22323 diff changeset	504 %!error <NaN value encountered>
fba8a6c742c6 fminsearch.m: Fix error in "guarded_eval". Markus Muetzel <markus.muetzel@gmx.de> parents: 22323 diff changeset	505 %! fminsearch (@(x) (NaN), 0, optimset ("FunValCheck", "on"));
fba8a6c742c6 fminsearch.m: Fix error in "guarded_eval". Markus Muetzel <markus.muetzel@gmx.de> parents: 22323 diff changeset	506 %!error <Inf value encountered>
fba8a6c742c6 fminsearch.m: Fix error in "guarded_eval". Markus Muetzel <markus.muetzel@gmx.de> parents: 22323 diff changeset	507 %! fminsearch (@(x) (Inf), 0, optimset ("FunValCheck", "on"));
23113 9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	508
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	509 ## Test input validation
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	510 %!error fminsearch ()
9241a0fa7873 Additional output arguments for fminsearch (bug #44220). Markus Muetzel <markus.muetzel@gmx.de> parents: 23085 diff changeset	511 %!error fminsearch (1)