Mercurial > octave
annotate scripts/optimization/fminunc.m @ 23219:3ac9f9ecfae5 stable
maint: Update copyright dates.
author | John W. Eaton <jwe@octave.org> |
---|---|
date | Wed, 22 Feb 2017 12:39:29 -0500 |
parents | e9a0469dedd9 |
children | 092078913d54 |
rev | line source |
---|---|
23219
3ac9f9ecfae5
maint: Update copyright dates.
John W. Eaton <jwe@octave.org>
parents:
23083
diff
changeset
|
1 ## Copyright (C) 2008-2017 VZLU Prague, a.s. |
9084
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
2 ## |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
3 ## This file is part of Octave. |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
4 ## |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
5 ## Octave is free software; you can redistribute it and/or modify it |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
6 ## under the terms of the GNU General Public License as published by |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
7 ## the Free Software Foundation; either version 3 of the License, or (at |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
8 ## your option) any later version. |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
9 ## |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
10 ## Octave is distributed in the hope that it will be useful, but |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
11 ## WITHOUT ANY WARRANTY; without even the implied warranty of |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
12 ## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
13 ## General Public License for more details. |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
14 ## |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
15 ## You should have received a copy of the GNU General Public License |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
16 ## along with Octave; see the file COPYING. If not, see |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
17 ## <http://www.gnu.org/licenses/>. |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
18 ## |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
19 ## Author: Jaroslav Hajek <highegg@gmail.com> |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
20 |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
21 ## -*- texinfo -*- |
20852
516bb87ea72e
2015 Code Sprint: remove class of function from docstring for all m-files.
Rik <rik@octave.org>
parents:
20735
diff
changeset
|
22 ## @deftypefn {} {} fminunc (@var{fcn}, @var{x0}) |
516bb87ea72e
2015 Code Sprint: remove class of function from docstring for all m-files.
Rik <rik@octave.org>
parents:
20735
diff
changeset
|
23 ## @deftypefnx {} {} fminunc (@var{fcn}, @var{x0}, @var{options}) |
516bb87ea72e
2015 Code Sprint: remove class of function from docstring for all m-files.
Rik <rik@octave.org>
parents:
20735
diff
changeset
|
24 ## @deftypefnx {} {[@var{x}, @var{fval}, @var{info}, @var{output}, @var{grad}, @var{hess}] =} fminunc (@var{fcn}, @dots{}) |
10793
be55736a0783
Grammarcheck the documentation from m-files.
Rik <octave@nomad.inbox5.com>
parents:
10687
diff
changeset
|
25 ## Solve an unconstrained optimization problem defined by the function |
be55736a0783
Grammarcheck the documentation from m-files.
Rik <octave@nomad.inbox5.com>
parents:
10687
diff
changeset
|
26 ## @var{fcn}. |
14895
e0525ecf156e
Add new function fminsearch.m
Andy Adler <andy@analyti.ca>
parents:
14868
diff
changeset
|
27 ## |
20165
f1d0f506ee78
doc: Update more docstrings to have one sentence summary as first line.
Rik <rik@octave.org>
parents:
19833
diff
changeset
|
28 ## @var{fcn} should accept a vector (array) defining the unknown variables, and |
f1d0f506ee78
doc: Update more docstrings to have one sentence summary as first line.
Rik <rik@octave.org>
parents:
19833
diff
changeset
|
29 ## return the objective function value, optionally with gradient. |
18609 | 30 ## @code{fminunc} attempts to determine a vector @var{x} such that |
20165
f1d0f506ee78
doc: Update more docstrings to have one sentence summary as first line.
Rik <rik@octave.org>
parents:
19833
diff
changeset
|
31 ## @code{@var{fcn} (@var{x})} is a local minimum. |
f1d0f506ee78
doc: Update more docstrings to have one sentence summary as first line.
Rik <rik@octave.org>
parents:
19833
diff
changeset
|
32 ## |
f1d0f506ee78
doc: Update more docstrings to have one sentence summary as first line.
Rik <rik@octave.org>
parents:
19833
diff
changeset
|
33 ## @var{x0} determines a starting guess. The shape of @var{x0} is preserved in |
f1d0f506ee78
doc: Update more docstrings to have one sentence summary as first line.
Rik <rik@octave.org>
parents:
19833
diff
changeset
|
34 ## all calls to @var{fcn}, but otherwise is treated as a column vector. |
f1d0f506ee78
doc: Update more docstrings to have one sentence summary as first line.
Rik <rik@octave.org>
parents:
19833
diff
changeset
|
35 ## |
f1d0f506ee78
doc: Update more docstrings to have one sentence summary as first line.
Rik <rik@octave.org>
parents:
19833
diff
changeset
|
36 ## @var{options} is a structure specifying additional options. Currently, |
f1d0f506ee78
doc: Update more docstrings to have one sentence summary as first line.
Rik <rik@octave.org>
parents:
19833
diff
changeset
|
37 ## @code{fminunc} recognizes these options: |
17281
bc924baa2c4e
doc: Add new @qcode macro for code samples which are quoted.
Rik <rik@octave.org>
parents:
14895
diff
changeset
|
38 ## @qcode{"FunValCheck"}, @qcode{"OutputFcn"}, @qcode{"TolX"}, |
bc924baa2c4e
doc: Add new @qcode macro for code samples which are quoted.
Rik <rik@octave.org>
parents:
14895
diff
changeset
|
39 ## @qcode{"TolFun"}, @qcode{"MaxIter"}, @qcode{"MaxFunEvals"}, |
20165
f1d0f506ee78
doc: Update more docstrings to have one sentence summary as first line.
Rik <rik@octave.org>
parents:
19833
diff
changeset
|
40 ## @qcode{"GradObj"}, @qcode{"FinDiffType"}, @qcode{"TypicalX"}, |
f1d0f506ee78
doc: Update more docstrings to have one sentence summary as first line.
Rik <rik@octave.org>
parents:
19833
diff
changeset
|
41 ## @qcode{"AutoScaling"}. |
9084
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
42 ## |
20165
f1d0f506ee78
doc: Update more docstrings to have one sentence summary as first line.
Rik <rik@octave.org>
parents:
19833
diff
changeset
|
43 ## If @qcode{"GradObj"} is @qcode{"on"}, it specifies that @var{fcn}, when |
21546
f7f97d7e9294
doc: Wrap m-file docstrings to 79 characters + newline (80 total).
Rik <rik@octave.org>
parents:
21178
diff
changeset
|
44 ## called with two output arguments, also returns the Jacobian matrix of |
f7f97d7e9294
doc: Wrap m-file docstrings to 79 characters + newline (80 total).
Rik <rik@octave.org>
parents:
21178
diff
changeset
|
45 ## partial first derivatives at the requested point. @code{TolX} specifies |
f7f97d7e9294
doc: Wrap m-file docstrings to 79 characters + newline (80 total).
Rik <rik@octave.org>
parents:
21178
diff
changeset
|
46 ## the termination tolerance for the unknown variables @var{x}, while |
f7f97d7e9294
doc: Wrap m-file docstrings to 79 characters + newline (80 total).
Rik <rik@octave.org>
parents:
21178
diff
changeset
|
47 ## @code{TolFun} is a tolerance for the objective function value @var{fval}. |
f7f97d7e9294
doc: Wrap m-file docstrings to 79 characters + newline (80 total).
Rik <rik@octave.org>
parents:
21178
diff
changeset
|
48 ## The default is @code{1e-7} for both options. |
11587
c792872f8942
all script files: untabify and strip trailing whitespace
John W. Eaton <jwe@octave.org>
parents:
11523
diff
changeset
|
49 ## |
18609 | 50 ## For a description of the other options, see @code{optimset}. |
9084
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
51 ## |
18609 | 52 ## On return, @var{x} is the location of the minimum and @var{fval} contains |
20165
f1d0f506ee78
doc: Update more docstrings to have one sentence summary as first line.
Rik <rik@octave.org>
parents:
19833
diff
changeset
|
53 ## the value of the objective function at @var{x}. |
f1d0f506ee78
doc: Update more docstrings to have one sentence summary as first line.
Rik <rik@octave.org>
parents:
19833
diff
changeset
|
54 ## |
f1d0f506ee78
doc: Update more docstrings to have one sentence summary as first line.
Rik <rik@octave.org>
parents:
19833
diff
changeset
|
55 ## @var{info} may be one of the following values: |
11587
c792872f8942
all script files: untabify and strip trailing whitespace
John W. Eaton <jwe@octave.org>
parents:
11523
diff
changeset
|
56 ## |
9084
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
57 ## @table @asis |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
58 ## @item 1 |
10793
be55736a0783
Grammarcheck the documentation from m-files.
Rik <octave@nomad.inbox5.com>
parents:
10687
diff
changeset
|
59 ## Converged to a solution point. Relative gradient error is less than |
18609 | 60 ## specified by @code{TolFun}. |
10821
693e22af08ae
Grammarcheck documentation of m-files
Rik <octave@nomad.inbox5.com>
parents:
10793
diff
changeset
|
61 ## |
9084
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
62 ## @item 2 |
18609 | 63 ## Last relative step size was less than @code{TolX}. |
10821
693e22af08ae
Grammarcheck documentation of m-files
Rik <octave@nomad.inbox5.com>
parents:
10793
diff
changeset
|
64 ## |
9084
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
65 ## @item 3 |
18609 | 66 ## Last relative change in function value was less than @code{TolFun}. |
10821
693e22af08ae
Grammarcheck documentation of m-files
Rik <octave@nomad.inbox5.com>
parents:
10793
diff
changeset
|
67 ## |
9084
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
68 ## @item 0 |
18816
e275d15c27b5
doc: Periodic spellcheck of documentation.
Rik <rik@octave.org>
parents:
18609
diff
changeset
|
69 ## Iteration limit exceeded---either maximum number of algorithm iterations |
18609 | 70 ## @code{MaxIter} or maximum number of function evaluations @code{MaxFunEvals}. |
71 ## | |
72 ## @item -1 | |
18816
e275d15c27b5
doc: Periodic spellcheck of documentation.
Rik <rik@octave.org>
parents:
18609
diff
changeset
|
73 ## Algorithm terminated by @code{OutputFcn}. |
10821
693e22af08ae
Grammarcheck documentation of m-files
Rik <octave@nomad.inbox5.com>
parents:
10793
diff
changeset
|
74 ## |
9084
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
75 ## @item -3 |
11587
c792872f8942
all script files: untabify and strip trailing whitespace
John W. Eaton <jwe@octave.org>
parents:
11523
diff
changeset
|
76 ## The trust region radius became excessively small. |
9084
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
77 ## @end table |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
78 ## |
21546
f7f97d7e9294
doc: Wrap m-file docstrings to 79 characters + newline (80 total).
Rik <rik@octave.org>
parents:
21178
diff
changeset
|
79 ## Optionally, @code{fminunc} can return a structure with convergence |
f7f97d7e9294
doc: Wrap m-file docstrings to 79 characters + newline (80 total).
Rik <rik@octave.org>
parents:
21178
diff
changeset
|
80 ## statistics (@var{output}), the output gradient (@var{grad}) at the |
f7f97d7e9294
doc: Wrap m-file docstrings to 79 characters + newline (80 total).
Rik <rik@octave.org>
parents:
21178
diff
changeset
|
81 ## solution @var{x}, and approximate Hessian (@var{hess}) at the solution |
f7f97d7e9294
doc: Wrap m-file docstrings to 79 characters + newline (80 total).
Rik <rik@octave.org>
parents:
21178
diff
changeset
|
82 ## @var{x}. |
9627
5bcfa0b346e8
fix extra outputs in fminunc
Jaroslav Hajek <highegg@gmail.com>
parents:
9623
diff
changeset
|
83 ## |
20695
6faaab833605
fminunc.m: Clean up function to meet Octave coding standards.
Rik <rik@octave.org>
parents:
20694
diff
changeset
|
84 ## Application Notes: If the objective function is a single nonlinear equation |
6faaab833605
fminunc.m: Clean up function to meet Octave coding standards.
Rik <rik@octave.org>
parents:
20694
diff
changeset
|
85 ## of one variable then using @code{fminbnd} is usually a better choice. |
20165
f1d0f506ee78
doc: Update more docstrings to have one sentence summary as first line.
Rik <rik@octave.org>
parents:
19833
diff
changeset
|
86 ## |
22092
d3cef63f79ac
* fminunc.m: Fix incorrect reference to fminunc as fminsearch (bug #48479)
Mike Miller <mtmiller@octave.org>
parents:
21758
diff
changeset
|
87 ## The algorithm used by @code{fminunc} is a gradient search which depends |
20165
f1d0f506ee78
doc: Update more docstrings to have one sentence summary as first line.
Rik <rik@octave.org>
parents:
19833
diff
changeset
|
88 ## on the objective function being differentiable. If the function has |
f1d0f506ee78
doc: Update more docstrings to have one sentence summary as first line.
Rik <rik@octave.org>
parents:
19833
diff
changeset
|
89 ## discontinuities it may be better to use a derivative-free algorithm such as |
f1d0f506ee78
doc: Update more docstrings to have one sentence summary as first line.
Rik <rik@octave.org>
parents:
19833
diff
changeset
|
90 ## @code{fminsearch}. |
14895
e0525ecf156e
Add new function fminsearch.m
Andy Adler <andy@analyti.ca>
parents:
14868
diff
changeset
|
91 ## @seealso{fminbnd, fminsearch, optimset} |
9084
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
92 ## @end deftypefn |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
93 |
13027
b9a89ca0fb75
prevent optimization functions from setting ans in workspace at startup
John W. Eaton <jwe@octave.org>
parents:
12578
diff
changeset
|
94 ## PKG_ADD: ## Discard result to avoid polluting workspace with ans at startup. |
b9a89ca0fb75
prevent optimization functions from setting ans in workspace at startup
John W. Eaton <jwe@octave.org>
parents:
12578
diff
changeset
|
95 ## PKG_ADD: [~] = __all_opts__ ("fminunc"); |
9084
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
96 |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
97 function [x, fval, info, output, grad, hess] = fminunc (fcn, x0, options = struct ()) |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
98 |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
99 ## Get default options if requested. |
20695
6faaab833605
fminunc.m: Clean up function to meet Octave coding standards.
Rik <rik@octave.org>
parents:
20694
diff
changeset
|
100 if (nargin == 1 && strcmp (fcn, "defaults")) |
14552
86854d032a37
maint: miscellaneous style fixes for .m files
John W. Eaton <jwe@octave.org>
parents:
14386
diff
changeset
|
101 x = optimset ("MaxIter", 400, "MaxFunEvals", Inf, |
86854d032a37
maint: miscellaneous style fixes for .m files
John W. Eaton <jwe@octave.org>
parents:
14386
diff
changeset
|
102 "GradObj", "off", "TolX", 1e-7, "TolFun", 1e-7, |
86854d032a37
maint: miscellaneous style fixes for .m files
John W. Eaton <jwe@octave.org>
parents:
14386
diff
changeset
|
103 "OutputFcn", [], "FunValCheck", "off", |
86854d032a37
maint: miscellaneous style fixes for .m files
John W. Eaton <jwe@octave.org>
parents:
14386
diff
changeset
|
104 "FinDiffType", "central", |
86854d032a37
maint: miscellaneous style fixes for .m files
John W. Eaton <jwe@octave.org>
parents:
14386
diff
changeset
|
105 "TypicalX", [], "AutoScaling", "off"); |
9084
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
106 return; |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
107 endif |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
108 |
21171
2935d56203a4
Fix regressions caused by ismatrix definition change (partial fix bug #47036).
Rik <rik@octave.org>
parents:
20165
diff
changeset
|
109 if (nargin < 2 || nargin > 3 || ! isnumeric (x0)) |
9084
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
110 print_usage (); |
11587
c792872f8942
all script files: untabify and strip trailing whitespace
John W. Eaton <jwe@octave.org>
parents:
11523
diff
changeset
|
111 endif |
9084
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
112 |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
113 if (ischar (fcn)) |
9464
e598248a060d
safer str2func use in optim functions
Jaroslav Hajek <highegg@gmail.com>
parents:
9212
diff
changeset
|
114 fcn = str2func (fcn, "global"); |
9084
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
115 endif |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
116 |
20695
6faaab833605
fminunc.m: Clean up function to meet Octave coding standards.
Rik <rik@octave.org>
parents:
20694
diff
changeset
|
117 xsz = size (x0); |
9084
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
118 n = numel (x0); |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
119 |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
120 has_grad = strcmpi (optimget (options, "GradObj", "off"), "on"); |
9212
6feb27c38da1
support central differences in fminunc and fsolve
Jaroslav Hajek <highegg@gmail.com>
parents:
9207
diff
changeset
|
121 cdif = strcmpi (optimget (options, "FinDiffType", "central"), "central"); |
9084
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
122 maxiter = optimget (options, "MaxIter", 400); |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
123 maxfev = optimget (options, "MaxFunEvals", Inf); |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
124 outfcn = optimget (options, "OutputFcn"); |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
125 |
20695
6faaab833605
fminunc.m: Clean up function to meet Octave coding standards.
Rik <rik@octave.org>
parents:
20694
diff
changeset
|
126 ## Get scaling matrix using the TypicalX option. If set to "auto", the |
6faaab833605
fminunc.m: Clean up function to meet Octave coding standards.
Rik <rik@octave.org>
parents:
20694
diff
changeset
|
127 ## scaling matrix is estimated using the Jacobian. |
10201
5c66978f3fdf
support TypicalX and AutoScaling in fsolve/fminunc, don't autoscale by default
Jaroslav Hajek <highegg@gmail.com>
parents:
9899
diff
changeset
|
128 typicalx = optimget (options, "TypicalX"); |
5c66978f3fdf
support TypicalX and AutoScaling in fsolve/fminunc, don't autoscale by default
Jaroslav Hajek <highegg@gmail.com>
parents:
9899
diff
changeset
|
129 if (isempty (typicalx)) |
5c66978f3fdf
support TypicalX and AutoScaling in fsolve/fminunc, don't autoscale by default
Jaroslav Hajek <highegg@gmail.com>
parents:
9899
diff
changeset
|
130 typicalx = ones (n, 1); |
5c66978f3fdf
support TypicalX and AutoScaling in fsolve/fminunc, don't autoscale by default
Jaroslav Hajek <highegg@gmail.com>
parents:
9899
diff
changeset
|
131 endif |
5c66978f3fdf
support TypicalX and AutoScaling in fsolve/fminunc, don't autoscale by default
Jaroslav Hajek <highegg@gmail.com>
parents:
9899
diff
changeset
|
132 autoscale = strcmpi (optimget (options, "AutoScaling", "off"), "on"); |
5c66978f3fdf
support TypicalX and AutoScaling in fsolve/fminunc, don't autoscale by default
Jaroslav Hajek <highegg@gmail.com>
parents:
9899
diff
changeset
|
133 if (! autoscale) |
5c66978f3fdf
support TypicalX and AutoScaling in fsolve/fminunc, don't autoscale by default
Jaroslav Hajek <highegg@gmail.com>
parents:
9899
diff
changeset
|
134 dg = 1 ./ typicalx; |
5c66978f3fdf
support TypicalX and AutoScaling in fsolve/fminunc, don't autoscale by default
Jaroslav Hajek <highegg@gmail.com>
parents:
9899
diff
changeset
|
135 endif |
5c66978f3fdf
support TypicalX and AutoScaling in fsolve/fminunc, don't autoscale by default
Jaroslav Hajek <highegg@gmail.com>
parents:
9899
diff
changeset
|
136 |
9084
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
137 funvalchk = strcmpi (optimget (options, "FunValCheck", "off"), "on"); |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
138 |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
139 if (funvalchk) |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
140 ## Replace fcn with a guarded version. |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
141 fcn = @(x) guarded_eval (fcn, x); |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
142 endif |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
143 |
20695
6faaab833605
fminunc.m: Clean up function to meet Octave coding standards.
Rik <rik@octave.org>
parents:
20694
diff
changeset
|
144 ## These defaults are rather stringent. I think that normally, user |
9084
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
145 ## prefers accuracy to performance. |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
146 |
10201
5c66978f3fdf
support TypicalX and AutoScaling in fsolve/fminunc, don't autoscale by default
Jaroslav Hajek <highegg@gmail.com>
parents:
9899
diff
changeset
|
147 tolx = optimget (options, "TolX", 1e-7); |
5c66978f3fdf
support TypicalX and AutoScaling in fsolve/fminunc, don't autoscale by default
Jaroslav Hajek <highegg@gmail.com>
parents:
9899
diff
changeset
|
148 tolf = optimget (options, "TolFun", 1e-7); |
9084
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
149 |
9623
bc0739d02724
update initial TR step for fsolve and fminunc
Jaroslav Hajek <highegg@gmail.com>
parents:
9464
diff
changeset
|
150 factor = 0.1; |
9084
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
151 ## FIXME: TypicalX corresponds to user scaling (???) |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
152 autodg = true; |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
153 |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
154 niter = 1; |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
155 nfev = 0; |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
156 |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
157 x = x0(:); |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
158 info = 0; |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
159 |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
160 ## Initial evaluation. |
20695
6faaab833605
fminunc.m: Clean up function to meet Octave coding standards.
Rik <rik@octave.org>
parents:
20694
diff
changeset
|
161 fval = fcn (reshape (x, xsz)); |
9084
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
162 n = length (x); |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
163 |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
164 if (! isempty (outfcn)) |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
165 optimvalues.iter = niter; |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
166 optimvalues.funccount = nfev; |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
167 optimvalues.fval = fval; |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
168 optimvalues.searchdirection = zeros (n, 1); |
20695
6faaab833605
fminunc.m: Clean up function to meet Octave coding standards.
Rik <rik@octave.org>
parents:
20694
diff
changeset
|
169 state = "init"; |
9084
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
170 stop = outfcn (x, optimvalues, state); |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
171 if (stop) |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
172 info = -1; |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
173 break; |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
174 endif |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
175 endif |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
176 |
21099
52af4092f863
For optimization scripts, correctly choose tolerance (eps) based on class of fun and X0.
Rik <rik@octave.org>
parents:
20852
diff
changeset
|
177 if (isa (x0, "single") || isa (fval, "single")) |
52af4092f863
For optimization scripts, correctly choose tolerance (eps) based on class of fun and X0.
Rik <rik@octave.org>
parents:
20852
diff
changeset
|
178 macheps = eps ("single"); |
52af4092f863
For optimization scripts, correctly choose tolerance (eps) based on class of fun and X0.
Rik <rik@octave.org>
parents:
20852
diff
changeset
|
179 else |
52af4092f863
For optimization scripts, correctly choose tolerance (eps) based on class of fun and X0.
Rik <rik@octave.org>
parents:
20852
diff
changeset
|
180 macheps = eps ("double"); |
52af4092f863
For optimization scripts, correctly choose tolerance (eps) based on class of fun and X0.
Rik <rik@octave.org>
parents:
20852
diff
changeset
|
181 endif |
52af4092f863
For optimization scripts, correctly choose tolerance (eps) based on class of fun and X0.
Rik <rik@octave.org>
parents:
20852
diff
changeset
|
182 |
9084
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
183 nsuciter = 0; |
9199
399884c9d4a1
import the step adaptation strategy from fsolve to fminunc
Jaroslav Hajek <highegg@gmail.com>
parents:
9084
diff
changeset
|
184 lastratio = 0; |
9084
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
185 |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
186 grad = []; |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
187 |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
188 ## Outer loop. |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
189 while (niter < maxiter && nfev < maxfev && ! info) |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
190 |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
191 grad0 = grad; |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
192 |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
193 ## Calculate function value and gradient (possibly via FD). |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
194 if (has_grad) |
20695
6faaab833605
fminunc.m: Clean up function to meet Octave coding standards.
Rik <rik@octave.org>
parents:
20694
diff
changeset
|
195 [fval, grad] = fcn (reshape (x, xsz)); |
9084
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
196 grad = grad(:); |
20695
6faaab833605
fminunc.m: Clean up function to meet Octave coding standards.
Rik <rik@octave.org>
parents:
20694
diff
changeset
|
197 nfev += 1; |
9084
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
198 else |
20695
6faaab833605
fminunc.m: Clean up function to meet Octave coding standards.
Rik <rik@octave.org>
parents:
20694
diff
changeset
|
199 grad = __fdjac__ (fcn, reshape (x, xsz), fval, typicalx, cdif)(:); |
9212
6feb27c38da1
support central differences in fminunc and fsolve
Jaroslav Hajek <highegg@gmail.com>
parents:
9207
diff
changeset
|
200 nfev += (1 + cdif) * length (x); |
9084
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
201 endif |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
202 |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
203 if (niter == 1) |
11587
c792872f8942
all script files: untabify and strip trailing whitespace
John W. Eaton <jwe@octave.org>
parents:
11523
diff
changeset
|
204 ## Initialize by identity matrix. |
9084
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
205 hesr = eye (n); |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
206 else |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
207 ## Use the damped BFGS formula. |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
208 y = grad - grad0; |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
209 sBs = sumsq (w); |
20695
6faaab833605
fminunc.m: Clean up function to meet Octave coding standards.
Rik <rik@octave.org>
parents:
20694
diff
changeset
|
210 Bs = hesr' * w; |
6faaab833605
fminunc.m: Clean up function to meet Octave coding standards.
Rik <rik@octave.org>
parents:
20694
diff
changeset
|
211 sy = y' * s; |
9633
ecc2c556f844
simplify damped BFGS formula
Jaroslav Hajek <highegg@gmail.com>
parents:
9631
diff
changeset
|
212 theta = 0.8 / max (1 - sy / sBs, 0.8); |
9084
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
213 r = theta * y + (1-theta) * Bs; |
20695
6faaab833605
fminunc.m: Clean up function to meet Octave coding standards.
Rik <rik@octave.org>
parents:
20694
diff
changeset
|
214 hesr = cholupdate (hesr, r / sqrt (s' * r), "+"); |
9084
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
215 [hesr, info] = cholupdate (hesr, Bs / sqrt (sBs), "-"); |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
216 if (info) |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
217 hesr = eye (n); |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
218 endif |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
219 endif |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
220 |
10201
5c66978f3fdf
support TypicalX and AutoScaling in fsolve/fminunc, don't autoscale by default
Jaroslav Hajek <highegg@gmail.com>
parents:
9899
diff
changeset
|
221 if (autoscale) |
20695
6faaab833605
fminunc.m: Clean up function to meet Octave coding standards.
Rik <rik@octave.org>
parents:
20694
diff
changeset
|
222 ## Second derivatives approximate the Hessian. |
6faaab833605
fminunc.m: Clean up function to meet Octave coding standards.
Rik <rik@octave.org>
parents:
20694
diff
changeset
|
223 d2f = norm (hesr, "columns").'; |
10201
5c66978f3fdf
support TypicalX and AutoScaling in fsolve/fminunc, don't autoscale by default
Jaroslav Hajek <highegg@gmail.com>
parents:
9899
diff
changeset
|
224 if (niter == 1) |
5c66978f3fdf
support TypicalX and AutoScaling in fsolve/fminunc, don't autoscale by default
Jaroslav Hajek <highegg@gmail.com>
parents:
9899
diff
changeset
|
225 dg = d2f; |
5c66978f3fdf
support TypicalX and AutoScaling in fsolve/fminunc, don't autoscale by default
Jaroslav Hajek <highegg@gmail.com>
parents:
9899
diff
changeset
|
226 else |
5c66978f3fdf
support TypicalX and AutoScaling in fsolve/fminunc, don't autoscale by default
Jaroslav Hajek <highegg@gmail.com>
parents:
9899
diff
changeset
|
227 ## FIXME: maybe fixed lower and upper bounds? |
5c66978f3fdf
support TypicalX and AutoScaling in fsolve/fminunc, don't autoscale by default
Jaroslav Hajek <highegg@gmail.com>
parents:
9899
diff
changeset
|
228 dg = max (0.1*dg, d2f); |
5c66978f3fdf
support TypicalX and AutoScaling in fsolve/fminunc, don't autoscale by default
Jaroslav Hajek <highegg@gmail.com>
parents:
9899
diff
changeset
|
229 endif |
5c66978f3fdf
support TypicalX and AutoScaling in fsolve/fminunc, don't autoscale by default
Jaroslav Hajek <highegg@gmail.com>
parents:
9899
diff
changeset
|
230 endif |
5c66978f3fdf
support TypicalX and AutoScaling in fsolve/fminunc, don't autoscale by default
Jaroslav Hajek <highegg@gmail.com>
parents:
9899
diff
changeset
|
231 |
9084
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
232 if (niter == 1) |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
233 xn = norm (dg .* x); |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
234 ## FIXME: something better? |
9628
73e6ad869f08
further correct initial TR step strategy
Jaroslav Hajek <highegg@gmail.com>
parents:
9627
diff
changeset
|
235 delta = factor * max (xn, 1); |
9084
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
236 endif |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
237 |
18609 | 238 ## FIXME: why tolf*n*xn? If abs (e) ~ abs(x) * eps is a vector |
21751
b571fc85953f
maint: Use two spaces after period to indicate sentence break.
Rik <rik@octave.org>
parents:
21578
diff
changeset
|
239 ## of perturbations of x, then norm (hesr*e) <= eps*xn, i.e., by |
9084
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
240 ## tolf ~ eps we demand as much accuracy as we can expect. |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
241 if (norm (grad) <= tolf*n*xn) |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
242 info = 1; |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
243 break; |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
244 endif |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
245 |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
246 suc = false; |
9207
25f50d2d76b3
improve TR updating strategy for fminunc and fsolve
Jaroslav Hajek <highegg@gmail.com>
parents:
9199
diff
changeset
|
247 decfac = 0.5; |
25f50d2d76b3
improve TR updating strategy for fminunc and fsolve
Jaroslav Hajek <highegg@gmail.com>
parents:
9199
diff
changeset
|
248 |
9084
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
249 ## Inner loop. |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
250 while (! suc && niter <= maxiter && nfev < maxfev && ! info) |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
251 |
9631
00958d0c4e3c
split __dogleg__ > __doglegm__
Jaroslav Hajek <highegg@gmail.com>
parents:
9628
diff
changeset
|
252 s = - __doglegm__ (hesr, grad, dg, delta); |
9084
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
253 |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
254 sn = norm (dg .* s); |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
255 if (niter == 1) |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
256 delta = min (delta, sn); |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
257 endif |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
258 |
20695
6faaab833605
fminunc.m: Clean up function to meet Octave coding standards.
Rik <rik@octave.org>
parents:
20694
diff
changeset
|
259 fval1 = fcn (reshape (x + s, xsz))(:); |
6faaab833605
fminunc.m: Clean up function to meet Octave coding standards.
Rik <rik@octave.org>
parents:
20694
diff
changeset
|
260 nfev += 1; |
9084
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
261 |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
262 if (fval1 < fval) |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
263 ## Scaled actual reduction. |
19833
9fc020886ae9
maint: Clean up m-files to follow Octave coding conventions.
Rik <rik@octave.org>
parents:
19697
diff
changeset
|
264 actred = (fval - fval1) / (abs (fval1) + abs (fval)); |
9084
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
265 else |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
266 actred = -1; |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
267 endif |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
268 |
20695
6faaab833605
fminunc.m: Clean up function to meet Octave coding standards.
Rik <rik@octave.org>
parents:
20694
diff
changeset
|
269 w = hesr * s; |
9084
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
270 ## Scaled predicted reduction, and ratio. |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
271 t = 1/2 * sumsq (w) + grad'*s; |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
272 if (t < 0) |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
273 prered = -t/(abs (fval) + abs (fval + t)); |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
274 ratio = actred / prered; |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
275 else |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
276 prered = 0; |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
277 ratio = 0; |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
278 endif |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
279 |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
280 ## Update delta. |
14868
5d3a684236b0
maint: Use Octave coding conventions for cuddling parentheses in scripts directory
Rik <octave@nomad.inbox5.com>
parents:
14552
diff
changeset
|
281 if (ratio < min (max (0.1, 0.8*lastratio), 0.9)) |
9207
25f50d2d76b3
improve TR updating strategy for fminunc and fsolve
Jaroslav Hajek <highegg@gmail.com>
parents:
9199
diff
changeset
|
282 delta *= decfac; |
25f50d2d76b3
improve TR updating strategy for fminunc and fsolve
Jaroslav Hajek <highegg@gmail.com>
parents:
9199
diff
changeset
|
283 decfac ^= 1.4142; |
20695
6faaab833605
fminunc.m: Clean up function to meet Octave coding standards.
Rik <rik@octave.org>
parents:
20694
diff
changeset
|
284 if (delta <= 10*macheps*xn) |
9084
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
285 ## Trust region became uselessly small. |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
286 info = -3; |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
287 break; |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
288 endif |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
289 else |
9199
399884c9d4a1
import the step adaptation strategy from fsolve to fminunc
Jaroslav Hajek <highegg@gmail.com>
parents:
9084
diff
changeset
|
290 lastratio = ratio; |
9207
25f50d2d76b3
improve TR updating strategy for fminunc and fsolve
Jaroslav Hajek <highegg@gmail.com>
parents:
9199
diff
changeset
|
291 decfac = 0.5; |
9084
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
292 if (abs (1-ratio) <= 0.1) |
9207
25f50d2d76b3
improve TR updating strategy for fminunc and fsolve
Jaroslav Hajek <highegg@gmail.com>
parents:
9199
diff
changeset
|
293 delta = 1.4142*sn; |
9084
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
294 elseif (ratio >= 0.5) |
9207
25f50d2d76b3
improve TR updating strategy for fminunc and fsolve
Jaroslav Hajek <highegg@gmail.com>
parents:
9199
diff
changeset
|
295 delta = max (delta, 1.4142*sn); |
9084
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
296 endif |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
297 endif |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
298 |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
299 if (ratio >= 1e-4) |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
300 ## Successful iteration. |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
301 x += s; |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
302 xn = norm (dg .* x); |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
303 fval = fval1; |
20735
418ae0cb752f
Replace ++,-- with in-place operators for performance.
Rik <rik@octave.org>
parents:
20715
diff
changeset
|
304 nsuciter += 1; |
9084
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
305 suc = true; |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
306 endif |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
307 |
20695
6faaab833605
fminunc.m: Clean up function to meet Octave coding standards.
Rik <rik@octave.org>
parents:
20694
diff
changeset
|
308 niter += 1; |
9084
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
309 |
18609 | 310 ## FIXME: should outputfcn be called only after a successful iteration? |
9084
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
311 if (! isempty (outfcn)) |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
312 optimvalues.iter = niter; |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
313 optimvalues.funccount = nfev; |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
314 optimvalues.fval = fval; |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
315 optimvalues.searchdirection = s; |
20695
6faaab833605
fminunc.m: Clean up function to meet Octave coding standards.
Rik <rik@octave.org>
parents:
20694
diff
changeset
|
316 state = "iter"; |
9084
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
317 stop = outfcn (x, optimvalues, state); |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
318 if (stop) |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
319 info = -1; |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
320 break; |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
321 endif |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
322 endif |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
323 |
20695
6faaab833605
fminunc.m: Clean up function to meet Octave coding standards.
Rik <rik@octave.org>
parents:
20694
diff
changeset
|
324 ## Tests for termination conditions. A mysterious place, anything |
9084
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
325 ## can happen if you change something here... |
11587
c792872f8942
all script files: untabify and strip trailing whitespace
John W. Eaton <jwe@octave.org>
parents:
11523
diff
changeset
|
326 |
9084
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
327 ## The rule of thumb (which I'm not sure M*b is quite following) |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
328 ## is that for a tolerance that depends on scaling, only 0 makes |
20695
6faaab833605
fminunc.m: Clean up function to meet Octave coding standards.
Rik <rik@octave.org>
parents:
20694
diff
changeset
|
329 ## sense as a default value. But 0 usually means uselessly long |
9084
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
330 ## iterations, so we need scaling-independent tolerances wherever |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
331 ## possible. |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
332 |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
333 ## The following tests done only after successful step. |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
334 if (ratio >= 1e-4) |
20695
6faaab833605
fminunc.m: Clean up function to meet Octave coding standards.
Rik <rik@octave.org>
parents:
20694
diff
changeset
|
335 ## This one is classic. Note that we use scaled variables again, |
9084
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
336 ## but compare to scaled step, so nothing bad. |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
337 if (sn <= tolx*xn) |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
338 info = 2; |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
339 ## Again a classic one. |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
340 elseif (actred < tolf) |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
341 info = 3; |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
342 endif |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
343 endif |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
344 |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
345 endwhile |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
346 endwhile |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
347 |
20695
6faaab833605
fminunc.m: Clean up function to meet Octave coding standards.
Rik <rik@octave.org>
parents:
20694
diff
changeset
|
348 ## When info != 1, recalculate the gradient and Hessian using the final x. |
20694
a7dbc4fc3732
fminunc.m: Correctly calculate gradient and hessian even when fminunc fails (bug #40101)
José Vallet <jose.vallet@aalto.fi>
parents:
20165
diff
changeset
|
349 if (nargout > 4 && (info == -1 || info == 2 || info == 3)) |
20715
5b7643257978
Remove trailing whitespace at end of lines.
Rik <rik@octave.org>
parents:
20695
diff
changeset
|
350 grad0 = grad; |
20694
a7dbc4fc3732
fminunc.m: Correctly calculate gradient and hessian even when fminunc fails (bug #40101)
José Vallet <jose.vallet@aalto.fi>
parents:
20165
diff
changeset
|
351 if (has_grad) |
20695
6faaab833605
fminunc.m: Clean up function to meet Octave coding standards.
Rik <rik@octave.org>
parents:
20694
diff
changeset
|
352 [fval, grad] = fcn (reshape (x, xsz)); |
20694
a7dbc4fc3732
fminunc.m: Correctly calculate gradient and hessian even when fminunc fails (bug #40101)
José Vallet <jose.vallet@aalto.fi>
parents:
20165
diff
changeset
|
353 grad = grad(:); |
a7dbc4fc3732
fminunc.m: Correctly calculate gradient and hessian even when fminunc fails (bug #40101)
José Vallet <jose.vallet@aalto.fi>
parents:
20165
diff
changeset
|
354 else |
20695
6faaab833605
fminunc.m: Clean up function to meet Octave coding standards.
Rik <rik@octave.org>
parents:
20694
diff
changeset
|
355 grad = __fdjac__ (fcn, reshape (x, xsz), fval, typicalx, cdif)(:); |
20694
a7dbc4fc3732
fminunc.m: Correctly calculate gradient and hessian even when fminunc fails (bug #40101)
José Vallet <jose.vallet@aalto.fi>
parents:
20165
diff
changeset
|
356 endif |
9084
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
357 |
20694
a7dbc4fc3732
fminunc.m: Correctly calculate gradient and hessian even when fminunc fails (bug #40101)
José Vallet <jose.vallet@aalto.fi>
parents:
20165
diff
changeset
|
358 if (nargout > 5) |
a7dbc4fc3732
fminunc.m: Correctly calculate gradient and hessian even when fminunc fails (bug #40101)
José Vallet <jose.vallet@aalto.fi>
parents:
20165
diff
changeset
|
359 ## Use the damped BFGS formula. |
a7dbc4fc3732
fminunc.m: Correctly calculate gradient and hessian even when fminunc fails (bug #40101)
José Vallet <jose.vallet@aalto.fi>
parents:
20165
diff
changeset
|
360 y = grad - grad0; |
a7dbc4fc3732
fminunc.m: Correctly calculate gradient and hessian even when fminunc fails (bug #40101)
José Vallet <jose.vallet@aalto.fi>
parents:
20165
diff
changeset
|
361 sBs = sumsq (w); |
a7dbc4fc3732
fminunc.m: Correctly calculate gradient and hessian even when fminunc fails (bug #40101)
José Vallet <jose.vallet@aalto.fi>
parents:
20165
diff
changeset
|
362 Bs = hesr' * w; |
a7dbc4fc3732
fminunc.m: Correctly calculate gradient and hessian even when fminunc fails (bug #40101)
José Vallet <jose.vallet@aalto.fi>
parents:
20165
diff
changeset
|
363 sy = y' * s; |
a7dbc4fc3732
fminunc.m: Correctly calculate gradient and hessian even when fminunc fails (bug #40101)
José Vallet <jose.vallet@aalto.fi>
parents:
20165
diff
changeset
|
364 theta = 0.8 / max (1 - sy / sBs, 0.8); |
a7dbc4fc3732
fminunc.m: Correctly calculate gradient and hessian even when fminunc fails (bug #40101)
José Vallet <jose.vallet@aalto.fi>
parents:
20165
diff
changeset
|
365 r = theta * y + (1-theta) * Bs; |
a7dbc4fc3732
fminunc.m: Correctly calculate gradient and hessian even when fminunc fails (bug #40101)
José Vallet <jose.vallet@aalto.fi>
parents:
20165
diff
changeset
|
366 hesr = cholupdate (hesr, r / sqrt (s' * r), "+"); |
a7dbc4fc3732
fminunc.m: Correctly calculate gradient and hessian even when fminunc fails (bug #40101)
José Vallet <jose.vallet@aalto.fi>
parents:
20165
diff
changeset
|
367 hesr = cholupdate (hesr, Bs / sqrt (sBs), "-"); |
a7dbc4fc3732
fminunc.m: Correctly calculate gradient and hessian even when fminunc fails (bug #40101)
José Vallet <jose.vallet@aalto.fi>
parents:
20165
diff
changeset
|
368 endif |
a7dbc4fc3732
fminunc.m: Correctly calculate gradient and hessian even when fminunc fails (bug #40101)
José Vallet <jose.vallet@aalto.fi>
parents:
20165
diff
changeset
|
369 ## Return the gradient in the same shape as x |
20695
6faaab833605
fminunc.m: Clean up function to meet Octave coding standards.
Rik <rik@octave.org>
parents:
20694
diff
changeset
|
370 grad = reshape (grad, xsz); |
20694
a7dbc4fc3732
fminunc.m: Correctly calculate gradient and hessian even when fminunc fails (bug #40101)
José Vallet <jose.vallet@aalto.fi>
parents:
20165
diff
changeset
|
371 endif |
a7dbc4fc3732
fminunc.m: Correctly calculate gradient and hessian even when fminunc fails (bug #40101)
José Vallet <jose.vallet@aalto.fi>
parents:
20165
diff
changeset
|
372 |
9084
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
373 ## Restore original shapes. |
20695
6faaab833605
fminunc.m: Clean up function to meet Octave coding standards.
Rik <rik@octave.org>
parents:
20694
diff
changeset
|
374 x = reshape (x, xsz); |
9084
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
375 |
20695
6faaab833605
fminunc.m: Clean up function to meet Octave coding standards.
Rik <rik@octave.org>
parents:
20694
diff
changeset
|
376 if (nargout > 3) |
6faaab833605
fminunc.m: Clean up function to meet Octave coding standards.
Rik <rik@octave.org>
parents:
20694
diff
changeset
|
377 output.iterations = niter; |
6faaab833605
fminunc.m: Clean up function to meet Octave coding standards.
Rik <rik@octave.org>
parents:
20694
diff
changeset
|
378 output.successful = nsuciter; |
6faaab833605
fminunc.m: Clean up function to meet Octave coding standards.
Rik <rik@octave.org>
parents:
20694
diff
changeset
|
379 output.funcCount = nfev; |
6faaab833605
fminunc.m: Clean up function to meet Octave coding standards.
Rik <rik@octave.org>
parents:
20694
diff
changeset
|
380 endif |
9084
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
381 |
9627
5bcfa0b346e8
fix extra outputs in fminunc
Jaroslav Hajek <highegg@gmail.com>
parents:
9623
diff
changeset
|
382 if (nargout > 5) |
5bcfa0b346e8
fix extra outputs in fminunc
Jaroslav Hajek <highegg@gmail.com>
parents:
9623
diff
changeset
|
383 hess = hesr'*hesr; |
5bcfa0b346e8
fix extra outputs in fminunc
Jaroslav Hajek <highegg@gmail.com>
parents:
9623
diff
changeset
|
384 endif |
5bcfa0b346e8
fix extra outputs in fminunc
Jaroslav Hajek <highegg@gmail.com>
parents:
9623
diff
changeset
|
385 |
9084
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
386 endfunction |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
387 |
18609 | 388 ## A helper function that evaluates a function and checks for bad results. |
9084
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
389 function [fx, gx] = guarded_eval (fun, x) |
21758
ffad2baa90f7
maint: Use newlines to make code more readable.
Rik <rik@octave.org>
parents:
21751
diff
changeset
|
390 |
9084
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
391 if (nargout > 1) |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
392 [fx, gx] = fun (x); |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
393 else |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
394 fx = fun (x); |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
395 gx = []; |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
396 endif |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
397 |
14383
07c55bceca23
Fix guarded_eval() subfunction in fminunc (bug #35534).
Olaf Till <olaf.till@uni-jena.de>
parents:
14138
diff
changeset
|
398 if (! (isreal (fx) && isreal (gx))) |
11587
c792872f8942
all script files: untabify and strip trailing whitespace
John W. Eaton <jwe@octave.org>
parents:
11523
diff
changeset
|
399 error ("fminunc:notreal", "fminunc: non-real value encountered"); |
9084
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
400 elseif (any (isnan (fx(:)))) |
11587
c792872f8942
all script files: untabify and strip trailing whitespace
John W. Eaton <jwe@octave.org>
parents:
11523
diff
changeset
|
401 error ("fminunc:isnan", "fminunc: NaN value encountered"); |
14386
59aab666f2bf
Extend "FunValCheck" option to optimization routines to detect Inf values.
Rik <octave@nomad.inbox5.com>
parents:
14383
diff
changeset
|
402 elseif (any (isinf (fx(:)))) |
59aab666f2bf
Extend "FunValCheck" option to optimization routines to detect Inf values.
Rik <octave@nomad.inbox5.com>
parents:
14383
diff
changeset
|
403 error ("fminunc:isinf", "fminunc: Inf value encountered"); |
9084
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
404 endif |
21758
ffad2baa90f7
maint: Use newlines to make code more readable.
Rik <rik@octave.org>
parents:
21751
diff
changeset
|
405 |
9084
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
406 endfunction |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
407 |
14386
59aab666f2bf
Extend "FunValCheck" option to optimization routines to detect Inf values.
Rik <octave@nomad.inbox5.com>
parents:
14383
diff
changeset
|
408 |
20695
6faaab833605
fminunc.m: Clean up function to meet Octave coding standards.
Rik <rik@octave.org>
parents:
20694
diff
changeset
|
409 %!function f = __rosenb__ (x) |
9084
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
410 %! n = length (x); |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
411 %! f = sumsq (1 - x(1:n-1)) + 100 * sumsq (x(2:n) - x(1:n-1).^2); |
13305
63463570d9fe
Add %!endfunction block keyword to test.m
Rik <octave@nomad.inbox5.com>
parents:
13027
diff
changeset
|
412 %!endfunction |
19833
9fc020886ae9
maint: Clean up m-files to follow Octave coding conventions.
Rik <rik@octave.org>
parents:
19697
diff
changeset
|
413 %! |
9084
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
414 %!test |
20695
6faaab833605
fminunc.m: Clean up function to meet Octave coding standards.
Rik <rik@octave.org>
parents:
20694
diff
changeset
|
415 %! [x, fval, info, out] = fminunc (@__rosenb__, [5, -5]); |
9084
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
416 %! tol = 2e-5; |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
417 %! assert (info > 0); |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
418 %! assert (x, ones (1, 2), tol); |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
419 %! assert (fval, 0, tol); |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
420 %!test |
20695
6faaab833605
fminunc.m: Clean up function to meet Octave coding standards.
Rik <rik@octave.org>
parents:
20694
diff
changeset
|
421 %! [x, fval, info, out] = fminunc (@__rosenb__, zeros (1, 4)); |
9084
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
422 %! tol = 2e-5; |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
423 %! assert (info > 0); |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
424 %! assert (x, ones (1, 4), tol); |
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
425 %! assert (fval, 0, tol); |
19833
9fc020886ae9
maint: Clean up m-files to follow Octave coding conventions.
Rik <rik@octave.org>
parents:
19697
diff
changeset
|
426 |
9fc020886ae9
maint: Clean up m-files to follow Octave coding conventions.
Rik <rik@octave.org>
parents:
19697
diff
changeset
|
427 ## Test FunValCheck works correctly |
14386
59aab666f2bf
Extend "FunValCheck" option to optimization routines to detect Inf values.
Rik <octave@nomad.inbox5.com>
parents:
14383
diff
changeset
|
428 %!assert (fminunc (@(x) x^2, 1, optimset ("FunValCheck", "on")), 0, eps) |
59aab666f2bf
Extend "FunValCheck" option to optimization routines to detect Inf values.
Rik <octave@nomad.inbox5.com>
parents:
14383
diff
changeset
|
429 %!error <non-real value> fminunc (@(x) x + i, 1, optimset ("FunValCheck", "on")) |
59aab666f2bf
Extend "FunValCheck" option to optimization routines to detect Inf values.
Rik <octave@nomad.inbox5.com>
parents:
14383
diff
changeset
|
430 %!error <NaN value> fminunc (@(x) x + NaN, 1, optimset ("FunValCheck", "on")) |
59aab666f2bf
Extend "FunValCheck" option to optimization routines to detect Inf values.
Rik <octave@nomad.inbox5.com>
parents:
14383
diff
changeset
|
431 %!error <Inf value> fminunc (@(x) x + Inf, 1, optimset ("FunValCheck", "on")) |
59aab666f2bf
Extend "FunValCheck" option to optimization routines to detect Inf values.
Rik <octave@nomad.inbox5.com>
parents:
14383
diff
changeset
|
432 |
9084
b7210faa3ed0
implement fminunc using factored trust-region BFGS
Jaroslav Hajek <highegg@gmail.com>
parents:
diff
changeset
|
433 |
9899
9f25290a35e8
more private function and subfunction changes
John W. Eaton <jwe@octave.org>
parents:
9758
diff
changeset
|
434 ## Solve the double dogleg trust-region minimization problem: |
9f25290a35e8
more private function and subfunction changes
John W. Eaton <jwe@octave.org>
parents:
9758
diff
changeset
|
435 ## Minimize 1/2*norm(r*x)^2 subject to the constraint norm(d.*x) <= delta, |
9f25290a35e8
more private function and subfunction changes
John W. Eaton <jwe@octave.org>
parents:
9758
diff
changeset
|
436 ## x being a convex combination of the gauss-newton and scaled gradient. |
9f25290a35e8
more private function and subfunction changes
John W. Eaton <jwe@octave.org>
parents:
9758
diff
changeset
|
437 |
21578
683a1beee538
maint: Use "FIXME:" for all code blocks needing further attention.
Rik <rik@octave.org>
parents:
21546
diff
changeset
|
438 ## FIXME: error checks |
683a1beee538
maint: Use "FIXME:" for all code blocks needing further attention.
Rik <rik@octave.org>
parents:
21546
diff
changeset
|
439 ## FIXME: handle singularity, or leave it up to mldivide? |
9899
9f25290a35e8
more private function and subfunction changes
John W. Eaton <jwe@octave.org>
parents:
9758
diff
changeset
|
440 |
9f25290a35e8
more private function and subfunction changes
John W. Eaton <jwe@octave.org>
parents:
9758
diff
changeset
|
441 function x = __doglegm__ (r, g, d, delta) |
21758
ffad2baa90f7
maint: Use newlines to make code more readable.
Rik <rik@octave.org>
parents:
21751
diff
changeset
|
442 |
9899
9f25290a35e8
more private function and subfunction changes
John W. Eaton <jwe@octave.org>
parents:
9758
diff
changeset
|
443 ## Get Gauss-Newton direction. |
9f25290a35e8
more private function and subfunction changes
John W. Eaton <jwe@octave.org>
parents:
9758
diff
changeset
|
444 b = r' \ g; |
9f25290a35e8
more private function and subfunction changes
John W. Eaton <jwe@octave.org>
parents:
9758
diff
changeset
|
445 x = r \ b; |
9f25290a35e8
more private function and subfunction changes
John W. Eaton <jwe@octave.org>
parents:
9758
diff
changeset
|
446 xn = norm (d .* x); |
9f25290a35e8
more private function and subfunction changes
John W. Eaton <jwe@octave.org>
parents:
9758
diff
changeset
|
447 if (xn > delta) |
9f25290a35e8
more private function and subfunction changes
John W. Eaton <jwe@octave.org>
parents:
9758
diff
changeset
|
448 ## GN is too big, get scaled gradient. |
9f25290a35e8
more private function and subfunction changes
John W. Eaton <jwe@octave.org>
parents:
9758
diff
changeset
|
449 s = g ./ d; |
9f25290a35e8
more private function and subfunction changes
John W. Eaton <jwe@octave.org>
parents:
9758
diff
changeset
|
450 sn = norm (s); |
9f25290a35e8
more private function and subfunction changes
John W. Eaton <jwe@octave.org>
parents:
9758
diff
changeset
|
451 if (sn > 0) |
9f25290a35e8
more private function and subfunction changes
John W. Eaton <jwe@octave.org>
parents:
9758
diff
changeset
|
452 ## Normalize and rescale. |
9f25290a35e8
more private function and subfunction changes
John W. Eaton <jwe@octave.org>
parents:
9758
diff
changeset
|
453 s = (s / sn) ./ d; |
9f25290a35e8
more private function and subfunction changes
John W. Eaton <jwe@octave.org>
parents:
9758
diff
changeset
|
454 ## Get the line minimizer in s direction. |
9f25290a35e8
more private function and subfunction changes
John W. Eaton <jwe@octave.org>
parents:
9758
diff
changeset
|
455 tn = norm (r*s); |
9f25290a35e8
more private function and subfunction changes
John W. Eaton <jwe@octave.org>
parents:
9758
diff
changeset
|
456 snm = (sn / tn) / tn; |
9f25290a35e8
more private function and subfunction changes
John W. Eaton <jwe@octave.org>
parents:
9758
diff
changeset
|
457 if (snm < delta) |
10549 | 458 ## Get the dogleg path minimizer. |
9899
9f25290a35e8
more private function and subfunction changes
John W. Eaton <jwe@octave.org>
parents:
9758
diff
changeset
|
459 bn = norm (b); |
9f25290a35e8
more private function and subfunction changes
John W. Eaton <jwe@octave.org>
parents:
9758
diff
changeset
|
460 dxn = delta/xn; snmd = snm/delta; |
9f25290a35e8
more private function and subfunction changes
John W. Eaton <jwe@octave.org>
parents:
9758
diff
changeset
|
461 t = (bn/sn) * (bn/xn) * snmd; |
9f25290a35e8
more private function and subfunction changes
John W. Eaton <jwe@octave.org>
parents:
9758
diff
changeset
|
462 t -= dxn * snmd^2 - sqrt ((t-dxn)^2 + (1-dxn^2)*(1-snmd^2)); |
9f25290a35e8
more private function and subfunction changes
John W. Eaton <jwe@octave.org>
parents:
9758
diff
changeset
|
463 alpha = dxn*(1-snmd^2) / t; |
9f25290a35e8
more private function and subfunction changes
John W. Eaton <jwe@octave.org>
parents:
9758
diff
changeset
|
464 else |
9f25290a35e8
more private function and subfunction changes
John W. Eaton <jwe@octave.org>
parents:
9758
diff
changeset
|
465 alpha = 0; |
9f25290a35e8
more private function and subfunction changes
John W. Eaton <jwe@octave.org>
parents:
9758
diff
changeset
|
466 endif |
9f25290a35e8
more private function and subfunction changes
John W. Eaton <jwe@octave.org>
parents:
9758
diff
changeset
|
467 else |
9f25290a35e8
more private function and subfunction changes
John W. Eaton <jwe@octave.org>
parents:
9758
diff
changeset
|
468 alpha = delta / xn; |
9f25290a35e8
more private function and subfunction changes
John W. Eaton <jwe@octave.org>
parents:
9758
diff
changeset
|
469 snm = 0; |
9f25290a35e8
more private function and subfunction changes
John W. Eaton <jwe@octave.org>
parents:
9758
diff
changeset
|
470 endif |
9f25290a35e8
more private function and subfunction changes
John W. Eaton <jwe@octave.org>
parents:
9758
diff
changeset
|
471 ## Form the appropriate convex combination. |
9f25290a35e8
more private function and subfunction changes
John W. Eaton <jwe@octave.org>
parents:
9758
diff
changeset
|
472 x = alpha * x + ((1-alpha) * min (snm, delta)) * s; |
9f25290a35e8
more private function and subfunction changes
John W. Eaton <jwe@octave.org>
parents:
9758
diff
changeset
|
473 endif |
21758
ffad2baa90f7
maint: Use newlines to make code more readable.
Rik <rik@octave.org>
parents:
21751
diff
changeset
|
474 |
9899
9f25290a35e8
more private function and subfunction changes
John W. Eaton <jwe@octave.org>
parents:
9758
diff
changeset
|
475 endfunction |