Mercurial > forge

annotate main/optim/inst/bfgsmin.m @ 9930:d30cfca46e8a octave-forge

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
optim: upgrade license to GPLv3+ and mention on DESCRIPTION the other package licenses
author carandraug
date Fri, 30 Mar 2012 15:14:48 +0000
parents b11b5363d680
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
9930
d30cfca46e8a optim: upgrade license to GPLv3+ and mention on DESCRIPTION the other package licenses
carandraug
parents: 7870
diff changeset
1 ## Copyright (C) 2006 Michael Creel <michael.creel@uab.es>
2370
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
2 ##
9930
d30cfca46e8a optim: upgrade license to GPLv3+ and mention on DESCRIPTION the other package licenses
carandraug
parents: 7870
diff changeset
3 ## This program is free software; you can redistribute it and/or modify it under
d30cfca46e8a optim: upgrade license to GPLv3+ and mention on DESCRIPTION the other package licenses
carandraug
parents: 7870
diff changeset
4 ## the terms of the GNU General Public License as published by the Free Software
d30cfca46e8a optim: upgrade license to GPLv3+ and mention on DESCRIPTION the other package licenses
carandraug
parents: 7870
diff changeset
5 ## Foundation; either version 3 of the License, or (at your option) any later
d30cfca46e8a optim: upgrade license to GPLv3+ and mention on DESCRIPTION the other package licenses
carandraug
parents: 7870
diff changeset
6 ## version.
2370
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
7 ##
9930
d30cfca46e8a optim: upgrade license to GPLv3+ and mention on DESCRIPTION the other package licenses
carandraug
parents: 7870
diff changeset
8 ## This program is distributed in the hope that it will be useful, but WITHOUT
d30cfca46e8a optim: upgrade license to GPLv3+ and mention on DESCRIPTION the other package licenses
carandraug
parents: 7870
diff changeset
9 ## ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
d30cfca46e8a optim: upgrade license to GPLv3+ and mention on DESCRIPTION the other package licenses
carandraug
parents: 7870
diff changeset
10 ## FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more
d30cfca46e8a optim: upgrade license to GPLv3+ and mention on DESCRIPTION the other package licenses
carandraug
parents: 7870
diff changeset
11 ## details.
2370
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
12 ##
9930
d30cfca46e8a optim: upgrade license to GPLv3+ and mention on DESCRIPTION the other package licenses
carandraug
parents: 7870
diff changeset
13 ## You should have received a copy of the GNU General Public License along with
d30cfca46e8a optim: upgrade license to GPLv3+ and mention on DESCRIPTION the other package licenses
carandraug
parents: 7870
diff changeset
14 ## this program; if not, see <http://www.gnu.org/licenses/>.
2370
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
15
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
16 ## bfgsmin: bfgs or limited memory bfgs minimization of function
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
17 ##
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
18 ## Usage: [x, obj_value, convergence, iters] = bfgsmin(f, args, control)
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
19 ##
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
20 ## The function must be of the form
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
21 ## [value, return_2,..., return_m] = f(arg_1, arg_2,..., arg_n)
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
22 ## By default, minimization is w.r.t. arg_1, but it can be done
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
23 ## w.r.t. any argument that is a vector. Numeric derivatives are
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
24 ## used unless analytic derivatives are supplied. See bfgsmin_example.m
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
25 ## for methods.
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
26 ##
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
27 ## Arguments:
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
28 ## * f: name of function to minimize (string)
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
29 ## * args: a cell array that holds all arguments of the function
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
30 ## The argument with respect to which minimization is done
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
31 ## MUST be a vector
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
32 ## * control: an optional cell array of 1-8 elements. If a cell
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
33 ## array shorter than 8 elements is provided, the trailing elements
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
34 ## are provided with default values.
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
35 ## * elem 1: maximum iterations (positive integer, or -1 or Inf for unlimited (default))
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
36 ## * elem 2: verbosity
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
37 ## 0 = no screen output (default)
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
38 ## 1 = only final results
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
39 ## 2 = summary every iteration
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
40 ## 3 = detailed information
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
41 ## * elem 3: convergence criterion
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
42 ## 1 = strict (function, gradient and param change) (default)
2658
b3b306babb2c correct erroneous information in help string
mcreel
parents: 2370
diff changeset
43 ## 0 = weak - only function convergence required
2370
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
44 ## * elem 4: arg in f_args with respect to which minimization is done (default is first)
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
45 ## * elem 5: (optional) Memory limit for lbfgs. If it's a positive integer
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
46 ## then lbfgs will be use. Otherwise ordinary bfgs is used
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
47 ## * elem 6: function change tolerance, default 1e-12
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
48 ## * elem 7: parameter change tolerance, default 1e-6
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
49 ## * elem 8: gradient tolerance, default 1e-5
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
50 ##
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
51 ## Returns:
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
52 ## * x: the minimizer
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
53 ## * obj_value: the value of f() at x
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
54 ## * convergence: 1 if normal conv, other values if not
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
55 ## * iters: number of iterations performed
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
56 ##
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
57 ## Example: see bfgsmin_example.m
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
58
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
59 function [parameter, obj, convergence, iters] = bfgsmin(f, f_args, control)
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
60
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
61 # check number and types of arguments
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
62 if ((nargin < 2) || (nargin > 3))
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
63 usage("bfgsmin: you must supply 2 or 3 arguments");
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
64 endif
4886
cbeed5a759e9 clean up isstr and is_vector warnings, simplify examples
mcreel
parents: 4404
diff changeset
65 if (!ischar(f)) usage("bfgsmin: first argument must be string holding objective function name"); endif
2370
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
66 if (!iscell(f_args)) usage("bfgsmin: second argument must cell array of function arguments"); endif
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
67 if (nargin > 2)
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
68 if (!iscell(control))
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
69 usage("bfgsmin: 3rd argument must be a cell array of 1-8 elements");
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
70 endif
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
71 if (length(control) > 8)
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
72 usage("bfgsmin: 3rd argument must be a cell array of 1-8 elements");
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
73 endif
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
74 else control = {};
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
75 endif
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
76
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
77 # provide defaults for missing controls
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
78 if (length(control) == 0) control{1} = -1; endif # limit on iterations
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
79 if (length(control) == 1) control{2} = 0; endif # level of verbosity
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
80 if (length(control) == 2) control{3} = 1; endif # strong (function, gradient and parameter change) convergence required?
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
81 if (length(control) == 3) control{4} = 1; endif # argument with respect to which minimization is done
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
82 if (length(control) == 4) control{5} = 0; endif # memory for lbfgs: 0 uses ordinary bfgs
6880
604e95e37fcb slacken default function convergence tolerance, it was too strict in comparison to gradient and param change tolerances
mcreel
parents: 4886
diff changeset
83 if (length(control) == 5) control{6} = 1e-10; endif # tolerance for function convergence
2370
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
84 if (length(control) == 6) control{7} = 1e-6; endif # tolerance for parameter convergence
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
85 if (length(control) == 7) control{8} = 1e-5; endif # tolerance for gradient convergence
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
86
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
87 # validity checks on all controls
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
88 tmp = control{1};
7870
b11b5363d680 Get rid of some warnings with Octave-3.3.54+
i7tiol
parents: 7139
diff changeset
89 if (((tmp !=Inf) && (tmp != -1)) && (tmp > 0 && (mod(tmp,1) != 0)))
2370
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
90 usage("bfgsmin: 1st element of 3rd argument (iteration limit) must be Inf or positive integer");
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
91 endif
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
92 tmp = control{2};
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
93 if ((tmp < 0) || (tmp > 3) || (mod(tmp,1) != 0))
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
94 usage("bfgsmin: 2nd element of 3rd argument (verbosity level) must be 0, 1, 2, or 3");
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
95 endif
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
96 tmp = control{3};
7870
b11b5363d680 Get rid of some warnings with Octave-3.3.54+
i7tiol
parents: 7139
diff changeset
97 if ((tmp != 0) && (tmp != 1))
2370
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
98 usage("bfgsmin: 3rd element of 3rd argument (strong/weak convergence) must be 0 (weak) or 1 (strong)");
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
99 endif
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
100 tmp = control{4};
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
101 if ((tmp < 1) || (tmp > length(f_args)) || (mod(tmp,1) != 0))
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
102 usage("bfgsmin: 4th element of 3rd argument (arg with respect to which minimization is done) must be an integer that indicates one of the elements of f_args");
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
103 endif
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
104 tmp = control{5};
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
105 if ((tmp < 0) || (mod(tmp,1) != 0))
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
106 usage("bfgsmin: 5th element of 3rd argument (memory for lbfgs must be zero (regular bfgs) or a positive integer");
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
107 endif
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
108 tmp = control{6};
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
109 if (tmp < 0)
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
110 usage("bfgsmin: 6th element of 3rd argument (tolerance for function convergence) must be a positive real number");
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
111 endif
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
112 tmp = control{7};
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
113 if (tmp < 0)
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
114 usage("bfgsmin: 7th element of 3rd argument (tolerance for parameter convergence) must be a positive real number");
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
115 endif
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
116 tmp = control{8};
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
117 if (tmp < 0)
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
118 usage("bfgsmin: 8th element of 3rd argument (tolerance for gradient convergence) must be a positive real number");
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
119 endif
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
120
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
121 # check that the parameter we minimize w.r.t. is a vector
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
122 minarg = control{4};
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
123 theta = f_args{minarg};
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
124 theta = theta(:);
4886
cbeed5a759e9 clean up isstr and is_vector warnings, simplify examples
mcreel
parents: 4404
diff changeset
125 if (!isvector(theta)) usage("bfgsmin: minimization must be done with respect to a vector of parameters"); endif
2370
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
126 f_args{minarg} = theta;
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
127
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
128 # now go ahead and do the minimization
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
129 [parameter, obj, convergence, iters] = __bfgsmin(f, f_args, control);
24d6a5cdedfe Changed the directory structure to match the package system
hauberg
parents:
diff changeset
130 endfunction