Mercurial > octave
annotate scripts/statistics/models/logistic_regression.m @ 21633:dcf8922b724b
Deprecate printf, puts, and scanf. Make fputs a M-file.
* libinterp/corefcn/file-io.cc: extracted printf, puts, and scanf to M-files.
* scripts/deprecated/printf.m: new function, equivalent to version file-io.cc.
* scripts/deprecated/puts.m: new function, equivalent to version file-io.cc.
* scripts/deprecated/scanf.m: new function, equivalent to version file-io.cc.
* scripts/deprecated/module.mk: added printf, puts, and scanf.
* scripts/io/fputs.m: new function, equivalent to version file-io.cc.
* scripts/io/module.mk: added fputs.
* doc/interpreter/io.txi: adapted documentation to current implementation.
* NEWS: announced changes.
* doc/interpreter/basics.txi: replaced printf by fprintf.
* doc/interpreter/stmt.txi: replaced printf by fprintf.
* doc/interpreter/var.txi: replaced printf by fprintf.
* doc/refcard/refcard.tex: removed printf and scanf.
* examples/code/@FIRfilter/display.m: replaced printf by fprintf.
* examples/code/@polynomial/display.m: replaced printf by fprintf.
* scripts/@ftp/display.m: replaced printf by fprintf.
* scripts/general/inputParser.m: replaced printf by fprintf.
* scripts/general/methods.m: replaced printf by fprintf.
* scripts/general/profexplore.m: replaced printf by fprintf.
* scripts/general/profshow.m: replaced printf by fprintf.
* scripts/help/help.m: replaced puts by fputs and printf by fprintf.
* scripts/help/lookfor.m: replaced puts by fputs and printf by fprintf.
* scripts/help/which.m: replaced printf by fprintf.
* scripts/image/imformats.m: replaced printf by fprintf.
* scripts/io/beep.m: replaced puts by fputs.
* scripts/io/textread.m: replaced printf by fprintf.
* scripts/java/javaclasspath.m: replaced printf by fprintf.
* scripts/java/javamem.m: replaced printf by fprintf.
* scripts/miscellaneous/dir.m: replaced printf by fprintf.
* scripts/miscellaneous/dos.m: replaced printf by fprintf.
* scripts/miscellaneous/fact.m: replaced printf by fprintf.
* scripts/miscellaneous/info.m: replaced printf by fprintf.
* scripts/miscellaneous/license.m: replaced printf by fprintf.
* scripts/miscellaneous/ls.m: replaced puts by fputs.
* scripts/miscellaneous/menu.m: replaced printf by fprintf.
* scripts/miscellaneous/mkoctfile.m: replaced printf by fprintf.
* scripts/miscellaneous/private/display_info_file.m:
* scripts/miscellaneous/unix.m: replaced printf by fprintf.
* scripts/miscellaneous/ver.m: replaced printf by fprintf.
* scripts/miscellaneous/what.m: replaced printf by fprintf.
* scripts/ode/ode23.m: replaced printf by fprintf.
* scripts/ode/ode45.m: replaced printf by fprintf.
* scripts/optimization/fminbnd.m: replaced printf by fprintf.
* scripts/optimization/optimset.m: replaced puts by fputs and printf by fprintf.
* scripts/optimization/sqp.m: replaced printf by fprintf.
* scripts/pkg/pkg.m: replaced printf by fprintf.
* scripts/pkg/private/describe.m: replaced printf by fprintf.
* scripts/pkg/private/install.m: replaced printf by fprintf.
* scripts/pkg/private/installed_packages.m: replaced printf by fprintf.
* scripts/pkg/private/list_forge_packages.m: replaced puts by fputs and printf by fprintf.
* scripts/pkg/private/rebuild.m: replaced printf by fprintf.
* scripts/plot/util/__gnuplot_drawnow__.m: replaced puts by fputs and printf by fprintf.
* scripts/signal/stft.m: replaced printf by fprintf.
* scripts/sparse/bicg.m: replaced printf by fprintf.
* scripts/sparse/bicgstab.m: replaced printf by fprintf.
* scripts/sparse/cgs.m: replaced printf by fprintf.
* scripts/sparse/pcg.m: replaced printf by fprintf.
* scripts/sparse/pcr.m: replaced printf by fprintf.
* scripts/sparse/qmr.m: replaced printf by fprintf.
* scripts/statistics/models/logistic_regression.m: replaced printf by fprintf.
* scripts/statistics/tests/anova.m: replaced printf by fprintf.
* scripts/statistics/tests/bartlett_test.m: replaced printf by fprintf.
* scripts/statistics/tests/chisquare_test_homogeneity.m: replaced printf by fprintf.
* scripts/statistics/tests/chisquare_test_independence.m: replaced printf by fprintf.
* scripts/statistics/tests/cor_test.m: replaced printf by fprintf.
* scripts/statistics/tests/f_test_regression.m: replaced printf by fprintf.
* scripts/statistics/tests/hotelling_test.m: replaced printf by fprintf.
* scripts/statistics/tests/hotelling_test_2.m: replaced printf by fprintf.
* scripts/statistics/tests/kolmogorov_smirnov_test.m: replaced printf by fprintf.
* scripts/statistics/tests/kolmogorov_smirnov_test_2.m: replaced printf by fprintf.
* scripts/statistics/tests/kruskal_wallis_test.m: replaced printf by fprintf.
* scripts/statistics/tests/manova.m: replaced printf by fprintf.
* scripts/statistics/tests/mcnemar_test.m: replaced printf by fprintf.
* scripts/statistics/tests/prop_test_2.m: replaced printf by fprintf.
* scripts/statistics/tests/run_test.m: replaced printf by fprintf.
* scripts/statistics/tests/sign_test.m: replaced printf by fprintf.
* scripts/statistics/tests/t_test.m: replaced printf by fprintf.
* scripts/statistics/tests/t_test_2.m: replaced printf by fprintf.
* scripts/statistics/tests/t_test_regression.m: replaced printf by fprintf.
* scripts/statistics/tests/u_test.m: replaced printf by fprintf.
* scripts/statistics/tests/var_test.m: replaced printf by fprintf.
* scripts/statistics/tests/welch_test.m: replaced printf by fprintf.
* scripts/statistics/tests/wilcoxon_test.m: replaced printf by fprintf.
* scripts/statistics/tests/z_test.m: replaced printf by fprintf.
* scripts/statistics/tests/z_test_2.m: replaced printf by fprintf.
* scripts/strings/strtok.m: replaced printf by fprintf.
* scripts/testfun/__run_test_suite__.m: replaced puts by fputs and printf by fprintf.
* scripts/testfun/demo.m: replaced printf by fprintf.
* scripts/testfun/example.m: replaced printf by fprintf.
* scripts/testfun/private/compare_plot_demos.m: replaced printf by fprintf.
* scripts/testfun/rundemos.m: replaced printf by fprintf.
* scripts/testfun/runtests.m: replaced puts by fputs and printf by fprintf.
* scripts/testfun/speed.m: replaced printf by fprintf.
* scripts/testfun/test.m: replaced printf by fprintf.
* scripts/time/calendar.m: replaced puts by fputs and printf by fprintf.
* test/io.tst: replaced printf by fprintf.
author | Kai T. Ohlhus <k.ohlhus@gmail.com> |
---|---|
date | Wed, 20 Apr 2016 17:53:10 +0200 |
parents | 516bb87ea72e |
children | 96518f623c91 |
rev | line source |
---|---|
19697
4197fc428c7d
maint: Update copyright notices for 2015.
John W. Eaton <jwe@octave.org>
parents:
17744
diff
changeset
|
1 ## Copyright (C) 1995-2015 Kurt Hornik |
3426 | 2 ## |
3922 | 3 ## This file is part of Octave. |
4 ## | |
5 ## Octave is free software; you can redistribute it and/or modify it | |
6 ## under the terms of the GNU General Public License as published by | |
7016 | 7 ## the Free Software Foundation; either version 3 of the License, or (at |
8 ## your option) any later version. | |
3426 | 9 ## |
3922 | 10 ## Octave is distributed in the hope that it will be useful, but |
3191 | 11 ## WITHOUT ANY WARRANTY; without even the implied warranty of |
12 ## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU | |
3426 | 13 ## General Public License for more details. |
14 ## | |
3191 | 15 ## You should have received a copy of the GNU General Public License |
7016 | 16 ## along with Octave; see the file COPYING. If not, see |
17 ## <http://www.gnu.org/licenses/>. | |
3191 | 18 |
3454 | 19 ## -*- texinfo -*- |
20852
516bb87ea72e
2015 Code Sprint: remove class of function from docstring for all m-files.
Rik <rik@octave.org>
parents:
20231
diff
changeset
|
20 ## @deftypefn {} {[@var{theta}, @var{beta}, @var{dev}, @var{dl}, @var{d2l}, @var{p}] =} logistic_regression (@var{y}, @var{x}, @var{print}, @var{theta}, @var{beta}) |
3454 | 21 ## Perform ordinal logistic regression. |
3191 | 22 ## |
3454 | 23 ## Suppose @var{y} takes values in @var{k} ordered categories, and let |
24 ## @code{gamma_i (@var{x})} be the cumulative probability that @var{y} | |
25 ## falls in one of the first @var{i} categories given the covariate | |
26 ## @var{x}. Then | |
27 ## | |
28 ## @example | |
29 ## [theta, beta] = logistic_regression (y, x) | |
30 ## @end example | |
31 ## | |
32 ## @noindent | |
3191 | 33 ## fits the model |
3454 | 34 ## |
35 ## @example | |
9051
1bf0ce0930be
Grammar check TexInfo in all .m files
Rik <rdrider0-list@yahoo.com>
parents:
7031
diff
changeset
|
36 ## logit (gamma_i (x)) = theta_i - beta' * x, i = 1 @dots{} k-1 |
3454 | 37 ## @end example |
3191 | 38 ## |
3454 | 39 ## The number of ordinal categories, @var{k}, is taken to be the number |
40 ## of distinct values of @code{round (@var{y})}. If @var{k} equals 2, | |
41 ## @var{y} is binary and the model is ordinary logistic regression. The | |
42 ## matrix @var{x} is assumed to have full column rank. | |
43 ## | |
44 ## Given @var{y} only, @code{theta = logistic_regression (y)} | |
3191 | 45 ## fits the model with baseline logit odds only. |
46 ## | |
47 ## The full form is | |
3454 | 48 ## |
49 ## @example | |
9051
1bf0ce0930be
Grammar check TexInfo in all .m files
Rik <rdrider0-list@yahoo.com>
parents:
7031
diff
changeset
|
50 ## @group |
3454 | 51 ## [theta, beta, dev, dl, d2l, gamma] |
52 ## = logistic_regression (y, x, print, theta, beta) | |
9051
1bf0ce0930be
Grammar check TexInfo in all .m files
Rik <rdrider0-list@yahoo.com>
parents:
7031
diff
changeset
|
53 ## @end group |
3454 | 54 ## @end example |
55 ## | |
56 ## @noindent | |
57 ## in which all output arguments and all input arguments except @var{y} | |
58 ## are optional. | |
3191 | 59 ## |
6754 | 60 ## Setting @var{print} to 1 requests summary information about the fitted |
3454 | 61 ## model to be displayed. Setting @var{print} to 2 requests information |
62 ## about convergence at each iteration. Other values request no | |
63 ## information to be displayed. The input arguments @var{theta} and | |
64 ## @var{beta} give initial estimates for @var{theta} and @var{beta}. | |
65 ## | |
66 ## The returned value @var{dev} holds minus twice the log-likelihood. | |
3191 | 67 ## |
3454 | 68 ## The returned values @var{dl} and @var{d2l} are the vector of first |
69 ## and the matrix of second derivatives of the log-likelihood with | |
70 ## respect to @var{theta} and @var{beta}. | |
3191 | 71 ## |
3454 | 72 ## @var{p} holds estimates for the conditional distribution of @var{y} |
73 ## given @var{x}. | |
74 ## @end deftypefn | |
3191 | 75 |
3426 | 76 ## Original for MATLAB written by Gordon K Smyth <gks@maths.uq.oz.au>, |
3191 | 77 ## U of Queensland, Australia, on Nov 19, 1990. Last revision Aug 3, |
78 ## 1992. | |
79 | |
3456 | 80 ## Author: Gordon K Smyth <gks@maths.uq.oz.au>, |
5428 | 81 ## Adapted-By: KH <Kurt.Hornik@wu-wien.ac.at> |
3456 | 82 ## Description: Ordinal logistic regression |
3191 | 83 |
84 ## Uses the auxiliary functions logistic_regression_derivatives and | |
85 ## logistic_regression_likelihood. | |
86 | |
12530
d70c99028ba3
Make helper functions for logistic_regression private.
Rik <octave@nomad.inbox5.com>
parents:
11523
diff
changeset
|
87 function [theta, beta, dev, dl, d2l, p] = logistic_regression (y, x, print, theta, beta) |
3426 | 88 |
3191 | 89 ## check input |
3426 | 90 y = round (vec (y)); |
91 [my, ny] = size (y); | |
3191 | 92 if (nargin < 2) |
3426 | 93 x = zeros (my, 0); |
3191 | 94 endif; |
3238 | 95 [mx, nx] = size (x); |
3191 | 96 if (mx != my) |
11472
1740012184f9
Use uppercase for variable names in error() strings to match Info documentation. Only m-files done.
Rik <octave@nomad.inbox5.com>
parents:
10635
diff
changeset
|
97 error ("logistic_regression: X and Y must have the same number of observations"); |
3191 | 98 endif |
3426 | 99 |
3191 | 100 ## initial calculations |
101 x = -x; | |
102 tol = 1e-6; incr = 10; decr = 2; | |
103 ymin = min (y); ymax = max (y); yrange = ymax - ymin; | |
104 z = (y * ones (1, yrange)) == ((y * 0 + 1) * (ymin : (ymax - 1))); | |
105 z1 = (y * ones (1, yrange)) == ((y * 0 + 1) * ((ymin + 1) : ymax)); | |
3426 | 106 z = z(:, any (z)); |
19833
9fc020886ae9
maint: Clean up m-files to follow Octave coding conventions.
Rik <rik@octave.org>
parents:
19697
diff
changeset
|
107 z1 = z1(:, any(z1)); |
3238 | 108 [mz, nz] = size (z); |
3426 | 109 |
3191 | 110 ## starting values |
111 if (nargin < 3) | |
3426 | 112 print = 0; |
3191 | 113 endif; |
3426 | 114 if (nargin < 4) |
115 beta = zeros (nx, 1); | |
3191 | 116 endif; |
3426 | 117 if (nargin < 5) |
118 g = cumsum (sum (z))' ./ my; | |
119 theta = log (g ./ (1 - g)); | |
3191 | 120 endif; |
121 tb = [theta; beta]; | |
122 | |
123 ## likelihood and derivatives at starting values | |
124 [g, g1, p, dev] = logistic_regression_likelihood (y, x, tb, z, z1); | |
125 [dl, d2l] = logistic_regression_derivatives (x, z, z1, g, g1, p); | |
126 epsilon = std (vec (d2l)) / 1000; | |
3426 | 127 |
3191 | 128 ## maximize likelihood using Levenberg modified Newton's method |
129 iter = 0; | |
130 while (abs (dl' * (d2l \ dl) / length (dl)) > tol) | |
20231
83792dd9bcc1
Use in-place operators in m-files where possible.
Rik <rik@octave.org>
parents:
19833
diff
changeset
|
131 iter += 1; |
3191 | 132 tbold = tb; |
133 devold = dev; | |
134 tb = tbold - d2l \ dl; | |
135 [g, g1, p, dev] = logistic_regression_likelihood (y, x, tb, z, z1); | |
136 if ((dev - devold) / (dl' * (tb - tbold)) < 0) | |
20231
83792dd9bcc1
Use in-place operators in m-files where possible.
Rik <rik@octave.org>
parents:
19833
diff
changeset
|
137 epsilon /= decr; |
3191 | 138 else |
139 while ((dev - devold) / (dl' * (tb - tbold)) > 0) | |
20231
83792dd9bcc1
Use in-place operators in m-files where possible.
Rik <rik@octave.org>
parents:
19833
diff
changeset
|
140 epsilon *= incr; |
3191 | 141 if (epsilon > 1e+15) |
10635
d1978e7364ad
Print name of function in error() string messages.
Rik <octave@nomad.inbox5.com>
parents:
9245
diff
changeset
|
142 error ("logistic_regression: epsilon too large"); |
3191 | 143 endif |
3426 | 144 tb = tbold - (d2l - epsilon * eye (size (d2l))) \ dl; |
145 [g, g1, p, dev] = logistic_regression_likelihood (y, x, tb, z, z1); | |
3191 | 146 disp ("epsilon"); disp (epsilon); |
147 endwhile | |
148 endif | |
149 [dl, d2l] = logistic_regression_derivatives (x, z, z1, g, g1, p); | |
150 if (print == 2) | |
151 disp ("Iteration"); disp (iter); | |
152 disp ("Deviance"); disp (dev); | |
153 disp ("First derivative"); disp (dl'); | |
154 disp ("Eigenvalues of second derivative"); disp (eig (d2l)'); | |
155 endif | |
156 endwhile | |
157 | |
158 ## tidy up output | |
159 | |
19833
9fc020886ae9
maint: Clean up m-files to follow Octave coding conventions.
Rik <rik@octave.org>
parents:
19697
diff
changeset
|
160 theta = tb(1 : nz, 1); |
9fc020886ae9
maint: Clean up m-files to follow Octave coding conventions.
Rik <rik@octave.org>
parents:
19697
diff
changeset
|
161 beta = tb((nz + 1) : (nz + nx), 1); |
3191 | 162 |
163 if (print >= 1) | |
21633
dcf8922b724b
Deprecate printf, puts, and scanf. Make fputs a M-file.
Kai T. Ohlhus <k.ohlhus@gmail.com>
parents:
20852
diff
changeset
|
164 fprintf ("\n"); |
dcf8922b724b
Deprecate printf, puts, and scanf. Make fputs a M-file.
Kai T. Ohlhus <k.ohlhus@gmail.com>
parents:
20852
diff
changeset
|
165 fprintf ("Logistic Regression Results:\n"); |
dcf8922b724b
Deprecate printf, puts, and scanf. Make fputs a M-file.
Kai T. Ohlhus <k.ohlhus@gmail.com>
parents:
20852
diff
changeset
|
166 fprintf ("\n"); |
dcf8922b724b
Deprecate printf, puts, and scanf. Make fputs a M-file.
Kai T. Ohlhus <k.ohlhus@gmail.com>
parents:
20852
diff
changeset
|
167 fprintf ("Number of Iterations: %d\n", iter); |
dcf8922b724b
Deprecate printf, puts, and scanf. Make fputs a M-file.
Kai T. Ohlhus <k.ohlhus@gmail.com>
parents:
20852
diff
changeset
|
168 fprintf ("Deviance: %f\n", dev); |
dcf8922b724b
Deprecate printf, puts, and scanf. Make fputs a M-file.
Kai T. Ohlhus <k.ohlhus@gmail.com>
parents:
20852
diff
changeset
|
169 fprintf ("Parameter Estimates:\n"); |
dcf8922b724b
Deprecate printf, puts, and scanf. Make fputs a M-file.
Kai T. Ohlhus <k.ohlhus@gmail.com>
parents:
20852
diff
changeset
|
170 fprintf (" Theta S.E.\n"); |
3426 | 171 se = sqrt (diag (inv (-d2l))); |
3191 | 172 for i = 1 : nz |
21633
dcf8922b724b
Deprecate printf, puts, and scanf. Make fputs a M-file.
Kai T. Ohlhus <k.ohlhus@gmail.com>
parents:
20852
diff
changeset
|
173 fprintf (" %8.4f %8.4f\n", tb (i), se (i)); |
3191 | 174 endfor |
175 if (nx > 0) | |
21633
dcf8922b724b
Deprecate printf, puts, and scanf. Make fputs a M-file.
Kai T. Ohlhus <k.ohlhus@gmail.com>
parents:
20852
diff
changeset
|
176 fprintf (" Beta S.E.\n"); |
3191 | 177 for i = (nz + 1) : (nz + nx) |
21633
dcf8922b724b
Deprecate printf, puts, and scanf. Make fputs a M-file.
Kai T. Ohlhus <k.ohlhus@gmail.com>
parents:
20852
diff
changeset
|
178 fprintf (" %8.4f %8.4f\n", tb (i), se (i)); |
3191 | 179 endfor |
180 endif | |
181 endif | |
182 | |
183 if (nargout == 6) | |
184 if (nx > 0) | |
185 e = ((x * beta) * ones (1, nz)) + ((y * 0 + 1) * theta'); | |
186 else | |
187 e = (y * 0 + 1) * theta'; | |
188 endif | |
3238 | 189 gamma = diff ([(y * 0), (exp (e) ./ (1 + exp (e))), (y * 0 + 1)]')'; |
3191 | 190 endif |
3426 | 191 |
3191 | 192 endfunction |
17338
1c89599167a6
maint: End m-files with 1 blank line.
Rik <rik@octave.org>
parents:
14138
diff
changeset
|
193 |