Mercurial > octave
annotate scripts/strings/strtok.m @ 23083:e9a0469dedd9 stable
maint: strip extra trailing newlines from files.
author | John W. Eaton <jwe@octave.org> |
---|---|
date | Fri, 20 Jan 2017 12:19:08 -0500 |
parents | bac0d6f07a3e |
children | ef4d915df748 3ac9f9ecfae5 |
rev | line source |
---|---|
22323
bac0d6f07a3e
maint: Update copyright notices for 2016.
John W. Eaton <jwe@octave.org>
parents:
21634
diff
changeset
|
1 ## Copyright (C) 2000-2016 Paul Kienzle |
5827 | 2 ## |
3 ## This file is part of Octave. | |
4 ## | |
5 ## Octave is free software; you can redistribute it and/or modify it | |
6 ## under the terms of the GNU General Public License as published by | |
7016 | 7 ## the Free Software Foundation; either version 3 of the License, or (at |
8 ## your option) any later version. | |
5827 | 9 ## |
10 ## Octave is distributed in the hope that it will be useful, but | |
11 ## WITHOUT ANY WARRANTY; without even the implied warranty of | |
12 ## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU | |
13 ## General Public License for more details. | |
14 ## | |
15 ## You should have received a copy of the GNU General Public License | |
7016 | 16 ## along with Octave; see the file COPYING. If not, see |
17 ## <http://www.gnu.org/licenses/>. | |
5827 | 18 |
19 ## -*- texinfo -*- | |
20852
516bb87ea72e
2015 Code Sprint: remove class of function from docstring for all m-files.
Rik <rik@octave.org>
parents:
20164
diff
changeset
|
20 ## @deftypefn {} {[@var{tok}, @var{rem}] =} strtok (@var{str}) |
516bb87ea72e
2015 Code Sprint: remove class of function from docstring for all m-files.
Rik <rik@octave.org>
parents:
20164
diff
changeset
|
21 ## @deftypefnx {} {[@var{tok}, @var{rem}] =} strtok (@var{str}, @var{delim}) |
11587
c792872f8942
all script files: untabify and strip trailing whitespace
John W. Eaton <jwe@octave.org>
parents:
11523
diff
changeset
|
22 ## |
19593
446c46af4b42
strip trailing whitespace from most source files
John W. Eaton <jwe@octave.org>
parents:
17744
diff
changeset
|
23 ## Find all characters in the string @var{str} up to, but not including, the |
20164
df437a52bcaf
doc: Update more docstrings to have one sentence summary as first line.
Rik <rik@octave.org>
parents:
19833
diff
changeset
|
24 ## first character which is in the string @var{delim}. |
df437a52bcaf
doc: Update more docstrings to have one sentence summary as first line.
Rik <rik@octave.org>
parents:
19833
diff
changeset
|
25 ## |
df437a52bcaf
doc: Update more docstrings to have one sentence summary as first line.
Rik <rik@octave.org>
parents:
19833
diff
changeset
|
26 ## @var{str} may also be a cell array of strings in which case the function |
df437a52bcaf
doc: Update more docstrings to have one sentence summary as first line.
Rik <rik@octave.org>
parents:
19833
diff
changeset
|
27 ## executes on every individual string and returns a cell array of tokens and |
df437a52bcaf
doc: Update more docstrings to have one sentence summary as first line.
Rik <rik@octave.org>
parents:
19833
diff
changeset
|
28 ## remainders. |
df437a52bcaf
doc: Update more docstrings to have one sentence summary as first line.
Rik <rik@octave.org>
parents:
19833
diff
changeset
|
29 ## |
df437a52bcaf
doc: Update more docstrings to have one sentence summary as first line.
Rik <rik@octave.org>
parents:
19833
diff
changeset
|
30 ## Leading delimiters are ignored. If @var{delim} is not specified, |
df437a52bcaf
doc: Update more docstrings to have one sentence summary as first line.
Rik <rik@octave.org>
parents:
19833
diff
changeset
|
31 ## whitespace is assumed. |
df437a52bcaf
doc: Update more docstrings to have one sentence summary as first line.
Rik <rik@octave.org>
parents:
19833
diff
changeset
|
32 ## |
df437a52bcaf
doc: Update more docstrings to have one sentence summary as first line.
Rik <rik@octave.org>
parents:
19833
diff
changeset
|
33 ## If @var{rem} is requested, it contains the remainder of the string, starting |
df437a52bcaf
doc: Update more docstrings to have one sentence summary as first line.
Rik <rik@octave.org>
parents:
19833
diff
changeset
|
34 ## at the first delimiter. |
13790
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
35 ## |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
36 ## Examples: |
8442
502e58a0d44f
Fix docstrings, add examples, references and tests to string functions
Thorsten Meyer <thorsten.meyier@gmx.de>
parents:
8202
diff
changeset
|
37 ## |
502e58a0d44f
Fix docstrings, add examples, references and tests to string functions
Thorsten Meyer <thorsten.meyier@gmx.de>
parents:
8202
diff
changeset
|
38 ## @example |
502e58a0d44f
Fix docstrings, add examples, references and tests to string functions
Thorsten Meyer <thorsten.meyier@gmx.de>
parents:
8202
diff
changeset
|
39 ## @group |
502e58a0d44f
Fix docstrings, add examples, references and tests to string functions
Thorsten Meyer <thorsten.meyier@gmx.de>
parents:
8202
diff
changeset
|
40 ## strtok ("this is the life") |
502e58a0d44f
Fix docstrings, add examples, references and tests to string functions
Thorsten Meyer <thorsten.meyier@gmx.de>
parents:
8202
diff
changeset
|
41 ## @result{} "this" |
5827 | 42 ## |
8442
502e58a0d44f
Fix docstrings, add examples, references and tests to string functions
Thorsten Meyer <thorsten.meyier@gmx.de>
parents:
8202
diff
changeset
|
43 ## [tok, rem] = strtok ("14*27+31", "+-*/") |
502e58a0d44f
Fix docstrings, add examples, references and tests to string functions
Thorsten Meyer <thorsten.meyier@gmx.de>
parents:
8202
diff
changeset
|
44 ## @result{} |
502e58a0d44f
Fix docstrings, add examples, references and tests to string functions
Thorsten Meyer <thorsten.meyier@gmx.de>
parents:
8202
diff
changeset
|
45 ## tok = 14 |
502e58a0d44f
Fix docstrings, add examples, references and tests to string functions
Thorsten Meyer <thorsten.meyier@gmx.de>
parents:
8202
diff
changeset
|
46 ## rem = *27+31 |
502e58a0d44f
Fix docstrings, add examples, references and tests to string functions
Thorsten Meyer <thorsten.meyier@gmx.de>
parents:
8202
diff
changeset
|
47 ## @end group |
502e58a0d44f
Fix docstrings, add examples, references and tests to string functions
Thorsten Meyer <thorsten.meyier@gmx.de>
parents:
8202
diff
changeset
|
48 ## @end example |
13790
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
49 ## @seealso{index, strsplit, strchr, isspace} |
5827 | 50 ## @end deftypefn |
51 | |
52 function [tok, rem] = strtok (str, delim) | |
53 | |
13790
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
54 if (nargin < 1 || nargin > 2) |
5827 | 55 print_usage (); |
13790
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
56 elseif (! (ischar (str) || iscellstr (str))) |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
57 error ("strtok: STR must be a string or cell array of strings."); |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
58 elseif (ischar (str) && ! isvector (str) &&! isempty (str)) |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
59 error ("strtok: STR cannot be a 2-D character array."); |
5827 | 60 endif |
61 | |
62 if (nargin < 2 || isempty (delim)) | |
13790
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
63 ws_delim = true; |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
64 else |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
65 ws_delim = false; |
5827 | 66 endif |
67 | |
68 if (isempty (str)) | |
69 tok = rem = ""; | |
13790
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
70 elseif (ischar (str)) |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
71 if (ws_delim) |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
72 idx = isspace (str); |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
73 elseif (length (delim) <= 7) |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
74 ## Build index of delimiters incrementally for low N. |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
75 idx = str == delim(1); |
19593
446c46af4b42
strip trailing whitespace from most source files
John W. Eaton <jwe@octave.org>
parents:
17744
diff
changeset
|
76 for i = 2:length (delim) |
13790
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
77 idx |= str == delim(i); |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
78 endfor |
5827 | 79 else |
13790
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
80 ## Index the str into a mask of valid values. Faster for large N. |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
81 f = false (256, 1); |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
82 ## This is slower than it could be because of the +1 issue. |
14868
5d3a684236b0
maint: Use Octave coding conventions for cuddling parentheses in scripts directory
Rik <octave@nomad.inbox5.com>
parents:
14363
diff
changeset
|
83 f(uint8 (delim)+1) = true; |
13790
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
84 ## Default goes via double -- unnecessarily long. |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
85 si = uint32 (str); |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
86 ## in-place is faster than str+1 |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
87 ++si; |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
88 idx = f(si); |
5827 | 89 endif |
13790
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
90 |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
91 idx_dlim = find (idx, 1); |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
92 idx_nodlim = find (! idx, 1); |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
93 if (isempty (idx_dlim)) |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
94 ## No delimiter. Return whole string. |
5827 | 95 tok = str; |
96 rem = ""; | |
13790
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
97 elseif (idx_dlim > idx_nodlim) |
19593
446c46af4b42
strip trailing whitespace from most source files
John W. Eaton <jwe@octave.org>
parents:
17744
diff
changeset
|
98 ## Normal case. No leading delimiters and at least 1 delimiter in STR. |
13790
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
99 tok = str(1:idx_dlim-1); |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
100 rem = str(idx_dlim:end); |
5827 | 101 else |
13790
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
102 ## Leading delimiter found. |
19593
446c46af4b42
strip trailing whitespace from most source files
John W. Eaton <jwe@octave.org>
parents:
17744
diff
changeset
|
103 idx_dlim = find (idx(idx_nodlim+1:end), 1); |
13790
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
104 if (isempty (idx_dlim)) |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
105 ## No further delimiters. Return STR stripped of delimiter prefix. |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
106 tok = str(idx_nodlim:end); |
10549 | 107 rem = ""; |
5827 | 108 else |
13790
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
109 ## Strip delimiter prefix. Return STR up to 1st delimiter |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
110 tok = str(idx_nodlim:(idx_dlim + idx_nodlim -1)); |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
111 rem = str((idx_dlim + idx_nodlim):end); |
5827 | 112 endif |
113 endif | |
13790
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
114 else # Cell array of strings |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
115 if (ws_delim) |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
116 delim = '\s'; |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
117 endif |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
118 ptn = [ '^[' delim ']*','([^' delim ']+)','([' delim '].*)$' ]; |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
119 matches = regexp (str, ptn, "tokens"); |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
120 eidx = cellfun ("isempty", matches); |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
121 midx = ! eidx; |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
122 tok = cell (size (str)); |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
123 tok(eidx) = regexprep (str(eidx), [ '^[' delim ']+' ], ''); |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
124 ## Unwrap doubly nested cell array from regexp |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
125 tmp = [matches{midx}]; |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
126 if (! isempty (tmp)) |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
127 tmp = [tmp{:}]; |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
128 endif |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
129 tok(midx) = tmp(1:2:end); |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
130 if (isargout (2)) |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
131 rem = cell (size (str)); |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
132 rem(eidx) = {""}; |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
133 rem(midx) = tmp(2:2:end); |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
134 endif |
5827 | 135 endif |
136 | |
137 endfunction | |
138 | |
13790
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
139 |
5827 | 140 %!demo |
14237
11949c9795a0
Revamp %!demos in m-files to use Octave coding conventions on spacing, etc.
Rik <octave@nomad.inbox5.com>
parents:
14138
diff
changeset
|
141 %! strtok ("this is the life") |
21580
ecce63c99c3f
maint: Add semicolons to terminate code in %! blocks.
Rik <rik@octave.org>
parents:
21317
diff
changeset
|
142 %! # split at the first space, returning "this" |
5827 | 143 |
144 %!demo | |
145 %! s = "14*27+31" | |
13790
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
146 %! while (1) |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
147 %! [t, s] = strtok (s, "+-*/"); |
21634
96518f623c91
Backed out changeset dcf8922b724b
Mike Miller <mtmiller@octave.org>
parents:
21633
diff
changeset
|
148 %! printf ("<%s>", t); |
13790
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
149 %! if (isempty (s)) |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
150 %! break; |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
151 %! endif |
21634
96518f623c91
Backed out changeset dcf8922b724b
Mike Miller <mtmiller@octave.org>
parents:
21633
diff
changeset
|
152 %! printf ("<%s>", s(1)); |
5827 | 153 %! endwhile |
21634
96518f623c91
Backed out changeset dcf8922b724b
Mike Miller <mtmiller@octave.org>
parents:
21633
diff
changeset
|
154 %! printf ("\n"); |
5827 | 155 %! % ---------------------------------------------------- |
156 %! % Demonstrates processing of an entire string split on | |
13790
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
157 %! % a variety of delimiters. Tokens and delimiters are |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
158 %! % printed one after another in angle brackets. |
5827 | 159 |
19833
9fc020886ae9
maint: Clean up m-files to follow Octave coding conventions.
Rik <rik@octave.org>
parents:
19697
diff
changeset
|
160 ## Test the tokens for all cases |
13790
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
161 %!assert (strtok (""), ""); # no string |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
162 %!assert (strtok ("this"), "this"); # no delimiter in string |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
163 %!assert (strtok ("this "), "this"); # delimiter at end |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
164 %!assert (strtok ("this is"), "this"); # delimiter in middle |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
165 %!assert (strtok (" this"), "this"); # delimiter at start |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
166 %!assert (strtok (" this "), "this"); # delimiter at start and end |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
167 %!assert (strtok (" "), ""(1:0)); # delimiter only |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
168 |
19833
9fc020886ae9
maint: Clean up m-files to follow Octave coding conventions.
Rik <rik@octave.org>
parents:
19697
diff
changeset
|
169 ## Test the remainder for all cases |
13790
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
170 %!test [t,r] = strtok (""); assert (r, ""); |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
171 %!test [t,r] = strtok ("this"); assert (r, ""); |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
172 %!test [t,r] = strtok ("this "); assert (r, " "); |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
173 %!test [t,r] = strtok ("this is"); assert (r, " is"); |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
174 %!test [t,r] = strtok (" this"); assert (r, ""); |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
175 %!test [t,r] = strtok (" this "); assert (r, " "); |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
176 %!test [t,r] = strtok (" "); assert (r, ""); |
5827 | 177 |
19833
9fc020886ae9
maint: Clean up m-files to follow Octave coding conventions.
Rik <rik@octave.org>
parents:
19697
diff
changeset
|
178 ## Test all tokens and remainders with cell array input |
13790
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
179 %!test |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
180 %! str = {"", "this", "this ", "this is", " this", " this ", " "}; |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
181 %! [t, r] = strtok (str); |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
182 %! assert (t{1}, ""); |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
183 %! assert (r{1}, ""); |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
184 %! assert (t{2}, "this"); |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
185 %! assert (r{2}, ""); |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
186 %! assert (t{3}, "this"); |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
187 %! assert (r{3}, " "); |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
188 %! assert (t{4}, "this"); |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
189 %! assert (r{4}, " is"); |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
190 %! assert (t{5}, "this"); |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
191 %! assert (r{5}, ""); |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
192 %! assert (t{6}, "this"); |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
193 %! assert (r{6}, " "); |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
194 %! assert (t{7}, ""); |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
195 %! assert (r{7}, ""); |
5827 | 196 |
19833
9fc020886ae9
maint: Clean up m-files to follow Octave coding conventions.
Rik <rik@octave.org>
parents:
19697
diff
changeset
|
197 ## Simple check for 2, 3, and 4 delimeters |
14363
f3d52523cde1
Use Octave coding conventions in all m-file %!test blocks
Rik <octave@nomad.inbox5.com>
parents:
14237
diff
changeset
|
198 %!assert (strtok ("this is", "i "), "th") |
f3d52523cde1
Use Octave coding conventions in all m-file %!test blocks
Rik <octave@nomad.inbox5.com>
parents:
14237
diff
changeset
|
199 %!assert (strtok ("this is", "ij "), "th") |
f3d52523cde1
Use Octave coding conventions in all m-file %!test blocks
Rik <octave@nomad.inbox5.com>
parents:
14237
diff
changeset
|
200 %!assert (strtok ("this is", "ijk "), "th") |
5827 | 201 |
19833
9fc020886ae9
maint: Clean up m-files to follow Octave coding conventions.
Rik <rik@octave.org>
parents:
19697
diff
changeset
|
202 ## Test all cases for 8 delimiters since a different |
13790
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
203 %!# algorithm is used when more than 7 delimiters |
14363
f3d52523cde1
Use Octave coding conventions in all m-file %!test blocks
Rik <octave@nomad.inbox5.com>
parents:
14237
diff
changeset
|
204 %!assert (strtok ("","jklmnop "), "") |
f3d52523cde1
Use Octave coding conventions in all m-file %!test blocks
Rik <octave@nomad.inbox5.com>
parents:
14237
diff
changeset
|
205 %!assert (strtok ("this","jklmnop "), "this") |
f3d52523cde1
Use Octave coding conventions in all m-file %!test blocks
Rik <octave@nomad.inbox5.com>
parents:
14237
diff
changeset
|
206 %!assert (strtok ("this ","jklmnop "), "this") |
f3d52523cde1
Use Octave coding conventions in all m-file %!test blocks
Rik <octave@nomad.inbox5.com>
parents:
14237
diff
changeset
|
207 %!assert (strtok ("this is","jklmnop "), "this") |
f3d52523cde1
Use Octave coding conventions in all m-file %!test blocks
Rik <octave@nomad.inbox5.com>
parents:
14237
diff
changeset
|
208 %!assert (strtok (" this","jklmnop "), "this") |
f3d52523cde1
Use Octave coding conventions in all m-file %!test blocks
Rik <octave@nomad.inbox5.com>
parents:
14237
diff
changeset
|
209 %!assert (strtok (" this ","jklmnop "), "this") |
f3d52523cde1
Use Octave coding conventions in all m-file %!test blocks
Rik <octave@nomad.inbox5.com>
parents:
14237
diff
changeset
|
210 %!assert (strtok (" ","jklmnop "), ""(1:0)) |
8002
30f560a5fbc3
strtok.m: include TAB, LF, VT, FF, and CR in default set of delim characters
John W. Eaton <jwe@octave.org>
parents:
7017
diff
changeset
|
211 |
19833
9fc020886ae9
maint: Clean up m-files to follow Octave coding conventions.
Rik <rik@octave.org>
parents:
19697
diff
changeset
|
212 ## Test 'bad' string orientations |
13790
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
213 %!assert (strtok (" this ".'), "this".'); # delimiter at start and end |
21317
a4faec57f4c8
maint: remove semicolon after %!assert tests to follow Octave conventions.
Rik <rik@octave.org>
parents:
20852
diff
changeset
|
214 %!assert (strtok (" this ".',"jkl "), "this".') |
13790
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
215 |
19833
9fc020886ae9
maint: Clean up m-files to follow Octave coding conventions.
Rik <rik@octave.org>
parents:
19697
diff
changeset
|
216 ## Test with TAB, LF, VT, FF, and CR |
8002
30f560a5fbc3
strtok.m: include TAB, LF, VT, FF, and CR in default set of delim characters
John W. Eaton <jwe@octave.org>
parents:
7017
diff
changeset
|
217 %!test |
30f560a5fbc3
strtok.m: include TAB, LF, VT, FF, and CR in default set of delim characters
John W. Eaton <jwe@octave.org>
parents:
7017
diff
changeset
|
218 %! for ch = "\t\n\v\f\r" |
16994
333243133364
Use matrix concatenation for strings, rather than cstrcat(), for clarity and performance.
Rik <rik@octave.org>
parents:
14868
diff
changeset
|
219 %! [t, r] = strtok (["beg", ch, "end"]); |
8002
30f560a5fbc3
strtok.m: include TAB, LF, VT, FF, and CR in default set of delim characters
John W. Eaton <jwe@octave.org>
parents:
7017
diff
changeset
|
220 %! assert (t, "beg"); |
16994
333243133364
Use matrix concatenation for strings, rather than cstrcat(), for clarity and performance.
Rik <rik@octave.org>
parents:
14868
diff
changeset
|
221 %! assert (r, [ch, "end"]); |
8002
30f560a5fbc3
strtok.m: include TAB, LF, VT, FF, and CR in default set of delim characters
John W. Eaton <jwe@octave.org>
parents:
7017
diff
changeset
|
222 %! endfor |
13790
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
223 |
19833
9fc020886ae9
maint: Clean up m-files to follow Octave coding conventions.
Rik <rik@octave.org>
parents:
19697
diff
changeset
|
224 ## Test input validation |
13790
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
225 %!error strtok () |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
226 %!error strtok ("a", "b", "c") |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
227 %!error <STR must be a string> strtok (1, "b") |
fa94d6a93d45
strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents:
11587
diff
changeset
|
228 %!error <STR cannot be a 2-D> strtok (char ("hello", "world"), "l") |