annotate libinterp/parse-tree/lex.ll @ 16275:dd7de0345124

10/10 commits reworking the lexer This and the previous nine changesets completely revise the way significant whitespace is handled inside matrix and cell lists. Now instead of indefinite look ahead, we keep track of all tokens read and look back to previous tokens and ahead at most one character. If the surrounding context changes the parse, we unput the text for the current token and alter the start state, unput an additional character (typically a comma or semicolon) or both. Comments are now handled by a separate set of exlusive start states and there is no explicit handling of comments within any other patterns.
author John W. Eaton <jwe@octave.org>
date Mon, 11 Mar 2013 14:40:29 -0400
parents c5e5f6ccac5d
children 26fd237a3427 8cb65fd72164
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
1994
c2d20f365b84 [project @ 1996-03-03 01:19:32 by jwe]
jwe
parents: 1884
diff changeset
1 /*
1
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
2
14138
72c96de7a403 maint: update copyright notices for 2012
John W. Eaton <jwe@octave.org>
parents: 14074
diff changeset
3 Copyright (C) 1993-2012 John W. Eaton
1
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
4
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
5 This file is part of Octave.
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
6
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
7 Octave is free software; you can redistribute it and/or modify it
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
8 under the terms of the GNU General Public License as published by the
7016
93c65f2a5668 [project @ 2007-10-12 06:40:56 by jwe]
jwe
parents: 6547
diff changeset
9 Free Software Foundation; either version 3 of the License, or (at your
93c65f2a5668 [project @ 2007-10-12 06:40:56 by jwe]
jwe
parents: 6547
diff changeset
10 option) any later version.
1
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
11
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
12 Octave is distributed in the hope that it will be useful, but WITHOUT
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
13 ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
14 FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
15 for more details.
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
16
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
17 You should have received a copy of the GNU General Public License
7016
93c65f2a5668 [project @ 2007-10-12 06:40:56 by jwe]
jwe
parents: 6547
diff changeset
18 along with Octave; see the file COPYING. If not, see
93c65f2a5668 [project @ 2007-10-12 06:40:56 by jwe]
jwe
parents: 6547
diff changeset
19 <http://www.gnu.org/licenses/>.
1
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
20
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
21 */
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
22
16150
891a2a4df71f * lex.ll: Use C-style comments in preamble.
John W. Eaton <jwe@octave.org>
parents: 16149
diff changeset
23 /*
891a2a4df71f * lex.ll: Use C-style comments in preamble.
John W. Eaton <jwe@octave.org>
parents: 16149
diff changeset
24 We are using the pure parser interface and the reentrant lexer
891a2a4df71f * lex.ll: Use C-style comments in preamble.
John W. Eaton <jwe@octave.org>
parents: 16149
diff changeset
25 interface but the Octave parser and lexer are NOT properly
891a2a4df71f * lex.ll: Use C-style comments in preamble.
John W. Eaton <jwe@octave.org>
parents: 16149
diff changeset
26 reentrant because both still use many global variables. It should be
891a2a4df71f * lex.ll: Use C-style comments in preamble.
John W. Eaton <jwe@octave.org>
parents: 16149
diff changeset
27 safe to create a parser object and call it while anotehr parser
891a2a4df71f * lex.ll: Use C-style comments in preamble.
John W. Eaton <jwe@octave.org>
parents: 16149
diff changeset
28 object is active (to parse a callback function while the main
891a2a4df71f * lex.ll: Use C-style comments in preamble.
John W. Eaton <jwe@octave.org>
parents: 16149
diff changeset
29 interactive parser is waiting for input, for example) if you take
891a2a4df71f * lex.ll: Use C-style comments in preamble.
John W. Eaton <jwe@octave.org>
parents: 16149
diff changeset
30 care to properly save and restore (typically with an unwind_protect
891a2a4df71f * lex.ll: Use C-style comments in preamble.
John W. Eaton <jwe@octave.org>
parents: 16149
diff changeset
31 object) relevant global values before and after the nested call.
891a2a4df71f * lex.ll: Use C-style comments in preamble.
John W. Eaton <jwe@octave.org>
parents: 16149
diff changeset
32 */
16149
49dfba4fd3c5 use pure parser and reentrant lexer interfaces
John W. Eaton <jwe@octave.org>
parents: 16148
diff changeset
33
4753
9f0ce1536368 [project @ 2004-02-13 17:55:24 by jwe]
jwe
parents: 4678
diff changeset
34 %option prefix = "octave_"
16146
b6050fc0a2d9 use %option noyywrap
John W. Eaton <jwe@octave.org>
parents: 16139
diff changeset
35 %option noyywrap
16149
49dfba4fd3c5 use pure parser and reentrant lexer interfaces
John W. Eaton <jwe@octave.org>
parents: 16148
diff changeset
36 %option reentrant
49dfba4fd3c5 use pure parser and reentrant lexer interfaces
John W. Eaton <jwe@octave.org>
parents: 16148
diff changeset
37 %option bison-bridge
4753
9f0ce1536368 [project @ 2004-02-13 17:55:24 by jwe]
jwe
parents: 4678
diff changeset
38
10019
7ad32bf759c3 lex.cc: Use %top block to include config.h
John W. Eaton <jwe@octave.org>
parents: 9794
diff changeset
39 %top {
7ad32bf759c3 lex.cc: Use %top block to include config.h
John W. Eaton <jwe@octave.org>
parents: 9794
diff changeset
40 #ifdef HAVE_CONFIG_H
7ad32bf759c3 lex.cc: Use %top block to include config.h
John W. Eaton <jwe@octave.org>
parents: 9794
diff changeset
41 #include <config.h>
7ad32bf759c3 lex.cc: Use %top block to include config.h
John W. Eaton <jwe@octave.org>
parents: 9794
diff changeset
42 #endif
10411
479cc8a0a846 use gnulib namespace
John W. Eaton <jwe@octave.org>
parents: 10315
diff changeset
43
10019
7ad32bf759c3 lex.cc: Use %top block to include config.h
John W. Eaton <jwe@octave.org>
parents: 9794
diff changeset
44 }
7ad32bf759c3 lex.cc: Use %top block to include config.h
John W. Eaton <jwe@octave.org>
parents: 9794
diff changeset
45
4208
e96f52432059 [project @ 2002-12-03 19:48:57 by jwe]
jwe
parents: 4198
diff changeset
46 %s COMMAND_START
e96f52432059 [project @ 2002-12-03 19:48:57 by jwe]
jwe
parents: 4198
diff changeset
47 %s MATRIX_START
4240
8627d992beb8 [project @ 2002-12-25 23:07:03 by jwe]
jwe
parents: 4238
diff changeset
48
16224
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
49 %x INPUT_FILE_START
1
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
50
16228
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
51 %x BLOCK_COMMENT_START
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
52 %x LINE_COMMENT_START
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
53
1
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
54 %{
240
a99f28f5e351 [project @ 1993-11-30 20:24:36 by jwe]
jwe
parents: 216
diff changeset
55
1341
e4e4131b1648 [project @ 1995-09-04 00:13:33 by jwe]
jwe
parents: 1315
diff changeset
56 #include <cctype>
e4e4131b1648 [project @ 1995-09-04 00:13:33 by jwe]
jwe
parents: 1315
diff changeset
57 #include <cstring>
e4e4131b1648 [project @ 1995-09-04 00:13:33 by jwe]
jwe
parents: 1315
diff changeset
58
8950
d865363208d6 include <iosfwd> instead of <iostream> in header files
John W. Eaton <jwe@octave.org>
parents: 8920
diff changeset
59 #include <iostream>
8001
ff9e7873f8ea improve handling of command-style names in matrix_or_assign_lhs context
John W. Eaton <jwe@octave.org>
parents: 7898
diff changeset
60 #include <set>
5765
7ba9ad1fec11 [project @ 2006-04-17 05:05:15 by jwe]
jwe
parents: 5615
diff changeset
61 #include <sstream>
1823
5cdd59e7579a [project @ 1996-02-01 12:45:05 by jwe]
jwe
parents: 1755
diff changeset
62 #include <string>
4214
b9317f3973ec [project @ 2002-12-04 04:57:01 by jwe]
jwe
parents: 4208
diff changeset
63 #include <stack>
1823
5cdd59e7579a [project @ 1996-02-01 12:45:05 by jwe]
jwe
parents: 1755
diff changeset
64
4093
5a82e874999b [project @ 2002-10-08 23:48:45 by jwe]
jwe
parents: 4064
diff changeset
65 #include <sys/types.h>
5a82e874999b [project @ 2002-10-08 23:48:45 by jwe]
jwe
parents: 4064
diff changeset
66 #include <unistd.h>
5a82e874999b [project @ 2002-10-08 23:48:45 by jwe]
jwe
parents: 4064
diff changeset
67
2926
66ef74ee5d9f [project @ 1997-05-05 03:20:52 by jwe]
jwe
parents: 2900
diff changeset
68 #include "cmd-edit.h"
4153
6b96ce9f5743 [project @ 2002-11-06 20:38:49 by jwe]
jwe
parents: 4131
diff changeset
69 #include "quit.h"
4910
1242acab4246 [project @ 2004-07-12 17:14:13 by jwe]
jwe
parents: 4867
diff changeset
70 #include "lo-mappers.h"
2926
66ef74ee5d9f [project @ 1997-05-05 03:20:52 by jwe]
jwe
parents: 2900
diff changeset
71
16154
aa5e1e8dce66 * lex.ll: Change y.tab.h to oct-parse.h in comment.
John W. Eaton <jwe@octave.org>
parents: 16150
diff changeset
72 // These would be alphabetical, but oct-parse.h must be included before
aa5e1e8dce66 * lex.ll: Change y.tab.h to oct-parse.h in comment.
John W. Eaton <jwe@octave.org>
parents: 16150
diff changeset
73 // oct-gperf.h and oct-parse.h must be included after token.h and the tree
aa5e1e8dce66 * lex.ll: Change y.tab.h to oct-parse.h in comment.
John W. Eaton <jwe@octave.org>
parents: 16150
diff changeset
74 // class declarations. We can't include oct-parse.h in oct-gperf.h
1497
48a0b289f1be [project @ 1995-09-30 22:04:27 by jwe]
jwe
parents: 1489
diff changeset
75 // because it may not be protected to allow it to be included multiple
48a0b289f1be [project @ 1995-09-30 22:04:27 by jwe]
jwe
parents: 1489
diff changeset
76 // times.
48a0b289f1be [project @ 1995-09-30 22:04:27 by jwe]
jwe
parents: 1489
diff changeset
77
4264
4e2d2516da22 [project @ 2003-01-03 05:30:34 by jwe]
jwe
parents: 4248
diff changeset
78 #include "Cell.h"
3665
0689afb1d001 [project @ 2000-05-11 19:07:56 by jwe]
jwe
parents: 3598
diff changeset
79 #include "comment-list.h"
2181
138b3c98dc85 [project @ 1996-05-13 18:02:31 by jwe]
jwe
parents: 2167
diff changeset
80 #include "defun.h"
1355
94697d007075 [project @ 1995-09-05 20:04:15 by jwe]
jwe
parents: 1351
diff changeset
81 #include "error.h"
4910
1242acab4246 [project @ 2004-07-12 17:14:13 by jwe]
jwe
parents: 4867
diff changeset
82 #include "gripes.h"
1351
8256c4c57419 [project @ 1995-09-05 08:02:08 by jwe]
jwe
parents: 1341
diff changeset
83 #include "input.h"
1355
94697d007075 [project @ 1995-09-05 20:04:15 by jwe]
jwe
parents: 1351
diff changeset
84 #include "lex.h"
2891
1a30f46e1870 [project @ 1997-04-28 01:49:00 by jwe]
jwe
parents: 2877
diff changeset
85 #include "ov.h"
1355
94697d007075 [project @ 1995-09-05 20:04:15 by jwe]
jwe
parents: 1351
diff changeset
86 #include "parse.h"
14329
8d1ae996c122 also save and restore symbol table context in parser (bug #35448)
John W. Eaton <jwe@octave.org>
parents: 14294
diff changeset
87 #include "parse-private.h"
2987
ae20b3f37f5e [project @ 1997-05-16 07:05:15 by jwe]
jwe
parents: 2982
diff changeset
88 #include "pt-all.h"
2891
1a30f46e1870 [project @ 1997-04-28 01:49:00 by jwe]
jwe
parents: 2877
diff changeset
89 #include "symtab.h"
1a30f46e1870 [project @ 1997-04-28 01:49:00 by jwe]
jwe
parents: 2877
diff changeset
90 #include "token.h"
1a30f46e1870 [project @ 1997-04-28 01:49:00 by jwe]
jwe
parents: 2877
diff changeset
91 #include "toplev.h"
1355
94697d007075 [project @ 1995-09-05 20:04:15 by jwe]
jwe
parents: 1351
diff changeset
92 #include "utils.h"
94697d007075 [project @ 1995-09-05 20:04:15 by jwe]
jwe
parents: 1351
diff changeset
93 #include "variables.h"
9794
0d4613a736e9 convert build system to use automake and libtool
John W. Eaton <jwe@octave.org>
parents: 9476
diff changeset
94 #include <oct-parse.h>
2492
06595bc7f2d0 [project @ 1996-11-09 00:13:50 by jwe]
jwe
parents: 2375
diff changeset
95 #include <oct-gperf.h>
1
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
96
10426
4db7beace28e oct-parse.yy, lex.ll: use symbols via gnulib namespace
John W. Eaton <jwe@octave.org>
parents: 10411
diff changeset
97 #if defined (GNULIB_NAMESPACE)
4db7beace28e oct-parse.yy, lex.ll: use symbols via gnulib namespace
John W. Eaton <jwe@octave.org>
parents: 10411
diff changeset
98 // Calls to the following functions appear in the generated output from
4db7beace28e oct-parse.yy, lex.ll: use symbols via gnulib namespace
John W. Eaton <jwe@octave.org>
parents: 10411
diff changeset
99 // flex without the namespace tag. Redefine them so we will use them
4db7beace28e oct-parse.yy, lex.ll: use symbols via gnulib namespace
John W. Eaton <jwe@octave.org>
parents: 10411
diff changeset
100 // via the gnulib namespace.
4db7beace28e oct-parse.yy, lex.ll: use symbols via gnulib namespace
John W. Eaton <jwe@octave.org>
parents: 10411
diff changeset
101 #define fprintf GNULIB_NAMESPACE::fprintf
4db7beace28e oct-parse.yy, lex.ll: use symbols via gnulib namespace
John W. Eaton <jwe@octave.org>
parents: 10411
diff changeset
102 #define fwrite GNULIB_NAMESPACE::fwrite
14161
f4b5a2f899d3 check in change accidentally omitted from previuos changeset
John W. Eaton <jwe@octave.org>
parents: 14138
diff changeset
103 #define isatty GNULIB_NAMESPACE::isatty
10426
4db7beace28e oct-parse.yy, lex.ll: use symbols via gnulib namespace
John W. Eaton <jwe@octave.org>
parents: 10411
diff changeset
104 #define malloc GNULIB_NAMESPACE::malloc
4db7beace28e oct-parse.yy, lex.ll: use symbols via gnulib namespace
John W. Eaton <jwe@octave.org>
parents: 10411
diff changeset
105 #define realloc GNULIB_NAMESPACE::realloc
4db7beace28e oct-parse.yy, lex.ll: use symbols via gnulib namespace
John W. Eaton <jwe@octave.org>
parents: 10411
diff changeset
106 #endif
4db7beace28e oct-parse.yy, lex.ll: use symbols via gnulib namespace
John W. Eaton <jwe@octave.org>
parents: 10411
diff changeset
107
2716
2b5c27299cde [project @ 1997-02-22 08:40:55 by jwe]
jwe
parents: 2702
diff changeset
108 #if ! (defined (FLEX_SCANNER) \
2b5c27299cde [project @ 1997-02-22 08:40:55 by jwe]
jwe
parents: 2702
diff changeset
109 && defined (YY_FLEX_MAJOR_VERSION) && YY_FLEX_MAJOR_VERSION >= 2 \
2b5c27299cde [project @ 1997-02-22 08:40:55 by jwe]
jwe
parents: 2702
diff changeset
110 && defined (YY_FLEX_MINOR_VERSION) && YY_FLEX_MINOR_VERSION >= 5)
2b5c27299cde [project @ 1997-02-22 08:40:55 by jwe]
jwe
parents: 2702
diff changeset
111 #error lex.l requires flex version 2.5.4 or later
2b5c27299cde [project @ 1997-02-22 08:40:55 by jwe]
jwe
parents: 2702
diff changeset
112 #endif
2b5c27299cde [project @ 1997-02-22 08:40:55 by jwe]
jwe
parents: 2702
diff changeset
113
16158
7eb614760ddb rename lexical_feedback to octave_lexer
John W. Eaton <jwe@octave.org>
parents: 16154
diff changeset
114 #define YY_EXTRA_TYPE octave_lexer *
16149
49dfba4fd3c5 use pure parser and reentrant lexer interfaces
John W. Eaton <jwe@octave.org>
parents: 16148
diff changeset
115 #define curr_lexer yyextra
4753
9f0ce1536368 [project @ 2004-02-13 17:55:24 by jwe]
jwe
parents: 4678
diff changeset
116
9f0ce1536368 [project @ 2004-02-13 17:55:24 by jwe]
jwe
parents: 4678
diff changeset
117 // Arrange to get input via readline.
9f0ce1536368 [project @ 2004-02-13 17:55:24 by jwe]
jwe
parents: 4678
diff changeset
118
9f0ce1536368 [project @ 2004-02-13 17:55:24 by jwe]
jwe
parents: 4678
diff changeset
119 #ifdef YY_INPUT
9f0ce1536368 [project @ 2004-02-13 17:55:24 by jwe]
jwe
parents: 4678
diff changeset
120 #undef YY_INPUT
9f0ce1536368 [project @ 2004-02-13 17:55:24 by jwe]
jwe
parents: 4678
diff changeset
121 #endif
9f0ce1536368 [project @ 2004-02-13 17:55:24 by jwe]
jwe
parents: 4678
diff changeset
122 #define YY_INPUT(buf, result, max_size) \
16195
b52d2f9294b6 use class for reading lexer input
John W. Eaton <jwe@octave.org>
parents: 16194
diff changeset
123 result = curr_lexer->read (buf, max_size)
4753
9f0ce1536368 [project @ 2004-02-13 17:55:24 by jwe]
jwe
parents: 4678
diff changeset
124
9f0ce1536368 [project @ 2004-02-13 17:55:24 by jwe]
jwe
parents: 4678
diff changeset
125 // Try to avoid crashing out completely on fatal scanner errors.
9f0ce1536368 [project @ 2004-02-13 17:55:24 by jwe]
jwe
parents: 4678
diff changeset
126
9f0ce1536368 [project @ 2004-02-13 17:55:24 by jwe]
jwe
parents: 4678
diff changeset
127 #ifdef YY_FATAL_ERROR
9f0ce1536368 [project @ 2004-02-13 17:55:24 by jwe]
jwe
parents: 4678
diff changeset
128 #undef YY_FATAL_ERROR
9f0ce1536368 [project @ 2004-02-13 17:55:24 by jwe]
jwe
parents: 4678
diff changeset
129 #endif
9f0ce1536368 [project @ 2004-02-13 17:55:24 by jwe]
jwe
parents: 4678
diff changeset
130 #define YY_FATAL_ERROR(msg) \
16149
49dfba4fd3c5 use pure parser and reentrant lexer interfaces
John W. Eaton <jwe@octave.org>
parents: 16148
diff changeset
131 (yyget_extra (yyscanner))->fatal_error (msg)
4753
9f0ce1536368 [project @ 2004-02-13 17:55:24 by jwe]
jwe
parents: 4678
diff changeset
132
16275
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
133 #define CMD_OR_OP(PATTERN, TOK, COMPAT) \
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
134 \
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
135 do \
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
136 { \
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
137 curr_lexer->lexer_debug (PATTERN); \
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
138 \
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
139 if (curr_lexer->looks_like_command_arg ()) \
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
140 { \
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
141 yyless (0); \
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
142 curr_lexer->push_start_state (COMMAND_START); \
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
143 } \
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
144 else \
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
145 { \
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
146 return curr_lexer->handle_op_internal (TOK, false, COMPAT); \
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
147 } \
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
148 } \
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
149 while (0)
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
150
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
151 #define CMD_OR_UNARY_OP(PATTERN, TOK, COMPAT) \
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
152 \
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
153 do \
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
154 { \
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
155 curr_lexer->lexer_debug (PATTERN); \
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
156 \
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
157 if (curr_lexer->looks_like_command_arg ()) \
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
158 { \
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
159 yyless (0); \
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
160 curr_lexer->push_start_state (COMMAND_START); \
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
161 } \
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
162 else \
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
163 { \
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
164 int tok \
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
165 = (COMPAT \
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
166 ? curr_lexer->handle_unary_op (TOK) \
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
167 : curr_lexer->handle_incompatible_unary_op (TOK)); \
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
168 \
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
169 if (tok < 0) \
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
170 { \
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
171 yyless (0); \
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
172 curr_lexer->xunput (','); \
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
173 } \
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
174 else \
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
175 { \
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
176 return tok; \
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
177 } \
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
178 } \
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
179 } \
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
180 while (0)
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
181
7722
c3bb0b7a4261 lex.l: allow tokens to be displayed when parsed
John W. Eaton <jwe@octave.org>
parents: 7720
diff changeset
182 static bool Vdisplay_tokens = false;
c3bb0b7a4261 lex.l: allow tokens to be displayed when parsed
John W. Eaton <jwe@octave.org>
parents: 7720
diff changeset
183
4910
1242acab4246 [project @ 2004-07-12 17:14:13 by jwe]
jwe
parents: 4867
diff changeset
184 static unsigned int Vtoken_count = 0;
1242acab4246 [project @ 2004-07-12 17:14:13 by jwe]
jwe
parents: 4867
diff changeset
185
8535
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
186 // Internal variable for lexer debugging state.
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
187 static bool lexer_debug_flag = false;
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
188
146
edfa5a96c5f1 [project @ 1993-10-05 04:02:15 by jwe]
jwe
parents: 143
diff changeset
189 // Forward declarations for functions defined at the bottom of this
16114
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
190 // file that are needed inside the lexer actions.
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
191
3523
b80bbb43a1a9 [project @ 2000-02-02 10:25:52 by jwe]
jwe
parents: 3503
diff changeset
192 static std::string strip_trailing_whitespace (char *s);
1
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
193
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
194 %}
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
195
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
196 D [0-9]
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
197 S [ \t]
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
198 NL ((\n)|(\r)|(\r\n))
16261
b45a90cdb0ae 3/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16259
diff changeset
199 CONT ((\.\.\.)|(\\))
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
200 Im [iIjJ]
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
201 CCHAR [#%]
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
202 IDENT ([_$a-zA-Z][_$a-zA-Z0-9]*)
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
203 EXPON ([DdEe][+-]?{D}+)
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
204 NUMBER (({D}+\.?{D}*{EXPON}?)|(\.{D}+{EXPON}?)|(0[xX][0-9a-fA-F]+))
16228
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
205
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
206 ANY_INCLUDING_NL (.|{NL})
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
207
1
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
208 %%
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
209
9474
25ed2d6aacf6 Parse nested functions more accurately.
David Grundberg <individ@acc.umu.se>
parents: 9324
diff changeset
210 %{
25ed2d6aacf6 Parse nested functions more accurately.
David Grundberg <individ@acc.umu.se>
parents: 9324
diff changeset
211 // Make script and function files start with a bogus token. This makes
25ed2d6aacf6 Parse nested functions more accurately.
David Grundberg <individ@acc.umu.se>
parents: 9324
diff changeset
212 // the parser go down a special path.
25ed2d6aacf6 Parse nested functions more accurately.
David Grundberg <individ@acc.umu.se>
parents: 9324
diff changeset
213 %}
25ed2d6aacf6 Parse nested functions more accurately.
David Grundberg <individ@acc.umu.se>
parents: 9324
diff changeset
214
16228
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
215 <INPUT_FILE_START>{ANY_INCLUDING_NL} {
16229
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
216 curr_lexer->lexer_debug ("<INPUT_FILE_START>{ANY_INCLUDING_NL}");
16224
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
217
16125
96a58f197f93 allow xunput to be called without buffer argument
John W. Eaton <jwe@octave.org>
parents: 16124
diff changeset
218 curr_lexer->xunput (yytext[0]);
16203
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
219
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
220 // May be reset later if we see "function" or "classdef" appears
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
221 // as the first token.
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
222 curr_lexer->reading_script_file = true;
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
223
16224
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
224 curr_lexer->pop_start_state ();
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
225
16229
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
226 return curr_lexer->show_token (INPUT_FILE);
4238
a5a68c0afe56 [project @ 2002-12-25 21:04:33 by jwe]
jwe
parents: 4237
diff changeset
227 }
a5a68c0afe56 [project @ 2002-12-25 21:04:33 by jwe]
jwe
parents: 4237
diff changeset
228
968
9351572b7210 [project @ 1994-12-11 21:53:00 by jwe]
jwe
parents: 967
diff changeset
229 %{
8746
5dd06f19e9be handle commands in the lexer
John W. Eaton <jwe@octave.org>
parents: 8745
diff changeset
230 // Help and other command-style functions.
968
9351572b7210 [project @ 1994-12-11 21:53:00 by jwe]
jwe
parents: 967
diff changeset
231 %}
9351572b7210 [project @ 1994-12-11 21:53:00 by jwe]
jwe
parents: 967
diff changeset
232
4208
e96f52432059 [project @ 2002-12-03 19:48:57 by jwe]
jwe
parents: 4198
diff changeset
233 <COMMAND_START>{NL} {
16229
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
234 curr_lexer->lexer_debug ("<COMMAND_START>{NL}");
8535
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
235
16110
7302f8a4df83 use pointer for global lexical_feedback structure
John W. Eaton <jwe@octave.org>
parents: 16106
diff changeset
236 curr_lexer->input_line_number++;
7302f8a4df83 use pointer for global lexical_feedback structure
John W. Eaton <jwe@octave.org>
parents: 16106
diff changeset
237 curr_lexer->current_input_column = 1;
7302f8a4df83 use pointer for global lexical_feedback structure
John W. Eaton <jwe@octave.org>
parents: 16106
diff changeset
238
7302f8a4df83 use pointer for global lexical_feedback structure
John W. Eaton <jwe@octave.org>
parents: 16106
diff changeset
239 curr_lexer->looking_for_object_index = false;
7302f8a4df83 use pointer for global lexical_feedback structure
John W. Eaton <jwe@octave.org>
parents: 16106
diff changeset
240 curr_lexer->at_beginning_of_statement = true;
8745
6dc61981d18b better handling of object indexing in lexer
John W. Eaton <jwe@octave.org>
parents: 8701
diff changeset
241
16224
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
242 curr_lexer->pop_start_state ();
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
243
16229
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
244 return curr_lexer->count_token ('\n');
967
b228d6cd59a1 [project @ 1994-12-10 00:15:41 by jwe]
jwe
parents: 947
diff changeset
245 }
1
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
246
4208
e96f52432059 [project @ 2002-12-03 19:48:57 by jwe]
jwe
parents: 4198
diff changeset
247 <COMMAND_START>[\;\,] {
16229
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
248 curr_lexer->lexer_debug ("<COMMAND_START>[\\;\\,]");
8535
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
249
16110
7302f8a4df83 use pointer for global lexical_feedback structure
John W. Eaton <jwe@octave.org>
parents: 16106
diff changeset
250 curr_lexer->looking_for_object_index = false;
7302f8a4df83 use pointer for global lexical_feedback structure
John W. Eaton <jwe@octave.org>
parents: 16106
diff changeset
251 curr_lexer->at_beginning_of_statement = true;
5102
b04b30d30c66 [project @ 2004-12-28 01:59:05 by jwe]
jwe
parents: 5088
diff changeset
252
16224
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
253 curr_lexer->pop_start_state ();
5102
b04b30d30c66 [project @ 2004-12-28 01:59:05 by jwe]
jwe
parents: 5088
diff changeset
254
b04b30d30c66 [project @ 2004-12-28 01:59:05 by jwe]
jwe
parents: 5088
diff changeset
255 if (strcmp (yytext, ",") == 0)
16229
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
256 return curr_lexer->handle_token (',');
967
b228d6cd59a1 [project @ 1994-12-10 00:15:41 by jwe]
jwe
parents: 947
diff changeset
257 else
16229
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
258 return curr_lexer->handle_token (';');
967
b228d6cd59a1 [project @ 1994-12-10 00:15:41 by jwe]
jwe
parents: 947
diff changeset
259 }
1
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
260
4208
e96f52432059 [project @ 2002-12-03 19:48:57 by jwe]
jwe
parents: 4198
diff changeset
261 <COMMAND_START>[\"\'] {
16229
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
262 curr_lexer->lexer_debug ("<COMMAND_START>[\\\"\\']");
8535
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
263
16110
7302f8a4df83 use pointer for global lexical_feedback structure
John W. Eaton <jwe@octave.org>
parents: 16106
diff changeset
264 curr_lexer->at_beginning_of_statement = false;
7302f8a4df83 use pointer for global lexical_feedback structure
John W. Eaton <jwe@octave.org>
parents: 16106
diff changeset
265
7302f8a4df83 use pointer for global lexical_feedback structure
John W. Eaton <jwe@octave.org>
parents: 16106
diff changeset
266 curr_lexer->current_input_column++;
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
267 int tok = curr_lexer->handle_string (yytext[0]);
8745
6dc61981d18b better handling of object indexing in lexer
John W. Eaton <jwe@octave.org>
parents: 8701
diff changeset
268
16257
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
269 return curr_lexer->count_token_internal (tok);
975
e7165acbf96f [project @ 1994-12-12 20:30:19 by jwe]
jwe
parents: 973
diff changeset
270 }
e7165acbf96f [project @ 1994-12-12 20:30:19 by jwe]
jwe
parents: 973
diff changeset
271
4923
b11223097ad4 [project @ 2004-08-02 16:33:11 by jwe]
jwe
parents: 4910
diff changeset
272 <COMMAND_START>[^#% \t\r\n\;\,\"\'][^ \t\r\n\;\,]*{S}* {
16229
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
273 curr_lexer->lexer_debug ("<COMMAND_START>[^#% \\t\\r\\n\\;\\,\\\"\\'][^ \\t\\r\\n\\;\\,]*{S}*");
8535
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
274
3523
b80bbb43a1a9 [project @ 2000-02-02 10:25:52 by jwe]
jwe
parents: 3503
diff changeset
275 std::string tok = strip_trailing_whitespace (yytext);
8745
6dc61981d18b better handling of object indexing in lexer
John W. Eaton <jwe@octave.org>
parents: 8701
diff changeset
276
16110
7302f8a4df83 use pointer for global lexical_feedback structure
John W. Eaton <jwe@octave.org>
parents: 16106
diff changeset
277 curr_lexer->looking_for_object_index = false;
7302f8a4df83 use pointer for global lexical_feedback structure
John W. Eaton <jwe@octave.org>
parents: 16106
diff changeset
278 curr_lexer->at_beginning_of_statement = false;
8745
6dc61981d18b better handling of object indexing in lexer
John W. Eaton <jwe@octave.org>
parents: 8701
diff changeset
279
16231
2b15ae55c721 put all tokens in the token cache
John W. Eaton <jwe@octave.org>
parents: 16230
diff changeset
280 return curr_lexer->handle_token (tok, SQ_STRING);
967
b228d6cd59a1 [project @ 1994-12-10 00:15:41 by jwe]
jwe
parents: 947
diff changeset
281 }
1
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
282
16257
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
283 <MATRIX_START>{S}* {
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
284 curr_lexer->lexer_debug ("<MATRIX_START>{S}*");
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
285
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
286 curr_lexer->mark_previous_token_trailing_space ();
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
287 }
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
288
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
289 <MATRIX_START>{NL} {
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
290 curr_lexer->lexer_debug ("<MATRIX_START>{NL}");
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
291
16267
15f55df088e7 6/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16265
diff changeset
292 if (curr_lexer->nesting_level.is_paren ())
15f55df088e7 6/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16265
diff changeset
293 curr_lexer->gripe_matlab_incompatible ("bare newline inside parentheses");
15f55df088e7 6/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16265
diff changeset
294 else
15f55df088e7 6/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16265
diff changeset
295 {
15f55df088e7 6/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16265
diff changeset
296 int tok = curr_lexer->previous_token_value ();
15f55df088e7 6/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16265
diff changeset
297
15f55df088e7 6/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16265
diff changeset
298 if (! (tok == ';' || tok == '[' || tok == '{'))
15f55df088e7 6/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16265
diff changeset
299 curr_lexer->xunput (';');
15f55df088e7 6/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16265
diff changeset
300 }
16257
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
301 }
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
302
968
9351572b7210 [project @ 1994-12-11 21:53:00 by jwe]
jwe
parents: 967
diff changeset
303 %{
1
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
304 // For this and the next two rules, we're looking at ']', and we
15466
d174210ce1ec use ' instead of ` in error messages, warnings and most comments
John W. Eaton <jwe@octave.org>
parents: 14161
diff changeset
305 // need to know if the next token is '=' or '=='.
1
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
306 //
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
307 // It would have been so much easier if the delimiters were simply
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
308 // different for the expression on the left hand side of the equals
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
309 // operator.
971
fe71abb43457 [project @ 1994-12-12 15:10:22 by jwe]
jwe
parents: 970
diff changeset
310 //
fe71abb43457 [project @ 1994-12-12 15:10:22 by jwe]
jwe
parents: 970
diff changeset
311 // It's also a pain in the ass to decide whether to insert a comma
fe71abb43457 [project @ 1994-12-12 15:10:22 by jwe]
jwe
parents: 970
diff changeset
312 // after seeing a ']' character...
7723
74f5e0c7de9e first pass at handling block comments
John W. Eaton <jwe@octave.org>
parents: 7722
diff changeset
313
74f5e0c7de9e first pass at handling block comments
John W. Eaton <jwe@octave.org>
parents: 7722
diff changeset
314 // FIXME -- we need to handle block comments here.
968
9351572b7210 [project @ 1994-12-11 21:53:00 by jwe]
jwe
parents: 967
diff changeset
315 %}
9351572b7210 [project @ 1994-12-11 21:53:00 by jwe]
jwe
parents: 967
diff changeset
316
16257
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
317 <MATRIX_START>\] {
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
318 curr_lexer->lexer_debug ("<MATRIX_START>\\]");
8535
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
319
16110
7302f8a4df83 use pointer for global lexical_feedback structure
John W. Eaton <jwe@octave.org>
parents: 16106
diff changeset
320 curr_lexer->looking_at_object_index.pop_front ();
7302f8a4df83 use pointer for global lexical_feedback structure
John W. Eaton <jwe@octave.org>
parents: 16106
diff changeset
321
7302f8a4df83 use pointer for global lexical_feedback structure
John W. Eaton <jwe@octave.org>
parents: 16106
diff changeset
322 curr_lexer->looking_for_object_index = true;
7302f8a4df83 use pointer for global lexical_feedback structure
John W. Eaton <jwe@octave.org>
parents: 16106
diff changeset
323 curr_lexer->at_beginning_of_statement = false;
8745
6dc61981d18b better handling of object indexing in lexer
John W. Eaton <jwe@octave.org>
parents: 8701
diff changeset
324
16259
0b5ab09dfce4 2/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16257
diff changeset
325 int tok_to_return = curr_lexer->handle_close_bracket (']');
8745
6dc61981d18b better handling of object indexing in lexer
John W. Eaton <jwe@octave.org>
parents: 8701
diff changeset
326
16257
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
327 return curr_lexer->count_token (']');
4608
22ca4cc02525 [project @ 2003-11-14 03:55:04 by jwe]
jwe
parents: 4476
diff changeset
328 }
22ca4cc02525 [project @ 2003-11-14 03:55:04 by jwe]
jwe
parents: 4476
diff changeset
329
7723
74f5e0c7de9e first pass at handling block comments
John W. Eaton <jwe@octave.org>
parents: 7722
diff changeset
330 %{
74f5e0c7de9e first pass at handling block comments
John W. Eaton <jwe@octave.org>
parents: 7722
diff changeset
331 // FIXME -- we need to handle block comments here.
74f5e0c7de9e first pass at handling block comments
John W. Eaton <jwe@octave.org>
parents: 7722
diff changeset
332 %}
74f5e0c7de9e first pass at handling block comments
John W. Eaton <jwe@octave.org>
parents: 7722
diff changeset
333
16257
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
334 <MATRIX_START>\} {
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
335 curr_lexer->lexer_debug ("<MATRIX_START>\\}*");
8535
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
336
16110
7302f8a4df83 use pointer for global lexical_feedback structure
John W. Eaton <jwe@octave.org>
parents: 16106
diff changeset
337 curr_lexer->looking_at_object_index.pop_front ();
7302f8a4df83 use pointer for global lexical_feedback structure
John W. Eaton <jwe@octave.org>
parents: 16106
diff changeset
338
7302f8a4df83 use pointer for global lexical_feedback structure
John W. Eaton <jwe@octave.org>
parents: 16106
diff changeset
339 curr_lexer->looking_for_object_index = true;
7302f8a4df83 use pointer for global lexical_feedback structure
John W. Eaton <jwe@octave.org>
parents: 16106
diff changeset
340 curr_lexer->at_beginning_of_statement = false;
8745
6dc61981d18b better handling of object indexing in lexer
John W. Eaton <jwe@octave.org>
parents: 8701
diff changeset
341
16259
0b5ab09dfce4 2/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16257
diff changeset
342 int tok_to_return = curr_lexer->handle_close_bracket ('}');
8745
6dc61981d18b better handling of object indexing in lexer
John W. Eaton <jwe@octave.org>
parents: 8701
diff changeset
343
16257
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
344 return curr_lexer->count_token ('}');
967
b228d6cd59a1 [project @ 1994-12-10 00:15:41 by jwe]
jwe
parents: 947
diff changeset
345 }
1
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
346
16257
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
347 \[ {
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
348 curr_lexer->lexer_debug ("\\[");
8535
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
349
16275
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
350 bool unput_comma = false;
16263
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
351
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
352 if (curr_lexer->whitespace_is_significant ()
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
353 && curr_lexer->space_follows_previous_token ())
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
354 {
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
355 int tok = curr_lexer->previous_token_value ();
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
356
16267
15f55df088e7 6/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16265
diff changeset
357 if (! (tok == '[' || tok == '{'
16263
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
358 || curr_lexer->previous_token_is_binop ()))
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
359 unput_comma = true;
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
360 }
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
361
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
362 if (unput_comma)
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
363 {
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
364 yyless (0);
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
365 curr_lexer->xunput (',');
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
366 }
5615
d01f07aeaec5 [project @ 2006-02-10 21:30:41 by jwe]
jwe
parents: 5570
diff changeset
367 else
16263
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
368 {
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
369 curr_lexer->nesting_level.bracket ();
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
370
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
371 curr_lexer->looking_at_object_index.push_front (false);
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
372
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
373 curr_lexer->current_input_column += yyleng;
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
374 curr_lexer->looking_for_object_index = false;
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
375 curr_lexer->at_beginning_of_statement = false;
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
376
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
377 if (curr_lexer->defining_func
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
378 && ! curr_lexer->parsed_function_name.top ())
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
379 curr_lexer->looking_at_return_list = true;
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
380 else
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
381 curr_lexer->looking_at_matrix_or_assign_lhs = true;
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
382
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
383 curr_lexer->decrement_promptflag ();
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
384
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
385 curr_lexer->bracketflag++;
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
386
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
387 curr_lexer->push_start_state (MATRIX_START);
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
388
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
389 return curr_lexer->count_token ('[');
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
390 }
967
b228d6cd59a1 [project @ 1994-12-10 00:15:41 by jwe]
jwe
parents: 947
diff changeset
391 }
1
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
392
968
9351572b7210 [project @ 1994-12-11 21:53:00 by jwe]
jwe
parents: 967
diff changeset
393 \] {
16229
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
394 curr_lexer->lexer_debug ("\\]");
8535
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
395
16110
7302f8a4df83 use pointer for global lexical_feedback structure
John W. Eaton <jwe@octave.org>
parents: 16106
diff changeset
396 curr_lexer->nesting_level.remove ();
7302f8a4df83 use pointer for global lexical_feedback structure
John W. Eaton <jwe@octave.org>
parents: 16106
diff changeset
397
7302f8a4df83 use pointer for global lexical_feedback structure
John W. Eaton <jwe@octave.org>
parents: 16106
diff changeset
398 curr_lexer->looking_at_object_index.pop_front ();
7302f8a4df83 use pointer for global lexical_feedback structure
John W. Eaton <jwe@octave.org>
parents: 16106
diff changeset
399
7302f8a4df83 use pointer for global lexical_feedback structure
John W. Eaton <jwe@octave.org>
parents: 16106
diff changeset
400 curr_lexer->looking_for_object_index = true;
7302f8a4df83 use pointer for global lexical_feedback structure
John W. Eaton <jwe@octave.org>
parents: 16106
diff changeset
401 curr_lexer->at_beginning_of_statement = false;
8745
6dc61981d18b better handling of object indexing in lexer
John W. Eaton <jwe@octave.org>
parents: 8701
diff changeset
402
16229
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
403 return curr_lexer->handle_token (']');
968
9351572b7210 [project @ 1994-12-11 21:53:00 by jwe]
jwe
parents: 967
diff changeset
404 }
9351572b7210 [project @ 1994-12-11 21:53:00 by jwe]
jwe
parents: 967
diff changeset
405
9351572b7210 [project @ 1994-12-11 21:53:00 by jwe]
jwe
parents: 967
diff changeset
406 %{
16228
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
407 // Gobble comments.
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
408 %}
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
409
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
410 %{
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
411 // Start of a block comment. If the comment marker appears immediately
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
412 // after a block of full-line comments, finish the full line comment
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
413 // block.
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
414 %}
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
415
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
416 ^{S}*{CCHAR}\{{S}*{NL} {
16229
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
417 curr_lexer->lexer_debug ("^{S}*{CCHAR}\{{S}*{NL}");
16228
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
418
16253
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
419 yyless (0);
16228
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
420
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
421 if (curr_lexer->start_state () == LINE_COMMENT_START)
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
422 {
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
423 if (! curr_lexer->comment_text.empty ())
16253
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
424 curr_lexer->finish_comment (octave_comment_elt::full_line);
16228
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
425
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
426 curr_lexer->pop_start_state ();
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
427 }
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
428
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
429 curr_lexer->push_start_state (BLOCK_COMMENT_START);
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
430
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
431 }
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
432
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
433 <BLOCK_COMMENT_START>^{S}*{CCHAR}\{{S}*{NL} {
16229
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
434 curr_lexer->lexer_debug ("<BLOCK_COMMENT_START>^{S}*{CCHAR}\{{S}*{NL}");
16228
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
435
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
436 curr_lexer->input_line_number++;
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
437 curr_lexer->current_input_column = 1;
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
438
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
439 if (curr_lexer->block_comment_nesting_level)
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
440 curr_lexer->comment_text = "\n";
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
441
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
442 curr_lexer->block_comment_nesting_level++;
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
443 }
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
444
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
445 %{
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
446 // End of a block comment. If this block comment is nested inside
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
447 // another, wait for the outermost block comment block to be closed
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
448 // before storing the comment.
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
449 %}
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
450
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
451 <BLOCK_COMMENT_START>^{S}*{CCHAR}\}{S}*{NL} {
16229
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
452 curr_lexer->lexer_debug ("<BLOCK_COMMENT_START>^{S}*{CCHAR}\\}{S}*{NL}");
16228
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
453
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
454 curr_lexer->input_line_number++;
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
455 curr_lexer->current_input_column = 1;
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
456
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
457 if (curr_lexer->block_comment_nesting_level > 1)
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
458 curr_lexer->comment_text = "\n";
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
459 else
16253
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
460 curr_lexer->finish_comment (octave_comment_elt::block);
16228
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
461
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
462 curr_lexer->block_comment_nesting_level--;
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
463 curr_lexer->pop_start_state ();
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
464 }
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
465
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
466 %{
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
467 // Body of a block comment.
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
468 %}
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
469
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
470 <BLOCK_COMMENT_START>.*{NL} {
16229
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
471 curr_lexer->lexer_debug ("<BLOCK_COMMENT_START>.*{NL}");
16228
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
472
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
473 curr_lexer->input_line_number++;
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
474 curr_lexer->current_input_column = 1;
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
475 curr_lexer->comment_text += yytext;
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
476 }
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
477
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
478 %{
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
479 // Full-line or end-of-line comment.
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
480 %}
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
481
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
482 {S}*{CCHAR}.*{NL} {
16229
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
483 curr_lexer->lexer_debug ("{S}*{CCHAR}.*{NL}");
16228
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
484
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
485 curr_lexer->push_start_state (LINE_COMMENT_START);
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
486 yyless (0);
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
487 }
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
488
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
489 <LINE_COMMENT_START>{S}*{CCHAR}.*{NL} {
16229
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
490 curr_lexer->lexer_debug ("<LINE_COMMENT_START>{S}*{CCHAR}.*{NL}");
16228
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
491
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
492 bool full_line_comment = curr_lexer->current_input_column == 1;
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
493 curr_lexer->input_line_number++;
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
494 curr_lexer->current_input_column = 1;
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
495
16255
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
496 bool have_space = false;
16228
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
497 size_t len = yyleng;
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
498 size_t i = 0;
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
499 while (i < len)
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
500 {
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
501 char c = yytext[i];
16255
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
502 if (c == ' ' || c == '\t')
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
503 {
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
504 have_space = true;
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
505 i++;
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
506 }
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
507 else
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
508 break;
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
509 }
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
510
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
511 while (i < len)
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
512 {
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
513 char c = yytext[i];
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
514 if (c == '#' || c == '%')
16228
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
515 i++;
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
516 else
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
517 break;
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
518 }
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
519
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
520 curr_lexer->comment_text += &yytext[i];
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
521
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
522 if (! full_line_comment)
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
523 {
16255
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
524 if (have_space)
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
525 curr_lexer->mark_previous_token_trailing_space ();
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
526
16253
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
527 curr_lexer->finish_comment (octave_comment_elt::end_of_line);
16228
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
528
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
529 curr_lexer->pop_start_state ();
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
530 }
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
531 }
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
532
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
533 %{
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
534 // End of a block of full-line comments.
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
535 %}
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
536
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
537 <LINE_COMMENT_START>{ANY_INCLUDING_NL} {
16229
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
538 curr_lexer->lexer_debug ("<LINE_COMMENT_START>{ANY_INCLUDING_NL}");
16228
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
539
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
540 curr_lexer->xunput (yytext[0]);
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
541
16253
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
542 curr_lexer->finish_comment (octave_comment_elt::full_line);
16228
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
543
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
544 curr_lexer->pop_start_state ();
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
545 }
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
546
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
547 %{
968
9351572b7210 [project @ 1994-12-11 21:53:00 by jwe]
jwe
parents: 967
diff changeset
548 // Imaginary numbers.
9351572b7210 [project @ 1994-12-11 21:53:00 by jwe]
jwe
parents: 967
diff changeset
549 %}
9351572b7210 [project @ 1994-12-11 21:53:00 by jwe]
jwe
parents: 967
diff changeset
550
9351572b7210 [project @ 1994-12-11 21:53:00 by jwe]
jwe
parents: 967
diff changeset
551 {NUMBER}{Im} {
16229
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
552 curr_lexer->lexer_debug ("{NUMBER}{Im}");
8535
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
553
16263
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
554 int tok = curr_lexer->previous_token_value ();
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
555
16257
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
556 if (curr_lexer->whitespace_is_significant ()
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
557 && curr_lexer->space_follows_previous_token ()
16263
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
558 && ! (tok == '[' || tok == '{'
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
559 || curr_lexer->previous_token_is_binop ()))
16257
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
560 {
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
561 yyless (0);
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
562 unput (',');
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
563 }
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
564 else
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
565 {
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
566 curr_lexer->handle_number ();
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
567 return curr_lexer->count_token_internal (IMAG_NUM);
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
568 }
968
9351572b7210 [project @ 1994-12-11 21:53:00 by jwe]
jwe
parents: 967
diff changeset
569 }
9351572b7210 [project @ 1994-12-11 21:53:00 by jwe]
jwe
parents: 967
diff changeset
570
9351572b7210 [project @ 1994-12-11 21:53:00 by jwe]
jwe
parents: 967
diff changeset
571 %{
15466
d174210ce1ec use ' instead of ` in error messages, warnings and most comments
John W. Eaton <jwe@octave.org>
parents: 14161
diff changeset
572 // Real numbers. Don't grab the '.' part of a dot operator as part of
968
9351572b7210 [project @ 1994-12-11 21:53:00 by jwe]
jwe
parents: 967
diff changeset
573 // the constant.
9351572b7210 [project @ 1994-12-11 21:53:00 by jwe]
jwe
parents: 967
diff changeset
574 %}
9351572b7210 [project @ 1994-12-11 21:53:00 by jwe]
jwe
parents: 967
diff changeset
575
8535
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
576 {D}+/\.[\*/\\^\'] |
968
9351572b7210 [project @ 1994-12-11 21:53:00 by jwe]
jwe
parents: 967
diff changeset
577 {NUMBER} {
16229
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
578 curr_lexer->lexer_debug ("{D}+/\\.[\\*/\\^\\']|{NUMBER}");
16257
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
579
16275
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
580 int tok = curr_lexer->previous_token_value ();
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
581
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
582 if (curr_lexer->whitespace_is_significant ()
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
583 && curr_lexer->space_follows_previous_token ()
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
584 && ! (tok == '[' || tok == '{'
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
585 || curr_lexer->previous_token_is_binop ()))
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
586 {
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
587 yyless (0);
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
588 unput (',');
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
589 }
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
590 else
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
591 {
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
592 curr_lexer->handle_number ();
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
593 return curr_lexer->count_token_internal (NUM);
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
594 }
968
9351572b7210 [project @ 1994-12-11 21:53:00 by jwe]
jwe
parents: 967
diff changeset
595 }
9351572b7210 [project @ 1994-12-11 21:53:00 by jwe]
jwe
parents: 967
diff changeset
596
9351572b7210 [project @ 1994-12-11 21:53:00 by jwe]
jwe
parents: 967
diff changeset
597 %{
9351572b7210 [project @ 1994-12-11 21:53:00 by jwe]
jwe
parents: 967
diff changeset
598 // Eat whitespace. Whitespace inside matrix constants is handled by
4208
e96f52432059 [project @ 2002-12-03 19:48:57 by jwe]
jwe
parents: 4198
diff changeset
599 // the <MATRIX_START> start state code above.
968
9351572b7210 [project @ 1994-12-11 21:53:00 by jwe]
jwe
parents: 967
diff changeset
600 %}
9351572b7210 [project @ 1994-12-11 21:53:00 by jwe]
jwe
parents: 967
diff changeset
601
967
b228d6cd59a1 [project @ 1994-12-10 00:15:41 by jwe]
jwe
parents: 947
diff changeset
602 {S}* {
16110
7302f8a4df83 use pointer for global lexical_feedback structure
John W. Eaton <jwe@octave.org>
parents: 16106
diff changeset
603 curr_lexer->current_input_column += yyleng;
16255
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
604
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
605 curr_lexer->mark_previous_token_trailing_space ();
967
b228d6cd59a1 [project @ 1994-12-10 00:15:41 by jwe]
jwe
parents: 947
diff changeset
606 }
b228d6cd59a1 [project @ 1994-12-10 00:15:41 by jwe]
jwe
parents: 947
diff changeset
607
968
9351572b7210 [project @ 1994-12-11 21:53:00 by jwe]
jwe
parents: 967
diff changeset
608 %{
9351572b7210 [project @ 1994-12-11 21:53:00 by jwe]
jwe
parents: 967
diff changeset
609 // Continuation lines. Allow comments after continuations.
9351572b7210 [project @ 1994-12-11 21:53:00 by jwe]
jwe
parents: 967
diff changeset
610 %}
9351572b7210 [project @ 1994-12-11 21:53:00 by jwe]
jwe
parents: 967
diff changeset
611
967
b228d6cd59a1 [project @ 1994-12-10 00:15:41 by jwe]
jwe
parents: 947
diff changeset
612 {CONT}{S}*{NL} |
16253
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
613 {CONT}{S}*{CCHAR}.*{NL} {
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
614 curr_lexer->lexer_debug ("{CONT}{S}*{NL}|{CONT}{S}*{CCHAR}.*{NL}");
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
615
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
616 curr_lexer->handle_continuation ();
967
b228d6cd59a1 [project @ 1994-12-10 00:15:41 by jwe]
jwe
parents: 947
diff changeset
617 }
1
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
618
968
9351572b7210 [project @ 1994-12-11 21:53:00 by jwe]
jwe
parents: 967
diff changeset
619 %{
9351572b7210 [project @ 1994-12-11 21:53:00 by jwe]
jwe
parents: 967
diff changeset
620 // End of file.
9351572b7210 [project @ 1994-12-11 21:53:00 by jwe]
jwe
parents: 967
diff changeset
621 %}
9351572b7210 [project @ 1994-12-11 21:53:00 by jwe]
jwe
parents: 967
diff changeset
622
967
b228d6cd59a1 [project @ 1994-12-10 00:15:41 by jwe]
jwe
parents: 947
diff changeset
623 <<EOF>> {
16164
c5bfdc4c0963 move end_of_input flag from octve_parser class to octave_lexer class
John W. Eaton <jwe@octave.org>
parents: 16158
diff changeset
624 return curr_lexer->handle_end_of_input ();
967
b228d6cd59a1 [project @ 1994-12-10 00:15:41 by jwe]
jwe
parents: 947
diff changeset
625 }
1
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
626
968
9351572b7210 [project @ 1994-12-11 21:53:00 by jwe]
jwe
parents: 967
diff changeset
627 %{
16259
0b5ab09dfce4 2/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16257
diff changeset
628 // Identifiers.
968
9351572b7210 [project @ 1994-12-11 21:53:00 by jwe]
jwe
parents: 967
diff changeset
629 %}
9351572b7210 [project @ 1994-12-11 21:53:00 by jwe]
jwe
parents: 967
diff changeset
630
16257
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
631 {IDENT} {
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
632 curr_lexer->lexer_debug ("{IDENT}");
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
633
16259
0b5ab09dfce4 2/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16257
diff changeset
634 int tok = curr_lexer->previous_token_value ();
0b5ab09dfce4 2/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16257
diff changeset
635
16257
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
636 if (curr_lexer->whitespace_is_significant ()
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
637 && curr_lexer->space_follows_previous_token ()
16259
0b5ab09dfce4 2/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16257
diff changeset
638 && ! (tok == '[' || tok == '{'
0b5ab09dfce4 2/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16257
diff changeset
639 || curr_lexer->previous_token_is_binop ()))
16257
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
640 {
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
641 yyless (0);
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
642 unput (',');
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
643 }
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
644 else
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
645 {
16263
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
646 if (! curr_lexer->looking_at_decl_list
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
647 && curr_lexer->previous_token_may_be_command ())
16257
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
648 {
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
649 yyless (0);
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
650 curr_lexer->push_start_state (COMMAND_START);
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
651 }
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
652 else
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
653 {
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
654 int id_tok = curr_lexer->handle_identifier ();
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
655
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
656 if (id_tok >= 0)
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
657 return curr_lexer->count_token_internal (id_tok);
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
658 }
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
659 }
967
b228d6cd59a1 [project @ 1994-12-10 00:15:41 by jwe]
jwe
parents: 947
diff changeset
660 }
1
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
661
968
9351572b7210 [project @ 1994-12-11 21:53:00 by jwe]
jwe
parents: 967
diff changeset
662 %{
11586
12df7854fa7c strip trailing whitespace from source files
John W. Eaton <jwe@octave.org>
parents: 11523
diff changeset
663 // Superclass method identifiers.
9476
d9b25c5b8ee5 handle classdef syntax in lexer and parser
Ryan Rusaw
parents: 9474
diff changeset
664 %}
d9b25c5b8ee5 handle classdef syntax in lexer and parser
Ryan Rusaw
parents: 9474
diff changeset
665
16261
b45a90cdb0ae 3/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16259
diff changeset
666 {IDENT}@{IDENT} |
b45a90cdb0ae 3/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16259
diff changeset
667 {IDENT}@{IDENT}.{IDENT} {
b45a90cdb0ae 3/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16259
diff changeset
668 curr_lexer->lexer_debug ("{IDENT}@{IDENT}|{IDENT}@{IDENT}.{IDENT}");
9476
d9b25c5b8ee5 handle classdef syntax in lexer and parser
Ryan Rusaw
parents: 9474
diff changeset
669
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
670 int id_tok = curr_lexer->handle_superclass_identifier ();
9476
d9b25c5b8ee5 handle classdef syntax in lexer and parser
Ryan Rusaw
parents: 9474
diff changeset
671
d9b25c5b8ee5 handle classdef syntax in lexer and parser
Ryan Rusaw
parents: 9474
diff changeset
672 if (id_tok >= 0)
d9b25c5b8ee5 handle classdef syntax in lexer and parser
Ryan Rusaw
parents: 9474
diff changeset
673 {
16110
7302f8a4df83 use pointer for global lexical_feedback structure
John W. Eaton <jwe@octave.org>
parents: 16106
diff changeset
674 curr_lexer->looking_for_object_index = true;
9476
d9b25c5b8ee5 handle classdef syntax in lexer and parser
Ryan Rusaw
parents: 9474
diff changeset
675
16257
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
676 return curr_lexer->count_token_internal (SUPERCLASSREF);
9476
d9b25c5b8ee5 handle classdef syntax in lexer and parser
Ryan Rusaw
parents: 9474
diff changeset
677 }
d9b25c5b8ee5 handle classdef syntax in lexer and parser
Ryan Rusaw
parents: 9474
diff changeset
678 }
11586
12df7854fa7c strip trailing whitespace from source files
John W. Eaton <jwe@octave.org>
parents: 11523
diff changeset
679
9476
d9b25c5b8ee5 handle classdef syntax in lexer and parser
Ryan Rusaw
parents: 9474
diff changeset
680 %{
d9b25c5b8ee5 handle classdef syntax in lexer and parser
Ryan Rusaw
parents: 9474
diff changeset
681 // Metaclass query
11586
12df7854fa7c strip trailing whitespace from source files
John W. Eaton <jwe@octave.org>
parents: 11523
diff changeset
682 %}
12df7854fa7c strip trailing whitespace from source files
John W. Eaton <jwe@octave.org>
parents: 11523
diff changeset
683
16261
b45a90cdb0ae 3/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16259
diff changeset
684 \?{IDENT} |
b45a90cdb0ae 3/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16259
diff changeset
685 \?{IDENT}\.{IDENT} {
b45a90cdb0ae 3/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16259
diff changeset
686 curr_lexer->lexer_debug ("\\?{IDENT}|\\?{IDENT}\\.{IDENT}");
11586
12df7854fa7c strip trailing whitespace from source files
John W. Eaton <jwe@octave.org>
parents: 11523
diff changeset
687
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
688 int id_tok = curr_lexer->handle_meta_identifier ();
9476
d9b25c5b8ee5 handle classdef syntax in lexer and parser
Ryan Rusaw
parents: 9474
diff changeset
689
d9b25c5b8ee5 handle classdef syntax in lexer and parser
Ryan Rusaw
parents: 9474
diff changeset
690 if (id_tok >= 0)
d9b25c5b8ee5 handle classdef syntax in lexer and parser
Ryan Rusaw
parents: 9474
diff changeset
691 {
16110
7302f8a4df83 use pointer for global lexical_feedback structure
John W. Eaton <jwe@octave.org>
parents: 16106
diff changeset
692 curr_lexer->looking_for_object_index = true;
9476
d9b25c5b8ee5 handle classdef syntax in lexer and parser
Ryan Rusaw
parents: 9474
diff changeset
693
16257
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
694 return curr_lexer->count_token_internal (METAQUERY);
9476
d9b25c5b8ee5 handle classdef syntax in lexer and parser
Ryan Rusaw
parents: 9474
diff changeset
695 }
11586
12df7854fa7c strip trailing whitespace from source files
John W. Eaton <jwe@octave.org>
parents: 11523
diff changeset
696 }
9476
d9b25c5b8ee5 handle classdef syntax in lexer and parser
Ryan Rusaw
parents: 9474
diff changeset
697
4930
bdb307dc8613 [project @ 2004-08-05 04:55:26 by jwe]
jwe
parents: 4923
diff changeset
698 "@" {
16229
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
699 curr_lexer->lexer_debug ("@");
8535
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
700
16110
7302f8a4df83 use pointer for global lexical_feedback structure
John W. Eaton <jwe@octave.org>
parents: 16106
diff changeset
701 curr_lexer->current_input_column++;
7302f8a4df83 use pointer for global lexical_feedback structure
John W. Eaton <jwe@octave.org>
parents: 16106
diff changeset
702
7302f8a4df83 use pointer for global lexical_feedback structure
John W. Eaton <jwe@octave.org>
parents: 16106
diff changeset
703 curr_lexer->looking_at_function_handle++;
7302f8a4df83 use pointer for global lexical_feedback structure
John W. Eaton <jwe@octave.org>
parents: 16106
diff changeset
704 curr_lexer->looking_for_object_index = false;
7302f8a4df83 use pointer for global lexical_feedback structure
John W. Eaton <jwe@octave.org>
parents: 16106
diff changeset
705 curr_lexer->at_beginning_of_statement = false;
8745
6dc61981d18b better handling of object indexing in lexer
John W. Eaton <jwe@octave.org>
parents: 8701
diff changeset
706
16229
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
707 return curr_lexer->count_token ('@');
4342
813effe14ee1 [project @ 2003-02-20 08:35:55 by jwe]
jwe
parents: 4323
diff changeset
708 }
813effe14ee1 [project @ 2003-02-20 08:35:55 by jwe]
jwe
parents: 4323
diff changeset
709
813effe14ee1 [project @ 2003-02-20 08:35:55 by jwe]
jwe
parents: 4323
diff changeset
710 %{
968
9351572b7210 [project @ 1994-12-11 21:53:00 by jwe]
jwe
parents: 967
diff changeset
711 // A new line character. New line characters inside matrix constants
4208
e96f52432059 [project @ 2002-12-03 19:48:57 by jwe]
jwe
parents: 4198
diff changeset
712 // are handled by the <MATRIX_START> start state code above. If closest
985
4b483cf9f6b0 [project @ 1994-12-14 18:06:38 by jwe]
jwe
parents: 975
diff changeset
713 // nesting is inside parentheses, don't return a row separator.
968
9351572b7210 [project @ 1994-12-11 21:53:00 by jwe]
jwe
parents: 967
diff changeset
714 %}
9351572b7210 [project @ 1994-12-11 21:53:00 by jwe]
jwe
parents: 967
diff changeset
715
967
b228d6cd59a1 [project @ 1994-12-10 00:15:41 by jwe]
jwe
parents: 947
diff changeset
716 {NL} {
16229
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
717 curr_lexer->lexer_debug ("{NL}");
8535
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
718
16110
7302f8a4df83 use pointer for global lexical_feedback structure
John W. Eaton <jwe@octave.org>
parents: 16106
diff changeset
719 curr_lexer->input_line_number++;
7302f8a4df83 use pointer for global lexical_feedback structure
John W. Eaton <jwe@octave.org>
parents: 16106
diff changeset
720 curr_lexer->current_input_column = 1;
7302f8a4df83 use pointer for global lexical_feedback structure
John W. Eaton <jwe@octave.org>
parents: 16106
diff changeset
721
7302f8a4df83 use pointer for global lexical_feedback structure
John W. Eaton <jwe@octave.org>
parents: 16106
diff changeset
722 if (curr_lexer->nesting_level.none ())
8746
5dd06f19e9be handle commands in the lexer
John W. Eaton <jwe@octave.org>
parents: 8745
diff changeset
723 {
16110
7302f8a4df83 use pointer for global lexical_feedback structure
John W. Eaton <jwe@octave.org>
parents: 16106
diff changeset
724 curr_lexer->at_beginning_of_statement = true;
16229
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
725 return curr_lexer->count_token ('\n');
8746
5dd06f19e9be handle commands in the lexer
John W. Eaton <jwe@octave.org>
parents: 8745
diff changeset
726 }
16110
7302f8a4df83 use pointer for global lexical_feedback structure
John W. Eaton <jwe@octave.org>
parents: 16106
diff changeset
727 else if (curr_lexer->nesting_level.is_paren ())
8746
5dd06f19e9be handle commands in the lexer
John W. Eaton <jwe@octave.org>
parents: 8745
diff changeset
728 {
16110
7302f8a4df83 use pointer for global lexical_feedback structure
John W. Eaton <jwe@octave.org>
parents: 16106
diff changeset
729 curr_lexer->at_beginning_of_statement = false;
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
730 curr_lexer->gripe_matlab_incompatible ("bare newline inside parentheses");
8746
5dd06f19e9be handle commands in the lexer
John W. Eaton <jwe@octave.org>
parents: 8745
diff changeset
731 }
16110
7302f8a4df83 use pointer for global lexical_feedback structure
John W. Eaton <jwe@octave.org>
parents: 16106
diff changeset
732 else if (curr_lexer->nesting_level.is_bracket_or_brace ())
985
4b483cf9f6b0 [project @ 1994-12-14 18:06:38 by jwe]
jwe
parents: 975
diff changeset
733 return LEXICAL_ERROR;
967
b228d6cd59a1 [project @ 1994-12-10 00:15:41 by jwe]
jwe
parents: 947
diff changeset
734 }
1
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
735
968
9351572b7210 [project @ 1994-12-11 21:53:00 by jwe]
jwe
parents: 967
diff changeset
736 %{
9351572b7210 [project @ 1994-12-11 21:53:00 by jwe]
jwe
parents: 967
diff changeset
737 // Single quote can either be the beginning of a string or a transpose
11586
12df7854fa7c strip trailing whitespace from source files
John W. Eaton <jwe@octave.org>
parents: 11523
diff changeset
738 // operator.
968
9351572b7210 [project @ 1994-12-11 21:53:00 by jwe]
jwe
parents: 967
diff changeset
739 %}
9351572b7210 [project @ 1994-12-11 21:53:00 by jwe]
jwe
parents: 967
diff changeset
740
967
b228d6cd59a1 [project @ 1994-12-10 00:15:41 by jwe]
jwe
parents: 947
diff changeset
741 "'" {
16229
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
742 curr_lexer->lexer_debug ("'");
8535
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
743
16275
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
744 if (curr_lexer->previous_token_may_be_command ()
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
745 && curr_lexer->space_follows_previous_token ())
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
746 {
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
747 yyless (0);
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
748 curr_lexer->push_start_state (COMMAND_START);
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
749 }
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
750 else
967
b228d6cd59a1 [project @ 1994-12-10 00:15:41 by jwe]
jwe
parents: 947
diff changeset
751 {
16275
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
752 int tok = curr_lexer->previous_token_value ();
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
753
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
754 bool transpose = false;
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
755
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
756 if (curr_lexer->whitespace_is_significant ())
16257
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
757 {
16275
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
758 if (curr_lexer->space_follows_previous_token ())
16257
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
759 {
16275
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
760 if (tok == '[' || tok == '{'
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
761 || curr_lexer->previous_token_is_binop ())
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
762 {
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
763 curr_lexer->current_input_column++;
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
764 int retval = curr_lexer->handle_string ('\'');
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
765 return curr_lexer->count_token_internal (retval);
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
766 }
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
767 else
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
768 {
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
769 yyless (0);
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
770 curr_lexer->xunput (',');
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
771 }
16257
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
772 }
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
773 else
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
774 {
16275
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
775 if (tok == '[' || tok == '{'
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
776 || curr_lexer->previous_token_is_binop ()
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
777 || curr_lexer->previous_token_is_keyword ())
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
778 {
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
779 curr_lexer->current_input_column++;
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
780 int retval = curr_lexer->handle_string ('\'');
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
781 return curr_lexer->count_token_internal (retval);
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
782 }
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
783 else
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
784 return curr_lexer->count_token (QUOTE);
16257
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
785 }
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
786 }
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
787 else
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
788 {
16275
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
789 if (! tok || tok == '[' || tok == '{' || tok == '('
16267
15f55df088e7 6/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16265
diff changeset
790 || curr_lexer->previous_token_is_binop ()
15f55df088e7 6/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16265
diff changeset
791 || curr_lexer->previous_token_is_keyword ())
16257
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
792 {
16261
b45a90cdb0ae 3/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16259
diff changeset
793 curr_lexer->current_input_column++;
16257
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
794 int retval = curr_lexer->handle_string ('\'');
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
795 return curr_lexer->count_token_internal (retval);
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
796 }
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
797 else
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
798 return curr_lexer->count_token (QUOTE);
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
799 }
967
b228d6cd59a1 [project @ 1994-12-10 00:15:41 by jwe]
jwe
parents: 947
diff changeset
800 }
b228d6cd59a1 [project @ 1994-12-10 00:15:41 by jwe]
jwe
parents: 947
diff changeset
801 }
1
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
802
968
9351572b7210 [project @ 1994-12-11 21:53:00 by jwe]
jwe
parents: 967
diff changeset
803 %{
971
fe71abb43457 [project @ 1994-12-12 15:10:22 by jwe]
jwe
parents: 970
diff changeset
804 // Double quotes always begin strings.
fe71abb43457 [project @ 1994-12-12 15:10:22 by jwe]
jwe
parents: 970
diff changeset
805 %}
fe71abb43457 [project @ 1994-12-12 15:10:22 by jwe]
jwe
parents: 970
diff changeset
806
973
46673c918034 [project @ 1994-12-12 17:50:07 by jwe]
jwe
parents: 972
diff changeset
807 \" {
16229
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
808 curr_lexer->lexer_debug ("\"");
8535
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
809
16275
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
810 if (curr_lexer->previous_token_may_be_command ()
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
811 && curr_lexer->space_follows_previous_token ())
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
812 {
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
813 yyless (0);
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
814 curr_lexer->push_start_state (COMMAND_START);
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
815 }
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
816 else
16261
b45a90cdb0ae 3/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16259
diff changeset
817 {
16275
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
818 int tok = curr_lexer->previous_token_value ();
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
819
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
820 bool transpose = false;
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
821
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
822 if (curr_lexer->whitespace_is_significant ())
16261
b45a90cdb0ae 3/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16259
diff changeset
823 {
16275
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
824 if (curr_lexer->space_follows_previous_token ())
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
825 {
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
826 if (tok == '[' || tok == '{'
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
827 || curr_lexer->previous_token_is_binop ())
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
828 {
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
829 curr_lexer->current_input_column++;
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
830 int retval = curr_lexer->handle_string ('"');
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
831 return curr_lexer->count_token_internal (retval);
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
832 }
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
833 else
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
834 {
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
835 yyless (0);
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
836 curr_lexer->xunput (',');
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
837 }
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
838 }
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
839 else
16261
b45a90cdb0ae 3/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16259
diff changeset
840 {
b45a90cdb0ae 3/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16259
diff changeset
841 curr_lexer->current_input_column++;
b45a90cdb0ae 3/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16259
diff changeset
842 int retval = curr_lexer->handle_string ('"');
b45a90cdb0ae 3/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16259
diff changeset
843 return curr_lexer->count_token_internal (retval);
b45a90cdb0ae 3/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16259
diff changeset
844 }
b45a90cdb0ae 3/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16259
diff changeset
845 }
b45a90cdb0ae 3/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16259
diff changeset
846 else
b45a90cdb0ae 3/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16259
diff changeset
847 {
b45a90cdb0ae 3/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16259
diff changeset
848 curr_lexer->current_input_column++;
b45a90cdb0ae 3/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16259
diff changeset
849 int retval = curr_lexer->handle_string ('"');
b45a90cdb0ae 3/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16259
diff changeset
850 return curr_lexer->count_token_internal (retval);
b45a90cdb0ae 3/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16259
diff changeset
851 }
b45a90cdb0ae 3/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16259
diff changeset
852 }
b45a90cdb0ae 3/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16259
diff changeset
853 }
971
fe71abb43457 [project @ 1994-12-12 15:10:22 by jwe]
jwe
parents: 970
diff changeset
854
fe71abb43457 [project @ 1994-12-12 15:10:22 by jwe]
jwe
parents: 970
diff changeset
855 %{
968
9351572b7210 [project @ 1994-12-11 21:53:00 by jwe]
jwe
parents: 967
diff changeset
856 // Other operators.
9351572b7210 [project @ 1994-12-11 21:53:00 by jwe]
jwe
parents: 967
diff changeset
857 %}
9351572b7210 [project @ 1994-12-11 21:53:00 by jwe]
jwe
parents: 967
diff changeset
858
16275
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
859 ":" { CMD_OR_OP (":", ':', true); }
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
860 ".+" { CMD_OR_OP (".+", EPLUS, false); }
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
861 ".-" { CMD_OR_OP (".-", EMINUS, false); }
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
862 ".*" { CMD_OR_OP (".*", EMUL, true); }
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
863 "./" { CMD_OR_OP ("./", EDIV, true); }
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
864 ".\\" { CMD_OR_OP (".\\", ELEFTDIV, true); }
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
865 ".^" { CMD_OR_OP (".^", EPOW, true); }
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
866 ".**" { CMD_OR_OP (".**", EPOW, false); }
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
867 "<=" { CMD_OR_OP ("<=", EXPR_LE, true); }
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
868 "==" { CMD_OR_OP ("==", EXPR_EQ, true); }
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
869 "~=" { CMD_OR_OP ("~=", EXPR_NE, true); }
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
870 "!=" { CMD_OR_OP ("!=", EXPR_NE, false); }
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
871 ">=" { CMD_OR_OP (">=", EXPR_GE, true); }
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
872 "&" { CMD_OR_OP ("&", EXPR_AND, true); }
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
873 "|" { CMD_OR_OP ("|", EXPR_OR, true); }
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
874 "<" { CMD_OR_OP ("<", EXPR_LT, true); }
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
875 ">" { CMD_OR_OP (">", EXPR_GT, true); }
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
876 "*" { CMD_OR_OP ("*", '*', true); }
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
877 "/" { CMD_OR_OP ("/", '/', true); }
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
878
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
879 %{
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
880 // In Matlab, '\' may also trigger command syntax.
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
881 %}
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
882
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
883 "\\" { return curr_lexer->handle_op ("\\", LEFTDIV); }
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
884
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
885 "^" { CMD_OR_OP ("^", POW, true); }
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
886 "**" { CMD_OR_OP ("**", POW, false); }
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
887 "&&" { CMD_OR_OP ("&&", EXPR_AND_AND, true); }
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
888 "||" { CMD_OR_OP ("||", EXPR_OR_OR, true); }
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
889 "<<" { CMD_OR_OP ("<<", LSHIFT, false); }
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
890 ">>" { CMD_OR_OP (">>", RSHIFT, false); }
16265
71ee3afedb69 5/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16263
diff changeset
891
71ee3afedb69 5/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16263
diff changeset
892 ";" {
71ee3afedb69 5/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16263
diff changeset
893 bool at_beginning_of_statement
71ee3afedb69 5/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16263
diff changeset
894 = (! (curr_lexer->whitespace_is_significant ()
71ee3afedb69 5/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16263
diff changeset
895 || curr_lexer->looking_at_object_index.front ()));
71ee3afedb69 5/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16263
diff changeset
896
16275
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
897 return curr_lexer->handle_op (";", ';', at_beginning_of_statement);
16263
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
898 }
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
899
16275
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
900 "+" { CMD_OR_UNARY_OP ("+", '+', true); }
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
901 "-" { CMD_OR_UNARY_OP ("-", '-', true); }
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
902
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
903 "~" { CMD_OR_UNARY_OP ("~", EXPR_NOT, true); }
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
904 "!" { CMD_OR_UNARY_OP ("!", EXPR_NOT, false); }
16263
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
905
16229
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
906 "," {
16265
71ee3afedb69 5/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16263
diff changeset
907 bool at_beginning_of_statement
71ee3afedb69 5/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16263
diff changeset
908 = (! (curr_lexer->whitespace_is_significant ()
71ee3afedb69 5/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16263
diff changeset
909 || curr_lexer->looking_at_object_index.front ()));
71ee3afedb69 5/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16263
diff changeset
910
16275
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
911 return curr_lexer->handle_op (",", ',', at_beginning_of_statement);
16229
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
912 }
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
913
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
914 ".'" {
16275
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
915 return curr_lexer->handle_op (".'", TRANSPOSE, false);
16229
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
916 }
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
917
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
918 "++" {
16275
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
919 curr_lexer->lexer_debug ("++");
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
920
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
921 int tok = curr_lexer->handle_incompatible_unary_op (PLUS_PLUS, false);
16263
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
922
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
923 if (tok < 0)
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
924 {
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
925 yyless (0);
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
926 curr_lexer->xunput (',');
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
927 }
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
928 else
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
929 return tok;
16229
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
930 }
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
931
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
932 "--" {
16275
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
933 curr_lexer->lexer_debug ("--");
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
934
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
935 int tok = curr_lexer->handle_incompatible_unary_op (MINUS_MINUS, false);
16263
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
936
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
937 if (tok < 0)
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
938 {
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
939 yyless (0);
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
940 curr_lexer->xunput (',');
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
941 }
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
942 else
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
943 return tok;
967
b228d6cd59a1 [project @ 1994-12-10 00:15:41 by jwe]
jwe
parents: 947
diff changeset
944 }
1
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
945
967
b228d6cd59a1 [project @ 1994-12-10 00:15:41 by jwe]
jwe
parents: 947
diff changeset
946 "(" {
16229
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
947 curr_lexer->lexer_debug ("(");
8535
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
948
16263
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
949 bool unput_comma = false;
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
950
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
951 if (curr_lexer->whitespace_is_significant ()
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
952 && curr_lexer->space_follows_previous_token ())
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
953 {
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
954 int tok = curr_lexer->previous_token_value ();
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
955
16267
15f55df088e7 6/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16265
diff changeset
956 if (! (tok == '[' || tok == '{'
16263
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
957 || curr_lexer->previous_token_is_binop ()))
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
958 unput_comma = true;
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
959 }
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
960
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
961 if (unput_comma)
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
962 {
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
963 yyless (0);
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
964 curr_lexer->xunput (',');
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
965 }
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
966 else
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
967 {
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
968 // If we are looking for an object index, then push TRUE for
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
969 // looking_at_object_index. Otherwise, just push whatever state
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
970 // is current (so that we can pop it off the stack when we find
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
971 // the matching close paren).
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
972
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
973 curr_lexer->looking_at_object_index.push_front
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
974 (curr_lexer->looking_for_object_index);
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
975
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
976 curr_lexer->looking_at_indirect_ref = false;
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
977 curr_lexer->looking_for_object_index = false;
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
978 curr_lexer->at_beginning_of_statement = false;
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
979
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
980 curr_lexer->nesting_level.paren ();
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
981 curr_lexer->decrement_promptflag ();
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
982
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
983 return curr_lexer->handle_token ('(');
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
984 }
967
b228d6cd59a1 [project @ 1994-12-10 00:15:41 by jwe]
jwe
parents: 947
diff changeset
985 }
b228d6cd59a1 [project @ 1994-12-10 00:15:41 by jwe]
jwe
parents: 947
diff changeset
986
b228d6cd59a1 [project @ 1994-12-10 00:15:41 by jwe]
jwe
parents: 947
diff changeset
987 ")" {
16229
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
988 curr_lexer->lexer_debug (")");
8535
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
989
16110
7302f8a4df83 use pointer for global lexical_feedback structure
John W. Eaton <jwe@octave.org>
parents: 16106
diff changeset
990 curr_lexer->nesting_level.remove ();
7302f8a4df83 use pointer for global lexical_feedback structure
John W. Eaton <jwe@octave.org>
parents: 16106
diff changeset
991 curr_lexer->current_input_column++;
7302f8a4df83 use pointer for global lexical_feedback structure
John W. Eaton <jwe@octave.org>
parents: 16106
diff changeset
992
7302f8a4df83 use pointer for global lexical_feedback structure
John W. Eaton <jwe@octave.org>
parents: 16106
diff changeset
993 curr_lexer->looking_at_object_index.pop_front ();
7302f8a4df83 use pointer for global lexical_feedback structure
John W. Eaton <jwe@octave.org>
parents: 16106
diff changeset
994
7302f8a4df83 use pointer for global lexical_feedback structure
John W. Eaton <jwe@octave.org>
parents: 16106
diff changeset
995 curr_lexer->looking_for_object_index = true;
7302f8a4df83 use pointer for global lexical_feedback structure
John W. Eaton <jwe@octave.org>
parents: 16106
diff changeset
996 curr_lexer->at_beginning_of_statement = false;
7302f8a4df83 use pointer for global lexical_feedback structure
John W. Eaton <jwe@octave.org>
parents: 16106
diff changeset
997
7302f8a4df83 use pointer for global lexical_feedback structure
John W. Eaton <jwe@octave.org>
parents: 16106
diff changeset
998 if (curr_lexer->looking_at_anon_fcn_args)
16265
71ee3afedb69 5/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16263
diff changeset
999 {
71ee3afedb69 5/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16263
diff changeset
1000 curr_lexer->looking_at_anon_fcn_args = false;
71ee3afedb69 5/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16263
diff changeset
1001 curr_lexer->nesting_level.anon_fcn_body ();
71ee3afedb69 5/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16263
diff changeset
1002 }
13237
1bfca2bbea8b fix parsing of anonymous functions inside cell array lists.
John W. Eaton <jwe@octave.org>
parents: 12801
diff changeset
1003
16229
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
1004 return curr_lexer->count_token (')');
967
b228d6cd59a1 [project @ 1994-12-10 00:15:41 by jwe]
jwe
parents: 947
diff changeset
1005 }
b228d6cd59a1 [project @ 1994-12-10 00:15:41 by jwe]
jwe
parents: 947
diff changeset
1006
8745
6dc61981d18b better handling of object indexing in lexer
John W. Eaton <jwe@octave.org>
parents: 8701
diff changeset
1007 "." {
16229
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
1008 curr_lexer->lexer_debug (".");
8745
6dc61981d18b better handling of object indexing in lexer
John W. Eaton <jwe@octave.org>
parents: 8701
diff changeset
1009
16110
7302f8a4df83 use pointer for global lexical_feedback structure
John W. Eaton <jwe@octave.org>
parents: 16106
diff changeset
1010 curr_lexer->looking_for_object_index = false;
7302f8a4df83 use pointer for global lexical_feedback structure
John W. Eaton <jwe@octave.org>
parents: 16106
diff changeset
1011 curr_lexer->at_beginning_of_statement = false;
8745
6dc61981d18b better handling of object indexing in lexer
John W. Eaton <jwe@octave.org>
parents: 8701
diff changeset
1012
16229
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
1013 return curr_lexer->handle_token ('.');
8745
6dc61981d18b better handling of object indexing in lexer
John W. Eaton <jwe@octave.org>
parents: 8701
diff changeset
1014 }
8535
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1015
16229
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
1016 %{
16257
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
1017 // = and op= operators.
16229
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
1018 %}
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
1019
16273
c5e5f6ccac5d 9/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16271
diff changeset
1020 "=" { return curr_lexer->handle_op ("=", '='); }
c5e5f6ccac5d 9/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16271
diff changeset
1021 "+=" { return curr_lexer->handle_incompatible_op ("+=", ADD_EQ); }
c5e5f6ccac5d 9/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16271
diff changeset
1022 "-=" { return curr_lexer->handle_incompatible_op ("-=", SUB_EQ); }
c5e5f6ccac5d 9/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16271
diff changeset
1023 "*=" { return curr_lexer->handle_incompatible_op ("*=", MUL_EQ); }
c5e5f6ccac5d 9/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16271
diff changeset
1024 "/=" { return curr_lexer->handle_incompatible_op ("/=", DIV_EQ); }
c5e5f6ccac5d 9/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16271
diff changeset
1025 "\\=" { return curr_lexer->handle_incompatible_op ("\\=", LEFTDIV_EQ); }
c5e5f6ccac5d 9/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16271
diff changeset
1026 ".+=" { return curr_lexer->handle_incompatible_op (".+=", ADD_EQ); }
c5e5f6ccac5d 9/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16271
diff changeset
1027 ".-=" { return curr_lexer->handle_incompatible_op (".-=", SUB_EQ); }
c5e5f6ccac5d 9/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16271
diff changeset
1028 ".*=" { return curr_lexer->handle_incompatible_op (".*=", EMUL_EQ); }
c5e5f6ccac5d 9/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16271
diff changeset
1029 "./=" { return curr_lexer->handle_incompatible_op ("./=", EDIV_EQ); }
c5e5f6ccac5d 9/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16271
diff changeset
1030 ".\\=" { return curr_lexer->handle_incompatible_op (".\\=", ELEFTDIV_EQ); }
c5e5f6ccac5d 9/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16271
diff changeset
1031 "^=" { return curr_lexer->handle_incompatible_op ("^=", POW_EQ); }
c5e5f6ccac5d 9/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16271
diff changeset
1032 "**=" { return curr_lexer->handle_incompatible_op ("^=", POW_EQ); }
c5e5f6ccac5d 9/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16271
diff changeset
1033 ".^=" { return curr_lexer->handle_incompatible_op (".^=", EPOW_EQ); }
c5e5f6ccac5d 9/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16271
diff changeset
1034 ".**=" { return curr_lexer->handle_incompatible_op (".^=", EPOW_EQ); }
c5e5f6ccac5d 9/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16271
diff changeset
1035 "&=" { return curr_lexer->handle_incompatible_op ("&=", AND_EQ); }
c5e5f6ccac5d 9/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16271
diff changeset
1036 "|=" { return curr_lexer->handle_incompatible_op ("|=", OR_EQ); }
c5e5f6ccac5d 9/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16271
diff changeset
1037 "<<=" { return curr_lexer->handle_incompatible_op ("<<=", LSHIFT_EQ); }
c5e5f6ccac5d 9/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16271
diff changeset
1038 ">>=" { return curr_lexer->handle_incompatible_op (">>=", RSHIFT_EQ); }
16257
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
1039
16275
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
1040 %{
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
1041 // In Matlab, '{' may also trigger command syntax.
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
1042 %}
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
1043
16257
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
1044 "{" {
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
1045 curr_lexer->lexer_debug ("{");
8535
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1046
16263
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
1047 bool unput_comma = false;
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
1048
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
1049 if (curr_lexer->whitespace_is_significant ()
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
1050 && curr_lexer->space_follows_previous_token ())
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
1051 {
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
1052 int tok = curr_lexer->previous_token_value ();
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
1053
16267
15f55df088e7 6/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16265
diff changeset
1054 if (! (tok == '[' || tok == '{'
16263
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
1055 || curr_lexer->previous_token_is_binop ()))
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
1056 unput_comma = true;
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
1057 }
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
1058
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
1059 if (unput_comma)
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
1060 {
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
1061 yyless (0);
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
1062 curr_lexer->xunput (',');
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
1063 }
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
1064 else
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
1065 {
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
1066 curr_lexer->nesting_level.brace ();
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
1067
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
1068 curr_lexer->looking_at_object_index.push_front
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
1069 (curr_lexer->looking_for_object_index);
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
1070
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
1071 curr_lexer->current_input_column += yyleng;
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
1072 curr_lexer->looking_for_object_index = false;
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
1073 curr_lexer->at_beginning_of_statement = false;
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
1074
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
1075 curr_lexer->decrement_promptflag ();
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
1076
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
1077 curr_lexer->braceflag++;
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
1078
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
1079 curr_lexer->push_start_state (MATRIX_START);
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
1080
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
1081 return curr_lexer->count_token ('{');
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
1082 }
3351
8623649c967c [project @ 1999-11-15 16:17:01 by jwe]
jwe
parents: 3332
diff changeset
1083 }
8623649c967c [project @ 1999-11-15 16:17:01 by jwe]
jwe
parents: 3332
diff changeset
1084
8623649c967c [project @ 1999-11-15 16:17:01 by jwe]
jwe
parents: 3332
diff changeset
1085 "}" {
16229
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
1086 curr_lexer->lexer_debug ("}");
8535
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1087
16110
7302f8a4df83 use pointer for global lexical_feedback structure
John W. Eaton <jwe@octave.org>
parents: 16106
diff changeset
1088 curr_lexer->looking_at_object_index.pop_front ();
7302f8a4df83 use pointer for global lexical_feedback structure
John W. Eaton <jwe@octave.org>
parents: 16106
diff changeset
1089
7302f8a4df83 use pointer for global lexical_feedback structure
John W. Eaton <jwe@octave.org>
parents: 16106
diff changeset
1090 curr_lexer->looking_for_object_index = true;
7302f8a4df83 use pointer for global lexical_feedback structure
John W. Eaton <jwe@octave.org>
parents: 16106
diff changeset
1091 curr_lexer->at_beginning_of_statement = false;
7302f8a4df83 use pointer for global lexical_feedback structure
John W. Eaton <jwe@octave.org>
parents: 16106
diff changeset
1092
7302f8a4df83 use pointer for global lexical_feedback structure
John W. Eaton <jwe@octave.org>
parents: 16106
diff changeset
1093 curr_lexer->nesting_level.remove ();
3351
8623649c967c [project @ 1999-11-15 16:17:01 by jwe]
jwe
parents: 3332
diff changeset
1094
16229
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
1095 return curr_lexer->handle_token ('}');
3351
8623649c967c [project @ 1999-11-15 16:17:01 by jwe]
jwe
parents: 3332
diff changeset
1096 }
8623649c967c [project @ 1999-11-15 16:17:01 by jwe]
jwe
parents: 3332
diff changeset
1097
968
9351572b7210 [project @ 1994-12-11 21:53:00 by jwe]
jwe
parents: 967
diff changeset
1098 %{
2066
a75d2a5d6783 [project @ 1996-04-11 21:20:43 by jwe]
jwe
parents: 2042
diff changeset
1099 // Unrecognized input is a lexical error.
968
9351572b7210 [project @ 1994-12-11 21:53:00 by jwe]
jwe
parents: 967
diff changeset
1100 %}
1
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
1101
2042
31f4f775b8c6 [project @ 1996-03-28 09:06:28 by jwe]
jwe
parents: 1994
diff changeset
1102 . {
16229
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
1103 curr_lexer->lexer_debug (".");
8535
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1104
16125
96a58f197f93 allow xunput to be called without buffer argument
John W. Eaton <jwe@octave.org>
parents: 16124
diff changeset
1105 curr_lexer->xunput (yytext[0]);
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
1106
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
1107 int c = curr_lexer->text_yyinput ();
4248
eef64f3f9a4c [project @ 2002-12-31 00:00:20 by jwe]
jwe
parents: 4241
diff changeset
1108
eef64f3f9a4c [project @ 2002-12-31 00:00:20 by jwe]
jwe
parents: 4241
diff changeset
1109 if (c != EOF)
4240
8627d992beb8 [project @ 2002-12-25 23:07:03 by jwe]
jwe
parents: 4238
diff changeset
1110 {
16110
7302f8a4df83 use pointer for global lexical_feedback structure
John W. Eaton <jwe@octave.org>
parents: 16106
diff changeset
1111 curr_lexer->current_input_column++;
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1112
15466
d174210ce1ec use ' instead of ` in error messages, warnings and most comments
John W. Eaton <jwe@octave.org>
parents: 14161
diff changeset
1113 error ("invalid character '%s' (ASCII %d) near line %d, column %d",
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1114 undo_string_escape (static_cast<char> (c)), c,
16110
7302f8a4df83 use pointer for global lexical_feedback structure
John W. Eaton <jwe@octave.org>
parents: 16106
diff changeset
1115 curr_lexer->input_line_number, curr_lexer->current_input_column);
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1116
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1117 return LEXICAL_ERROR;
4240
8627d992beb8 [project @ 2002-12-25 23:07:03 by jwe]
jwe
parents: 4238
diff changeset
1118 }
8627d992beb8 [project @ 2002-12-25 23:07:03 by jwe]
jwe
parents: 4238
diff changeset
1119 else
16164
c5bfdc4c0963 move end_of_input flag from octve_parser class to octave_lexer class
John W. Eaton <jwe@octave.org>
parents: 16158
diff changeset
1120 return curr_lexer->handle_end_of_input ();
2066
a75d2a5d6783 [project @ 1996-04-11 21:20:43 by jwe]
jwe
parents: 2042
diff changeset
1121 }
1
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
1122
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
1123 %%
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
1124
8447
adab48231a03 make input_line_number work again
John W. Eaton <jwe@octave.org>
parents: 8312
diff changeset
1125 static void
8535
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1126 display_character (char c)
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1127 {
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1128 if (isgraph (c))
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1129 std::cerr << c;
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1130 else
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1131 switch (c)
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1132 {
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1133 case 0:
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1134 std::cerr << "NUL";
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1135 break;
8535
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1136
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1137 case 1:
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1138 std::cerr << "SOH";
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1139 break;
8535
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1140
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1141 case 2:
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1142 std::cerr << "STX";
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1143 break;
8535
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1144
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1145 case 3:
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1146 std::cerr << "ETX";
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1147 break;
8535
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1148
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1149 case 4:
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1150 std::cerr << "EOT";
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1151 break;
8535
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1152
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1153 case 5:
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1154 std::cerr << "ENQ";
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1155 break;
8535
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1156
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1157 case 6:
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1158 std::cerr << "ACK";
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1159 break;
8535
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1160
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1161 case 7:
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1162 std::cerr << "\\a";
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1163 break;
8535
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1164
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1165 case 8:
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1166 std::cerr << "\\b";
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1167 break;
8535
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1168
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1169 case 9:
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1170 std::cerr << "\\t";
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1171 break;
8535
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1172
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1173 case 10:
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1174 std::cerr << "\\n";
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1175 break;
8535
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1176
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1177 case 11:
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1178 std::cerr << "\\v";
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1179 break;
8535
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1180
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1181 case 12:
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1182 std::cerr << "\\f";
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1183 break;
8535
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1184
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1185 case 13:
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1186 std::cerr << "\\r";
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1187 break;
8535
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1188
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1189 case 14:
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1190 std::cerr << "SO";
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1191 break;
8535
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1192
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1193 case 15:
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1194 std::cerr << "SI";
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1195 break;
8535
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1196
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1197 case 16:
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1198 std::cerr << "DLE";
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1199 break;
8535
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1200
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1201 case 17:
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1202 std::cerr << "DC1";
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1203 break;
8535
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1204
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1205 case 18:
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1206 std::cerr << "DC2";
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1207 break;
8535
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1208
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1209 case 19:
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1210 std::cerr << "DC3";
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1211 break;
8535
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1212
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1213 case 20:
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1214 std::cerr << "DC4";
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1215 break;
8535
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1216
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1217 case 21:
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1218 std::cerr << "NAK";
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1219 break;
8535
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1220
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1221 case 22:
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1222 std::cerr << "SYN";
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1223 break;
8535
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1224
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1225 case 23:
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1226 std::cerr << "ETB";
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1227 break;
8535
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1228
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1229 case 24:
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1230 std::cerr << "CAN";
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1231 break;
8535
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1232
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1233 case 25:
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1234 std::cerr << "EM";
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1235 break;
8535
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1236
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1237 case 26:
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1238 std::cerr << "SUB";
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1239 break;
8535
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1240
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1241 case 27:
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1242 std::cerr << "ESC";
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1243 break;
8535
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1244
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1245 case 28:
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1246 std::cerr << "FS";
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1247 break;
8535
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1248
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1249 case 29:
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1250 std::cerr << "GS";
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1251 break;
8535
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1252
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1253 case 30:
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1254 std::cerr << "RS";
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1255 break;
8535
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1256
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1257 case 31:
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1258 std::cerr << "US";
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1259 break;
8535
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1260
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1261 case 32:
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1262 std::cerr << "SPACE";
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1263 break;
8535
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1264
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1265 case 127:
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1266 std::cerr << "DEL";
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1267 break;
8535
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1268 }
75e6ab186761 lexer debugging functions
John W. Eaton <jwe@octave.org>
parents: 8504
diff changeset
1269 }
8745
6dc61981d18b better handling of object indexing in lexer
John W. Eaton <jwe@octave.org>
parents: 8701
diff changeset
1270
13973
2c664266e9d0 clean up parser memory on exit
John W. Eaton <jwe@octave.org>
parents: 13260
diff changeset
1271 void
2c664266e9d0 clean up parser memory on exit
John W. Eaton <jwe@octave.org>
parents: 13260
diff changeset
1272 cleanup_parser (void)
2c664266e9d0 clean up parser memory on exit
John W. Eaton <jwe@octave.org>
parents: 13260
diff changeset
1273 {
1
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
1274 }
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
1275
16114
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1276 // Return 1 if the given character matches any character in the given
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1277 // string.
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1278
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1279 static bool
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1280 match_any (char c, const char *s)
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1281 {
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1282 char tmp;
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1283 while ((tmp = *s++) != '\0')
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1284 {
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1285 if (c == tmp)
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1286 return true;
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1287 }
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1288 return false;
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1289 }
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1290
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1291 bool
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1292 is_keyword (const std::string& s)
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1293 {
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1294 // Parsing function names like "set.property_name" inside
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1295 // classdef-style class definitions is simplified by handling the
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1296 // "set" and "get" portions of the names using the same mechanism as
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1297 // is used for keywords. However, they are not really keywords in
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1298 // the language, so omit them from the list of possible keywords.
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1299
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1300 return (octave_kw_hash::in_word_set (s.c_str (), s.length ()) != 0
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1301 && ! (s == "set" || s == "get"));
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1302 }
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1303
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1304 DEFUN (iskeyword, args, ,
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1305 "-*- texinfo -*-\n\
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1306 @deftypefn {Built-in Function} {} iskeyword ()\n\
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1307 @deftypefnx {Built-in Function} {} iskeyword (@var{name})\n\
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1308 Return true if @var{name} is an Octave keyword. If @var{name}\n\
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1309 is omitted, return a list of keywords.\n\
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1310 @seealso{isvarname, exist}\n\
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1311 @end deftypefn")
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1312 {
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1313 octave_value retval;
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1314
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1315 int argc = args.length () + 1;
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1316
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1317 string_vector argv = args.make_argv ("iskeyword");
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1318
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1319 if (error_state)
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1320 return retval;
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1321
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1322 if (argc == 1)
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1323 {
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1324 // Neither set and get are keywords. See the note in the
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1325 // is_keyword function for additional details.
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1326
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1327 string_vector lst (TOTAL_KEYWORDS);
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1328
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1329 int j = 0;
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1330
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1331 for (int i = 0; i < TOTAL_KEYWORDS; i++)
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1332 {
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1333 std::string tmp = wordlist[i].name;
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1334
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1335 if (! (tmp == "set" || tmp == "get"))
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1336 lst[j++] = tmp;
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1337 }
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1338
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1339 lst.resize (j);
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1340
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1341 retval = Cell (lst.sort ());
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1342 }
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1343 else if (argc == 2)
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1344 {
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1345 retval = is_keyword (argv[1]);
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1346 }
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1347 else
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1348 print_usage ();
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1349
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1350 return retval;
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1351 }
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1352
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1353 /*
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1354
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1355 %!assert (iskeyword ("for"))
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1356 %!assert (iskeyword ("fort"), false)
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1357 %!assert (iskeyword ("fft"), false)
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1358
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1359 */
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1360
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1361 // Used to delete trailing white space from tokens.
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1362
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1363 static std::string
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1364 strip_trailing_whitespace (char *s)
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1365 {
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1366 std::string retval = s;
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1367
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1368 size_t pos = retval.find_first_of (" \t");
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1369
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1370 if (pos != std::string::npos)
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1371 retval.resize (pos);
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1372
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1373 return retval;
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1374 }
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1375
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1376 DEFUN (__display_tokens__, args, nargout,
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1377 "-*- texinfo -*-\n\
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1378 @deftypefn {Built-in Function} {} __display_tokens__ ()\n\
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1379 Query or set the internal variable that determines whether Octave's\n\
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1380 lexer displays tokens as they are read.\n\
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1381 @end deftypefn")
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1382 {
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1383 return SET_INTERNAL_VARIABLE (display_tokens);
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1384 }
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1385
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1386 DEFUN (__token_count__, , ,
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1387 "-*- texinfo -*-\n\
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1388 @deftypefn {Built-in Function} {} __token_count__ ()\n\
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1389 Number of language tokens processed since Octave startup.\n\
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1390 @end deftypefn")
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1391 {
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1392 return octave_value (Vtoken_count);
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1393 }
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1394
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1395 DEFUN (__lexer_debug_flag__, args, nargout,
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1396 "-*- texinfo -*-\n\
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1397 @deftypefn {Built-in Function} {@var{old_val} =} __lexer_debug_flag__ (@var{new_val}))\n\
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1398 Undocumented internal function.\n\
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1399 @end deftypefn")
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1400 {
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1401 octave_value retval;
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1402
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1403 retval = set_internal_variable (lexer_debug_flag, args, nargout,
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1404 "__lexer_debug_flag__");
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1405
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1406 return retval;
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1407 }
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1408
16117
771186909285 maint: move class definition in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16116
diff changeset
1409 class
771186909285 maint: move class definition in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16116
diff changeset
1410 flex_stream_reader : public stream_reader
771186909285 maint: move class definition in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16116
diff changeset
1411 {
771186909285 maint: move class definition in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16116
diff changeset
1412 public:
16158
7eb614760ddb rename lexical_feedback to octave_lexer
John W. Eaton <jwe@octave.org>
parents: 16154
diff changeset
1413 flex_stream_reader (octave_lexer *l, char *buf_arg)
16117
771186909285 maint: move class definition in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16116
diff changeset
1414 : stream_reader (), lexer (l), buf (buf_arg)
771186909285 maint: move class definition in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16116
diff changeset
1415 { }
771186909285 maint: move class definition in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16116
diff changeset
1416
771186909285 maint: move class definition in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16116
diff changeset
1417 int getc (void) { return lexer->text_yyinput (); }
771186909285 maint: move class definition in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16116
diff changeset
1418 int ungetc (int c) { lexer->xunput (c, buf); return 0; }
771186909285 maint: move class definition in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16116
diff changeset
1419
771186909285 maint: move class definition in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16116
diff changeset
1420 private:
771186909285 maint: move class definition in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16116
diff changeset
1421
771186909285 maint: move class definition in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16116
diff changeset
1422 // No copying!
771186909285 maint: move class definition in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16116
diff changeset
1423
771186909285 maint: move class definition in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16116
diff changeset
1424 flex_stream_reader (const flex_stream_reader&);
771186909285 maint: move class definition in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16116
diff changeset
1425
771186909285 maint: move class definition in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16116
diff changeset
1426 flex_stream_reader& operator = (const flex_stream_reader&);
771186909285 maint: move class definition in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16116
diff changeset
1427
16158
7eb614760ddb rename lexical_feedback to octave_lexer
John W. Eaton <jwe@octave.org>
parents: 16154
diff changeset
1428 octave_lexer *lexer;
16117
771186909285 maint: move class definition in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16116
diff changeset
1429
771186909285 maint: move class definition in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16116
diff changeset
1430 char *buf;
771186909285 maint: move class definition in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16116
diff changeset
1431 };
771186909285 maint: move class definition in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16116
diff changeset
1432
16193
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16183
diff changeset
1433 lexical_feedback::~lexical_feedback (void)
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16183
diff changeset
1434 {
16230
4bf907906134 use a queue to hold tokens in the lexer
John W. Eaton <jwe@octave.org>
parents: 16229
diff changeset
1435 tokens.clear ();
16194
b7ca669af528 reset lexical_feedback state before starting a new parse
John W. Eaton <jwe@octave.org>
parents: 16193
diff changeset
1436 }
b7ca669af528 reset lexical_feedback state before starting a new parse
John W. Eaton <jwe@octave.org>
parents: 16193
diff changeset
1437
b7ca669af528 reset lexical_feedback state before starting a new parse
John W. Eaton <jwe@octave.org>
parents: 16193
diff changeset
1438 void
b7ca669af528 reset lexical_feedback state before starting a new parse
John W. Eaton <jwe@octave.org>
parents: 16193
diff changeset
1439 lexical_feedback::init (void)
b7ca669af528 reset lexical_feedback state before starting a new parse
John W. Eaton <jwe@octave.org>
parents: 16193
diff changeset
1440 {
b7ca669af528 reset lexical_feedback state before starting a new parse
John W. Eaton <jwe@octave.org>
parents: 16193
diff changeset
1441 // The closest paren, brace, or bracket nesting is not an object
b7ca669af528 reset lexical_feedback state before starting a new parse
John W. Eaton <jwe@octave.org>
parents: 16193
diff changeset
1442 // index.
b7ca669af528 reset lexical_feedback state before starting a new parse
John W. Eaton <jwe@octave.org>
parents: 16193
diff changeset
1443 looking_at_object_index.push_front (false);
b7ca669af528 reset lexical_feedback state before starting a new parse
John W. Eaton <jwe@octave.org>
parents: 16193
diff changeset
1444 }
b7ca669af528 reset lexical_feedback state before starting a new parse
John W. Eaton <jwe@octave.org>
parents: 16193
diff changeset
1445
b7ca669af528 reset lexical_feedback state before starting a new parse
John W. Eaton <jwe@octave.org>
parents: 16193
diff changeset
1446 void
b7ca669af528 reset lexical_feedback state before starting a new parse
John W. Eaton <jwe@octave.org>
parents: 16193
diff changeset
1447 lexical_feedback::reset (void)
b7ca669af528 reset lexical_feedback state before starting a new parse
John W. Eaton <jwe@octave.org>
parents: 16193
diff changeset
1448 {
b7ca669af528 reset lexical_feedback state before starting a new parse
John W. Eaton <jwe@octave.org>
parents: 16193
diff changeset
1449 end_of_input = false;
b7ca669af528 reset lexical_feedback state before starting a new parse
John W. Eaton <jwe@octave.org>
parents: 16193
diff changeset
1450 at_beginning_of_statement = true;
b7ca669af528 reset lexical_feedback state before starting a new parse
John W. Eaton <jwe@octave.org>
parents: 16193
diff changeset
1451 looking_at_anon_fcn_args = false;
b7ca669af528 reset lexical_feedback state before starting a new parse
John W. Eaton <jwe@octave.org>
parents: 16193
diff changeset
1452 looking_at_return_list = false;
b7ca669af528 reset lexical_feedback state before starting a new parse
John W. Eaton <jwe@octave.org>
parents: 16193
diff changeset
1453 looking_at_parameter_list = false;
b7ca669af528 reset lexical_feedback state before starting a new parse
John W. Eaton <jwe@octave.org>
parents: 16193
diff changeset
1454 looking_at_decl_list = false;
b7ca669af528 reset lexical_feedback state before starting a new parse
John W. Eaton <jwe@octave.org>
parents: 16193
diff changeset
1455 looking_at_initializer_expression = false;
b7ca669af528 reset lexical_feedback state before starting a new parse
John W. Eaton <jwe@octave.org>
parents: 16193
diff changeset
1456 looking_at_matrix_or_assign_lhs = false;
b7ca669af528 reset lexical_feedback state before starting a new parse
John W. Eaton <jwe@octave.org>
parents: 16193
diff changeset
1457 looking_for_object_index = false;
b7ca669af528 reset lexical_feedback state before starting a new parse
John W. Eaton <jwe@octave.org>
parents: 16193
diff changeset
1458 looking_at_indirect_ref = false;
b7ca669af528 reset lexical_feedback state before starting a new parse
John W. Eaton <jwe@octave.org>
parents: 16193
diff changeset
1459 parsing_class_method = false;
b7ca669af528 reset lexical_feedback state before starting a new parse
John W. Eaton <jwe@octave.org>
parents: 16193
diff changeset
1460 maybe_classdef_get_set_method = false;
b7ca669af528 reset lexical_feedback state before starting a new parse
John W. Eaton <jwe@octave.org>
parents: 16193
diff changeset
1461 parsing_classdef = false;
16203
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
1462 force_script = false;
16199
810a71122c25 move more global variables to octave_lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16195
diff changeset
1463 reading_fcn_file = false;
810a71122c25 move more global variables to octave_lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16195
diff changeset
1464 reading_script_file = false;
810a71122c25 move more global variables to octave_lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16195
diff changeset
1465 reading_classdef_file = false;
16194
b7ca669af528 reset lexical_feedback state before starting a new parse
John W. Eaton <jwe@octave.org>
parents: 16193
diff changeset
1466 input_line_number = 1;
b7ca669af528 reset lexical_feedback state before starting a new parse
John W. Eaton <jwe@octave.org>
parents: 16193
diff changeset
1467 current_input_column = 1;
b7ca669af528 reset lexical_feedback state before starting a new parse
John W. Eaton <jwe@octave.org>
parents: 16193
diff changeset
1468 bracketflag = 0;
b7ca669af528 reset lexical_feedback state before starting a new parse
John W. Eaton <jwe@octave.org>
parents: 16193
diff changeset
1469 braceflag = 0;
b7ca669af528 reset lexical_feedback state before starting a new parse
John W. Eaton <jwe@octave.org>
parents: 16193
diff changeset
1470 looping = 0;
b7ca669af528 reset lexical_feedback state before starting a new parse
John W. Eaton <jwe@octave.org>
parents: 16193
diff changeset
1471 defining_func = 0;
b7ca669af528 reset lexical_feedback state before starting a new parse
John W. Eaton <jwe@octave.org>
parents: 16193
diff changeset
1472 looking_at_function_handle = 0;
b7ca669af528 reset lexical_feedback state before starting a new parse
John W. Eaton <jwe@octave.org>
parents: 16193
diff changeset
1473 block_comment_nesting_level = 0;
16203
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
1474 token_count = 0;
16207
0467d68ca891 move current_input_line to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16203
diff changeset
1475 current_input_line = "";
16228
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
1476 comment_text = "";
16203
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
1477 help_text = "";
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
1478 fcn_file_name = "";
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
1479 fcn_file_full_name = "";
16194
b7ca669af528 reset lexical_feedback state before starting a new parse
John W. Eaton <jwe@octave.org>
parents: 16193
diff changeset
1480 looking_at_object_index.clear ();
b7ca669af528 reset lexical_feedback state before starting a new parse
John W. Eaton <jwe@octave.org>
parents: 16193
diff changeset
1481 looking_at_object_index.push_front (false);
b7ca669af528 reset lexical_feedback state before starting a new parse
John W. Eaton <jwe@octave.org>
parents: 16193
diff changeset
1482
b7ca669af528 reset lexical_feedback state before starting a new parse
John W. Eaton <jwe@octave.org>
parents: 16193
diff changeset
1483 while (! parsed_function_name.empty ())
b7ca669af528 reset lexical_feedback state before starting a new parse
John W. Eaton <jwe@octave.org>
parents: 16193
diff changeset
1484 parsed_function_name.pop ();
b7ca669af528 reset lexical_feedback state before starting a new parse
John W. Eaton <jwe@octave.org>
parents: 16193
diff changeset
1485
b7ca669af528 reset lexical_feedback state before starting a new parse
John W. Eaton <jwe@octave.org>
parents: 16193
diff changeset
1486 pending_local_variables.clear ();
b7ca669af528 reset lexical_feedback state before starting a new parse
John W. Eaton <jwe@octave.org>
parents: 16193
diff changeset
1487
b7ca669af528 reset lexical_feedback state before starting a new parse
John W. Eaton <jwe@octave.org>
parents: 16193
diff changeset
1488 nesting_level.reset ();
b7ca669af528 reset lexical_feedback state before starting a new parse
John W. Eaton <jwe@octave.org>
parents: 16193
diff changeset
1489
16230
4bf907906134 use a queue to hold tokens in the lexer
John W. Eaton <jwe@octave.org>
parents: 16229
diff changeset
1490 tokens.clear ();
16194
b7ca669af528 reset lexical_feedback state before starting a new parse
John W. Eaton <jwe@octave.org>
parents: 16193
diff changeset
1491 }
b7ca669af528 reset lexical_feedback state before starting a new parse
John W. Eaton <jwe@octave.org>
parents: 16193
diff changeset
1492
16255
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
1493 int
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
1494 lexical_feedback::previous_token_value (void) const
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
1495 {
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
1496 const token *tok = tokens.front ();
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
1497 return tok ? tok->token_value () : 0;
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
1498 }
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
1499
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
1500 bool
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
1501 lexical_feedback::previous_token_value_is (int tok_val) const
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
1502 {
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
1503 const token *tok = tokens.front ();
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
1504 return tok ? tok->token_value_is (tok_val) : false;
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
1505 }
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
1506
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
1507 void
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
1508 lexical_feedback::mark_previous_token_trailing_space (void)
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
1509 {
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
1510 token *tok = tokens.front ();
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
1511 if (tok && ! previous_token_value_is ('\n'))
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
1512 tok->mark_trailing_space ();
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
1513 }
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
1514
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
1515 bool
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
1516 lexical_feedback::space_follows_previous_token (void) const
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
1517 {
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
1518 const token *tok = tokens.front ();
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
1519 return tok ? tok->space_follows_token () : false;
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
1520 }
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
1521
16257
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
1522 bool
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
1523 lexical_feedback::previous_token_is_binop (void) const
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
1524 {
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
1525 int tok = previous_token_value ();
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
1526
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
1527 return (tok == '+' || tok == '-' || tok == '@'
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
1528 || tok == ',' || tok == ';' || tok == '*' || tok == '/'
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
1529 || tok == ':' || tok == '=' || tok == ADD_EQ
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
1530 || tok == AND_EQ || tok == DIV_EQ || tok == EDIV
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
1531 || tok == EDIV_EQ || tok == ELEFTDIV || tok == ELEFTDIV_EQ
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
1532 || tok == EMINUS || tok == EMUL || tok == EMUL_EQ
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
1533 || tok == EPOW || tok == EPOW_EQ || tok == EXPR_AND
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
1534 || tok == EXPR_AND_AND || tok == EXPR_EQ || tok == EXPR_GE
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
1535 || tok == EXPR_GT || tok == EXPR_LE || tok == EXPR_LT
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
1536 || tok == EXPR_NE || tok == EXPR_NOT || tok == EXPR_OR
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
1537 || tok == EXPR_OR_OR || tok == LEFTDIV || tok == LEFTDIV_EQ
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
1538 || tok == LSHIFT || tok == LSHIFT_EQ || tok == MUL_EQ
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
1539 || tok == OR_EQ || tok == POW || tok == POW_EQ
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
1540 || tok == RSHIFT || tok == RSHIFT_EQ || tok == SUB_EQ);
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
1541 }
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
1542
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
1543 bool
16267
15f55df088e7 6/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16265
diff changeset
1544 lexical_feedback::previous_token_is_keyword (void) const
15f55df088e7 6/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16265
diff changeset
1545 {
15f55df088e7 6/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16265
diff changeset
1546 const token *tok = tokens.front ();
15f55df088e7 6/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16265
diff changeset
1547 return tok ? tok->is_keyword () : false;
15f55df088e7 6/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16265
diff changeset
1548 }
15f55df088e7 6/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16265
diff changeset
1549
15f55df088e7 6/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16265
diff changeset
1550 bool
16257
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
1551 lexical_feedback::previous_token_may_be_command (void) const
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
1552 {
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
1553 const token *tok = tokens.front ();
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
1554 return tok ? tok->may_be_command () : false;
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
1555 }
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
1556
16228
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
1557 static bool
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
1558 looks_like_copyright (const std::string& s)
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
1559 {
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
1560 bool retval = false;
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
1561
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
1562 if (! s.empty ())
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
1563 {
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
1564 size_t offset = s.find_first_not_of (" \t");
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
1565
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
1566 retval = (s.substr (offset, 9) == "Copyright" || s.substr (offset, 6) == "Author");
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
1567 }
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
1568
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
1569 return retval;
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
1570 }
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
1571
16194
b7ca669af528 reset lexical_feedback state before starting a new parse
John W. Eaton <jwe@octave.org>
parents: 16193
diff changeset
1572 void
16195
b52d2f9294b6 use class for reading lexer input
John W. Eaton <jwe@octave.org>
parents: 16194
diff changeset
1573 octave_lexer::input_buffer::fill (const std::string& input, bool eof_arg)
16183
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16166
diff changeset
1574 {
16195
b52d2f9294b6 use class for reading lexer input
John W. Eaton <jwe@octave.org>
parents: 16194
diff changeset
1575 buffer = input;
16183
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16166
diff changeset
1576 chars_left = buffer.length ();
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16166
diff changeset
1577 pos = buffer.c_str ();
16195
b52d2f9294b6 use class for reading lexer input
John W. Eaton <jwe@octave.org>
parents: 16194
diff changeset
1578 eof = eof_arg;
16183
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16166
diff changeset
1579 }
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16166
diff changeset
1580
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16166
diff changeset
1581 int
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16166
diff changeset
1582 octave_lexer::input_buffer::copy_chunk (char *buf, size_t max_size)
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16166
diff changeset
1583 {
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16166
diff changeset
1584 static const char * const eol = "\n";
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16166
diff changeset
1585
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16166
diff changeset
1586 size_t len = max_size > chars_left ? chars_left : max_size;
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16166
diff changeset
1587 assert (len > 0);
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16166
diff changeset
1588
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16166
diff changeset
1589 memcpy (buf, pos, len);
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16166
diff changeset
1590
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16166
diff changeset
1591 chars_left -= len;
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16166
diff changeset
1592 pos += len;
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16166
diff changeset
1593
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16166
diff changeset
1594 // Make sure input ends with a new line character.
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16166
diff changeset
1595 if (chars_left == 0 && buf[len-1] != '\n')
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16166
diff changeset
1596 {
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16166
diff changeset
1597 if (len < max_size)
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16166
diff changeset
1598 {
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16166
diff changeset
1599 // There is enough room to plug the newline character in
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16166
diff changeset
1600 // the buffer.
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16166
diff changeset
1601 buf[len++] = '\n';
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16166
diff changeset
1602 }
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16166
diff changeset
1603 else
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16166
diff changeset
1604 {
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16166
diff changeset
1605 // There isn't enough room to plug the newline character
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16166
diff changeset
1606 // in the buffer so arrange to have it returned on the next
16195
b52d2f9294b6 use class for reading lexer input
John W. Eaton <jwe@octave.org>
parents: 16194
diff changeset
1607 // call to octave_lexer::read.
16183
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16166
diff changeset
1608 pos = eol;
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16166
diff changeset
1609 chars_left = 1;
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16166
diff changeset
1610 }
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16166
diff changeset
1611 }
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16166
diff changeset
1612
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16166
diff changeset
1613 return len;
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16166
diff changeset
1614 }
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16166
diff changeset
1615
16158
7eb614760ddb rename lexical_feedback to octave_lexer
John W. Eaton <jwe@octave.org>
parents: 16154
diff changeset
1616 octave_lexer::~octave_lexer (void)
16114
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1617 {
16149
49dfba4fd3c5 use pure parser and reentrant lexer interfaces
John W. Eaton <jwe@octave.org>
parents: 16148
diff changeset
1618 yylex_destroy (scanner);
16114
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1619 }
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1620
16123
a484e39d1f22 maint: move function definition in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16122
diff changeset
1621 void
16158
7eb614760ddb rename lexical_feedback to octave_lexer
John W. Eaton <jwe@octave.org>
parents: 16154
diff changeset
1622 octave_lexer::init (void)
16149
49dfba4fd3c5 use pure parser and reentrant lexer interfaces
John W. Eaton <jwe@octave.org>
parents: 16148
diff changeset
1623 {
49dfba4fd3c5 use pure parser and reentrant lexer interfaces
John W. Eaton <jwe@octave.org>
parents: 16148
diff changeset
1624 yylex_init (&scanner);
49dfba4fd3c5 use pure parser and reentrant lexer interfaces
John W. Eaton <jwe@octave.org>
parents: 16148
diff changeset
1625
16158
7eb614760ddb rename lexical_feedback to octave_lexer
John W. Eaton <jwe@octave.org>
parents: 16154
diff changeset
1626 // Make octave_lexer object available through yyextra in
16149
49dfba4fd3c5 use pure parser and reentrant lexer interfaces
John W. Eaton <jwe@octave.org>
parents: 16148
diff changeset
1627 // flex-generated lexer.
49dfba4fd3c5 use pure parser and reentrant lexer interfaces
John W. Eaton <jwe@octave.org>
parents: 16148
diff changeset
1628 yyset_extra (this, scanner);
16224
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
1629
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
1630 clear_start_state ();
16149
49dfba4fd3c5 use pure parser and reentrant lexer interfaces
John W. Eaton <jwe@octave.org>
parents: 16148
diff changeset
1631 }
49dfba4fd3c5 use pure parser and reentrant lexer interfaces
John W. Eaton <jwe@octave.org>
parents: 16148
diff changeset
1632
49dfba4fd3c5 use pure parser and reentrant lexer interfaces
John W. Eaton <jwe@octave.org>
parents: 16148
diff changeset
1633 // Inside Flex-generated functions, yyg is the scanner cast to its real
16224
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
1634 // type. Some flex macros that we use in octave_lexer member functions
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
1635 // (for example, BEGIN) use yyg. If we could perform the actions of
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
1636 // these macros with functions instead, we could eliminate the
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
1637 // OCTAVE_YYG macro.
16149
49dfba4fd3c5 use pure parser and reentrant lexer interfaces
John W. Eaton <jwe@octave.org>
parents: 16148
diff changeset
1638
49dfba4fd3c5 use pure parser and reentrant lexer interfaces
John W. Eaton <jwe@octave.org>
parents: 16148
diff changeset
1639 #define OCTAVE_YYG \
49dfba4fd3c5 use pure parser and reentrant lexer interfaces
John W. Eaton <jwe@octave.org>
parents: 16148
diff changeset
1640 struct yyguts_t *yyg = static_cast<struct yyguts_t*> (scanner)
49dfba4fd3c5 use pure parser and reentrant lexer interfaces
John W. Eaton <jwe@octave.org>
parents: 16148
diff changeset
1641
49dfba4fd3c5 use pure parser and reentrant lexer interfaces
John W. Eaton <jwe@octave.org>
parents: 16148
diff changeset
1642 void
16158
7eb614760ddb rename lexical_feedback to octave_lexer
John W. Eaton <jwe@octave.org>
parents: 16154
diff changeset
1643 octave_lexer::reset (void)
16123
a484e39d1f22 maint: move function definition in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16122
diff changeset
1644 {
a484e39d1f22 maint: move function definition in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16122
diff changeset
1645 // Start off on the right foot.
16224
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
1646 clear_start_state ();
16123
a484e39d1f22 maint: move function definition in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16122
diff changeset
1647
a484e39d1f22 maint: move function definition in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16122
diff changeset
1648 parser_symtab_context.clear ();
a484e39d1f22 maint: move function definition in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16122
diff changeset
1649
a484e39d1f22 maint: move function definition in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16122
diff changeset
1650 // We do want a prompt by default.
16209
e7ff32e7cf82 move global promptflag variable to octave_reader class
John W. Eaton <jwe@octave.org>
parents: 16207
diff changeset
1651 promptflag (1);
16123
a484e39d1f22 maint: move function definition in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16122
diff changeset
1652
a484e39d1f22 maint: move function definition in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16122
diff changeset
1653 // Only ask for input from stdin if we are expecting interactive
a484e39d1f22 maint: move function definition in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16122
diff changeset
1654 // input.
a484e39d1f22 maint: move function definition in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16122
diff changeset
1655
a484e39d1f22 maint: move function definition in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16122
diff changeset
1656 if (! quitting_gracefully
a484e39d1f22 maint: move function definition in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16122
diff changeset
1657 && (interactive || forced_interactive)
a484e39d1f22 maint: move function definition in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16122
diff changeset
1658 && ! (reading_fcn_file
a484e39d1f22 maint: move function definition in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16122
diff changeset
1659 || reading_classdef_file
a484e39d1f22 maint: move function definition in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16122
diff changeset
1660 || reading_script_file
16203
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
1661 || input_from_eval_string ()))
16149
49dfba4fd3c5 use pure parser and reentrant lexer interfaces
John W. Eaton <jwe@octave.org>
parents: 16148
diff changeset
1662 yyrestart (stdin, scanner);
16123
a484e39d1f22 maint: move function definition in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16122
diff changeset
1663
16209
e7ff32e7cf82 move global promptflag variable to octave_reader class
John W. Eaton <jwe@octave.org>
parents: 16207
diff changeset
1664 input_reader.reset ();
e7ff32e7cf82 move global promptflag variable to octave_reader class
John W. Eaton <jwe@octave.org>
parents: 16207
diff changeset
1665
16194
b7ca669af528 reset lexical_feedback state before starting a new parse
John W. Eaton <jwe@octave.org>
parents: 16193
diff changeset
1666 lexical_feedback::reset ();
16123
a484e39d1f22 maint: move function definition in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16122
diff changeset
1667 }
a484e39d1f22 maint: move function definition in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16122
diff changeset
1668
16124
3be725cd195b move more lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16123
diff changeset
1669 void
16203
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
1670 octave_lexer::prep_for_file (void)
16124
3be725cd195b move more lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16123
diff changeset
1671 {
16203
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
1672 reading_script_file = true;
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
1673
16224
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
1674 push_start_state (INPUT_FILE_START);
16124
3be725cd195b move more lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16123
diff changeset
1675 }
3be725cd195b move more lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16123
diff changeset
1676
16121
bdf365c5c9bf maint: move function definition in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16120
diff changeset
1677 int
16195
b52d2f9294b6 use class for reading lexer input
John W. Eaton <jwe@octave.org>
parents: 16194
diff changeset
1678 octave_lexer::read (char *buf, unsigned max_size)
16121
bdf365c5c9bf maint: move function definition in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16120
diff changeset
1679 {
bdf365c5c9bf maint: move function definition in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16120
diff changeset
1680 int status = 0;
bdf365c5c9bf maint: move function definition in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16120
diff changeset
1681
16183
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16166
diff changeset
1682 if (input_buf.empty ())
16195
b52d2f9294b6 use class for reading lexer input
John W. Eaton <jwe@octave.org>
parents: 16194
diff changeset
1683 {
b52d2f9294b6 use class for reading lexer input
John W. Eaton <jwe@octave.org>
parents: 16194
diff changeset
1684 bool eof = false;
16207
0467d68ca891 move current_input_line to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16203
diff changeset
1685 current_input_line = input_reader.get_input (eof);
0467d68ca891 move current_input_line to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16203
diff changeset
1686 input_buf.fill (current_input_line, eof);
16195
b52d2f9294b6 use class for reading lexer input
John W. Eaton <jwe@octave.org>
parents: 16194
diff changeset
1687 }
16183
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16166
diff changeset
1688
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16166
diff changeset
1689 if (! input_buf.empty ())
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16166
diff changeset
1690 status = input_buf.copy_chunk (buf, max_size);
16121
bdf365c5c9bf maint: move function definition in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16120
diff changeset
1691 else
bdf365c5c9bf maint: move function definition in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16120
diff changeset
1692 {
bdf365c5c9bf maint: move function definition in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16120
diff changeset
1693 status = YY_NULL;
bdf365c5c9bf maint: move function definition in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16120
diff changeset
1694
16183
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16166
diff changeset
1695 if (! input_buf.at_eof ())
16195
b52d2f9294b6 use class for reading lexer input
John W. Eaton <jwe@octave.org>
parents: 16194
diff changeset
1696 fatal_error ("octave_lexer::read () in flex scanner failed");
16121
bdf365c5c9bf maint: move function definition in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16120
diff changeset
1697 }
bdf365c5c9bf maint: move function definition in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16120
diff changeset
1698
bdf365c5c9bf maint: move function definition in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16120
diff changeset
1699 return status;
bdf365c5c9bf maint: move function definition in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16120
diff changeset
1700 }
bdf365c5c9bf maint: move function definition in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16120
diff changeset
1701
16164
c5bfdc4c0963 move end_of_input flag from octve_parser class to octave_lexer class
John W. Eaton <jwe@octave.org>
parents: 16158
diff changeset
1702 int
c5bfdc4c0963 move end_of_input flag from octve_parser class to octave_lexer class
John W. Eaton <jwe@octave.org>
parents: 16158
diff changeset
1703 octave_lexer::handle_end_of_input (void)
c5bfdc4c0963 move end_of_input flag from octve_parser class to octave_lexer class
John W. Eaton <jwe@octave.org>
parents: 16158
diff changeset
1704 {
16229
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
1705 lexer_debug ("<<EOF>>");
16164
c5bfdc4c0963 move end_of_input flag from octve_parser class to octave_lexer class
John W. Eaton <jwe@octave.org>
parents: 16158
diff changeset
1706
c5bfdc4c0963 move end_of_input flag from octve_parser class to octave_lexer class
John W. Eaton <jwe@octave.org>
parents: 16158
diff changeset
1707 if (block_comment_nesting_level != 0)
c5bfdc4c0963 move end_of_input flag from octve_parser class to octave_lexer class
John W. Eaton <jwe@octave.org>
parents: 16158
diff changeset
1708 {
c5bfdc4c0963 move end_of_input flag from octve_parser class to octave_lexer class
John W. Eaton <jwe@octave.org>
parents: 16158
diff changeset
1709 warning ("block comment open at end of input");
c5bfdc4c0963 move end_of_input flag from octve_parser class to octave_lexer class
John W. Eaton <jwe@octave.org>
parents: 16158
diff changeset
1710
c5bfdc4c0963 move end_of_input flag from octve_parser class to octave_lexer class
John W. Eaton <jwe@octave.org>
parents: 16158
diff changeset
1711 if ((reading_fcn_file || reading_script_file || reading_classdef_file)
16203
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
1712 && ! fcn_file_name.empty ())
16164
c5bfdc4c0963 move end_of_input flag from octve_parser class to octave_lexer class
John W. Eaton <jwe@octave.org>
parents: 16158
diff changeset
1713 warning ("near line %d of file '%s.m'",
16203
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
1714 input_line_number, fcn_file_name.c_str ());
16164
c5bfdc4c0963 move end_of_input flag from octve_parser class to octave_lexer class
John W. Eaton <jwe@octave.org>
parents: 16158
diff changeset
1715 }
c5bfdc4c0963 move end_of_input flag from octve_parser class to octave_lexer class
John W. Eaton <jwe@octave.org>
parents: 16158
diff changeset
1716
16229
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
1717 return handle_token (END_OF_INPUT);
16164
c5bfdc4c0963 move end_of_input flag from octve_parser class to octave_lexer class
John W. Eaton <jwe@octave.org>
parents: 16158
diff changeset
1718 }
c5bfdc4c0963 move end_of_input flag from octve_parser class to octave_lexer class
John W. Eaton <jwe@octave.org>
parents: 16158
diff changeset
1719
16128
210039e91ad6 localize use of yytext with lexical_feedback member function
John W. Eaton <jwe@octave.org>
parents: 16126
diff changeset
1720 char *
16158
7eb614760ddb rename lexical_feedback to octave_lexer
John W. Eaton <jwe@octave.org>
parents: 16154
diff changeset
1721 octave_lexer::flex_yytext (void)
16128
210039e91ad6 localize use of yytext with lexical_feedback member function
John W. Eaton <jwe@octave.org>
parents: 16126
diff changeset
1722 {
16149
49dfba4fd3c5 use pure parser and reentrant lexer interfaces
John W. Eaton <jwe@octave.org>
parents: 16148
diff changeset
1723 return yyget_text (scanner);
16128
210039e91ad6 localize use of yytext with lexical_feedback member function
John W. Eaton <jwe@octave.org>
parents: 16126
diff changeset
1724 }
210039e91ad6 localize use of yytext with lexical_feedback member function
John W. Eaton <jwe@octave.org>
parents: 16126
diff changeset
1725
16130
421dea028bbf localize use of yyleng with lexical_feedback member function
John W. Eaton <jwe@octave.org>
parents: 16129
diff changeset
1726 int
16158
7eb614760ddb rename lexical_feedback to octave_lexer
John W. Eaton <jwe@octave.org>
parents: 16154
diff changeset
1727 octave_lexer::flex_yyleng (void)
16130
421dea028bbf localize use of yyleng with lexical_feedback member function
John W. Eaton <jwe@octave.org>
parents: 16129
diff changeset
1728 {
16149
49dfba4fd3c5 use pure parser and reentrant lexer interfaces
John W. Eaton <jwe@octave.org>
parents: 16148
diff changeset
1729 return yyget_leng (scanner);
16130
421dea028bbf localize use of yyleng with lexical_feedback member function
John W. Eaton <jwe@octave.org>
parents: 16129
diff changeset
1730 }
421dea028bbf localize use of yyleng with lexical_feedback member function
John W. Eaton <jwe@octave.org>
parents: 16129
diff changeset
1731
16114
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1732 int
16158
7eb614760ddb rename lexical_feedback to octave_lexer
John W. Eaton <jwe@octave.org>
parents: 16154
diff changeset
1733 octave_lexer::text_yyinput (void)
16114
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1734 {
16149
49dfba4fd3c5 use pure parser and reentrant lexer interfaces
John W. Eaton <jwe@octave.org>
parents: 16148
diff changeset
1735 int c = yyinput (scanner);
16114
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1736
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1737 if (lexer_debug_flag)
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1738 {
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1739 std::cerr << "I: ";
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1740 display_character (c);
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1741 std::cerr << std::endl;
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1742 }
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1743
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1744 // Convert CRLF into just LF and single CR into LF.
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1745
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1746 if (c == '\r')
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1747 {
16149
49dfba4fd3c5 use pure parser and reentrant lexer interfaces
John W. Eaton <jwe@octave.org>
parents: 16148
diff changeset
1748 c = yyinput (scanner);
16114
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1749
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1750 if (lexer_debug_flag)
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1751 {
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1752 std::cerr << "I: ";
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1753 display_character (c);
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1754 std::cerr << std::endl;
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1755 }
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1756
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1757 if (c != '\n')
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1758 {
16125
96a58f197f93 allow xunput to be called without buffer argument
John W. Eaton <jwe@octave.org>
parents: 16124
diff changeset
1759 xunput (c);
16114
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1760 c = '\n';
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1761 }
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1762 }
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1763
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1764 if (c == '\n')
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1765 input_line_number++;
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1766
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1767 return c;
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1768 }
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1769
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1770 void
16158
7eb614760ddb rename lexical_feedback to octave_lexer
John W. Eaton <jwe@octave.org>
parents: 16154
diff changeset
1771 octave_lexer::xunput (char c, char *buf)
16114
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1772 {
16164
c5bfdc4c0963 move end_of_input flag from octve_parser class to octave_lexer class
John W. Eaton <jwe@octave.org>
parents: 16158
diff changeset
1773 if (c != EOF)
16114
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1774 {
16164
c5bfdc4c0963 move end_of_input flag from octve_parser class to octave_lexer class
John W. Eaton <jwe@octave.org>
parents: 16158
diff changeset
1775 if (lexer_debug_flag)
c5bfdc4c0963 move end_of_input flag from octve_parser class to octave_lexer class
John W. Eaton <jwe@octave.org>
parents: 16158
diff changeset
1776 {
c5bfdc4c0963 move end_of_input flag from octve_parser class to octave_lexer class
John W. Eaton <jwe@octave.org>
parents: 16158
diff changeset
1777 std::cerr << "U: ";
c5bfdc4c0963 move end_of_input flag from octve_parser class to octave_lexer class
John W. Eaton <jwe@octave.org>
parents: 16158
diff changeset
1778 display_character (c);
c5bfdc4c0963 move end_of_input flag from octve_parser class to octave_lexer class
John W. Eaton <jwe@octave.org>
parents: 16158
diff changeset
1779 std::cerr << std::endl;
c5bfdc4c0963 move end_of_input flag from octve_parser class to octave_lexer class
John W. Eaton <jwe@octave.org>
parents: 16158
diff changeset
1780 }
c5bfdc4c0963 move end_of_input flag from octve_parser class to octave_lexer class
John W. Eaton <jwe@octave.org>
parents: 16158
diff changeset
1781
c5bfdc4c0963 move end_of_input flag from octve_parser class to octave_lexer class
John W. Eaton <jwe@octave.org>
parents: 16158
diff changeset
1782 if (c == '\n')
c5bfdc4c0963 move end_of_input flag from octve_parser class to octave_lexer class
John W. Eaton <jwe@octave.org>
parents: 16158
diff changeset
1783 input_line_number--;
c5bfdc4c0963 move end_of_input flag from octve_parser class to octave_lexer class
John W. Eaton <jwe@octave.org>
parents: 16158
diff changeset
1784
c5bfdc4c0963 move end_of_input flag from octve_parser class to octave_lexer class
John W. Eaton <jwe@octave.org>
parents: 16158
diff changeset
1785 yyunput (c, buf, scanner);
16114
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1786 }
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1787 }
73a21ade0b6b * lex.ll: Reorder function definitions.
John W. Eaton <jwe@octave.org>
parents: 16113
diff changeset
1788
16125
96a58f197f93 allow xunput to be called without buffer argument
John W. Eaton <jwe@octave.org>
parents: 16124
diff changeset
1789 void
16158
7eb614760ddb rename lexical_feedback to octave_lexer
John W. Eaton <jwe@octave.org>
parents: 16154
diff changeset
1790 octave_lexer::xunput (char c)
16125
96a58f197f93 allow xunput to be called without buffer argument
John W. Eaton <jwe@octave.org>
parents: 16124
diff changeset
1791 {
16128
210039e91ad6 localize use of yytext with lexical_feedback member function
John W. Eaton <jwe@octave.org>
parents: 16126
diff changeset
1792 char *yytxt = flex_yytext ();
16149
49dfba4fd3c5 use pure parser and reentrant lexer interfaces
John W. Eaton <jwe@octave.org>
parents: 16148
diff changeset
1793
16128
210039e91ad6 localize use of yytext with lexical_feedback member function
John W. Eaton <jwe@octave.org>
parents: 16126
diff changeset
1794 xunput (c, yytxt);
16125
96a58f197f93 allow xunput to be called without buffer argument
John W. Eaton <jwe@octave.org>
parents: 16124
diff changeset
1795 }
96a58f197f93 allow xunput to be called without buffer argument
John W. Eaton <jwe@octave.org>
parents: 16124
diff changeset
1796
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
1797 bool
16275
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
1798 octave_lexer::looking_at_space (void)
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
1799 {
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
1800 int c = text_yyinput ();
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
1801 xunput (c);
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
1802 return (c == ' ' || c == '\t');
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
1803 }
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
1804
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
1805 bool
16158
7eb614760ddb rename lexical_feedback to octave_lexer
John W. Eaton <jwe@octave.org>
parents: 16154
diff changeset
1806 octave_lexer::inside_any_object_index (void)
8745
6dc61981d18b better handling of object indexing in lexer
John W. Eaton <jwe@octave.org>
parents: 8701
diff changeset
1807 {
6dc61981d18b better handling of object indexing in lexer
John W. Eaton <jwe@octave.org>
parents: 8701
diff changeset
1808 bool retval = false;
6dc61981d18b better handling of object indexing in lexer
John W. Eaton <jwe@octave.org>
parents: 8701
diff changeset
1809
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
1810 for (std::list<bool>::const_iterator i = looking_at_object_index.begin ();
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
1811 i != looking_at_object_index.end (); i++)
8745
6dc61981d18b better handling of object indexing in lexer
John W. Eaton <jwe@octave.org>
parents: 8701
diff changeset
1812 {
6dc61981d18b better handling of object indexing in lexer
John W. Eaton <jwe@octave.org>
parents: 8701
diff changeset
1813 if (*i)
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1814 {
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1815 retval = true;
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1816 break;
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1817 }
8745
6dc61981d18b better handling of object indexing in lexer
John W. Eaton <jwe@octave.org>
parents: 8701
diff changeset
1818 }
6dc61981d18b better handling of object indexing in lexer
John W. Eaton <jwe@octave.org>
parents: 8701
diff changeset
1819
6dc61981d18b better handling of object indexing in lexer
John W. Eaton <jwe@octave.org>
parents: 8701
diff changeset
1820 return retval;
6dc61981d18b better handling of object indexing in lexer
John W. Eaton <jwe@octave.org>
parents: 8701
diff changeset
1821 }
6dc61981d18b better handling of object indexing in lexer
John W. Eaton <jwe@octave.org>
parents: 8701
diff changeset
1822
4238
a5a68c0afe56 [project @ 2002-12-25 21:04:33 by jwe]
jwe
parents: 4237
diff changeset
1823 // Handle keywords. Return -1 if the keyword should be ignored.
767
42731861ee09 [project @ 1994-10-05 21:26:54 by jwe]
jwe
parents: 747
diff changeset
1824
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
1825 int
16158
7eb614760ddb rename lexical_feedback to octave_lexer
John W. Eaton <jwe@octave.org>
parents: 16154
diff changeset
1826 octave_lexer::is_keyword_token (const std::string& s)
1
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
1827 {
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
1828 int l = input_line_number;
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
1829 int c = current_input_column;
3805
44386b0e53da [project @ 2001-03-01 16:54:31 by jwe]
jwe
parents: 3802
diff changeset
1830
1823
5cdd59e7579a [project @ 1996-02-01 12:45:05 by jwe]
jwe
parents: 1755
diff changeset
1831 int len = s.length ();
922
708827b941b4 [project @ 1994-11-14 22:14:13 by jwe]
jwe
parents: 916
diff changeset
1832
5088
7830f271a53f [project @ 2004-12-01 19:54:45 by jwe]
jwe
parents: 5035
diff changeset
1833 const octave_kw *kw = octave_kw_hash::in_word_set (s.c_str (), len);
191
b6b4d8c513fe [project @ 1993-10-29 23:41:39 by jwe]
jwe
parents: 186
diff changeset
1834
1497
48a0b289f1be [project @ 1995-09-30 22:04:27 by jwe]
jwe
parents: 1489
diff changeset
1835 if (kw)
143
7849db4b6dbc [project @ 1993-10-04 02:36:45 by jwe]
jwe
parents: 120
diff changeset
1836 {
16126
5c885c13bfa3 localize use of yylval.tok_val in lexical_feedback member functions
John W. Eaton <jwe@octave.org>
parents: 16125
diff changeset
1837 token *tok_val = 0;
1497
48a0b289f1be [project @ 1995-09-30 22:04:27 by jwe]
jwe
parents: 1489
diff changeset
1838
48a0b289f1be [project @ 1995-09-30 22:04:27 by jwe]
jwe
parents: 1489
diff changeset
1839 switch (kw->kw_id)
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1840 {
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1841 case break_kw:
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1842 case catch_kw:
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1843 case continue_kw:
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1844 case else_kw:
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1845 case otherwise_kw:
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1846 case return_kw:
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1847 case unwind_protect_cleanup_kw:
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
1848 at_beginning_of_statement = true;
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1849 break;
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1850
14294
9e3983c8963c deprecate the static keyword
John W. Eaton <jwe@octave.org>
parents: 14292
diff changeset
1851 case static_kw:
9e3983c8963c deprecate the static keyword
John W. Eaton <jwe@octave.org>
parents: 14292
diff changeset
1852 if ((reading_fcn_file || reading_script_file
9e3983c8963c deprecate the static keyword
John W. Eaton <jwe@octave.org>
parents: 14292
diff changeset
1853 || reading_classdef_file)
16203
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
1854 && ! fcn_file_full_name.empty ())
14294
9e3983c8963c deprecate the static keyword
John W. Eaton <jwe@octave.org>
parents: 14292
diff changeset
1855 warning_with_id ("Octave:deprecated-keyword",
15468
6437fa7263dd use ' instead of ` in error messages, warnings and most comments
John W. Eaton <jwe@octave.org>
parents: 15467
diff changeset
1856 "the 'static' keyword is obsolete and will be removed from a future version of Octave; please use 'persistent' instead; near line %d of file '%s'",
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
1857 input_line_number,
16203
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
1858 fcn_file_full_name.c_str ());
14294
9e3983c8963c deprecate the static keyword
John W. Eaton <jwe@octave.org>
parents: 14292
diff changeset
1859 else
9e3983c8963c deprecate the static keyword
John W. Eaton <jwe@octave.org>
parents: 14292
diff changeset
1860 warning_with_id ("Octave:deprecated-keyword",
15468
6437fa7263dd use ' instead of ` in error messages, warnings and most comments
John W. Eaton <jwe@octave.org>
parents: 15467
diff changeset
1861 "the 'static' keyword is obsolete and will be removed from a future version of Octave; please use 'persistent' instead; near line %d",
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
1862 input_line_number);
14294
9e3983c8963c deprecate the static keyword
John W. Eaton <jwe@octave.org>
parents: 14292
diff changeset
1863 // fall through ...
9e3983c8963c deprecate the static keyword
John W. Eaton <jwe@octave.org>
parents: 14292
diff changeset
1864
9e3983c8963c deprecate the static keyword
John W. Eaton <jwe@octave.org>
parents: 14292
diff changeset
1865 case persistent_kw:
16263
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
1866 case global_kw:
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
1867 looking_at_decl_list = true;
14294
9e3983c8963c deprecate the static keyword
John W. Eaton <jwe@octave.org>
parents: 14292
diff changeset
1868 break;
9e3983c8963c deprecate the static keyword
John W. Eaton <jwe@octave.org>
parents: 14292
diff changeset
1869
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1870 case case_kw:
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1871 case elseif_kw:
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1872 case until_kw:
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1873 break;
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1874
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1875 case end_kw:
14645
cbc00b587f36 also allow "end" indexing in classdef files
John W. Eaton <jwe@octave.org>
parents: 14642
diff changeset
1876 if (inside_any_object_index ()
cbc00b587f36 also allow "end" indexing in classdef files
John W. Eaton <jwe@octave.org>
parents: 14642
diff changeset
1877 || (! reading_classdef_file
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
1878 && (defining_func
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
1879 && ! (looking_at_return_list
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
1880 || parsed_function_name.top ()))))
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1881 return 0;
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1882
16255
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
1883 tok_val = new token (end_kw, token::simple_end, l, c);
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
1884 at_beginning_of_statement = true;
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1885 break;
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1886
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1887 case end_try_catch_kw:
16255
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
1888 tok_val = new token (end_try_catch_kw, token::try_catch_end, l, c);
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
1889 at_beginning_of_statement = true;
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1890 break;
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1891
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1892 case end_unwind_protect_kw:
16255
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
1893 tok_val = new token (end_unwind_protect_kw,
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
1894 token::unwind_protect_end, l, c);
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
1895 at_beginning_of_statement = true;
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1896 break;
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1897
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1898 case endfor_kw:
16255
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
1899 tok_val = new token (endfor_kw, token::for_end, l, c);
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
1900 at_beginning_of_statement = true;
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1901 break;
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1902
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1903 case endfunction_kw:
16255
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
1904 tok_val = new token (endfunction_kw, token::function_end, l, c);
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
1905 at_beginning_of_statement = true;
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1906 break;
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1907
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1908 case endif_kw:
16255
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
1909 tok_val = new token (endif_kw, token::if_end, l, c);
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
1910 at_beginning_of_statement = true;
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1911 break;
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1912
13246
0c69a564f2be additional parfor changes
John W. Eaton <jwe@octave.org>
parents: 13245
diff changeset
1913 case endparfor_kw:
16255
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
1914 tok_val = new token (endparfor_kw, token::parfor_end, l, c);
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
1915 at_beginning_of_statement = true;
13246
0c69a564f2be additional parfor changes
John W. Eaton <jwe@octave.org>
parents: 13245
diff changeset
1916 break;
0c69a564f2be additional parfor changes
John W. Eaton <jwe@octave.org>
parents: 13245
diff changeset
1917
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1918 case endswitch_kw:
16255
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
1919 tok_val = new token (endswitch_kw, token::switch_end, l, c);
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
1920 at_beginning_of_statement = true;
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1921 break;
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1922
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1923 case endwhile_kw:
16255
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
1924 tok_val = new token (endwhile_kw, token::while_end, l, c);
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
1925 at_beginning_of_statement = true;
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1926 break;
11586
12df7854fa7c strip trailing whitespace from source files
John W. Eaton <jwe@octave.org>
parents: 11523
diff changeset
1927
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1928 case endclassdef_kw:
16255
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
1929 tok_val = new token (endclassdef_kw, token::classdef_end, l, c);
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
1930 at_beginning_of_statement = true;
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1931 break;
11586
12df7854fa7c strip trailing whitespace from source files
John W. Eaton <jwe@octave.org>
parents: 11523
diff changeset
1932
13249
7861a5fd3479 accept enumeration keyword
John W. Eaton <jwe@octave.org>
parents: 13246
diff changeset
1933 case endenumeration_kw:
16267
15f55df088e7 6/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16265
diff changeset
1934 tok_val = new token (endenumeration_kw, token::enumeration_end, l, c);
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
1935 at_beginning_of_statement = true;
13249
7861a5fd3479 accept enumeration keyword
John W. Eaton <jwe@octave.org>
parents: 13246
diff changeset
1936 break;
7861a5fd3479 accept enumeration keyword
John W. Eaton <jwe@octave.org>
parents: 13246
diff changeset
1937
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1938 case endevents_kw:
16255
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
1939 tok_val = new token (endevents_kw, token::events_end, l, c);
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
1940 at_beginning_of_statement = true;
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1941 break;
11586
12df7854fa7c strip trailing whitespace from source files
John W. Eaton <jwe@octave.org>
parents: 11523
diff changeset
1942
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1943 case endmethods_kw:
16255
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
1944 tok_val = new token (endmethods_kw, token::methods_end, l, c);
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
1945 at_beginning_of_statement = true;
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1946 break;
11586
12df7854fa7c strip trailing whitespace from source files
John W. Eaton <jwe@octave.org>
parents: 11523
diff changeset
1947
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1948 case endproperties_kw:
16255
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
1949 tok_val = new token (endproperties_kw, token::properties_end, l, c);
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
1950 at_beginning_of_statement = true;
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1951 break;
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1952
13245
027a2186cd90 parfor keyword and infrastructure, but handle parfor as normal for loop for now
John W. Eaton <jwe@octave.org>
parents: 13237
diff changeset
1953
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1954 case for_kw:
13245
027a2186cd90 parfor keyword and infrastructure, but handle parfor as normal for loop for now
John W. Eaton <jwe@octave.org>
parents: 13237
diff changeset
1955 case parfor_kw:
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1956 case while_kw:
16209
e7ff32e7cf82 move global promptflag variable to octave_reader class
John W. Eaton <jwe@octave.org>
parents: 16207
diff changeset
1957 decrement_promptflag ();
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
1958 looping++;
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1959 break;
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1960
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1961 case do_kw:
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
1962 at_beginning_of_statement = true;
16209
e7ff32e7cf82 move global promptflag variable to octave_reader class
John W. Eaton <jwe@octave.org>
parents: 16207
diff changeset
1963 decrement_promptflag ();
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
1964 looping++;
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1965 break;
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1966
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1967 case try_kw:
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1968 case unwind_protect_kw:
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
1969 at_beginning_of_statement = true;
16209
e7ff32e7cf82 move global promptflag variable to octave_reader class
John W. Eaton <jwe@octave.org>
parents: 16207
diff changeset
1970 decrement_promptflag ();
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1971 break;
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1972
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1973 case if_kw:
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1974 case switch_kw:
16209
e7ff32e7cf82 move global promptflag variable to octave_reader class
John W. Eaton <jwe@octave.org>
parents: 16207
diff changeset
1975 decrement_promptflag ();
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1976 break;
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1977
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1978 case get_kw:
11586
12df7854fa7c strip trailing whitespace from source files
John W. Eaton <jwe@octave.org>
parents: 11523
diff changeset
1979 case set_kw:
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1980 // 'get' and 'set' are keywords in classdef method
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1981 // declarations.
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
1982 if (! maybe_classdef_get_set_method)
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1983 return 0;
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1984 break;
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1985
13249
7861a5fd3479 accept enumeration keyword
John W. Eaton <jwe@octave.org>
parents: 13246
diff changeset
1986 case enumeration_kw:
7861a5fd3479 accept enumeration keyword
John W. Eaton <jwe@octave.org>
parents: 13246
diff changeset
1987 case events_kw:
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1988 case methods_kw:
13249
7861a5fd3479 accept enumeration keyword
John W. Eaton <jwe@octave.org>
parents: 13246
diff changeset
1989 case properties_kw:
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1990 // 'properties', 'methods' and 'events' are keywords for
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1991 // classdef blocks.
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
1992 if (! parsing_classdef)
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1993 return 0;
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1994 // fall through ...
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1995
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1996 case classdef_kw:
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
1997 // 'classdef' is always a keyword.
16209
e7ff32e7cf82 move global promptflag variable to octave_reader class
John W. Eaton <jwe@octave.org>
parents: 16207
diff changeset
1998 decrement_promptflag ();
16203
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
1999
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
2000 if (! force_script && token_count == 0 && input_from_file ())
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
2001 {
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
2002 reading_classdef_file = true;
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
2003 reading_script_file = false;
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
2004 }
11586
12df7854fa7c strip trailing whitespace from source files
John W. Eaton <jwe@octave.org>
parents: 11523
diff changeset
2005 break;
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2006
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2007 case function_kw:
16209
e7ff32e7cf82 move global promptflag variable to octave_reader class
John W. Eaton <jwe@octave.org>
parents: 16207
diff changeset
2008 decrement_promptflag ();
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2009
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
2010 defining_func++;
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
2011 parsed_function_name.push (false);
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2012
16203
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
2013 if (! force_script && token_count == 0 && input_from_file ())
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
2014 {
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
2015 reading_fcn_file = true;
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
2016 reading_script_file = false;
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
2017 }
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
2018
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2019 if (! (reading_fcn_file || reading_script_file
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2020 || reading_classdef_file))
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
2021 input_line_number = 1;
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2022 break;
1497
48a0b289f1be [project @ 1995-09-30 22:04:27 by jwe]
jwe
parents: 1489
diff changeset
2023
3174
390d5e396682 [project @ 1998-05-04 16:36:17 by jwe]
jwe
parents: 3165
diff changeset
2024 case magic_file_kw:
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2025 {
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2026 if ((reading_fcn_file || reading_script_file
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2027 || reading_classdef_file)
16203
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
2028 && ! fcn_file_full_name.empty ())
16267
15f55df088e7 6/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16265
diff changeset
2029 tok_val = new token (magic_file_kw, true,
15f55df088e7 6/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16265
diff changeset
2030 fcn_file_full_name, l, c);
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2031 else
16255
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
2032 tok_val = new token (magic_file_kw, "stdin", l, c);
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2033 }
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2034 break;
3174
390d5e396682 [project @ 1998-05-04 16:36:17 by jwe]
jwe
parents: 3165
diff changeset
2035
390d5e396682 [project @ 1998-05-04 16:36:17 by jwe]
jwe
parents: 3165
diff changeset
2036 case magic_line_kw:
16255
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
2037 tok_val = new token (magic_line_kw, static_cast<double> (l),
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
2038 "", l, c);
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2039 break;
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2040
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2041 default:
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2042 panic_impossible ();
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2043 }
1497
48a0b289f1be [project @ 1995-09-30 22:04:27 by jwe]
jwe
parents: 1489
diff changeset
2044
16126
5c885c13bfa3 localize use of yylval.tok_val in lexical_feedback member functions
John W. Eaton <jwe@octave.org>
parents: 16125
diff changeset
2045 if (! tok_val)
16267
15f55df088e7 6/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16265
diff changeset
2046 tok_val = new token (kw->tok, true, l, c);
16126
5c885c13bfa3 localize use of yylval.tok_val in lexical_feedback member functions
John W. Eaton <jwe@octave.org>
parents: 16125
diff changeset
2047
16148
10abbc493f50 delete unnecessary X-> references inside class X member functions
John W. Eaton <jwe@octave.org>
parents: 16146
diff changeset
2048 push_token (tok_val);
1497
48a0b289f1be [project @ 1995-09-30 22:04:27 by jwe]
jwe
parents: 1489
diff changeset
2049
48a0b289f1be [project @ 1995-09-30 22:04:27 by jwe]
jwe
parents: 1489
diff changeset
2050 return kw->tok;
143
7849db4b6dbc [project @ 1993-10-04 02:36:45 by jwe]
jwe
parents: 120
diff changeset
2051 }
1
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
2052
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
2053 return 0;
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
2054 }
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
2055
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
2056 bool
16158
7eb614760ddb rename lexical_feedback to octave_lexer
John W. Eaton <jwe@octave.org>
parents: 16154
diff changeset
2057 octave_lexer::is_variable (const std::string& name)
2702
7a820bb7f406 [project @ 1997-02-19 19:22:34 by jwe]
jwe
parents: 2621
diff changeset
2058 {
8001
ff9e7873f8ea improve handling of command-style names in matrix_or_assign_lhs context
John W. Eaton <jwe@octave.org>
parents: 7898
diff changeset
2059 return (symbol_table::is_variable (name)
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
2060 || (pending_local_variables.find (name)
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
2061 != pending_local_variables.end ()));
2702
7a820bb7f406 [project @ 1997-02-19 19:22:34 by jwe]
jwe
parents: 2621
diff changeset
2062 }
7a820bb7f406 [project @ 1997-02-19 19:22:34 by jwe]
jwe
parents: 2621
diff changeset
2063
16257
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
2064 bool
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
2065 octave_lexer::whitespace_is_significant (void)
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
2066 {
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
2067 return (nesting_level.is_bracket ()
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
2068 || (nesting_level.is_brace ()
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
2069 && ! looking_at_object_index.front ()));
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
2070 }
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
2071
3220
3deb1105fbc1 [project @ 1998-11-19 00:06:30 by jwe]
jwe
parents: 3208
diff changeset
2072 static inline bool
3deb1105fbc1 [project @ 1998-11-19 00:06:30 by jwe]
jwe
parents: 3208
diff changeset
2073 looks_like_hex (const char *s, int len)
3deb1105fbc1 [project @ 1998-11-19 00:06:30 by jwe]
jwe
parents: 3208
diff changeset
2074 {
3deb1105fbc1 [project @ 1998-11-19 00:06:30 by jwe]
jwe
parents: 3208
diff changeset
2075 return (len > 2 && s[0] == '0' && (s[1] == 'x' || s[1] == 'X'));
3deb1105fbc1 [project @ 1998-11-19 00:06:30 by jwe]
jwe
parents: 3208
diff changeset
2076 }
3deb1105fbc1 [project @ 1998-11-19 00:06:30 by jwe]
jwe
parents: 3208
diff changeset
2077
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
2078 void
16158
7eb614760ddb rename lexical_feedback to octave_lexer
John W. Eaton <jwe@octave.org>
parents: 16154
diff changeset
2079 octave_lexer::handle_number (void)
972
3e25eb05b6c6 [project @ 1994-12-12 15:53:47 by jwe]
jwe
parents: 971
diff changeset
2080 {
3220
3deb1105fbc1 [project @ 1998-11-19 00:06:30 by jwe]
jwe
parents: 3208
diff changeset
2081 double value = 0.0;
3deb1105fbc1 [project @ 1998-11-19 00:06:30 by jwe]
jwe
parents: 3208
diff changeset
2082 int nread = 0;
3deb1105fbc1 [project @ 1998-11-19 00:06:30 by jwe]
jwe
parents: 3208
diff changeset
2083
16128
210039e91ad6 localize use of yytext with lexical_feedback member function
John W. Eaton <jwe@octave.org>
parents: 16126
diff changeset
2084 char *yytxt = flex_yytext ();
210039e91ad6 localize use of yytext with lexical_feedback member function
John W. Eaton <jwe@octave.org>
parents: 16126
diff changeset
2085
210039e91ad6 localize use of yytext with lexical_feedback member function
John W. Eaton <jwe@octave.org>
parents: 16126
diff changeset
2086 if (looks_like_hex (yytxt, strlen (yytxt)))
3220
3deb1105fbc1 [project @ 1998-11-19 00:06:30 by jwe]
jwe
parents: 3208
diff changeset
2087 {
3deb1105fbc1 [project @ 1998-11-19 00:06:30 by jwe]
jwe
parents: 3208
diff changeset
2088 unsigned long ival;
3598
0ae310231c46 [project @ 2000-02-19 08:07:08 by jwe]
jwe
parents: 3575
diff changeset
2089
16128
210039e91ad6 localize use of yytext with lexical_feedback member function
John W. Eaton <jwe@octave.org>
parents: 16126
diff changeset
2090 nread = sscanf (yytxt, "%lx", &ival);
3598
0ae310231c46 [project @ 2000-02-19 08:07:08 by jwe]
jwe
parents: 3575
diff changeset
2091
3220
3deb1105fbc1 [project @ 1998-11-19 00:06:30 by jwe]
jwe
parents: 3208
diff changeset
2092 value = static_cast<double> (ival);
3deb1105fbc1 [project @ 1998-11-19 00:06:30 by jwe]
jwe
parents: 3208
diff changeset
2093 }
3deb1105fbc1 [project @ 1998-11-19 00:06:30 by jwe]
jwe
parents: 3208
diff changeset
2094 else
3598
0ae310231c46 [project @ 2000-02-19 08:07:08 by jwe]
jwe
parents: 3575
diff changeset
2095 {
16128
210039e91ad6 localize use of yytext with lexical_feedback member function
John W. Eaton <jwe@octave.org>
parents: 16126
diff changeset
2096 char *tmp = strsave (yytxt);
3598
0ae310231c46 [project @ 2000-02-19 08:07:08 by jwe]
jwe
parents: 3575
diff changeset
2097
0ae310231c46 [project @ 2000-02-19 08:07:08 by jwe]
jwe
parents: 3575
diff changeset
2098 char *idx = strpbrk (tmp, "Dd");
2621
337a09dd1c06 [project @ 1997-01-24 21:49:41 by jwe]
jwe
parents: 2614
diff changeset
2099
3598
0ae310231c46 [project @ 2000-02-19 08:07:08 by jwe]
jwe
parents: 3575
diff changeset
2100 if (idx)
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2101 *idx = 'e';
3598
0ae310231c46 [project @ 2000-02-19 08:07:08 by jwe]
jwe
parents: 3575
diff changeset
2102
0ae310231c46 [project @ 2000-02-19 08:07:08 by jwe]
jwe
parents: 3575
diff changeset
2103 nread = sscanf (tmp, "%lf", &value);
0ae310231c46 [project @ 2000-02-19 08:07:08 by jwe]
jwe
parents: 3575
diff changeset
2104
0ae310231c46 [project @ 2000-02-19 08:07:08 by jwe]
jwe
parents: 3575
diff changeset
2105 delete [] tmp;
0ae310231c46 [project @ 2000-02-19 08:07:08 by jwe]
jwe
parents: 3575
diff changeset
2106 }
972
3e25eb05b6c6 [project @ 1994-12-12 15:53:47 by jwe]
jwe
parents: 971
diff changeset
2107
1826
b14829582cc4 [project @ 1996-02-02 03:05:07 by jwe]
jwe
parents: 1823
diff changeset
2108 // If yytext doesn't contain a valid number, we are in deep doo doo.
985
4b483cf9f6b0 [project @ 1994-12-14 18:06:38 by jwe]
jwe
parents: 975
diff changeset
2109
972
3e25eb05b6c6 [project @ 1994-12-12 15:53:47 by jwe]
jwe
parents: 971
diff changeset
2110 assert (nread == 1);
3e25eb05b6c6 [project @ 1994-12-12 15:53:47 by jwe]
jwe
parents: 971
diff changeset
2111
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
2112 looking_for_object_index = false;
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
2113 at_beginning_of_statement = false;
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
2114
16255
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
2115 push_token (new token (NUM, value, yytxt, input_line_number,
16148
10abbc493f50 delete unnecessary X-> references inside class X member functions
John W. Eaton <jwe@octave.org>
parents: 16146
diff changeset
2116 current_input_column));
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
2117
16130
421dea028bbf localize use of yyleng with lexical_feedback member function
John W. Eaton <jwe@octave.org>
parents: 16129
diff changeset
2118 current_input_column += flex_yyleng ();
972
3e25eb05b6c6 [project @ 1994-12-12 15:53:47 by jwe]
jwe
parents: 971
diff changeset
2119 }
3e25eb05b6c6 [project @ 1994-12-12 15:53:47 by jwe]
jwe
parents: 971
diff changeset
2120
16253
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2121 void
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2122 octave_lexer::handle_continuation (void)
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2123 {
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2124 char *yytxt = flex_yytext ();
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2125 int yylng = flex_yyleng ();
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2126
16257
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
2127 int offset = 1;
16253
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2128 if (yytxt[0] == '\\')
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2129 gripe_matlab_incompatible_continuation ();
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2130 else
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2131 offset = 3;
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2132
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2133 bool have_space = false;
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2134 while (offset < yylng)
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2135 {
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2136 char c = yytxt[offset];
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2137 if (c == ' ' || c == '\t')
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2138 {
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2139 have_space = true;
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2140 offset++;
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2141 }
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2142 else
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2143 break;
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2144 }
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2145
16255
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
2146 if (have_space)
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
2147 mark_previous_token_trailing_space ();
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
2148
16253
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2149 bool have_comment = false;
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2150 while (offset < yylng)
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2151 {
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2152 char c = yytxt[offset];
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2153 if (c == '#' || c == '%')
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2154 {
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2155 have_comment = true;
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2156 offset++;
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2157 }
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2158 else
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2159 break;
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2160 }
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2161
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2162 if (have_comment)
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2163 {
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2164 comment_text = &yytxt[offset];
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2165
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2166 finish_comment (octave_comment_elt::end_of_line, true);
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2167 }
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2168
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2169 decrement_promptflag ();
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2170 input_line_number++;
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2171 current_input_column = 1;
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2172 }
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2173
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2174 void
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2175 octave_lexer::finish_comment (octave_comment_elt::comment_type typ,
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2176 bool looking_at_continuation)
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2177 {
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2178 bool copyright = looks_like_copyright (comment_text);
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2179
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2180 if (nesting_level.none () && help_text.empty ()
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2181 && ! comment_text.empty () && ! copyright)
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2182 help_text = comment_text;
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2183
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2184 if (copyright)
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2185 typ = octave_comment_elt::copyright;
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2186
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2187 octave_comment_buffer::append (comment_text, typ);
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2188
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2189 comment_text = "";
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2190
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2191 at_beginning_of_statement = true;
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2192
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2193 if (! looking_at_continuation)
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2194 xunput ('\n');
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2195 }
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
2196
1001
641c05eaed01 [project @ 1994-12-23 06:17:30 by jwe]
jwe
parents: 991
diff changeset
2197 // We have seen a backslash and need to find out if it should be
641c05eaed01 [project @ 1994-12-23 06:17:30 by jwe]
jwe
parents: 991
diff changeset
2198 // treated as a continuation character. If so, this eats it, up to
641c05eaed01 [project @ 1994-12-23 06:17:30 by jwe]
jwe
parents: 991
diff changeset
2199 // and including the new line character.
641c05eaed01 [project @ 1994-12-23 06:17:30 by jwe]
jwe
parents: 991
diff changeset
2200 //
973
46673c918034 [project @ 1994-12-12 17:50:07 by jwe]
jwe
parents: 972
diff changeset
2201 // Match whitespace only, followed by a comment character or newline.
46673c918034 [project @ 1994-12-12 17:50:07 by jwe]
jwe
parents: 972
diff changeset
2202 // Once a comment character is found, discard all input until newline.
46673c918034 [project @ 1994-12-12 17:50:07 by jwe]
jwe
parents: 972
diff changeset
2203 // If non-whitespace characters are found before comment
46673c918034 [project @ 1994-12-12 17:50:07 by jwe]
jwe
parents: 972
diff changeset
2204 // characters, return 0. Otherwise, return 1.
46673c918034 [project @ 1994-12-12 17:50:07 by jwe]
jwe
parents: 972
diff changeset
2205
7723
74f5e0c7de9e first pass at handling block comments
John W. Eaton <jwe@octave.org>
parents: 7722
diff changeset
2206 // FIXME -- we need to handle block comments here.
74f5e0c7de9e first pass at handling block comments
John W. Eaton <jwe@octave.org>
parents: 7722
diff changeset
2207
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
2208 bool
16158
7eb614760ddb rename lexical_feedback to octave_lexer
John W. Eaton <jwe@octave.org>
parents: 16154
diff changeset
2209 octave_lexer::have_continuation (bool trailing_comments_ok)
973
46673c918034 [project @ 1994-12-12 17:50:07 by jwe]
jwe
parents: 972
diff changeset
2210 {
5765
7ba9ad1fec11 [project @ 2006-04-17 05:05:15 by jwe]
jwe
parents: 5615
diff changeset
2211 std::ostringstream buf;
973
46673c918034 [project @ 1994-12-12 17:50:07 by jwe]
jwe
parents: 972
diff changeset
2212
3665
0689afb1d001 [project @ 2000-05-11 19:07:56 by jwe]
jwe
parents: 3598
diff changeset
2213 std::string comment_buf;
0689afb1d001 [project @ 2000-05-11 19:07:56 by jwe]
jwe
parents: 3598
diff changeset
2214
2857
5448c8cba402 [project @ 1997-03-31 07:06:10 by jwe]
jwe
parents: 2856
diff changeset
2215 bool in_comment = false;
3665
0689afb1d001 [project @ 2000-05-11 19:07:56 by jwe]
jwe
parents: 3598
diff changeset
2216 bool beginning_of_comment = false;
0689afb1d001 [project @ 2000-05-11 19:07:56 by jwe]
jwe
parents: 3598
diff changeset
2217
0689afb1d001 [project @ 2000-05-11 19:07:56 by jwe]
jwe
parents: 3598
diff changeset
2218 int c = 0;
0689afb1d001 [project @ 2000-05-11 19:07:56 by jwe]
jwe
parents: 3598
diff changeset
2219
7728
13820b9f5fd9 more consistent handling of CR/CRLF/LF line endings in lexer and parser
John W. Eaton <jwe@octave.org>
parents: 7723
diff changeset
2220 while ((c = text_yyinput ()) != EOF)
973
46673c918034 [project @ 1994-12-12 17:50:07 by jwe]
jwe
parents: 972
diff changeset
2221 {
3802
4f1a26a730fd [project @ 2001-02-26 20:26:14 by jwe]
jwe
parents: 3796
diff changeset
2222 buf << static_cast<char> (c);
973
46673c918034 [project @ 1994-12-12 17:50:07 by jwe]
jwe
parents: 972
diff changeset
2223
46673c918034 [project @ 1994-12-12 17:50:07 by jwe]
jwe
parents: 972
diff changeset
2224 switch (c)
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2225 {
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2226 case ' ':
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2227 case '\t':
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2228 if (in_comment)
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2229 {
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2230 comment_buf += static_cast<char> (c);
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2231 beginning_of_comment = false;
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2232 }
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2233 break;
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2234
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2235 case '%':
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2236 case '#':
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2237 if (trailing_comments_ok)
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2238 {
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2239 if (in_comment)
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2240 {
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2241 if (! beginning_of_comment)
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2242 comment_buf += static_cast<char> (c);
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2243 }
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2244 else
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2245 {
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2246 maybe_gripe_matlab_incompatible_comment (c);
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2247 in_comment = true;
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2248 beginning_of_comment = true;
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2249 }
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2250 }
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2251 else
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2252 goto cleanup;
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2253 break;
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2254
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2255 case '\n':
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2256 if (in_comment)
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2257 {
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2258 comment_buf += static_cast<char> (c);
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2259 octave_comment_buffer::append (comment_buf);
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2260 }
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
2261 current_input_column = 0;
16209
e7ff32e7cf82 move global promptflag variable to octave_reader class
John W. Eaton <jwe@octave.org>
parents: 16207
diff changeset
2262 decrement_promptflag ();
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2263 gripe_matlab_incompatible_continuation ();
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2264 return true;
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2265
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2266 default:
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2267 if (in_comment)
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2268 {
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2269 comment_buf += static_cast<char> (c);
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2270 beginning_of_comment = false;
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2271 }
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2272 else
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2273 goto cleanup;
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2274 break;
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2275 }
973
46673c918034 [project @ 1994-12-12 17:50:07 by jwe]
jwe
parents: 972
diff changeset
2276 }
46673c918034 [project @ 1994-12-12 17:50:07 by jwe]
jwe
parents: 972
diff changeset
2277
16125
96a58f197f93 allow xunput to be called without buffer argument
John W. Eaton <jwe@octave.org>
parents: 16124
diff changeset
2278 xunput (c);
2857
5448c8cba402 [project @ 1997-03-31 07:06:10 by jwe]
jwe
parents: 2856
diff changeset
2279 return false;
973
46673c918034 [project @ 1994-12-12 17:50:07 by jwe]
jwe
parents: 972
diff changeset
2280
3096
ff8b4d6371b3 [project @ 1997-11-14 07:59:14 by jwe]
jwe
parents: 3079
diff changeset
2281 cleanup:
4051
b79da8779a0e [project @ 2002-08-17 19:38:32 by jwe]
jwe
parents: 4037
diff changeset
2282
5765
7ba9ad1fec11 [project @ 2006-04-17 05:05:15 by jwe]
jwe
parents: 5615
diff changeset
2283 std::string s = buf.str ();
4051
b79da8779a0e [project @ 2002-08-17 19:38:32 by jwe]
jwe
parents: 4037
diff changeset
2284
b79da8779a0e [project @ 2002-08-17 19:38:32 by jwe]
jwe
parents: 4037
diff changeset
2285 int len = s.length ();
b79da8779a0e [project @ 2002-08-17 19:38:32 by jwe]
jwe
parents: 4037
diff changeset
2286 while (len--)
16125
96a58f197f93 allow xunput to be called without buffer argument
John W. Eaton <jwe@octave.org>
parents: 16124
diff changeset
2287 xunput (s[len]);
3096
ff8b4d6371b3 [project @ 1997-11-14 07:59:14 by jwe]
jwe
parents: 3079
diff changeset
2288
2857
5448c8cba402 [project @ 1997-03-31 07:06:10 by jwe]
jwe
parents: 2856
diff changeset
2289 return false;
973
46673c918034 [project @ 1994-12-12 17:50:07 by jwe]
jwe
parents: 972
diff changeset
2290 }
46673c918034 [project @ 1994-12-12 17:50:07 by jwe]
jwe
parents: 972
diff changeset
2291
15466
d174210ce1ec use ' instead of ` in error messages, warnings and most comments
John W. Eaton <jwe@octave.org>
parents: 14161
diff changeset
2292 // We have seen a '.' and need to see if it is the start of a
1001
641c05eaed01 [project @ 1994-12-23 06:17:30 by jwe]
jwe
parents: 991
diff changeset
2293 // continuation. If so, this eats it, up to and including the new
641c05eaed01 [project @ 1994-12-23 06:17:30 by jwe]
jwe
parents: 991
diff changeset
2294 // line character.
641c05eaed01 [project @ 1994-12-23 06:17:30 by jwe]
jwe
parents: 991
diff changeset
2295
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
2296 bool
16158
7eb614760ddb rename lexical_feedback to octave_lexer
John W. Eaton <jwe@octave.org>
parents: 16154
diff changeset
2297 octave_lexer::have_ellipsis_continuation (bool trailing_comments_ok)
973
46673c918034 [project @ 1994-12-12 17:50:07 by jwe]
jwe
parents: 972
diff changeset
2298 {
7728
13820b9f5fd9 more consistent handling of CR/CRLF/LF line endings in lexer and parser
John W. Eaton <jwe@octave.org>
parents: 7723
diff changeset
2299 char c1 = text_yyinput ();
973
46673c918034 [project @ 1994-12-12 17:50:07 by jwe]
jwe
parents: 972
diff changeset
2300 if (c1 == '.')
46673c918034 [project @ 1994-12-12 17:50:07 by jwe]
jwe
parents: 972
diff changeset
2301 {
7728
13820b9f5fd9 more consistent handling of CR/CRLF/LF line endings in lexer and parser
John W. Eaton <jwe@octave.org>
parents: 7723
diff changeset
2302 char c2 = text_yyinput ();
1091
54abf1b3a8e9 [project @ 1995-02-02 00:30:14 by jwe]
jwe
parents: 1088
diff changeset
2303 if (c2 == '.' && have_continuation (trailing_comments_ok))
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2304 return true;
973
46673c918034 [project @ 1994-12-12 17:50:07 by jwe]
jwe
parents: 972
diff changeset
2305 else
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2306 {
16125
96a58f197f93 allow xunput to be called without buffer argument
John W. Eaton <jwe@octave.org>
parents: 16124
diff changeset
2307 xunput (c2);
96a58f197f93 allow xunput to be called without buffer argument
John W. Eaton <jwe@octave.org>
parents: 16124
diff changeset
2308 xunput (c1);
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2309 }
973
46673c918034 [project @ 1994-12-12 17:50:07 by jwe]
jwe
parents: 972
diff changeset
2310 }
46673c918034 [project @ 1994-12-12 17:50:07 by jwe]
jwe
parents: 972
diff changeset
2311 else
16125
96a58f197f93 allow xunput to be called without buffer argument
John W. Eaton <jwe@octave.org>
parents: 16124
diff changeset
2312 xunput (c1);
973
46673c918034 [project @ 1994-12-12 17:50:07 by jwe]
jwe
parents: 972
diff changeset
2313
2857
5448c8cba402 [project @ 1997-03-31 07:06:10 by jwe]
jwe
parents: 2856
diff changeset
2314 return false;
973
46673c918034 [project @ 1994-12-12 17:50:07 by jwe]
jwe
parents: 972
diff changeset
2315 }
46673c918034 [project @ 1994-12-12 17:50:07 by jwe]
jwe
parents: 972
diff changeset
2316
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
2317 int
16158
7eb614760ddb rename lexical_feedback to octave_lexer
John W. Eaton <jwe@octave.org>
parents: 16154
diff changeset
2318 octave_lexer::handle_string (char delim)
973
46673c918034 [project @ 1994-12-12 17:50:07 by jwe]
jwe
parents: 972
diff changeset
2319 {
5765
7ba9ad1fec11 [project @ 2006-04-17 05:05:15 by jwe]
jwe
parents: 5615
diff changeset
2320 std::ostringstream buf;
973
46673c918034 [project @ 1994-12-12 17:50:07 by jwe]
jwe
parents: 972
diff changeset
2321
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
2322 int bos_line = input_line_number;
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
2323 int bos_col = current_input_column;
3805
44386b0e53da [project @ 2001-03-01 16:54:31 by jwe]
jwe
parents: 3802
diff changeset
2324
973
46673c918034 [project @ 1994-12-12 17:50:07 by jwe]
jwe
parents: 972
diff changeset
2325 int c;
1031
59f5eb2d5eb3 [project @ 1995-01-15 21:11:11 by jwe]
jwe
parents: 1019
diff changeset
2326 int escape_pending = 0;
973
46673c918034 [project @ 1994-12-12 17:50:07 by jwe]
jwe
parents: 972
diff changeset
2327
7728
13820b9f5fd9 more consistent handling of CR/CRLF/LF line endings in lexer and parser
John W. Eaton <jwe@octave.org>
parents: 7723
diff changeset
2328 while ((c = text_yyinput ()) != EOF)
973
46673c918034 [project @ 1994-12-12 17:50:07 by jwe]
jwe
parents: 972
diff changeset
2329 {
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
2330 current_input_column++;
973
46673c918034 [project @ 1994-12-12 17:50:07 by jwe]
jwe
parents: 972
diff changeset
2331
3105
f936c7f5074f [project @ 1997-11-19 04:36:17 by jwe]
jwe
parents: 3103
diff changeset
2332 if (c == '\\')
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2333 {
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2334 if (delim == '\'' || escape_pending)
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2335 {
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2336 buf << static_cast<char> (c);
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2337 escape_pending = 0;
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2338 }
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2339 else
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2340 {
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2341 if (have_continuation (false))
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2342 escape_pending = 0;
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2343 else
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2344 {
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2345 buf << static_cast<char> (c);
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2346 escape_pending = 1;
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2347 }
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2348 }
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2349 continue;
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2350 }
973
46673c918034 [project @ 1994-12-12 17:50:07 by jwe]
jwe
parents: 972
diff changeset
2351 else if (c == '.')
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2352 {
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2353 if (delim == '\'' || ! have_ellipsis_continuation (false))
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2354 buf << static_cast<char> (c);
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2355 }
973
46673c918034 [project @ 1994-12-12 17:50:07 by jwe]
jwe
parents: 972
diff changeset
2356 else if (c == '\n')
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2357 {
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2358 error ("unterminated string constant");
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2359 break;
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2360 }
973
46673c918034 [project @ 1994-12-12 17:50:07 by jwe]
jwe
parents: 972
diff changeset
2361 else if (c == delim)
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2362 {
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2363 if (escape_pending)
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2364 buf << static_cast<char> (c);
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2365 else
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2366 {
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2367 c = text_yyinput ();
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2368 if (c == delim)
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2369 {
11586
12df7854fa7c strip trailing whitespace from source files
John W. Eaton <jwe@octave.org>
parents: 11523
diff changeset
2370 buf << static_cast<char> (c);
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2371 }
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2372 else
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2373 {
11586
12df7854fa7c strip trailing whitespace from source files
John W. Eaton <jwe@octave.org>
parents: 11523
diff changeset
2374 std::string s;
16125
96a58f197f93 allow xunput to be called without buffer argument
John W. Eaton <jwe@octave.org>
parents: 16124
diff changeset
2375 xunput (c);
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2376
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2377 if (delim == '\'')
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2378 s = buf.str ();
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2379 else
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2380 s = do_string_escapes (buf.str ());
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2381
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2382 if (delim == '"')
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2383 gripe_matlab_incompatible ("\" used as string delimiter");
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2384 else if (delim == '\'')
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2385 gripe_single_quote_string ();
3400
18366d37e7dd [project @ 1999-12-22 23:36:09 by jwe]
jwe
parents: 3388
diff changeset
2386
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
2387 looking_for_object_index = true;
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
2388 at_beginning_of_statement = false;
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2389
16255
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
2390 int tok = delim == '"' ? DQ_STRING : SQ_STRING;
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
2391
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
2392 push_token (new token (tok, s, bos_line, bos_col));
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
2393
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
2394 return tok;
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2395 }
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2396 }
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2397 }
973
46673c918034 [project @ 1994-12-12 17:50:07 by jwe]
jwe
parents: 972
diff changeset
2398 else
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2399 {
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2400 buf << static_cast<char> (c);
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2401 }
973
46673c918034 [project @ 1994-12-12 17:50:07 by jwe]
jwe
parents: 972
diff changeset
2402
1031
59f5eb2d5eb3 [project @ 1995-01-15 21:11:11 by jwe]
jwe
parents: 1019
diff changeset
2403 escape_pending = 0;
973
46673c918034 [project @ 1994-12-12 17:50:07 by jwe]
jwe
parents: 972
diff changeset
2404 }
46673c918034 [project @ 1994-12-12 17:50:07 by jwe]
jwe
parents: 972
diff changeset
2405
46673c918034 [project @ 1994-12-12 17:50:07 by jwe]
jwe
parents: 972
diff changeset
2406 return LEXICAL_ERROR;
46673c918034 [project @ 1994-12-12 17:50:07 by jwe]
jwe
parents: 972
diff changeset
2407 }
46673c918034 [project @ 1994-12-12 17:50:07 by jwe]
jwe
parents: 972
diff changeset
2408
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
2409 int
16259
0b5ab09dfce4 2/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16257
diff changeset
2410 octave_lexer::handle_close_bracket (int bracket_type)
971
fe71abb43457 [project @ 1994-12-12 15:10:22 by jwe]
jwe
parents: 970
diff changeset
2411 {
4612
d44675070f1a [project @ 2003-11-14 19:49:56 by jwe]
jwe
parents: 4608
diff changeset
2412 int retval = bracket_type;
3208
e8a7163701be [project @ 1998-11-03 05:12:47 by jwe]
jwe
parents: 3204
diff changeset
2413
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
2414 if (! nesting_level.none ())
971
fe71abb43457 [project @ 1994-12-12 15:10:22 by jwe]
jwe
parents: 970
diff changeset
2415 {
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
2416 nesting_level.remove ();
4613
d1786f2d8a3c [project @ 2003-11-14 22:46:19 by jwe]
jwe
parents: 4612
diff changeset
2417
d1786f2d8a3c [project @ 2003-11-14 22:46:19 by jwe]
jwe
parents: 4612
diff changeset
2418 if (bracket_type == ']')
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
2419 bracketflag--;
4613
d1786f2d8a3c [project @ 2003-11-14 22:46:19 by jwe]
jwe
parents: 4612
diff changeset
2420 else if (bracket_type == '}')
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
2421 braceflag--;
4613
d1786f2d8a3c [project @ 2003-11-14 22:46:19 by jwe]
jwe
parents: 4612
diff changeset
2422 else
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2423 panic_impossible ();
971
fe71abb43457 [project @ 1994-12-12 15:10:22 by jwe]
jwe
parents: 970
diff changeset
2424 }
fe71abb43457 [project @ 1994-12-12 15:10:22 by jwe]
jwe
parents: 970
diff changeset
2425
16224
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
2426 pop_start_state ();
1001
641c05eaed01 [project @ 1994-12-23 06:17:30 by jwe]
jwe
parents: 991
diff changeset
2427
3208
e8a7163701be [project @ 1998-11-03 05:12:47 by jwe]
jwe
parents: 3204
diff changeset
2428 return retval;
971
fe71abb43457 [project @ 1994-12-12 15:10:22 by jwe]
jwe
parents: 970
diff changeset
2429 }
fe71abb43457 [project @ 1994-12-12 15:10:22 by jwe]
jwe
parents: 970
diff changeset
2430
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
2431 bool
16158
7eb614760ddb rename lexical_feedback to octave_lexer
John W. Eaton <jwe@octave.org>
parents: 16154
diff changeset
2432 octave_lexer::next_token_can_follow_bin_op (void)
8746
5dd06f19e9be handle commands in the lexer
John W. Eaton <jwe@octave.org>
parents: 8745
diff changeset
2433 {
5dd06f19e9be handle commands in the lexer
John W. Eaton <jwe@octave.org>
parents: 8745
diff changeset
2434 std::stack<char> buf;
5dd06f19e9be handle commands in the lexer
John W. Eaton <jwe@octave.org>
parents: 8745
diff changeset
2435
5dd06f19e9be handle commands in the lexer
John W. Eaton <jwe@octave.org>
parents: 8745
diff changeset
2436 int c = EOF;
5dd06f19e9be handle commands in the lexer
John W. Eaton <jwe@octave.org>
parents: 8745
diff changeset
2437
5dd06f19e9be handle commands in the lexer
John W. Eaton <jwe@octave.org>
parents: 8745
diff changeset
2438 // Skip whitespace in current statement on current line
5dd06f19e9be handle commands in the lexer
John W. Eaton <jwe@octave.org>
parents: 8745
diff changeset
2439 while (true)
5dd06f19e9be handle commands in the lexer
John W. Eaton <jwe@octave.org>
parents: 8745
diff changeset
2440 {
5dd06f19e9be handle commands in the lexer
John W. Eaton <jwe@octave.org>
parents: 8745
diff changeset
2441 c = text_yyinput ();
5dd06f19e9be handle commands in the lexer
John W. Eaton <jwe@octave.org>
parents: 8745
diff changeset
2442
8767
026c6732ec7a lex.l (next_token_can_follow_bin_op): push all characters read on to buffer stack
John W. Eaton <jwe@octave.org>
parents: 8746
diff changeset
2443 buf.push (c);
026c6732ec7a lex.l (next_token_can_follow_bin_op): push all characters read on to buffer stack
John W. Eaton <jwe@octave.org>
parents: 8746
diff changeset
2444
026c6732ec7a lex.l (next_token_can_follow_bin_op): push all characters read on to buffer stack
John W. Eaton <jwe@octave.org>
parents: 8746
diff changeset
2445 if (match_any (c, ",;\n") || (c != ' ' && c != '\t'))
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2446 break;
8746
5dd06f19e9be handle commands in the lexer
John W. Eaton <jwe@octave.org>
parents: 8745
diff changeset
2447 }
5dd06f19e9be handle commands in the lexer
John W. Eaton <jwe@octave.org>
parents: 8745
diff changeset
2448
5dd06f19e9be handle commands in the lexer
John W. Eaton <jwe@octave.org>
parents: 8745
diff changeset
2449 // Restore input.
5dd06f19e9be handle commands in the lexer
John W. Eaton <jwe@octave.org>
parents: 8745
diff changeset
2450 while (! buf.empty ())
5dd06f19e9be handle commands in the lexer
John W. Eaton <jwe@octave.org>
parents: 8745
diff changeset
2451 {
16125
96a58f197f93 allow xunput to be called without buffer argument
John W. Eaton <jwe@octave.org>
parents: 16124
diff changeset
2452 xunput (buf.top ());
8746
5dd06f19e9be handle commands in the lexer
John W. Eaton <jwe@octave.org>
parents: 8745
diff changeset
2453
5dd06f19e9be handle commands in the lexer
John W. Eaton <jwe@octave.org>
parents: 8745
diff changeset
2454 buf.pop ();
5dd06f19e9be handle commands in the lexer
John W. Eaton <jwe@octave.org>
parents: 8745
diff changeset
2455 }
5dd06f19e9be handle commands in the lexer
John W. Eaton <jwe@octave.org>
parents: 8745
diff changeset
2456
5dd06f19e9be handle commands in the lexer
John W. Eaton <jwe@octave.org>
parents: 8745
diff changeset
2457 return (isalnum (c) || match_any (c, "!\"'(-[_{~"));
5dd06f19e9be handle commands in the lexer
John W. Eaton <jwe@octave.org>
parents: 8745
diff changeset
2458 }
5dd06f19e9be handle commands in the lexer
John W. Eaton <jwe@octave.org>
parents: 8745
diff changeset
2459
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
2460 bool
16158
7eb614760ddb rename lexical_feedback to octave_lexer
John W. Eaton <jwe@octave.org>
parents: 16154
diff changeset
2461 octave_lexer::looks_like_command_arg (void)
8746
5dd06f19e9be handle commands in the lexer
John W. Eaton <jwe@octave.org>
parents: 8745
diff changeset
2462 {
16275
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
2463 bool space_before = space_follows_previous_token ();
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
2464 bool space_after = looking_at_space ();
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
2465
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
2466 return (space_before && ! space_after
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
2467 && previous_token_may_be_command ());
8746
5dd06f19e9be handle commands in the lexer
John W. Eaton <jwe@octave.org>
parents: 8745
diff changeset
2468 }
5dd06f19e9be handle commands in the lexer
John W. Eaton <jwe@octave.org>
parents: 8745
diff changeset
2469
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
2470 int
16158
7eb614760ddb rename lexical_feedback to octave_lexer
John W. Eaton <jwe@octave.org>
parents: 16154
diff changeset
2471 octave_lexer::handle_superclass_identifier (void)
9476
d9b25c5b8ee5 handle classdef syntax in lexer and parser
Ryan Rusaw
parents: 9474
diff changeset
2472 {
11586
12df7854fa7c strip trailing whitespace from source files
John W. Eaton <jwe@octave.org>
parents: 11523
diff changeset
2473 std::string pkg;
16128
210039e91ad6 localize use of yytext with lexical_feedback member function
John W. Eaton <jwe@octave.org>
parents: 16126
diff changeset
2474 char *yytxt = flex_yytext ();
210039e91ad6 localize use of yytext with lexical_feedback member function
John W. Eaton <jwe@octave.org>
parents: 16126
diff changeset
2475 std::string meth = strip_trailing_whitespace (yytxt);
9476
d9b25c5b8ee5 handle classdef syntax in lexer and parser
Ryan Rusaw
parents: 9474
diff changeset
2476 size_t pos = meth.find ("@");
d9b25c5b8ee5 handle classdef syntax in lexer and parser
Ryan Rusaw
parents: 9474
diff changeset
2477 std::string cls = meth.substr (pos).substr (1);
11586
12df7854fa7c strip trailing whitespace from source files
John W. Eaton <jwe@octave.org>
parents: 11523
diff changeset
2478 meth = meth.substr (0, pos - 1);
12df7854fa7c strip trailing whitespace from source files
John W. Eaton <jwe@octave.org>
parents: 11523
diff changeset
2479
9476
d9b25c5b8ee5 handle classdef syntax in lexer and parser
Ryan Rusaw
parents: 9474
diff changeset
2480 pos = cls.find (".");
d9b25c5b8ee5 handle classdef syntax in lexer and parser
Ryan Rusaw
parents: 9474
diff changeset
2481 if (pos != std::string::npos)
11586
12df7854fa7c strip trailing whitespace from source files
John W. Eaton <jwe@octave.org>
parents: 11523
diff changeset
2482 {
9476
d9b25c5b8ee5 handle classdef syntax in lexer and parser
Ryan Rusaw
parents: 9474
diff changeset
2483 pkg = cls.substr (pos).substr (1);
d9b25c5b8ee5 handle classdef syntax in lexer and parser
Ryan Rusaw
parents: 9474
diff changeset
2484 cls = cls.substr (0, pos - 1);
d9b25c5b8ee5 handle classdef syntax in lexer and parser
Ryan Rusaw
parents: 9474
diff changeset
2485 }
11586
12df7854fa7c strip trailing whitespace from source files
John W. Eaton <jwe@octave.org>
parents: 11523
diff changeset
2486
9476
d9b25c5b8ee5 handle classdef syntax in lexer and parser
Ryan Rusaw
parents: 9474
diff changeset
2487 int kw_token = (is_keyword_token (meth) || is_keyword_token (cls)
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2488 || is_keyword_token (pkg));
9476
d9b25c5b8ee5 handle classdef syntax in lexer and parser
Ryan Rusaw
parents: 9474
diff changeset
2489 if (kw_token)
d9b25c5b8ee5 handle classdef syntax in lexer and parser
Ryan Rusaw
parents: 9474
diff changeset
2490 {
d9b25c5b8ee5 handle classdef syntax in lexer and parser
Ryan Rusaw
parents: 9474
diff changeset
2491 error ("method, class and package names may not be keywords");
d9b25c5b8ee5 handle classdef syntax in lexer and parser
Ryan Rusaw
parents: 9474
diff changeset
2492 return LEXICAL_ERROR;
d9b25c5b8ee5 handle classdef syntax in lexer and parser
Ryan Rusaw
parents: 9474
diff changeset
2493 }
11586
12df7854fa7c strip trailing whitespace from source files
John W. Eaton <jwe@octave.org>
parents: 11523
diff changeset
2494
16255
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
2495 push_token (new token (SUPERCLASSREF,
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
2496 meth.empty () ? 0 : &(symbol_table::insert (meth)),
16148
10abbc493f50 delete unnecessary X-> references inside class X member functions
John W. Eaton <jwe@octave.org>
parents: 16146
diff changeset
2497 cls.empty () ? 0 : &(symbol_table::insert (cls)),
10abbc493f50 delete unnecessary X-> references inside class X member functions
John W. Eaton <jwe@octave.org>
parents: 16146
diff changeset
2498 pkg.empty () ? 0 : &(symbol_table::insert (pkg)),
10abbc493f50 delete unnecessary X-> references inside class X member functions
John W. Eaton <jwe@octave.org>
parents: 16146
diff changeset
2499 input_line_number, current_input_column));
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
2500
16130
421dea028bbf localize use of yyleng with lexical_feedback member function
John W. Eaton <jwe@octave.org>
parents: 16129
diff changeset
2501 current_input_column += flex_yyleng ();
11586
12df7854fa7c strip trailing whitespace from source files
John W. Eaton <jwe@octave.org>
parents: 11523
diff changeset
2502
9476
d9b25c5b8ee5 handle classdef syntax in lexer and parser
Ryan Rusaw
parents: 9474
diff changeset
2503 return SUPERCLASSREF;
d9b25c5b8ee5 handle classdef syntax in lexer and parser
Ryan Rusaw
parents: 9474
diff changeset
2504 }
d9b25c5b8ee5 handle classdef syntax in lexer and parser
Ryan Rusaw
parents: 9474
diff changeset
2505
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
2506 int
16158
7eb614760ddb rename lexical_feedback to octave_lexer
John W. Eaton <jwe@octave.org>
parents: 16154
diff changeset
2507 octave_lexer::handle_meta_identifier (void)
9476
d9b25c5b8ee5 handle classdef syntax in lexer and parser
Ryan Rusaw
parents: 9474
diff changeset
2508 {
d9b25c5b8ee5 handle classdef syntax in lexer and parser
Ryan Rusaw
parents: 9474
diff changeset
2509 std::string pkg;
16128
210039e91ad6 localize use of yytext with lexical_feedback member function
John W. Eaton <jwe@octave.org>
parents: 16126
diff changeset
2510 char *yytxt = flex_yytext ();
210039e91ad6 localize use of yytext with lexical_feedback member function
John W. Eaton <jwe@octave.org>
parents: 16126
diff changeset
2511 std::string cls = strip_trailing_whitespace (yytxt).substr (1);
9476
d9b25c5b8ee5 handle classdef syntax in lexer and parser
Ryan Rusaw
parents: 9474
diff changeset
2512 size_t pos = cls.find (".");
11586
12df7854fa7c strip trailing whitespace from source files
John W. Eaton <jwe@octave.org>
parents: 11523
diff changeset
2513
9476
d9b25c5b8ee5 handle classdef syntax in lexer and parser
Ryan Rusaw
parents: 9474
diff changeset
2514 if (pos != std::string::npos)
11586
12df7854fa7c strip trailing whitespace from source files
John W. Eaton <jwe@octave.org>
parents: 11523
diff changeset
2515 {
9476
d9b25c5b8ee5 handle classdef syntax in lexer and parser
Ryan Rusaw
parents: 9474
diff changeset
2516 pkg = cls.substr (pos).substr (1);
d9b25c5b8ee5 handle classdef syntax in lexer and parser
Ryan Rusaw
parents: 9474
diff changeset
2517 cls = cls.substr (0, pos - 1);
d9b25c5b8ee5 handle classdef syntax in lexer and parser
Ryan Rusaw
parents: 9474
diff changeset
2518 }
11586
12df7854fa7c strip trailing whitespace from source files
John W. Eaton <jwe@octave.org>
parents: 11523
diff changeset
2519
9476
d9b25c5b8ee5 handle classdef syntax in lexer and parser
Ryan Rusaw
parents: 9474
diff changeset
2520 int kw_token = is_keyword_token (cls) || is_keyword_token (pkg);
d9b25c5b8ee5 handle classdef syntax in lexer and parser
Ryan Rusaw
parents: 9474
diff changeset
2521 if (kw_token)
d9b25c5b8ee5 handle classdef syntax in lexer and parser
Ryan Rusaw
parents: 9474
diff changeset
2522 {
d9b25c5b8ee5 handle classdef syntax in lexer and parser
Ryan Rusaw
parents: 9474
diff changeset
2523 error ("class and package names may not be keywords");
d9b25c5b8ee5 handle classdef syntax in lexer and parser
Ryan Rusaw
parents: 9474
diff changeset
2524 return LEXICAL_ERROR;
d9b25c5b8ee5 handle classdef syntax in lexer and parser
Ryan Rusaw
parents: 9474
diff changeset
2525 }
11586
12df7854fa7c strip trailing whitespace from source files
John W. Eaton <jwe@octave.org>
parents: 11523
diff changeset
2526
16255
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
2527 push_token (new token (METAQUERY,
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
2528 cls.empty () ? 0 : &(symbol_table::insert (cls)),
16148
10abbc493f50 delete unnecessary X-> references inside class X member functions
John W. Eaton <jwe@octave.org>
parents: 16146
diff changeset
2529 pkg.empty () ? 0 : &(symbol_table::insert (pkg)),
10abbc493f50 delete unnecessary X-> references inside class X member functions
John W. Eaton <jwe@octave.org>
parents: 16146
diff changeset
2530 input_line_number, current_input_column));
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
2531
16130
421dea028bbf localize use of yyleng with lexical_feedback member function
John W. Eaton <jwe@octave.org>
parents: 16129
diff changeset
2532 current_input_column += flex_yyleng ();
11586
12df7854fa7c strip trailing whitespace from source files
John W. Eaton <jwe@octave.org>
parents: 11523
diff changeset
2533
9476
d9b25c5b8ee5 handle classdef syntax in lexer and parser
Ryan Rusaw
parents: 9474
diff changeset
2534 return METAQUERY;
d9b25c5b8ee5 handle classdef syntax in lexer and parser
Ryan Rusaw
parents: 9474
diff changeset
2535 }
d9b25c5b8ee5 handle classdef syntax in lexer and parser
Ryan Rusaw
parents: 9474
diff changeset
2536
767
42731861ee09 [project @ 1994-10-05 21:26:54 by jwe]
jwe
parents: 747
diff changeset
2537 // Figure out exactly what kind of token to return when we have seen
4238
a5a68c0afe56 [project @ 2002-12-25 21:04:33 by jwe]
jwe
parents: 4237
diff changeset
2538 // an identifier. Handles keywords. Return -1 if the identifier
a5a68c0afe56 [project @ 2002-12-25 21:04:33 by jwe]
jwe
parents: 4237
diff changeset
2539 // should be ignored.
767
42731861ee09 [project @ 1994-10-05 21:26:54 by jwe]
jwe
parents: 747
diff changeset
2540
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
2541 int
16158
7eb614760ddb rename lexical_feedback to octave_lexer
John W. Eaton <jwe@octave.org>
parents: 16154
diff changeset
2542 octave_lexer::handle_identifier (void)
146
edfa5a96c5f1 [project @ 1993-10-05 04:02:15 by jwe]
jwe
parents: 143
diff changeset
2543 {
16128
210039e91ad6 localize use of yytext with lexical_feedback member function
John W. Eaton <jwe@octave.org>
parents: 16126
diff changeset
2544 char *yytxt = flex_yytext ();
210039e91ad6 localize use of yytext with lexical_feedback member function
John W. Eaton <jwe@octave.org>
parents: 16126
diff changeset
2545
16257
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
2546 std::string tok = yytxt;
16128
210039e91ad6 localize use of yytext with lexical_feedback member function
John W. Eaton <jwe@octave.org>
parents: 16126
diff changeset
2547
2970
b9e64477f703 [project @ 1997-05-15 18:04:21 by jwe]
jwe
parents: 2926
diff changeset
2548 // If we are expecting a structure element, avoid recognizing
b9e64477f703 [project @ 1997-05-15 18:04:21 by jwe]
jwe
parents: 2926
diff changeset
2549 // keywords and other special names and return STRUCT_ELT, which is
b9e64477f703 [project @ 1997-05-15 18:04:21 by jwe]
jwe
parents: 2926
diff changeset
2550 // a string that is also a valid identifier. But first, we have to
b9e64477f703 [project @ 1997-05-15 18:04:21 by jwe]
jwe
parents: 2926
diff changeset
2551 // decide whether to insert a comma.
747
6182de8cb3b7 [project @ 1994-09-30 15:09:36 by jwe]
jwe
parents: 661
diff changeset
2552
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
2553 if (looking_at_indirect_ref)
1072
0cd3ba9c1f61 [project @ 1995-01-26 19:22:22 by jwe]
jwe
parents: 1060
diff changeset
2554 {
16255
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
2555 push_token (new token (STRUCT_ELT, tok, input_line_number,
16148
10abbc493f50 delete unnecessary X-> references inside class X member functions
John W. Eaton <jwe@octave.org>
parents: 16146
diff changeset
2556 current_input_column));
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
2557
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
2558 looking_for_object_index = true;
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
2559
16130
421dea028bbf localize use of yyleng with lexical_feedback member function
John W. Eaton <jwe@octave.org>
parents: 16129
diff changeset
2560 current_input_column += flex_yyleng ();
2819
ae3cce876a8e [project @ 1997-03-17 23:27:45 by jwe]
jwe
parents: 2804
diff changeset
2561
2970
b9e64477f703 [project @ 1997-05-15 18:04:21 by jwe]
jwe
parents: 2926
diff changeset
2562 return STRUCT_ELT;
1072
0cd3ba9c1f61 [project @ 1995-01-26 19:22:22 by jwe]
jwe
parents: 1060
diff changeset
2563 }
747
6182de8cb3b7 [project @ 1994-09-30 15:09:36 by jwe]
jwe
parents: 661
diff changeset
2564
8746
5dd06f19e9be handle commands in the lexer
John W. Eaton <jwe@octave.org>
parents: 8745
diff changeset
2565 // The is_keyword_token may reset
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
2566 // at_beginning_of_statement. For example, if it sees
8746
5dd06f19e9be handle commands in the lexer
John W. Eaton <jwe@octave.org>
parents: 8745
diff changeset
2567 // an else token, then the next token is at the beginning of a
5dd06f19e9be handle commands in the lexer
John W. Eaton <jwe@octave.org>
parents: 8745
diff changeset
2568 // statement.
5dd06f19e9be handle commands in the lexer
John W. Eaton <jwe@octave.org>
parents: 8745
diff changeset
2569
16271
87069bd38107 8/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16269
diff changeset
2570 // May set at_beginning_of_statement to true.
4930
bdb307dc8613 [project @ 2004-08-05 04:55:26 by jwe]
jwe
parents: 4923
diff changeset
2571 int kw_token = is_keyword_token (tok);
bdb307dc8613 [project @ 2004-08-05 04:55:26 by jwe]
jwe
parents: 4923
diff changeset
2572
8746
5dd06f19e9be handle commands in the lexer
John W. Eaton <jwe@octave.org>
parents: 8745
diff changeset
2573 // If we found a keyword token, then the beginning_of_statement flag
5dd06f19e9be handle commands in the lexer
John W. Eaton <jwe@octave.org>
parents: 8745
diff changeset
2574 // is already set. Otherwise, we won't be at the beginning of a
5dd06f19e9be handle commands in the lexer
John W. Eaton <jwe@octave.org>
parents: 8745
diff changeset
2575 // statement.
5dd06f19e9be handle commands in the lexer
John W. Eaton <jwe@octave.org>
parents: 8745
diff changeset
2576
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
2577 if (looking_at_function_handle)
4930
bdb307dc8613 [project @ 2004-08-05 04:55:26 by jwe]
jwe
parents: 4923
diff changeset
2578 {
bdb307dc8613 [project @ 2004-08-05 04:55:26 by jwe]
jwe
parents: 4923
diff changeset
2579 if (kw_token)
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2580 {
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2581 error ("function handles may not refer to keywords");
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2582
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2583 return LEXICAL_ERROR;
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2584 }
4930
bdb307dc8613 [project @ 2004-08-05 04:55:26 by jwe]
jwe
parents: 4923
diff changeset
2585 else
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2586 {
16255
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
2587 push_token (new token (FCN_HANDLE, tok, input_line_number,
16148
10abbc493f50 delete unnecessary X-> references inside class X member functions
John W. Eaton <jwe@octave.org>
parents: 16146
diff changeset
2588 current_input_column));
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
2589
16130
421dea028bbf localize use of yyleng with lexical_feedback member function
John W. Eaton <jwe@octave.org>
parents: 16129
diff changeset
2590 current_input_column += flex_yyleng ();
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
2591 looking_for_object_index = true;
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2592
16257
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
2593 at_beginning_of_statement = false;
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
2594
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2595 return FCN_HANDLE;
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2596 }
4930
bdb307dc8613 [project @ 2004-08-05 04:55:26 by jwe]
jwe
parents: 4923
diff changeset
2597 }
bdb307dc8613 [project @ 2004-08-05 04:55:26 by jwe]
jwe
parents: 4923
diff changeset
2598
5102
b04b30d30c66 [project @ 2004-12-28 01:59:05 by jwe]
jwe
parents: 5088
diff changeset
2599 // If we have a regular keyword, return it.
7722
c3bb0b7a4261 lex.l: allow tokens to be displayed when parsed
John W. Eaton <jwe@octave.org>
parents: 7720
diff changeset
2600 // Keywords can be followed by identifiers.
146
edfa5a96c5f1 [project @ 1993-10-05 04:02:15 by jwe]
jwe
parents: 143
diff changeset
2601
edfa5a96c5f1 [project @ 1993-10-05 04:02:15 by jwe]
jwe
parents: 143
diff changeset
2602 if (kw_token)
edfa5a96c5f1 [project @ 1993-10-05 04:02:15 by jwe]
jwe
parents: 143
diff changeset
2603 {
7722
c3bb0b7a4261 lex.l: allow tokens to be displayed when parsed
John W. Eaton <jwe@octave.org>
parents: 7720
diff changeset
2604 if (kw_token >= 0)
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2605 {
16130
421dea028bbf localize use of yyleng with lexical_feedback member function
John W. Eaton <jwe@octave.org>
parents: 16129
diff changeset
2606 current_input_column += flex_yyleng ();
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
2607 looking_for_object_index = false;
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2608 }
7722
c3bb0b7a4261 lex.l: allow tokens to be displayed when parsed
John W. Eaton <jwe@octave.org>
parents: 7720
diff changeset
2609
c3bb0b7a4261 lex.l: allow tokens to be displayed when parsed
John W. Eaton <jwe@octave.org>
parents: 7720
diff changeset
2610 return kw_token;
146
edfa5a96c5f1 [project @ 1993-10-05 04:02:15 by jwe]
jwe
parents: 143
diff changeset
2611 }
edfa5a96c5f1 [project @ 1993-10-05 04:02:15 by jwe]
jwe
parents: 143
diff changeset
2612
4234
90e44267e8cf [project @ 2002-12-21 17:15:25 by jwe]
jwe
parents: 4233
diff changeset
2613 // Find the token in the symbol table. Beware the magic
90e44267e8cf [project @ 2002-12-21 17:15:25 by jwe]
jwe
parents: 4233
diff changeset
2614 // transformation of the end keyword...
90e44267e8cf [project @ 2002-12-21 17:15:25 by jwe]
jwe
parents: 4233
diff changeset
2615
90e44267e8cf [project @ 2002-12-21 17:15:25 by jwe]
jwe
parents: 4233
diff changeset
2616 if (tok == "end")
11586
12df7854fa7c strip trailing whitespace from source files
John W. Eaton <jwe@octave.org>
parents: 11523
diff changeset
2617 tok = "__end__";
146
edfa5a96c5f1 [project @ 1993-10-05 04:02:15 by jwe]
jwe
parents: 143
diff changeset
2618
16257
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
2619 token *tok_val = new token (NAME, &(symbol_table::insert (tok)),
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
2620 input_line_number, current_input_column);
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
2621
16269
488b0fef52c5 7/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16267
diff changeset
2622 if (at_beginning_of_statement
488b0fef52c5 7/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16267
diff changeset
2623 && (! (tok == "e"
488b0fef52c5 7/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16267
diff changeset
2624 || tok == "I" || tok == "i"
488b0fef52c5 7/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16267
diff changeset
2625 || tok == "J" || tok == "j"
488b0fef52c5 7/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16267
diff changeset
2626 || tok == "Inf" || tok == "inf"
488b0fef52c5 7/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16267
diff changeset
2627 || tok == "NaN" || tok == "nan")))
16257
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
2628 tok_val->mark_may_be_command ();
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
2629
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
2630 push_token (tok_val);
146
edfa5a96c5f1 [project @ 1993-10-05 04:02:15 by jwe]
jwe
parents: 143
diff changeset
2631
16130
421dea028bbf localize use of yyleng with lexical_feedback member function
John W. Eaton <jwe@octave.org>
parents: 16129
diff changeset
2632 current_input_column += flex_yyleng ();
146
edfa5a96c5f1 [project @ 1993-10-05 04:02:15 by jwe]
jwe
parents: 143
diff changeset
2633
9324
29563379fa9b don't look for object index after keyword
John W. Eaton <jwe@octave.org>
parents: 8974
diff changeset
2634 if (tok != "__end__")
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
2635 looking_for_object_index = true;
9324
29563379fa9b don't look for object index after keyword
John W. Eaton <jwe@octave.org>
parents: 8974
diff changeset
2636
16257
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
2637 at_beginning_of_statement = false;
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
2638
146
edfa5a96c5f1 [project @ 1993-10-05 04:02:15 by jwe]
jwe
parents: 143
diff changeset
2639 return NAME;
edfa5a96c5f1 [project @ 1993-10-05 04:02:15 by jwe]
jwe
parents: 143
diff changeset
2640 }
edfa5a96c5f1 [project @ 1993-10-05 04:02:15 by jwe]
jwe
parents: 143
diff changeset
2641
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
2642 void
16158
7eb614760ddb rename lexical_feedback to octave_lexer
John W. Eaton <jwe@octave.org>
parents: 16154
diff changeset
2643 octave_lexer::maybe_warn_separator_insert (char sep)
3388
42cb61dd0248 [project @ 1999-12-16 04:51:43 by jwe]
jwe
parents: 3351
diff changeset
2644 {
16203
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
2645 std::string nm = fcn_file_full_name;
3388
42cb61dd0248 [project @ 1999-12-16 04:51:43 by jwe]
jwe
parents: 3351
diff changeset
2646
5794
1138ced03f14 [project @ 2006-05-08 20:23:04 by jwe]
jwe
parents: 5783
diff changeset
2647 if (nm.empty ())
1138ced03f14 [project @ 2006-05-08 20:23:04 by jwe]
jwe
parents: 5783
diff changeset
2648 warning_with_id ("Octave:separator-insert",
15466
d174210ce1ec use ' instead of ` in error messages, warnings and most comments
John W. Eaton <jwe@octave.org>
parents: 14161
diff changeset
2649 "potential auto-insertion of '%c' near line %d",
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
2650 sep, input_line_number);
5794
1138ced03f14 [project @ 2006-05-08 20:23:04 by jwe]
jwe
parents: 5783
diff changeset
2651 else
1138ced03f14 [project @ 2006-05-08 20:23:04 by jwe]
jwe
parents: 5783
diff changeset
2652 warning_with_id ("Octave:separator-insert",
15466
d174210ce1ec use ' instead of ` in error messages, warnings and most comments
John W. Eaton <jwe@octave.org>
parents: 14161
diff changeset
2653 "potential auto-insertion of '%c' near line %d of file %s",
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
2654 sep, input_line_number, nm.c_str ());
3388
42cb61dd0248 [project @ 1999-12-16 04:51:43 by jwe]
jwe
parents: 3351
diff changeset
2655 }
42cb61dd0248 [project @ 1999-12-16 04:51:43 by jwe]
jwe
parents: 3351
diff changeset
2656
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
2657 void
16158
7eb614760ddb rename lexical_feedback to octave_lexer
John W. Eaton <jwe@octave.org>
parents: 16154
diff changeset
2658 octave_lexer::gripe_single_quote_string (void)
3400
18366d37e7dd [project @ 1999-12-22 23:36:09 by jwe]
jwe
parents: 3388
diff changeset
2659 {
16203
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
2660 std::string nm = fcn_file_full_name;
3400
18366d37e7dd [project @ 1999-12-22 23:36:09 by jwe]
jwe
parents: 3388
diff changeset
2661
5794
1138ced03f14 [project @ 2006-05-08 20:23:04 by jwe]
jwe
parents: 5783
diff changeset
2662 if (nm.empty ())
1138ced03f14 [project @ 2006-05-08 20:23:04 by jwe]
jwe
parents: 5783
diff changeset
2663 warning_with_id ("Octave:single-quote-string",
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2664 "single quote delimited string near line %d",
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
2665 input_line_number);
5794
1138ced03f14 [project @ 2006-05-08 20:23:04 by jwe]
jwe
parents: 5783
diff changeset
2666 else
1138ced03f14 [project @ 2006-05-08 20:23:04 by jwe]
jwe
parents: 5783
diff changeset
2667 warning_with_id ("Octave:single-quote-string",
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2668 "single quote delimited string near line %d of file %s",
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
2669 input_line_number, nm.c_str ());
3400
18366d37e7dd [project @ 1999-12-22 23:36:09 by jwe]
jwe
parents: 3388
diff changeset
2670 }
18366d37e7dd [project @ 1999-12-22 23:36:09 by jwe]
jwe
parents: 3388
diff changeset
2671
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
2672 void
16158
7eb614760ddb rename lexical_feedback to octave_lexer
John W. Eaton <jwe@octave.org>
parents: 16154
diff changeset
2673 octave_lexer::gripe_matlab_incompatible (const std::string& msg)
4037
7e0c73f17a5d [project @ 2002-08-13 02:02:26 by jwe]
jwe
parents: 4018
diff changeset
2674 {
16203
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
2675 std::string nm = fcn_file_full_name;
8974
fde2a916b2ac include line and file info in parser warnings
John W. Eaton <jwe@octave.org>
parents: 8950
diff changeset
2676
fde2a916b2ac include line and file info in parser warnings
John W. Eaton <jwe@octave.org>
parents: 8950
diff changeset
2677 if (nm.empty ())
fde2a916b2ac include line and file info in parser warnings
John W. Eaton <jwe@octave.org>
parents: 8950
diff changeset
2678 warning_with_id ("Octave:matlab-incompatible",
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2679 "potential Matlab compatibility problem: %s",
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2680 msg.c_str ());
8974
fde2a916b2ac include line and file info in parser warnings
John W. Eaton <jwe@octave.org>
parents: 8950
diff changeset
2681 else
fde2a916b2ac include line and file info in parser warnings
John W. Eaton <jwe@octave.org>
parents: 8950
diff changeset
2682 warning_with_id ("Octave:matlab-incompatible",
10315
57a59eae83cc untabify src C++ source files
John W. Eaton <jwe@octave.org>
parents: 10182
diff changeset
2683 "potential Matlab compatibility problem: %s near line %d offile %s",
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
2684 msg.c_str (), input_line_number, nm.c_str ());
4037
7e0c73f17a5d [project @ 2002-08-13 02:02:26 by jwe]
jwe
parents: 4018
diff changeset
2685 }
7e0c73f17a5d [project @ 2002-08-13 02:02:26 by jwe]
jwe
parents: 4018
diff changeset
2686
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
2687 void
16158
7eb614760ddb rename lexical_feedback to octave_lexer
John W. Eaton <jwe@octave.org>
parents: 16154
diff changeset
2688 octave_lexer::maybe_gripe_matlab_incompatible_comment (char c)
4037
7e0c73f17a5d [project @ 2002-08-13 02:02:26 by jwe]
jwe
parents: 4018
diff changeset
2689 {
7e0c73f17a5d [project @ 2002-08-13 02:02:26 by jwe]
jwe
parents: 4018
diff changeset
2690 if (c == '#')
7e0c73f17a5d [project @ 2002-08-13 02:02:26 by jwe]
jwe
parents: 4018
diff changeset
2691 gripe_matlab_incompatible ("# used as comment character");
7e0c73f17a5d [project @ 2002-08-13 02:02:26 by jwe]
jwe
parents: 4018
diff changeset
2692 }
7e0c73f17a5d [project @ 2002-08-13 02:02:26 by jwe]
jwe
parents: 4018
diff changeset
2693
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
2694 void
16158
7eb614760ddb rename lexical_feedback to octave_lexer
John W. Eaton <jwe@octave.org>
parents: 16154
diff changeset
2695 octave_lexer::gripe_matlab_incompatible_continuation (void)
4037
7e0c73f17a5d [project @ 2002-08-13 02:02:26 by jwe]
jwe
parents: 4018
diff changeset
2696 {
7e0c73f17a5d [project @ 2002-08-13 02:02:26 by jwe]
jwe
parents: 4018
diff changeset
2697 gripe_matlab_incompatible ("\\ used as line continuation marker");
7e0c73f17a5d [project @ 2002-08-13 02:02:26 by jwe]
jwe
parents: 4018
diff changeset
2698 }
7e0c73f17a5d [project @ 2002-08-13 02:02:26 by jwe]
jwe
parents: 4018
diff changeset
2699
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
2700 void
16158
7eb614760ddb rename lexical_feedback to octave_lexer
John W. Eaton <jwe@octave.org>
parents: 16154
diff changeset
2701 octave_lexer::gripe_matlab_incompatible_operator (const std::string& op)
4037
7e0c73f17a5d [project @ 2002-08-13 02:02:26 by jwe]
jwe
parents: 4018
diff changeset
2702 {
7e0c73f17a5d [project @ 2002-08-13 02:02:26 by jwe]
jwe
parents: 4018
diff changeset
2703 std::string t = op;
7e0c73f17a5d [project @ 2002-08-13 02:02:26 by jwe]
jwe
parents: 4018
diff changeset
2704 int n = t.length ();
7e0c73f17a5d [project @ 2002-08-13 02:02:26 by jwe]
jwe
parents: 4018
diff changeset
2705 if (t[n-1] == '\n')
7e0c73f17a5d [project @ 2002-08-13 02:02:26 by jwe]
jwe
parents: 4018
diff changeset
2706 t.resize (n-1);
7e0c73f17a5d [project @ 2002-08-13 02:02:26 by jwe]
jwe
parents: 4018
diff changeset
2707 gripe_matlab_incompatible (t + " used as operator");
7e0c73f17a5d [project @ 2002-08-13 02:02:26 by jwe]
jwe
parents: 4018
diff changeset
2708 }
16119
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2709
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2710 void
16158
7eb614760ddb rename lexical_feedback to octave_lexer
John W. Eaton <jwe@octave.org>
parents: 16154
diff changeset
2711 octave_lexer::push_token (token *tok)
16126
5c885c13bfa3 localize use of yylval.tok_val in lexical_feedback member functions
John W. Eaton <jwe@octave.org>
parents: 16125
diff changeset
2712 {
16149
49dfba4fd3c5 use pure parser and reentrant lexer interfaces
John W. Eaton <jwe@octave.org>
parents: 16148
diff changeset
2713 YYSTYPE *lval = yyget_lval (scanner);
49dfba4fd3c5 use pure parser and reentrant lexer interfaces
John W. Eaton <jwe@octave.org>
parents: 16148
diff changeset
2714 lval->tok_val = tok;
16230
4bf907906134 use a queue to hold tokens in the lexer
John W. Eaton <jwe@octave.org>
parents: 16229
diff changeset
2715 tokens.push (tok);
16126
5c885c13bfa3 localize use of yylval.tok_val in lexical_feedback member functions
John W. Eaton <jwe@octave.org>
parents: 16125
diff changeset
2716 }
5c885c13bfa3 localize use of yylval.tok_val in lexical_feedback member functions
John W. Eaton <jwe@octave.org>
parents: 16125
diff changeset
2717
5c885c13bfa3 localize use of yylval.tok_val in lexical_feedback member functions
John W. Eaton <jwe@octave.org>
parents: 16125
diff changeset
2718 token *
16158
7eb614760ddb rename lexical_feedback to octave_lexer
John W. Eaton <jwe@octave.org>
parents: 16154
diff changeset
2719 octave_lexer::current_token (void)
16126
5c885c13bfa3 localize use of yylval.tok_val in lexical_feedback member functions
John W. Eaton <jwe@octave.org>
parents: 16125
diff changeset
2720 {
16149
49dfba4fd3c5 use pure parser and reentrant lexer interfaces
John W. Eaton <jwe@octave.org>
parents: 16148
diff changeset
2721 YYSTYPE *lval = yyget_lval (scanner);
49dfba4fd3c5 use pure parser and reentrant lexer interfaces
John W. Eaton <jwe@octave.org>
parents: 16148
diff changeset
2722 return lval->tok_val;
16126
5c885c13bfa3 localize use of yylval.tok_val in lexical_feedback member functions
John W. Eaton <jwe@octave.org>
parents: 16125
diff changeset
2723 }
5c885c13bfa3 localize use of yylval.tok_val in lexical_feedback member functions
John W. Eaton <jwe@octave.org>
parents: 16125
diff changeset
2724
5c885c13bfa3 localize use of yylval.tok_val in lexical_feedback member functions
John W. Eaton <jwe@octave.org>
parents: 16125
diff changeset
2725 void
16158
7eb614760ddb rename lexical_feedback to octave_lexer
John W. Eaton <jwe@octave.org>
parents: 16154
diff changeset
2726 octave_lexer::display_token (int tok)
16119
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2727 {
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2728 switch (tok)
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2729 {
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2730 case '=': std::cerr << "'='\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2731 case ':': std::cerr << "':'\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2732 case '-': std::cerr << "'-'\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2733 case '+': std::cerr << "'+'\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2734 case '*': std::cerr << "'*'\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2735 case '/': std::cerr << "'/'\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2736 case ADD_EQ: std::cerr << "ADD_EQ\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2737 case SUB_EQ: std::cerr << "SUB_EQ\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2738 case MUL_EQ: std::cerr << "MUL_EQ\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2739 case DIV_EQ: std::cerr << "DIV_EQ\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2740 case LEFTDIV_EQ: std::cerr << "LEFTDIV_EQ\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2741 case POW_EQ: std::cerr << "POW_EQ\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2742 case EMUL_EQ: std::cerr << "EMUL_EQ\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2743 case EDIV_EQ: std::cerr << "EDIV_EQ\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2744 case ELEFTDIV_EQ: std::cerr << "ELEFTDIV_EQ\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2745 case EPOW_EQ: std::cerr << "EPOW_EQ\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2746 case AND_EQ: std::cerr << "AND_EQ\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2747 case OR_EQ: std::cerr << "OR_EQ\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2748 case LSHIFT_EQ: std::cerr << "LSHIFT_EQ\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2749 case RSHIFT_EQ: std::cerr << "RSHIFT_EQ\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2750 case LSHIFT: std::cerr << "LSHIFT\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2751 case RSHIFT: std::cerr << "RSHIFT\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2752 case EXPR_AND_AND: std::cerr << "EXPR_AND_AND\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2753 case EXPR_OR_OR: std::cerr << "EXPR_OR_OR\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2754 case EXPR_AND: std::cerr << "EXPR_AND\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2755 case EXPR_OR: std::cerr << "EXPR_OR\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2756 case EXPR_NOT: std::cerr << "EXPR_NOT\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2757 case EXPR_LT: std::cerr << "EXPR_LT\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2758 case EXPR_LE: std::cerr << "EXPR_LE\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2759 case EXPR_EQ: std::cerr << "EXPR_EQ\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2760 case EXPR_NE: std::cerr << "EXPR_NE\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2761 case EXPR_GE: std::cerr << "EXPR_GE\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2762 case EXPR_GT: std::cerr << "EXPR_GT\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2763 case LEFTDIV: std::cerr << "LEFTDIV\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2764 case EMUL: std::cerr << "EMUL\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2765 case EDIV: std::cerr << "EDIV\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2766 case ELEFTDIV: std::cerr << "ELEFTDIV\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2767 case EPLUS: std::cerr << "EPLUS\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2768 case EMINUS: std::cerr << "EMINUS\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2769 case QUOTE: std::cerr << "QUOTE\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2770 case TRANSPOSE: std::cerr << "TRANSPOSE\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2771 case PLUS_PLUS: std::cerr << "PLUS_PLUS\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2772 case MINUS_MINUS: std::cerr << "MINUS_MINUS\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2773 case POW: std::cerr << "POW\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2774 case EPOW: std::cerr << "EPOW\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2775
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2776 case NUM:
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2777 case IMAG_NUM:
16126
5c885c13bfa3 localize use of yylval.tok_val in lexical_feedback member functions
John W. Eaton <jwe@octave.org>
parents: 16125
diff changeset
2778 {
16148
10abbc493f50 delete unnecessary X-> references inside class X member functions
John W. Eaton <jwe@octave.org>
parents: 16146
diff changeset
2779 token *tok_val = current_token ();
16126
5c885c13bfa3 localize use of yylval.tok_val in lexical_feedback member functions
John W. Eaton <jwe@octave.org>
parents: 16125
diff changeset
2780 std::cerr << (tok == NUM ? "NUM" : "IMAG_NUM")
5c885c13bfa3 localize use of yylval.tok_val in lexical_feedback member functions
John W. Eaton <jwe@octave.org>
parents: 16125
diff changeset
2781 << " [" << tok_val->number () << "]\n";
5c885c13bfa3 localize use of yylval.tok_val in lexical_feedback member functions
John W. Eaton <jwe@octave.org>
parents: 16125
diff changeset
2782 }
16119
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2783 break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2784
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2785 case STRUCT_ELT:
16126
5c885c13bfa3 localize use of yylval.tok_val in lexical_feedback member functions
John W. Eaton <jwe@octave.org>
parents: 16125
diff changeset
2786 {
16148
10abbc493f50 delete unnecessary X-> references inside class X member functions
John W. Eaton <jwe@octave.org>
parents: 16146
diff changeset
2787 token *tok_val = current_token ();
16126
5c885c13bfa3 localize use of yylval.tok_val in lexical_feedback member functions
John W. Eaton <jwe@octave.org>
parents: 16125
diff changeset
2788 std::cerr << "STRUCT_ELT [" << tok_val->text () << "]\n";
5c885c13bfa3 localize use of yylval.tok_val in lexical_feedback member functions
John W. Eaton <jwe@octave.org>
parents: 16125
diff changeset
2789 }
5c885c13bfa3 localize use of yylval.tok_val in lexical_feedback member functions
John W. Eaton <jwe@octave.org>
parents: 16125
diff changeset
2790 break;
16119
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2791
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2792 case NAME:
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2793 {
16148
10abbc493f50 delete unnecessary X-> references inside class X member functions
John W. Eaton <jwe@octave.org>
parents: 16146
diff changeset
2794 token *tok_val = current_token ();
16126
5c885c13bfa3 localize use of yylval.tok_val in lexical_feedback member functions
John W. Eaton <jwe@octave.org>
parents: 16125
diff changeset
2795 symbol_table::symbol_record *sr = tok_val->sym_rec ();
16119
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2796 std::cerr << "NAME";
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2797 if (sr)
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2798 std::cerr << " [" << sr->name () << "]";
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2799 std::cerr << "\n";
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2800 }
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2801 break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2802
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2803 case END: std::cerr << "END\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2804
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2805 case DQ_STRING:
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2806 case SQ_STRING:
16126
5c885c13bfa3 localize use of yylval.tok_val in lexical_feedback member functions
John W. Eaton <jwe@octave.org>
parents: 16125
diff changeset
2807 {
16148
10abbc493f50 delete unnecessary X-> references inside class X member functions
John W. Eaton <jwe@octave.org>
parents: 16146
diff changeset
2808 token *tok_val = current_token ();
16126
5c885c13bfa3 localize use of yylval.tok_val in lexical_feedback member functions
John W. Eaton <jwe@octave.org>
parents: 16125
diff changeset
2809
5c885c13bfa3 localize use of yylval.tok_val in lexical_feedback member functions
John W. Eaton <jwe@octave.org>
parents: 16125
diff changeset
2810 std::cerr << (tok == DQ_STRING ? "DQ_STRING" : "SQ_STRING")
5c885c13bfa3 localize use of yylval.tok_val in lexical_feedback member functions
John W. Eaton <jwe@octave.org>
parents: 16125
diff changeset
2811 << " [" << tok_val->text () << "]\n";
5c885c13bfa3 localize use of yylval.tok_val in lexical_feedback member functions
John W. Eaton <jwe@octave.org>
parents: 16125
diff changeset
2812 }
16119
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2813 break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2814
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2815 case FOR: std::cerr << "FOR\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2816 case WHILE: std::cerr << "WHILE\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2817 case DO: std::cerr << "DO\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2818 case UNTIL: std::cerr << "UNTIL\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2819 case IF: std::cerr << "IF\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2820 case ELSEIF: std::cerr << "ELSEIF\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2821 case ELSE: std::cerr << "ELSE\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2822 case SWITCH: std::cerr << "SWITCH\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2823 case CASE: std::cerr << "CASE\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2824 case OTHERWISE: std::cerr << "OTHERWISE\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2825 case BREAK: std::cerr << "BREAK\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2826 case CONTINUE: std::cerr << "CONTINUE\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2827 case FUNC_RET: std::cerr << "FUNC_RET\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2828 case UNWIND: std::cerr << "UNWIND\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2829 case CLEANUP: std::cerr << "CLEANUP\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2830 case TRY: std::cerr << "TRY\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2831 case CATCH: std::cerr << "CATCH\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2832 case GLOBAL: std::cerr << "GLOBAL\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2833 case PERSISTENT: std::cerr << "PERSISTENT\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2834 case FCN_HANDLE: std::cerr << "FCN_HANDLE\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2835 case END_OF_INPUT: std::cerr << "END_OF_INPUT\n\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2836 case LEXICAL_ERROR: std::cerr << "LEXICAL_ERROR\n\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2837 case FCN: std::cerr << "FCN\n"; break;
16203
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
2838 case INPUT_FILE: std::cerr << "INPUT_FILE\n"; break;
16119
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2839 case SUPERCLASSREF: std::cerr << "SUPERCLASSREF\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2840 case METAQUERY: std::cerr << "METAQUERY\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2841 case GET: std::cerr << "GET\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2842 case SET: std::cerr << "SET\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2843 case PROPERTIES: std::cerr << "PROPERTIES\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2844 case METHODS: std::cerr << "METHODS\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2845 case EVENTS: std::cerr << "EVENTS\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2846 case CLASSDEF: std::cerr << "CLASSDEF\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2847 case '\n': std::cerr << "\\n\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2848 case '\r': std::cerr << "\\r\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2849 case '\t': std::cerr << "TAB\n"; break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2850 default:
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2851 {
16166
cb80b1d062b1 don't attempt to print unprintable tokens
John W. Eaton <jwe@octave.org>
parents: 16164
diff changeset
2852 if (tok < 256 && tok > 31)
16119
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2853 std::cerr << static_cast<char> (tok) << "\n";
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2854 else
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2855 std::cerr << "UNKNOWN(" << tok << ")\n";
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2856 }
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2857 break;
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2858 }
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2859 }
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2860
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2861 void
16158
7eb614760ddb rename lexical_feedback to octave_lexer
John W. Eaton <jwe@octave.org>
parents: 16154
diff changeset
2862 octave_lexer::fatal_error (const char *msg)
16129
053b0364b507 localize use of yy_fatal_error with lexical_feedback member function
John W. Eaton <jwe@octave.org>
parents: 16128
diff changeset
2863 {
053b0364b507 localize use of yy_fatal_error with lexical_feedback member function
John W. Eaton <jwe@octave.org>
parents: 16128
diff changeset
2864 error (msg);
053b0364b507 localize use of yy_fatal_error with lexical_feedback member function
John W. Eaton <jwe@octave.org>
parents: 16128
diff changeset
2865
053b0364b507 localize use of yy_fatal_error with lexical_feedback member function
John W. Eaton <jwe@octave.org>
parents: 16128
diff changeset
2866 OCTAVE_QUIT;
053b0364b507 localize use of yy_fatal_error with lexical_feedback member function
John W. Eaton <jwe@octave.org>
parents: 16128
diff changeset
2867
16149
49dfba4fd3c5 use pure parser and reentrant lexer interfaces
John W. Eaton <jwe@octave.org>
parents: 16148
diff changeset
2868 yy_fatal_error (msg, scanner);
16129
053b0364b507 localize use of yy_fatal_error with lexical_feedback member function
John W. Eaton <jwe@octave.org>
parents: 16128
diff changeset
2869 }
053b0364b507 localize use of yy_fatal_error with lexical_feedback member function
John W. Eaton <jwe@octave.org>
parents: 16128
diff changeset
2870
053b0364b507 localize use of yy_fatal_error with lexical_feedback member function
John W. Eaton <jwe@octave.org>
parents: 16128
diff changeset
2871 void
16229
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
2872 octave_lexer::lexer_debug (const char *pattern)
16119
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2873 {
16229
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
2874 if (lexer_debug_flag)
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
2875 {
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
2876 std::cerr << std::endl;
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
2877
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
2878 display_start_state ();
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
2879
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
2880 std::cerr << "P: " << pattern << std::endl;
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
2881 std::cerr << "T: " << flex_yytext () << std::endl;
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
2882 }
16119
b31eb56f4d84 maint: reorder class definitions in lex.ll
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
2883 }
16224
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
2884
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
2885 void
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
2886 octave_lexer::push_start_state (int state)
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
2887 {
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
2888 OCTAVE_YYG;
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
2889
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
2890 start_state_stack.push (state);
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
2891
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
2892 BEGIN (start_state ());
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
2893 }
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
2894
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
2895 void
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
2896 octave_lexer::pop_start_state (void)
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
2897 {
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
2898 OCTAVE_YYG;
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
2899
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
2900 start_state_stack.pop ();
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
2901
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
2902 BEGIN (start_state ());
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
2903 }
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
2904
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
2905 void
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
2906 octave_lexer::clear_start_state (void)
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
2907 {
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
2908 while (! start_state_stack.empty ())
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
2909 start_state_stack.pop ();
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
2910
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
2911 push_start_state (INITIAL);
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
2912 }
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
2913
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
2914 void
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
2915 octave_lexer::display_start_state (void) const
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
2916 {
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
2917 std::cerr << "S: ";
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
2918
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
2919 switch (start_state ())
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
2920 {
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
2921 case INITIAL:
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
2922 std::cerr << "INITIAL" << std::endl;
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
2923 break;
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
2924
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
2925 case COMMAND_START:
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
2926 std::cerr << "COMMAND_START" << std::endl;
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
2927 break;
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
2928
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
2929 case MATRIX_START:
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
2930 std::cerr << "MATRIX_START" << std::endl;
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
2931 break;
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
2932
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
2933 case INPUT_FILE_START:
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
2934 std::cerr << "INPUT_FILE_BEGIN" << std::endl;
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
2935 break;
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
2936
16228
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
2937 case BLOCK_COMMENT_START:
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
2938 std::cerr << "BLOCK_COMMENT_START" << std::endl;
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
2939 break;
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
2940
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
2941 case LINE_COMMENT_START:
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
2942 std::cerr << "LINE_COMMENT_START" << std::endl;
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
2943 break;
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
2944
16224
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
2945 default:
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
2946 std::cerr << "UNKNOWN START STATE!" << std::endl;
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
2947 break;
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
2948 }
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
2949 }
16229
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
2950
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
2951 int
16275
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
2952 octave_lexer::handle_op (const char *pattern, int tok, bool bos)
16229
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
2953 {
16263
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
2954 lexer_debug (pattern);
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
2955
16275
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
2956 return handle_op_internal (tok, bos, true);
16229
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
2957 }
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
2958
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
2959 int
16275
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
2960 octave_lexer::handle_incompatible_op (const char *pattern, int tok, bool bos)
16229
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
2961 {
16263
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
2962 lexer_debug (pattern);
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
2963
16275
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
2964 return handle_op_internal (tok, bos, false);
16229
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
2965 }
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
2966
16263
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
2967 bool
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
2968 octave_lexer::maybe_unput_comma_before_unary_op (int tok)
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
2969 {
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
2970 int prev_tok = previous_token_value ();
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
2971
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
2972 bool unput_comma = false;
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
2973
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
2974 if (whitespace_is_significant () && space_follows_previous_token ())
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
2975 {
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
2976 int c = text_yyinput ();
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
2977 xunput (c);
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
2978
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
2979 bool space_after = (c == ' ' || c == '\t');
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
2980
16267
15f55df088e7 6/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16265
diff changeset
2981 if (! (prev_tok == '[' || prev_tok == '{'
16263
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
2982 || previous_token_is_binop ()
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
2983 || ((tok == '+' || tok == '-') && space_after)))
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
2984 unput_comma = true;
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
2985 }
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
2986
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
2987 return unput_comma;
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
2988 }
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
2989
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
2990 int
16275
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
2991 octave_lexer::handle_unary_op (int tok, bool bos)
16263
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
2992 {
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
2993 return maybe_unput_comma_before_unary_op (tok)
16275
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
2994 ? -1 : handle_op_internal (tok, bos, true);
16263
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
2995 }
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
2996
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
2997 int
16275
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
2998 octave_lexer::handle_incompatible_unary_op (int tok, bool bos)
16263
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
2999 {
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
3000 return maybe_unput_comma_before_unary_op (tok)
16275
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
3001 ? -1 : handle_op_internal (tok, bos, false);
16263
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
3002 }
9acb86e6ac90 4/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16261
diff changeset
3003
16229
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
3004 int
16275
dd7de0345124 10/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16273
diff changeset
3005 octave_lexer::handle_op_internal (int tok, bool bos, bool compat)
16229
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
3006 {
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
3007 if (! compat)
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
3008 gripe_matlab_incompatible_operator (flex_yytext ());
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
3009
16255
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
3010 push_token (new token (tok, input_line_number, current_input_column));
16229
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
3011
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
3012 current_input_column += flex_yyleng ();
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
3013 looking_for_object_index = false;
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
3014 at_beginning_of_statement = bos;
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
3015
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
3016 return count_token (tok);
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
3017 }
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
3018
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
3019 int
16231
2b15ae55c721 put all tokens in the token cache
John W. Eaton <jwe@octave.org>
parents: 16230
diff changeset
3020 octave_lexer::handle_token (const std::string& name, int tok)
16229
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
3021 {
16255
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
3022 token *tok_val = new token (tok, name, input_line_number,
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
3023 current_input_column);
16231
2b15ae55c721 put all tokens in the token cache
John W. Eaton <jwe@octave.org>
parents: 16230
diff changeset
3024
2b15ae55c721 put all tokens in the token cache
John W. Eaton <jwe@octave.org>
parents: 16230
diff changeset
3025 return handle_token (tok, tok_val);
16229
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
3026 }
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
3027
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
3028 int
16231
2b15ae55c721 put all tokens in the token cache
John W. Eaton <jwe@octave.org>
parents: 16230
diff changeset
3029 octave_lexer::handle_token (int tok, token *tok_val)
16229
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
3030 {
16231
2b15ae55c721 put all tokens in the token cache
John W. Eaton <jwe@octave.org>
parents: 16230
diff changeset
3031 if (! tok_val)
16255
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
3032 tok_val = new token (tok, input_line_number, current_input_column);
16231
2b15ae55c721 put all tokens in the token cache
John W. Eaton <jwe@octave.org>
parents: 16230
diff changeset
3033
2b15ae55c721 put all tokens in the token cache
John W. Eaton <jwe@octave.org>
parents: 16230
diff changeset
3034 push_token (tok_val);
2b15ae55c721 put all tokens in the token cache
John W. Eaton <jwe@octave.org>
parents: 16230
diff changeset
3035
16229
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
3036 current_input_column += flex_yyleng ();
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
3037
16257
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
3038 return count_token_internal (tok);
16229
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
3039 }
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
3040
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
3041 int
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
3042 octave_lexer::count_token (int tok)
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
3043 {
16257
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
3044 token *tok_val = new token (tok, input_line_number, current_input_column);
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
3045
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
3046 push_token (tok_val);
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
3047
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
3048 return count_token_internal (tok);
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
3049 }
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
3050
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
3051 int
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
3052 octave_lexer::count_token_internal (int tok)
db7f07b22b9b 1/10 commits reworking the lexer
John W. Eaton <jwe@octave.org>
parents: 16255
diff changeset
3053 {
16229
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
3054 if (tok != '\n')
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
3055 {
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
3056 Vtoken_count++;
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
3057 token_count++;
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
3058 }
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
3059
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
3060 return show_token (tok);
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
3061 }
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
3062
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
3063 int
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
3064 octave_lexer::show_token (int tok)
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
3065 {
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
3066 if (Vdisplay_tokens)
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
3067 display_token (tok);
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
3068
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
3069 if (lexer_debug_flag)
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
3070 {
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
3071 std::cerr << "R: ";
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
3072 display_token (tok);
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
3073 std::cerr << std::endl;
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
3074 }
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
3075
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
3076 return tok;
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
3077 }