view libinterp/parse-tree/parse.h @ 16149:49dfba4fd3c5

use pure parser and reentrant lexer interfaces Making the Octave parser and lexer properly reentrant (and perhaps eventually thread safe as well) is still a work in progress. With the current set of changes the parser and lexer still use many global variables, so these changes alone do NOT make the Octave parser reentrant unless you take care to properly save and restore (typically with an unwind_protect object) relevant global values before and after calling the parser. Even if global variables are properly saved and restored, the parser will NOT be thread safe. * lex.ll: Use %option reentrant an %option bison-bridge. (yylval): Delete macro. (YY_EXTRA_TYPE, curr_lexer): New macros. Undefine curr_lexer (YY_FATAL_ERROR): Update decl for reentrant scanner. (lexical_feedback::reset): Update call to yyrestart for reentrant scanner interface. (lexical_feedback::fatal_error): Update call to yy_fatal_error for reentrant scanner interface. (lexical_feedback::text_yyinput): Update calls to yyinput and yyunput for reentrant scanner interface. (lexical_feedback::flex_yyleng): Use function interface to access yyleng. (lexical_feedback::flex_yytext): Use function interface to access yytext. (lexical_feedback::push_token, lexical_feedback::current_token): Use function interface to access yylval. * oct-parse.yy: Use %define api.pure, %parse-param, and %lex-param options. (curr_lexer): Define for syntax rules section. (scanner): New macro. * oct-parse.yy: Include oct-parse.h. (octave_lex): Declare. (yyerror): Update declaration for pure parser. * parse.h (octave_lex): Delete decl. * oct-parse.yy (octave_parser::run): Pass pointer to octave_parser object to octave_parse. * lex.ll (lexical_feedback::octave_read): Call fatal_error directly instead of using YY_FATAL_ERROR. * oct-parse.yy (parse_fcn_file): Pass line and column info for lexter to gobble_leading_whitespace. Access prep_for_script_file, prep_for_function_file, parsing_class_method, input_line_number, and current_input_column through curr_parser. * parse.h, oct-parse.yy (YY_BUFFER_STATE, create_buffer, current_buffer, switch_to_buffer, delete_buffer, clear_all_buffers): Delete. * toplev.cc (main_loop): Don't create new buffer for lexer. * input.cc (get_debug_input): Likewise. * oct-parse.yy (eval_string, parse_fcn_file): Likewise. * octave.cc (octave_initialize_interpreter): Likewise. * input.cc (get_debug_input): Likewise. * oct-parse.yy (eval_string, parse_fcn_file): Create parser as needed. * octave.cc (octave_initialize_interpreter): Likewise. * input.cc (get_debug_input): Likewise. * input.cc (input_even_hook): Allow function to run even if currently defining a function. * lex.h, lex.ll (curr_lexer): Delete global variable. * parse.h, oct-parse.yy (octave_parser::curr_lexer): New data member. (octave_parser::octave_parser): Create lexer here. (curr_parser): Delete global variable. * toplev.cc (main_loop): Don't protect global curr_lexer and curr_parser variables. * oct-parse.yy (eval_string, parse_fcn_file): Likewise. * input.cc (get_debug_input): Likewise. * lex.h, lex.ll (curr_lexer): Delete global variable. * parse.h, oct-parse.yy (CURR_LEXER): New temporary global. (octave_parser::octave_parser): Set global CURR_LEXER here. * toplev.cc (main_loop): Protect CURR_LEXER prior to constructing new parser object. * input.cc (get_debug_input): Likewise. * oct-parse.yy (eval_string, parse_fcn_file): Likewise. * lex.h, lex.ll (lexical_feedback::scanner): New data member. (lexical_feedback::init): Create it. Call yylex_set_extra to store pointer to lexical_feedback object in scanner data. (lexical_feedback::~lexical_feedback): Delete it. * lex.ll (YYG): New macro. (lexical_feedback::reset, lexical_feedback::prep_for_script_file, lexical_feedback::prep_for_function_file, lexical_feedback::process_comment, lexical_feedback::handle_close_bracket, lexical_feedback::handle_identifier, lexical_feedback::lexer_debug): Use it to access scanner data.
author John W. Eaton <jwe@octave.org>
date Wed, 27 Feb 2013 18:49:16 -0500
parents 26d65d677557
children a57c2c8c8163
line wrap: on
line source

/*

Copyright (C) 1993-2012 John W. Eaton

This file is part of Octave.

Octave is free software; you can redistribute it and/or modify it
under the terms of the GNU General Public License as published by the
Free Software Foundation; either version 3 of the License, or (at your
option) any later version.

Octave is distributed in the hope that it will be useful, but WITHOUT
ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
for more details.

You should have received a copy of the GNU General Public License
along with Octave; see the file COPYING.  If not, see
<http://www.gnu.org/licenses/>.

*/

#if !defined (octave_parse_h)
#define octave_parse_h 1

#include <cstdio>

#include <string>

#include <stack>

#include "lex.h"
#include "token.h"

class octave_comment_list;
class octave_function;
class octave_user_function;
class tree;
class tree_anon_fcn_handle;
class tree_argument_list;
class tree_cell;
class tree_colon_expression;
class tree_command;
class tree_constant;
class tree_decl_command;
class tree_decl_init_list;
class tree_expression;
class tree_fcn_handle;
class tree_function_def;
class tree_identifier;
class tree_if_clause;
class tree_if_command;
class tree_if_command_list;
class tree_index_expression;
class tree_matrix;
class tree_matrix;
class tree_parameter_list;
class tree_statement;
class tree_statement_list;
class tree_statement_listtree_statement;
class tree_switch_case;
class tree_switch_case_list;
class tree_switch_command;

#include "oct-obj.h"

// Nonzero means print parser debugging info (-d).
extern int octave_debug;

// Buffer for help text snagged from function files.
extern std::stack<std::string> help_buf;

// TRUE means we are using readline.
extern bool line_editing;

// TRUE means we printed messages about reading startup files.
extern bool reading_startup_message_printed;

// TRUE means input is coming from startup file.
extern bool input_from_startup_file;

// Name of the current class when we are parsing class methods or
// constructors.
extern std::string current_class_name;

extern OCTINTERP_API std::string
get_help_from_file (const std::string& nm, bool& symbol_found,
                    std::string& file);

extern OCTINTERP_API std::string
get_help_from_file (const std::string& nm, bool& symbol_found);

extern OCTINTERP_API std::string lookup_autoload (const std::string& nm);

extern OCTINTERP_API string_vector autoloaded_functions (void);

extern OCTINTERP_API string_vector reverse_lookup_autoload (const std::string& nm);

extern OCTINTERP_API octave_function *
load_fcn_from_file (const std::string& file_name,
                    const std::string& dir_name = std::string (),
                    const std::string& dispatch_type = std::string (),
                    const std::string& fcn_name = std::string (),
                    bool autoload = false);

extern OCTINTERP_API void
source_file (const std::string& file_name,
             const std::string& context = std::string (),
             bool verbose = false, bool require_file = true,
             const std::string& warn_for = std::string ());

extern OCTINTERP_API octave_value_list
feval (const std::string& name,
       const octave_value_list& args = octave_value_list (),
       int nargout = 0);

extern OCTINTERP_API octave_value_list
feval (octave_function *fcn,
       const octave_value_list& args = octave_value_list (),
       int nargout = 0);

extern OCTINTERP_API octave_value_list
feval (const octave_value_list& args, int nargout = 0);

extern OCTINTERP_API octave_value_list
eval_string (const std::string&, bool silent, int& parse_status, int hargout);

extern OCTINTERP_API octave_value
eval_string (const std::string&, bool silent, int& parse_status);

extern OCTINTERP_API void cleanup_statement_list (tree_statement_list **lst);

// Global access to currently active lexer.
// FIXME -- to be removed after more parser+lexer refactoring.
extern lexical_feedback *CURR_LEXER;

class
octave_parser
{
public:

  octave_parser (void)
    : end_of_input (false), curr_lexer (new lexical_feedback ())
  {
    CURR_LEXER = curr_lexer;
  }

  ~octave_parser (void)
  {
    delete curr_lexer;
  }

  void reset (void)
  {
    curr_lexer->reset ();
  }

  int run (void);

  // Error mesages for mismatched end tokens.
  void end_error (const char *type, token::end_tok_type ettype, int l, int c);

  // Check to see that end tokens are properly matched.
  bool end_token_ok (token *tok, token::end_tok_type expected);

  // Maybe print a warning if an assignment expression is used as the
  // test in a logical expression.
  void maybe_warn_assign_as_truth_value (tree_expression *expr);

  // Maybe print a warning about switch labels that aren't constants.
  void maybe_warn_variable_switch_label (tree_expression *expr);

  // Finish building a range.
  tree_expression *finish_colon_expression (tree_colon_expression *e);

  // Build a constant.
  tree_constant *make_constant (int op, token *tok_val);

  // Build a function handle.
  tree_fcn_handle *make_fcn_handle (token *tok_val);

  // Build an anonymous function handle.
  tree_anon_fcn_handle *
  make_anon_fcn_handle (tree_parameter_list *param_list, tree_statement *stmt);

  // Build a binary expression.
  tree_expression *
  make_binary_op (int op, tree_expression *op1, token *tok_val,
                  tree_expression *op2);

  // Build a boolean expression.
  tree_expression *
  make_boolean_op (int op, tree_expression *op1, token *tok_val,
                   tree_expression *op2);

  // Build a prefix expression.
  tree_expression *
  make_prefix_op (int op, tree_expression *op1, token *tok_val);

  // Build a postfix expression.
  tree_expression *
  make_postfix_op (int op, tree_expression *op1, token *tok_val);

  // Build an unwind-protect command.
  tree_command *
  make_unwind_command (token *unwind_tok, tree_statement_list *body,
                       tree_statement_list *cleanup, token *end_tok,
                       octave_comment_list *lc, octave_comment_list *mc);

  // Build a try-catch command.
  tree_command *
  make_try_command (token *try_tok, tree_statement_list *body,
                    tree_statement_list *cleanup, token *end_tok,
                    octave_comment_list *lc, octave_comment_list *mc);

  // Build a while command.
  tree_command *
  make_while_command (token *while_tok, tree_expression *expr,
                      tree_statement_list *body, token *end_tok,
                      octave_comment_list *lc);

  // Build a do-until command.
  tree_command *
  make_do_until_command (token *until_tok, tree_statement_list *body,
                         tree_expression *expr, octave_comment_list *lc);

  // Build a for command.
  tree_command *
  make_for_command (int tok_id, token *for_tok, tree_argument_list *lhs,
                    tree_expression *expr, tree_expression *maxproc,
                    tree_statement_list *body, token *end_tok,
                    octave_comment_list *lc);

  // Build a break command.
  tree_command *make_break_command (token *break_tok);

  // Build a continue command.
  tree_command *make_continue_command (token *continue_tok);

  // Build a return command.
  tree_command *make_return_command (token *return_tok);

  // Start an if command.
  tree_if_command_list *
  start_if_command (tree_expression *expr, tree_statement_list *list);

  // Finish an if command.
  tree_if_command *
  finish_if_command (token *if_tok, tree_if_command_list *list,
                     token *end_tok, octave_comment_list *lc);

  // Build an elseif clause.
  tree_if_clause *
  make_elseif_clause (token *elseif_tok, tree_expression *expr,
                      tree_statement_list *list, octave_comment_list *lc);

  // Finish a switch command.
  tree_switch_command *
  finish_switch_command (token *switch_tok, tree_expression *expr,
                         tree_switch_case_list *list, token *end_tok,
                         octave_comment_list *lc);

  // Build a switch case.
  tree_switch_case *
  make_switch_case (token *case_tok, tree_expression *expr,
                    tree_statement_list *list, octave_comment_list *lc);

  // Build an assignment to a variable.
  tree_expression *
  make_assign_op (int op, tree_argument_list *lhs, token *eq_tok,
                  tree_expression *rhs);

  // Define a script.
  void make_script (tree_statement_list *cmds, tree_statement *end_script);

  // Begin defining a function.
  octave_user_function *
  start_function (tree_parameter_list *param_list, tree_statement_list *body,
                  tree_statement *end_function);

  // Create a no-op statement for end_function.
  tree_statement *make_end (const std::string& type, int l, int c);

  // Do most of the work for defining a function.
  octave_user_function *
  frob_function (const std::string& fname, octave_user_function *fcn);

  // Finish defining a function.
  tree_function_def *
  finish_function (tree_parameter_list *ret_list,
                   octave_user_function *fcn, octave_comment_list *lc);

  // Reset state after parsing function.
  void
  recover_from_parsing_function (void);

  // Make an index expression.
  tree_index_expression *
  make_index_expression (tree_expression *expr,
                         tree_argument_list *args, char type);

  // Make an indirect reference expression.
  tree_index_expression *
  make_indirect_ref (tree_expression *expr, const std::string&);

  // Make an indirect reference expression with dynamic field name.
  tree_index_expression *
  make_indirect_ref (tree_expression *expr, tree_expression *field);

  // Make a declaration command.
  tree_decl_command *
  make_decl_command (int tok, token *tok_val, tree_decl_init_list *lst);

  // Validate argument list forming a matrix or cell row.
  tree_argument_list *validate_matrix_row (tree_argument_list *row);

  // Finish building a matrix list.
  tree_expression *finish_matrix (tree_matrix *m);

  // Finish building a cell list.
  tree_expression *finish_cell (tree_cell *c);

  // Maybe print a warning.  Duh.
  void maybe_warn_missing_semi (tree_statement_list *);

  // Set the print flag for a statement based on the separator type.
  tree_statement_list *
  set_stmt_print_flag (tree_statement_list *, char, bool);

  // Create a statement list.
  tree_statement_list *make_statement_list (tree_statement *stmt);

  // Append a statement to an existing statement list.
  tree_statement_list *
  append_statement_list (tree_statement_list *list, char sep,
                         tree_statement *stmt, bool warn_missing_semi);

  // Generic error messages.
  void bison_error (const char *s);

  // TRUE means that we have encountered EOF on the input stream.
  bool end_of_input;

  // State of the lexer.
  lexical_feedback *curr_lexer;

  // For unwind protect.
  static void cleanup (octave_parser *parser) { delete parser; }

private:

  // No copying!

  octave_parser (const octave_parser&);

  octave_parser& operator = (const octave_parser&);
};

#endif