instead of a closed set of languages. I also removed the offsets:
I simply use the current region to determine whether the
preprocessing directie starts at the beginning of a line. I also
removed scanning line indicators, to make the lexer simpler.
LexToken.mll: Moved the function [check_right_context] that
checks stylistic constraints from Lexer.mll to
LexToken.mll. While this triplicates code (as CameLIGO, PascaLIGO
and ReasonLIGO share the same constraints), the benefit is that
Lexer.mll becomes more generic and the signature for the TOKEN
module is simpler (no more exporting predicates, except for
EOF). In accordance with the change of the preprocessor, the
lexers and parsers for LIGO now depend on the kind of comments,
not a fixed set of syntaxes. This gives more versatility when
adding a new language: only the kinds of its comments are needed,
although Lexer.mll and Preproc.mll may have to be modified if
they do not already know the comment delimiters, for example line
comments starting with #.
****************************************************************
BUG: The exceptions coming from LexToken.mll when a stylistic
constraint is broken in [LexToken.check_right_context] are not
caught yet.
****************************************************************
Lexer.mll: I moved out as much as I could from the header into a
new module LexerLib. The aim is to make it easy to reuse as much
as possible of the lexer machinerie, when it cannot be used as
is.
* I added CLI option "--mono" to select the monolithic API of Menhir.
* I added a field "win" to the state of the lexer (a two-token
window for error reporting).
* I escaped LIGO strings before making them OCaml strings (for
example for printing).
Refactoring of parsing command-line arguments
* The type [options] is now abstract and implemented as an
object type to avoid struggling with scoping and type
inference when record types share some common field names.
Refactoring of ParserLog for PascaLIGO and CameLIGO
* The immediate motivation behind that refactoring was to
remove the use of a couple of global references. A
consequence is that we have a nicer and more compact code, by
threading a state. The files [pascaligo/Tests/pp.ligo] and
[ligodity/Tests/pp.mligo].
* Another consequence is that the choice of making strings from
AST nodes depends on the CLI (offsets? mode?). After this
refactoring, that choice is hardcoded in the simplifiers in a
few places (TODO), waiting for a general solution that would
have all CL options flow through the compiler.
* I removed the use of vendors [x_option.ml], [x_map.ml] and
[x_list.ml] when handling optional values. (Less dependencies
this way.)
Refactoring of the ASTs
* I removed the node [local_decl], which was set to [[]]
already in a previous commit (which removed local
declarations as being redundant, as statements could already
be instructions or declarations).
* I changed [StrLit] to [String] in the AST of CameLIGO and
ReasonLIGO.
* I also changed the type [fun_expr] so now either a block is
present, and therefore followed by the [with] keyword, or it
is not. (Before, the presence of a block was not enforced in
the type with the presence of the keyword.)
Notes
* [LexerMain.ml] and [ParserMain.ml] for CameLIGO and PascaLIGO
are almost identical and differ in the same way (language
name and file extension), which suggests that they should be
in the [shared] folder and instanciated as a functor in the
future (TODO).
* I removed the blank characters at the end of many lines in
the parser of ReasonLIGO.
