Skip to content

Conversation

@orycho
Copy link

@orycho orycho commented Jun 28, 2023

Completes the Tokenizer in the cpp directory, the parser and the evaluation.
Also adds support for hexadecimal integer literals, bitwise operators & (and), | (or) ^ (xor) and ~ (negation) for integers, as well aas && and || for booleans (short circuit logic).

Also includes several other changes to the original code:

  • Keyword token types clashed with C++ keywords, so capitalized all token types for uniformity
  • Avoiding copy into Lexer is a good idea, but having it rely on a string_view with the contents possibly being deallocated elsewhere is potentially hazardous. Instead, move the string into the lexer.
  • Conversely, tokens where overengineered using variants for literals: Made using them more dfficult, and special case for char optimizes the one case where the literal is not even neeed (and not present in eg. Rust enum). Replaced variant for literals with string_view (but remember, now the literal does not live beyond the lexer's lifetime)
  • Added constructor to token for easier coding.
  • replaced position based indexing into string with an iterator. Much more C++ style.
  • Lexer no longer throws exception on empty input, just returns an EOF token.
  • Reduced C++ standard needed to c++20 as the only c++23 feature used was not worth it (see block inside "#if __cplusplus < 202302L" in test.cc
  • Extensive suite of builtins
  • repl or script execution, command line arguments.

Now also fully implements "Writing a Compiler in Go" with all abive mentioned extensions.

@orycho
Copy link
Author

orycho commented Jun 29, 2023

Hmm, just realized the pull is based on my master branch head, and now it also includes the parser...

@orycho
Copy link
Author

orycho commented Jul 6, 2023

Monkey interpreter is now complete with

  • strings, arrays and hash maps
  • extended set of builtin functions

@orycho orycho changed the title Feat: Complete C++ Tokenizer Feat: Complete C++ Interpreter Jul 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant