> Doesn't work as well in languages where indentation is significant.
Store the bytecode, indents and dedents are present as <indent> and <dedent> token which are no different than e.g. <brace, open> and <brace, closed>.
Well technically those are in the tokeniser, the AST or bytecode would reify the structures they delimit, and if you stored the AST or bytecode the separator would be reified from there.
But the point is that for the machine indentation is no different than any other token (in fact Haskell is defined using braces and semi-colons, the common indentation-based version is an augmentation of the base language which semantically just inserts braces and semicolons)
Store the bytecode, indents and dedents are present as <indent> and <dedent> token which are no different than e.g. <brace, open> and <brace, closed>.
Well technically those are in the tokeniser, the AST or bytecode would reify the structures they delimit, and if you stored the AST or bytecode the separator would be reified from there.
But the point is that for the machine indentation is no different than any other token (in fact Haskell is defined using braces and semi-colons, the common indentation-based version is an augmentation of the base language which semantically just inserts braces and semicolons)