Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Well, yes and no. Lisp is an ASCII serialization of the Lisp AST. This isn't true for most other languages. There is no reason why source for most languages couldn't be serialized as a combination of the AST and the stream of lexical tokens. This would have a few interesting advantages. For example, everyone could have the source formatting of their preference.

EDIT: Another example -- someone could implement a language homomorphic with Python without significant whitespace and curly brackets.



Even for lisp, you would have to define an extended syntax for it and modify the parser to additionally read and store whitespaces, comments and newlines. This way you will be able to serialize your source to plaintext without loosing the original format. Non text sourcecode also allows for way smarter tooling, tree diffs on the AST, graphical repls and improved rendering of sourcecode.


Even for lisp, you would have to define an extended syntax for it and modify the parser to additionally read and store whitespaces, comments and newlines.

Note I said AST + stream of lexical tokens. If you store the whitespace and comments as tokens, such information can be preserved, including formatting hints.


Minor pedantry: Most variants of Lisp are not precisely a serialization of the AST. The read engine and macro engines have to pass over the code before it is truly an AST. For Common Lisp I believe technically either can be invoked at any time, but I am not a Common Lisp language lawyer.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: