http://front-trends.com/
Automatic tools can spit out super large scripts these days. Look at emscripten for examples, a script of several megabytes is not an exception (more like the rule). How do you process a script that is several megabytes large? Most parsers tend to want to parse everything in one sitting. These parsers run out of memory at some point. How can you do analysis or modifications to arbitrary scripts if you can’t even parse them? Enter a streaming parser. It is capable of parsing a script left to right without retaining more memory in the process. It can yield at any point where it needs more input and it can stream parsed tokens when one is available.
I’ve managed to write an automatic build script that transforms my JavaScript parser ZeParser from a regular sync parser to a streaming parser, working in ES5 JavaScript. In this talk I would like to explain how to go about that, what kind of problems you may encounter, and what the end result looks like.
Slides: https://drive.google.com/file/d/0B3hY2B8-FkPdaVJtelpwMVA5bTA/view