How CPython Parser Works, and How to Make It Work Better

Dive into the inner workings of CPython's parser and explore ways to enhance its performance in this EuroPython 2017 conference talk. Gain insights into the two parsers involved in processing Python source code, understand how they construct different types of syntax trees, and learn about potential optimizations to reduce memory footprint. Discover how modifying the structure of one of the trees can lead to significant memory savings with minimal changes to its consumers. Explore more invasive improvements that could yield even greater efficiency gains. Acquire the knowledge to hack the parser and potentially reduce Python's overall memory consumption by up to 30% in standard micro-benchmarks. Suitable for those fluent in C and familiar with CPython core internals, this talk provides a comprehensive look at parser optimization techniques and their impact on Python's performance.

Intro
What is parser
Grammar
Demonstration
Parser
AST
Formal grammar
Second parser
The catch
Benchmarks
Bug tracker
Validation
Rejection
Difficulties
Questions