perf: Prune function bodies from source before building AST - #489
perf: Prune function bodies from source before building AST#489pawamoy wants to merge 5 commits into
Conversation
|
@Jazzcort @ryanpetrello @mgorny @mcepl you've all packaged Griffe in the past for various distros, so I wanted to give you a heads up. Let me know if you'd prefer not being mentioned. This PR introduces a new extra called "faster" which adds Griffe stays pure Python, and doesn't change anything in its build process. Let me know if I can help, or if just giving you time to approach this is enough. Merging and releasing can wait a bit. |
|
Thanks for the heads up. Gentoo removed griffe along with MkDocs over 1.x being abandoned, and everyone switching to random forks, so we aren't affected. More generally, Rust dependency might be a problem but if it's optional, it's not much of a problem, and even less for a package without many reverse dependencies where we can just assume we can remove it from most platforms with no major loss. |
We use a Rust/Ruff-based lexer to prune function bodies in cases where we're sure we won't visit them. This makes
compile()much faster since it has much less code to parse and compile into an AST. Line numbers are kept correct by adding blank lines before a finalpass. See https://github.com/pawamoy/prune-source/.