undefined | Better HN

0 pointstakeda6y ago0 comments

So I did look it up, they do provide a warning though (in the first paragraph):

"The ast module helps Python applications to process trees of the Python abstract syntax grammar. The abstract syntax itself might change with each Python release; this module helps to find out programmatically what the current grammar looks like."

This module is a bit special, because as they say, it supposed to reflect current python grammar. If they change grammar and didn't make it reflect it it would lead to different kinds of issues.

0 comments

2 comments · 1 top-level

mehrdadn6y ago· 1 in thread

It's one thing to extend the AST and add a couple children because you added new syntax or something. That's what I think most sane people would expect from reading that quote as a language evolves. So yeah, don't assume a node has 4 children when it might have 5+ tomorrow. But it's another thing to outright remove a child entirely and to be so vague about it too. (They didn't even have it be a sentinel value like None, or even care to leave a useful error message in the stack trace! The message makes you think the code was broken all along and simply never triggered before.)

Lest you think informing users more clearly is a foreign concept to them, look at the dis module [1] for comparison. They're extremely clear the whole thing is implementation detail of CPython. If anything, after reading that, one would think your conclusion would be "ah, I should program against the AST then, not the bytecode". So you do that and then you're greeted with this nonsense! Obviously it's your fault for assuming there's anything stable to program against across minor releases.

[1] https://docs.python.org/3/library/dis.html

takedaOP6y ago

the child wasn't removed, it was renamed from arguments to posonlyargs, this is related to the change of allowing functions to specify that given arguments are always positional.

This module exposes internals of python, and providing such guarantees would cripple development, because it wouldn't even allow for refactoring the code.

Most languages don't run into this because they don't expose internals like that. You typically extract that yourself and you accept it can change every release.

As for dis, that's very different, the bytecode is just an optimization of the CPython, python code could work perfectly fine without it, the bytecode was introduced to make it faster and other implementation won't use I would imagine that Jython and IronPython most likely don't implement it since they have their own (JVM and .NET)

ast on the other hand is expected to be identical if a language claims to be compatible with a given version, and you would expect PyPy for example to also provide this package.

j / k navigate · click thread line to collapse