Regarding import cost, as it’s doing heavy IO traversing the file system, the cost heavily depends on how fast you can do IO in the hardware, and also the file system (and the OS).

So a fast SSD will help, and somewhat surprisingly putting it inside docker helps (in an HPC context, not so sure it’s implications here as we’re talking about a short scripts.)

But the context here is to port shell scripts to Python, I’m not sure how huge amounts of imports matters.

And it is probably an intrinsic problem of the language, unless we start talking about compiling (and somehow statically) not interpreting the Python program, whichever implementation of the language probably won’t help the situation.

Lastly, if high startup costs of the script becomes relevant, perhaps it is orchestrating wrong. This is an infamous problem of Julia, and their practice is then just keep the Julia instance alive and use it as “the shell”. Similarly, you can do so in Python. Ie rather than calling a script from the shell acting on millions of things, write a wrapper script that start the Python instance once. Memory leak could be a problem if it or its dependencies are not well written but even in that case you have ways to deal with that.