Embedding PyPy in a C application (opens in new tab)

"With only a few lines of code we have written a C program that fires up a PyPy interpreter and runs our Python code as if it was a native C function. Of course I've only shown you the basics here, but the technology is really powerful."

So, C speed with Python semantics?

Noob question, isn't Python already based on C?

There's interesting overlap between tools like `cffi` and `cython`.

This is one of the approaches I am pursuing to polish off something I put together recently: rapid prototyping of LD_PRELOADs in Python.

I presented an early approach at PyData London last week using LD_AUDIT and `cython`: https://www.youtube.com/watch?v=L3j2qw9XZCc

It doesn't seem that amazing to be able to write:

    @preload
    def open(pathname: 'const char*', flags: 'int') -> 'int':
        return __open__(pathname, flags)

After all, this is only slightly more convenient syntax than in C. However, imagine how much functionality you could quickly develop with access to the Python standard library.

Even using my clumsy `cython` and `LD_AUDIT` approach, I was able to rapidly develop libraries that interpose file-handling library calls and implement:

    # `less` is used as an example of a libc programme with no concept of Python, git, &c.

    $ COMMIT_HASH=abc123 gitit less test # interact with files as-of some commit in a git repo via pygit (libgit2)

    $ runit less test.py # redirect read() so that the contents of the file appears to be the results emitted of having run the file

    $ zipit less test.gz # redirect read()/write() to transparently handle gzip files

Each one of these took about half an hour to throw together, which is substantially faster than I could reliably develop them in pure-C (not to mention how much code I avoid having to write myself by relying on Python's rich stdlib & ecosystem.)

I used cffi recently to embed CPython into a fairly rigid Fortran program to support dynamic scripting. Was a piece of cake.

Does PyPy embedding get around the GIL?