PyOCI – Publish and install private Python packages using OCI/Docker registries (opens in new tab)

(github.com)

29 pointsrobinhoodexe5mo ago13 comments

13 comments

Would be nice to see at least a high-level overview of how it works under the hood. Is it doing anything interesting with reusable layers? Actually, thinking further that might be a moot point; I feel like running as a remote proxy loses some fun optimization chances. I could imagine a world where you install a package from the host's pip/uv, and then add it to a container image, and both of those are actually the same thing on disk. (Granted, that's likely harder to implement)

allexv5mo ago

Hi, I included a few details here: https://github.com/AllexVeldman/pyoci/blob/main/docs/design.... which I recognise does not contain much.

It's basically a single layer image with a multi-platform image index per tag(version) on top.

No smarts added, performance wise it's probably worse than running your own dedicated pypi index or any purpose-built system, my main goal is a private index for if you don't have access to one already.

yjftsjthsd-h5mo ago

(Thanks for replying!)

That's fair. I'm used to thinking of OCI images as being a particular optimization (there's a reason we don't ship containers around ass rootfs tarballs), but for your usecase it may well be a premature/pointless optimization.

allexv5mo ago

Thank you for posting my project here! Only noticed it just now due to the uptick in starts on github :)

globular-toast5mo ago

Or you could use GitLab which has support for Python packages without a proxy.

tuananh5mo ago

can we agree to use OCI for everything :D

deknos5mo ago

as long as we can convert the OCI to an bootable VM image, i am fine with that. But i also think, there's an size limit

yjftsjthsd-h5mo ago

There are still growing pains, but https://github.com/osbuild/bootc-image-builder exists and is likely to become exactly that in the general case (as it already is for the redhat family).

tetha5mo ago

Oh those size limits are pushed plenty by AI images, no worries. I recently had a good laugh when I found a docker image that was 2 - 3 times as big as the OS partition of a lot of our smaller servers.

And our OS image build order would reuse layers better than those.

1 more reply

tuananh5mo ago

i think it's already been done with bootable containers.

redhat has recently GA bootable container as well.

j / k navigate · click thread line to collapse

13 comments

yjftsjthsd-h5mo ago

allexv5mo ago

Hi, I included a few details here: https://github.com/AllexVeldman/pyoci/blob/main/docs/design.... which I recognise does not contain much.

It's basically a single layer image with a multi-platform image index per tag(version) on top.

yjftsjthsd-h5mo ago

(Thanks for replying!)

allexv5mo ago

Thank you for posting my project here! Only noticed it just now due to the uptick in starts on github :)

globular-toast5mo ago

Or you could use GitLab which has support for Python packages without a proxy.

tuananh5mo ago

can we agree to use OCI for everything :D

deknos5mo ago

as long as we can convert the OCI to an bootable VM image, i am fine with that. But i also think, there's an size limit

yjftsjthsd-h5mo ago

There are still growing pains, but https://github.com/osbuild/bootc-image-builder exists and is likely to become exactly that in the general case (as it already is for the redhat family).

tetha5mo ago

And our OS image build order would reuse layers better than those.

1 more reply

tuananh5mo ago

i think it's already been done with bootable containers.

redhat has recently GA bootable container as well.

j / k navigate · click thread line to collapse