This is one of the nicer ones.
It looks pretty conservative in it's use of Rust's advanced features. The code looks pretty easy to read and follow. There's actually a decent amount of comments (for rust code).
Not bad!
But then there's this Arc, Ref, Pinning and what not - how deep is that rabbit hole?
Lifetimes aren't bad, the learning curve is admittedly a bit high. Post-v1 rust significantly reduced the number of places you need them and a recent update allows you to elide them even more if memory serves.
Arc isn't any different than other languages, not sure what you're referring to by ref but a reference is just a pointer with added semantic guarantees, and Pin isn't necessary unless you're doing async (not a single Pin shows up in the kernel thus far and I can't imagine why I'd have one going forward).
What tends to make Rust complex is advanced use of traits, generics, iterators, closures, wrapper types, async, error types… You start getting these massive semi-autogenerated nested types, the syntax sugar starts generating complex logic for you in the background that you cannot see but have to keep in mind.
It’s tempting to use the advanced type system to encode and enforce complex API semantics, using Rust almost like a formal verifier / theorem prover. But things can easily become overwhelming down that rabbit hole.
So lemme ask: what other languages and project (open/closed, big/small, web/mobile/desktop, game/consumerapp/bizapp) have you experience with as to come to this conclusion?
The big concern I have however is hardware support, specifically networking hardware.
I think a very interesting approach would be to boot the machine with a FreeBSD or Linux kernel, just for the purposes of hardware as well as network support, and use a sort of Rust OS/abstraction layer for the rest, bypassing or simply not using the originally booted kernel for all user land specific stuff.
That's basically what you're getting with Docker containers and a shared kernel. AWS Lambda is doing something similar with dedicated kernels with Firecracker VMs
Basically single app directly to network (the world) and as little as possible else in between.
You can check out hardware support here: https://core.dpdk.org/supported/nics/
p.s.: i was wrong
>While we prioritize compatibility, it is important to note that Asterinas does not, nor will it in the future, support the loading of Linux kernel modules.
https://asterinas.github.io/book/kernel/linux-compatibility....
> While we prioritize compatibility, it is important to note that Asterinas does not, nor will it in the future, support the loading of Linux kernel modules.
They might not yet implement everything that's needed to boot a standard Linux userland but you could say boot straight into a web server built for Linux, instead of booting into init for example.
Thus it is a much more interesting project.
You still have kernel modules for microkernel-like functionality
> utilize the more productive Rust programming language
Nitpick: it’s 2024 and these ‘more productive’ comparisons are silly, completely unscientific, And a bit of a red flag for your project: The most productive language for a developer is the one they understand what is happening one layer below the level of abstraction they are working with. Unless you’re comparing something rating Ruby vs RiscV assembly, it’s just hocus-pocus.
FWIW that’s what the Linux compatibility layer in the BSDs does and also what WSL 1 did (https://jmmv.dev/2020/11/wsl-lost-potential.html).
It’s hard to get _everything_ perfectly right but not that difficult to get most of it working.
The company I work for has both rust and python projects (through partially pre "reasonable python type linting" using mypy and co.) and the general consensus there is "overall" rust is noticeable more productive (and stable in usage/reliable), especially if you have code which changes a lot.
A company I worked previous for had used rust in the very early days (around 1.0 days) and had one of this "let's throw up a huge prototype code base in a matter of days and then rewrite it later" (basically 90% of code had huge tech dept). But that code base stuck around way longer then intended, caused way less issues then expected. I had to maintain it a bit and in my experience with similar code in Python and Js (and a bit Jave) I expected it to be very painful but surprisingly it wasn't, like at all.
Similar comparing my experience massive time wastes due to having to debug soundness/UB issues in Rust, with experiences in C/C++ it's again way more productive.
So as long as you don't do bad stuff like over obsessing with the type system everything in my experience tells me using Rust is more productive (for many tasks, definitely not all task, there are some really grate frameworks doing a ton of work for you in some languages against which the rust ecosystem atm. can't compete).
---
> Most productive language for a developer is the one they understand what is happening one layer below the level of abstraction they are working with.
I strongly disagree, the most productive language is the one where the developer doesn't have to care much about what happens in a layer below in most cases. At least as long as you don't want obsess over micro optimizations not being worth the time and opportunity cost they come with for most companies/use cases.
You look at Linux's syscall table[0], read through the documentation to figure out the arguments, data types, flags, return values, etc., and then implement that in your kernel. The Linux ABI is just its "library" interface to userspace.
It's probably not that difficult; writing the rest of the kernel itself is more challenging, and, frankly, more interesting. Certainly matching behavior and semantics can be tricky sometimes, I'm sure. And I wouldn't be surprised if the initial implementation of some things (like io_uring, for example, if it's even supported yet) might be primitive and poorly optimized, or might even use other syscalls to do their work.
But it's doable. While Linux's internal ABI is unstable, the syscall interface is sacred. One of Torvalds' golden rules is you don't break userspace.
As well as some subset of the files expected in /dev, /proc, /sys, and similar, which are also part of the userspace ABI. And the startup mechanisms for processes, and the layout of AUXV...
It's absolutely doable, but the interface is wider than just the syscall layer.
Ah good times.
From the looks of it, this seems like a serious corporate backed project made by employees of the Ant Group, the chinese fintech giant. A more fair comparison would be with Google's Fuchsia OS (defunct) or Huawei's HarmonyOS. It may succeed, it may fail, but it's nothing like a couple of kids doing a passion project to learn Rust.
Its more of a research OS but still cool.
Unprivileged services can exploit known compiler bugs and do anything they want in safe Rust. How this affects their security model?
I guess it depends on what they mean by "easy". Certainly it's easier in the sense that you can just write code all day long, and not have to deal with the politics about Rust inside Linux, or deal with all the existing C interfaces, finding ways to wrap them in Rust in good, useful ways that leverage Rust's strengths but don't make it harder to evolve those C interfaces without trouble on the Rust side.
But the bulk of Linux is device drivers. You can build a kernel in Rust (like Asterinas) that can run all of a regular Linux userland without recompilation, and I imagine it's maybe not even that difficult to do so. But Asterinas only runs on x86_64 VMs right now, and won't run on real hardware. Getting to the point where it could -- especially on modern hardware -- might take years. Supporting all the architectures and various bits of hardware that Linux supports could take decades. I suppose limiting themselves to three or four architectures, and only supporting hardware made more recently could cut that down. But still, it's a daunting project.
> Currently, Asterinas only supports x86-64 VMs. However, our aim for 2024 is to make Asterinas production-ready on x86-64 VMs.
I'm confused.
> By 2024, we aim to achieve production-ready status for VM environments on x86-64. > In 2025 and beyond, we will expand our support for CPU architectures and hardware devices.
They've been working on it for a while so they can get rust into the linux kernel
There's no specification of that ABI, much less a compliance test suite. How complete is this compatibility?
https://asterinas.github.io/book/kernel/linux-compatibility....
Let's take example of network. There's IP address, gateway, DNS, routes etc. Depending on distribution we might see something like netplan reading config files and then calling ABI functions?
Or Linux kernel directly also reads some config files? Probably not...
Edit: looks like iproute2 uses NETLINK, but non-networking tools might use syscalls or device ioctls.
Sure is a lot of text to say: We try to use unsafe as little as possible.
Which is the minimum you'd expect anyways ¯\_(ツ)_/¯
0. https://asterinas.github.io/book/kernel/the-framekernel-arch...
Does it mean it can re-use the drivers written for hardware to run with linux ?
I guess the NT kernel needs to. Does Darwin?
As soon as I can financially retire, I'll make contributing to this my full time job!
> Currently, Asterinas only supports x86-64 VMs.
So no real hardware.
If this catches on and has generally been subject to significant third party code review with positive results, I'm not sure any backdoor is lower cost to use than an equivalent linux vulnerability. To be fair, I'm not sure it isn't either.
That's an interesting example because Huawei equipment is currently being removed by several Western countries (UK, Canada, US, Germany) specifically because it's Chinese.
https://www.nytimes.com/2024/07/11/business/huawei-germany-b...
https://www.cbc.ca/news/politics/huawei-5g-decision-1.631083...
https://www.gov.uk/government/news/huawei-to-be-removed-from...
https://www.reuters.com/business/media-telecom/us-open-progr...
> If everything goes well, Asterinas is now up and running inside a VM.
Seems like the developers are very confident about it too
This is all paraphrased from my memory, so take it with a grain of salt. I think the gist of it is still valid: Projects like Asterinas are interesting and have a place, but they will not replace Linux as we have it today.
(Asterinas, from what I understood, doesn't claim to replace Linux, but it a common expectation.)
> Torvalds seemed optimistic that "some clueless young person will decide 'how hard can it be?'" and start their own operating system in Rust or some other language. If they keep at it "for many, many decades", they may get somewhere; "I am looking forward to seeing that". Hohndel clarified that by "clueless", Torvalds was referring to his younger self; "Oh, absolutely, yeah, you have to be all kinds of stupid to say 'I can do this'", he said to more laughter. He could not have done it without the "literally tens of thousands of other people"; the "only reason I ever started was that I didn't know how hard it would be, but that's what makes it fun".
As the saying goes "We do this not because it is easy, but because we thought it would be easy."
Occasionally these are starts of great things.
It doesn't matter how the language gets deployed, if the runtime is on a container, a distroless container, or directly running on an hypervisor.
The runtime provides enough OS like services for the programming language purposes.
Provided you have virtio support you are ticking a lot of boxes already.
> [...] we accommodate the business need for proprietary kernel modules. Unlike GPL, the MPL permits the linking of MPL-covered files with proprietary code.
Glancing at the readme, it also looks like they are treating it as a big feature:
> Asterinas surpasses Linux in terms of developer friendliness. It empowers kernel developers to [...] choose between releasing their kernel modules as open source or keeping them proprietary, thanks to the flexibility offered by MPL.
Can't wait to glue some proprietary blobs to this new, secure rust kernel /s
Is that the new generation of curl | bashism in action?
curl | bash has an actual problem: potential execution of an incomplete script (which can be mitigated with function calling). And there’s the mostly theoretical problem of the server being pwned / sending malicious code just to you (which of course also applies to any other unsigned channel). Arbitrary code execution is never a problem unique to it, but people dunk on it all the time because they saw another person dunking on it in the past.
An example that illustrates this: https://lwn.net/Articles/22991/
(And wow, it's been 22 years already...?)
https://asterinas.github.io/book/osdk/guide/run-project.html
Have a look at AMD GPU driver. Massive, and full of 'stabilization/work around' code... happening all the time, for years.
I guess, the real "first thing first" is to design hardware, performant hardware on latest silicon process , with a, as simple as possible, modern, standard and stable hardware programing interface. Because, for many types of hardware, 'now we know how to do it properly' (command hardware ring buffers usually, or a good compromise for modern CPU architecture, like RISC-V).
Another angle of "cleanup", I guess it would be the removal of many of the C compiler extension (or "modern C") tantrums from linux, or at least proper alternatives with not-inline assembly to allow small and alternative compilers to step in.
Personally, I tend to write rv64 assembly (which I interpret on x86_64), but for the userland. If I code C, I push towards mostly "simple and plain C99".
The more I think about it, the more I get the following coming to my mind: 'hardware with simple standard interfaces' and standard assembly for the kernel.
