Yes, I built a prototype for it.
Thank you to the question.
As everybody knows Cloud is the rage in business
and enterprises, everything is moving into the cloud
and it gets even more with this whole AI stuff.
And yeah, in the cloud, you typically have your two kinds of system software stacks.
I can have VMs and you can have containers.
And then if I'll take the closer look into what the trusted computing base would be for these two
versions of solutions.
Of course, for the virtualization, you have the hypervisor,
as trusted computing base, and the individual gas OSs, as well as usually the kind of VM manager.
Any one of these can be compromised in any way you can get information leaks,
or in the case of the VM manager,
and the attacker could start to control the VMs on this host system,
which you typically don't want.
And if the hypervisor's compromised, you could breach the isolation of the VMs.
But even the gas OSs in the VM can be vulnerable to attacks, because if you leave,
if this is compromised, it can't attack as of the VMs.
Easily, it's still isolated, but it can leak the confidential information that we cite in the compromised VM.
And for containers, we have a similar picture just that we have one isolation layer.
Less, we have the host OS, which of course has to be trusted,
and usually your container manager, because even if your host OS is secure,
if it's for some reason, it's a back in the container manager that allows
an attacker to remotely control the containers on your host system,
you're also out of luck.
So let's have a look at how the container solutions look,
what are these solutions usually, and we can see about 80% is Docker.
Probably not such a big surprise.
And for the host OS, on Docker, we see 75% using Linux,
making about 71% of all container-based cloud infrastructure being based on the Linux system.
And on Linux, you usually initially didn't have any container support.
So newing the evolution of Linux, there are several mechanisms added to your usual
set of the usual traditional textbook unit system, which are C groups for restricting the usage of resources,
for example, to restrict the usage of certain CPU costs to specific containers.
Furthermore, you have namespaces, which reduce the visibility of the host fire system.
You can easily completely hide the host fire system,
but you can also, for example, usually if you're using Docker just for development,
you may have a shared folder for your individual project that you are working on.
And then, last but not least, we have second EBPF, which is used to restrict the source costs,
a container might call, because if you allow everything a source call to be performed outside of the container,
especially when it is allowed to do anything on the host system, you can easily break the isolation again.
And then, yeah, the guys from Kern concept and backhausen Institute had a joint paper called Met Eagle,
where they analyzed the security of these container solutions.
And first, they stated that about 2.7 million lines of code in Linux,
like 7.4 are for the container infrastructure alone.
And there they found eight critical vulnerabilities for the second mechanism,
three for three groups and 22 for namespaces.
Critical meanings that these are really exploitable vulnerabilities that can lead to a breach of the isolation of the containers.
And those, we have seen 33 critical TVs.
And yeah, one month thing now here, okay, containers, we know they are lightweight, but they are not sectors that use the ends.
But yeah, there we have first 50% being KVM, that's basically integrated part of the Linux kernel,
so in some capacity it's still a Linux system involved.
And we have 150,000 lines of code for KVM alone that's just the kernel part, not the virtual machine monitor,
which is usually a chemo system, chemo in user space that provides the actual emulation for the machine interface that the guest OS expects.
And there we see 24 critical vulnerabilities that have appeared since 2008 with a CBSS of more than 7.0,
according to the NIST vulnerability database, and for QM we have 77 critical vulnerabilities.
And if we move up the stack, we have, according to command linux.com,
we are over 90% of guest OS's in those cloud setups are linux OS's,
and then we have of course all the problems that the Linux kernel has in regards to security and vulnerabilities.
Being that we have a huge trusted computing base of 14 million lines of code,
of course not every kind of code is really actively used there, but it's a lot that will be used there,
because then we also have diverse drivers and other stuff.
And there we had more than 1,000 critical vulnerabilities,
I looked it up again in the NIST and national vulnerability database,
of course they are not all open yet. Now they've closed these issues,
maybe not all, but most of them, but it shows that we have a very complex and huge trusted computing base,
and due to the fact that it was originally was a unique system where you had a global namespace
with everything being visible and just restricted by file system permissions in order
to have containers, additional restriction techniques were introduced adding even more complexity
of an already complex system.
And these are prone to vulnerability that allows to breach the isolation.
So I might ask, we are, isn't there any better solution yet?
And I can recall Hermann Hattick, which I think most of you know,
are always stated when we had at the German SIG group meeting a talk regarding cloud and unique kernels
and hypervisor that yeah, we did this already 30 years ago,
and our solution of course was, yes, micro kernels.
So let's see how a micro kernel cloud architecture may look like.
So I'd like to introduce Erlan OS, which is my take on building a cloud infrastructure based on the
genet operating system framework.
And first, we have a set of components which comprise our system.
Each component is isolated, it's similar to an isolation to a container,
maybe even close to a virtual machine, depending on the isolation level,
the micro kernel may provide.
In genet, you have at least four different micro kernels you can choose.
And yeah, each of these components only interact with each other via RPCs by default.
And the good part is each RPC can only be used if a capability was granted to the
component that wants to invoke this RPC.
And yeah, the access to these capabilities in a genet system usually controlled by the parent components.
And again, for these parent components, it would be this component down here,
which is usually the initial components that are started in a system.
And those in no-minclature is not the one.
The genet usually has because in my system, I've modified the unit component of genet with
some additional features that I used for automatic resource partitioning.
So don't get confused by the names here.
And then, yeah, if we have, for example, this app 2 that wants now to get the,
oh, there's some small mistake on the slide.
Yeah, let's consider that here we would have a nick instead of tasking.
And app 2 wanted to now get the nick capability.
It would ask its parent, the screen component for this nick capability.
And if in the configuration of this green component,
the rule that allows app 2 to have that nick connection,
then app 2 gets the capability and then can now use nick that is not working.
And then you can have built trees in this kind with a parent component,
child components that may also have child components.
And in my system, I call them habitats because the components are originally called cells.
And they have habitat of resources where they will start in.
And these can also isolate different tenants or user sessions.
Or maybe if you want to compose more complex container or VM scenario,
we also have special libraries and not just one app.
Maybe additional apps you could isolate and building such a tree.
And this structure also guarantees that this components here, for example,
can't see the yellow or blue component or communicate with them,
then unless this component allows that,
and in order to for this component to allow contact with this blue and yellow component,
the gray component down here would have to allow it first.
That's basically a capability delegation,
or capabilities to decide here and then they are delegated down for subsets of resources.
So the advantages of such an architecture is that we do not have an global namespace,
so we don't need any restriction mechanisms that can possibly be breached.
We are isolated by default.
Then we have explicit access control to these capabilities.
And in case for capabilities, it's a proven security mechanism for decades now.
Now, and of course, since we're talking about microcolors,
we're talking about the kernel size of about a few thousands,
tens thousands of lines of code compared to the millions of lines of code for Linux.
So we have a very small across the computing base.
And in addition, our feature we get on microcolors,
since we execute all drivers and system services in user space,
based we have the advantage, if for example,
this blue one would fail or the FS server here would fail.
This could be recognized by the great component here,
or the blue component itself, and if F server could be restarted.
So you can have full content and a thought in a device
to have our system server does not crash the kernel and the whole operating system,
like for example, we saw a few years ago with some Windows update,
ironically, a security feature that will update a security feature,
crashed the Windows kernel on millions of machines and caused
quite some economic damage.
That's something we wouldn't have here.
And of course, even the fact that we have this all in user space,
we can have different variations of system services and drivers,
so that we also gain more modularities,
and you usually have with your monolithic kernel.
And as the guys from cell 4, of course,
we'll know you can certify or even formally prove
the correctness of your microcolors, which you can't do with limits.
It's a way to complex to do that.
So one might ask, yeah, if these microcolors are so good,
and we have them for 30 years by now.
Why does everybody still use Linux?
And I want to answer this question for my own experience and perspective,
and I think it boils down to two major challenges,
being compatibility and scalability performance.
And that's how I put together because, usually,
especially regarding microcolors,
I think you can't really separate those two.
And for compatibility, it stops with hardware support,
who can't use a microchor on a cell out,
wherever the hardware isn't supported.
And there, for example, we need numerous support.
This was one feature that Jeanette originally didn't have,
I edited my variation of Jeanette in the Nova microcolors.
This is especially needed for memory,
bound and memory intensive applications like databases,
because if you do not do this, you have to pay cash coherence,
traffic, and the Numa costs that you have,
then you communicate between different Numa regions,
and knowing the Numa topology can help you avoid that.
And of course, speaking of servers, we need drivers for nicks,
and I'm talking to server nicks,
and not your small Wi-Fi chip or onboard network cut in your laptop.
And of course, also rate controllers, hard disk, and stuff,
and increasingly more, especially since that I create,
we also need drivers for accelerators,
especially GPUs, NPUs, and so on.
And regarding security,
Windows already forces it,
TPM support, or SQX,
on-claves are a thing to further improve the security of your system,
and you need to support for that too in your microcolors.
One solution that a unit laps the tighter to go to the softest problem,
because writing all drivers from scratch for hardware,
may not even be documented at all by the hardware lender,
is a very tedious task,
and may even prove impossible for very big hardware like these GPUs.
I know that I had a group of students that really dared to take on the task
to write a driver for AMD GPUs,
or by the test from scratch with what was documented.
And after one year, we had to say,
okay, that doesn't work out.
So the idea here would be using Linux to have us
inside our components in user space.
And that's exactly how Gina does it with the device driver environment,
which is currently based on Linux 6.12.
And what it does is it provides a compatibility layer,
which implements some parts of the Linux kernel APIs
that is used by drivers, for example,
for PCI, discovery, memory mapping,
and memory management of your devices.
But yeah, even if your microcolors runs on the server,
and supports all your hardware,
this doesn't help you if you don't have this software
that you need for cloud operations.
And yeah, it boils down that most software that is used in cloud,
especially if it is something to do with web services,
we have some kind of public status applications,
or maybe even some runtime systems like JVM, Python, Ruby, and stuff.
And this, you have to park to your microcolour operating system.
But this is not all, you also need to support,
maybe for parking, at least to ease the use of your tool channel
that you have for your microcolour.
And yeah, the servers are usually not controlled
like a sculptor as directly on your device.
No administrator really sits at the server, connect,
keyboard to it in a monitor in these big server rooms,
but they are controlled remotely,
so you move some kind of tool support for remote management,
something like a Docker console, SSH, or something.
And this have to be part of two,
but fortunately, Jeanette helps a bit with their Goa toolkit
for porting to a party system,
party software, and it's also supported common build systems
and provides such nice features like automated debugging,
support testing and publishing all this in these nice packages.
And my system provides also a mocha where you can connect
while there's a network to your machines
that runs LNOS and manage the components at one time.
You can start and new components,
or destroy running components,
and also monitor the resource utilization.
But I have to, it's a prototype for academia.
It's very bad once at the moment,
I wouldn't say that it's already for production build set.
But that's from the future.
Yeah, then that was for the compatibility part.
Now the other big challenge is scalability and performance.
For example, here I measured the performance of the Jeanette heap,
data structure, which is a common used by all user space applications,
that do dynamic memory allocations.
And I performed the last memory allocation benchmark from Lassen,
which is a benchmark that performs memory allocations
of eight to one kilobit chunks randomly,
which should represent typical web server like memory allocations
that you have.
And yeah, in this case, I performed five million Mallock and three chords.
Each in this benchmark, and one can see,
if you run up the number of threats of your application,
that uses a heap,
you can see that the time it takes for allocation,
an integral block goes up,
which is not so good,
because even if you have five million allocations,
in total and would, yeah,
these two built it over 31 CPU cores,
you would expect it to be lower than with one CPU core.
But unfortunately, that's not the case.
And yeah, this boils down to some synchronization issues.
Choose in the DNA system.
But that's not the only bottleneck I discovered
by creating my own version of Genet for Cloud.
The other big problem is networking.
There I discovered that the default socket implementation
that Genet has doesn't scale well.
It performed very, very simple benchmark
that just measures the performance of the network stack
without any, yeah,
no noticeable impact from the application.
It was just simple, very simple TCP echo benchmark,
basically receiving a packet bar TCP
and just sending it directly back to the plot.
And compared against a state of the art,
the PDK based network stack that was developed
especially for these kinds of cloud operations,
called Caladan.
We can see, yeah, the tail latency,
that is in this case in 1999.9s,
percent tail tile of all latencies in the benchmark.
We see, yeah, Caladan, you can't even see anything there
because the Genet scalability is bottlenecked so much
that we have orders of magnitude here.
Here, if you have, yeah,
named reasons and number of connections and 32 connections
are not that much for solidifications.
So what's the reason for that?
I try to explain it as far as I've found out yet.
One possible reason is that from the architectural perspective
that in Genet and also other microcolonals
that follow the original F4 like design,
you have a server which has some entry point here,
which is completely single threaded,
and you have your client which performs an RPC
for an RPC object,
it has the capabilities for,
but it does, it's called a system call for this RPC,
which goes down into the kernel here,
and then this kernel triggers this entry point
and then performs this function.
But this entry point has to be synchronized
because if you allow your preferred access
or an RPC object,
you would have to think of it or you get
inconsistent data, especially if you have
what operations on this RPC object.
But now since we only have one entry point,
if you have 32 clients now that want to
concurrently access this entry point,
you see this will cause a traffic jam in this regard.
And what makes it worse,
at least I can say from the perspective of Genet,
it uses test and set locks
for locking internal data structures
that are used to lock the calling thread here
and for some resource management of the capabilities.
And these are known in research for not scaling well
when you have multi-core systems with many CPU cores,
which way above it we talk more about hundreds of CPU cores here.
And yeah, a quick solution would be to replace
this non-scalable locks with more scalable locks like
S locks or anything,
and or use weight-free synchronization as possible.
And an architectural attempt to first this problem
was proposed by the factored operating system,
where the idea is to not have a service with one
or three point.
But you have a fleet of service threads in your system.
Some, for example, this name service has several fleets.
And then all your requests to the service are distributed
over the fleet members,
which would be one architectural solution
to solve this bottleneck problem.
Yeah, and I have to fill up.
Yeah, the future won't matter.
I would envision would be, of course,
plotting some server applications,
advancing the tooling for my remote management system.
And one big chunk, the most important challenge for me
would be the redesign of genots network interface.
For example, according to such the idea,
like having fleets of genots,
a big router component, which is usually
a component handling or networking energy
in that system.
And on top of that, maybe the evaluation
and reasoned down of genots page locator,
I know from current concept, with their fiasco kernels,
they had a scalability issue with their original memory locator
and had a collaboration with the university of Hanover,
where they integrated a more scalable page locator,
which improves the performance of their system.
So maybe we can do that with genotool.
Yeah, and to finish my talk, yeah,
we have seen current cloud infrastructure
primarily based on Linux,
which has a huge and complex trusted computing base,
which is full with critical vulnerabilities,
not all the time, but you can expect them.
Yeah, and you know micro kernels of a greater
security, safety, and modularity,
but they see a little adoption,
at least, until now, for cloud operations,
this may be down to the effect of the lack of hardware
that was for server-grade hardware,
and a lack of tool and software support,
especially when it comes to server applications
that use legacy project style interfaces.
And we also, on top,
we have issues with the scalability of system services.
But there's ongoing work to overcome those challenges.
Genotlapse is more focused on providing better compatibility
with improving their go-a tool chain
and putting even more and more applications of libraries to genot,
and we also have gaphood,
which are more focused on improving the networking support,
and the genotl systems they use,
and can concept and back housing work with FRE,
more from what I know,
more focused on improving the architecture of micro kernels
and developing new concepts to make micro kernels more scalable.
So thank you for your attention,
and I hope that you'll get in touch with me.
Thank you.
As the IP statement, you know,
I appreciate any help.
Any questions?
Would it make sense to,
especially given the scalability of the network,
and we want to take a little more close to abandon the cloud
of our synchronous IPC,
and finally move to asynchronous IPC there.
I got the question,
whether it makes sense to,
we place synchronous IPC,
especially if we have that scalability issue with asynchronous IPC,
and, yeah, I have to say,
I am also,
of that opinion,
I think asynchronous IPCs would be something we could,
consider in the future, but of course,
yeah, all the systems you all form micro kernels,
are originally designed for synchronous IPCs,
which means that we have to change
a very fundamental part of the operating systems kernel,
and do lots of redesigned stuff,
because all your applications have to consider
that course asynchronous now,
and that's interesting to think about that,
but I don't think that it will be in the near future,
it's more important.
There are not just alpha micro kernels.
I know there are other micro kernels,
I could also look for example into other micro kernels,
and see whether there are micro-core candidates
that could work with Jeanette for my system here.
Now, as an example,
sample and in the general perspective,
yeah, if you already have an asynchronous
IPC-based micro-channel,
you could also try to build architecture like this on that.
Thank you.
Thanks.
Thanks a lot.
Thank you.