Hey, hello everyone, thank you for coming to my talk, and today we're going to take
about TPMs and the Linux kernel.
Everyone here knows about TPMs, what are they, okay, yeah, yeah, by the way, my name is Ignat,
I work for CloudFourer, anyone doesn't know CloudFourer, hey, good, so what is the TPM
it's a security chip present a door more than laptops and servers, right, yeah, it's a nice
passive device, it's not intrusive, so it's not like Intel management agents will
which dig into your memory if you don't ask it to do anything it won't, but it is still like
chip enough to put in every piece of hardware, but it's a foundation for platform integrity
authentication and remote as a station, and as a security chip, it can handle cryptographic
keys as security chips can do, so when you hear about TPMs and many talks, you might usually
talk about like remote at a station, right, so, but this talk is not about remote at a station,
it's about an application developers like who develops open source here, do you use TPMs to
store your keys? Why? This partially was actually inspired by two years ago James
Bottomly's talk, he introduced himself as like one of three people who works on TPMs, I think
in the world, right, so yeah, platform integrity is an operating system concept like where
application developers, what do we do? So this is my, sorry, this is my ideal mental model
and there's an application developer how to use TPM, my call TPM use, right? Not that simple,
right, like I run an operating system, so I cannot talk to hardware directly, so probably
I need to do some obstruction more than a link to you, I need to talk to the slash that TPMs
zero device, but it's not that simple, again, because TPMs are very low powered devices,
they can't have concurrent access, so something needs to serialize from multiple applications,
because everyone seems to want to use TPMs, so you need to have a thing called resource manager,
which kind of tracks and allows different applications to not to confuse the TPM,
and way to talk to the resource manager, you need some kind of obstruction layer TPM library,
right, like an interface, so let's focus on the resource manager, traditionally the first
iteration of resource manager was like a demon on the operating system, and it's kind of very
confusing for application developers, right, so like I talk to device, which is definitely present,
but if some service doesn't run, like I can talk to it, or is it crashed because it's written
and see or something like that, which is now like the Linux kernel handles it better, so
now kind of kernel has internal resource manager, so instead of talking to slash the FTPM,
you talk to slash the FTPM RM, which stands for resource manager, and yeah it kind of,
you kernel runs all the time and resource manager runs, if it crashes, your kernel crashes,
so you would notice, we're good here, TPM libraries, well James can talk about it many times,
so like there is like different variations of this, like if I'm an application developer,
I have to take the libraries, you have internal trusted software stack, IBM trusted software stack,
if you're doing on goal, you can use GoTPM, probably there is a rask rate flowing
around somewhere I never researched it, but yeah, but the point here is like all these interfaces
are different, so they're not standardized, so if you'd call your application against one librarian,
you want to distribute it on a Linux distro, which doesn't have your trusted software stack,
you kind of have to do either abstraction by on top of the abstraction layer or think of something else,
so it's confusing, if you go to this website, you will find this picture,
all this you have to understand to this one against TPM, right, which is,
and this is kind of my theory why not a lot of like really open source projects have to
keep in, but this is only part of the story, imagine I figured this all out and I write my code,
right, so this is a bare bones, deb and installation, so like I start talking to the resource manager
and I get permission denied even like I did all the interfaces, well like imagine I'm a go application,
and this is because they both the TPM device and resource manager by default is owned by route only,
right, so you need privileges, more confusingly if you do install something from the
infiltrust versus software stack on deb, you will see that the permission's changed, you're like,
now it's owned by the special TSS group and technically your application can become part of the
TSS group and talk to the TPM, but if you're writing, go, should you still package dependency
your deb and to install infiltrusted software stack just to get the proper permissions, well,
no, right, but again, TPMs are complex devices, you can do many things, what we're focusing
like, can I just get like cryptographic keys, right, for my use case, somehow?
So, and this is where there is another interesting thing which is mentioned in different
presentation Linux kernel key retention service, aka key rings or key store, actually I think
did the presentation about it two years ago here, I don't remember, and the whole point of it is,
you kind of have these multiple applications, you have the Linux kernel and you have three
system calls which kind of allows applications to store keys inside the kernel and the original purpose
was to kind of give keys to the kernel for using to internal crypto systems, like this
encryption and file system encryption, but it was late like standard that you can kind of like
also use these keys as objects, you can read them, you can use these keys to actually encrypt data,
and informally it's very similar to key rings or, you know, key store has key rings,
like collection of keys and key rings can contain keys, the actual objects that have
cryptographic data, and as kernel objects they also have like proper access control and everything,
and it's very very very similar to a file system where keys are files and key rings are
directly, so you can build this like hierarchy very easy. So, for application there wouldn't be
like somehow combine like TPMs and key stores, so like because key store is literally a set of
system calls available to any unprivileged application, if we can like the key store can be this
nice abstraction layer for the TPM keys, right, kernel kind of has that a little bit, so there is a
concept of a trusted key and the whole idea that you don't allow user space to have plain text cryptographic
material in the address space, so like imagine you have some secret or a key wrapped to the TPM
of your machine, you can use a kernel key store API or Cisco to insert that blob into the kernel,
and the kernel will automatically use your TPM to unwrap it, so in the kernel they object is plain
text and you can actually use it for encrypting stuff, but if you try to read it back it will
again wrap, so it's kind of like it's it's similar maybe to maybe a software HSM or something that
you can only get plain text keys inside the kernel, like a more restricted environment.
The restriction of this model is good for keys, but like we're from TPMs from time to time
as we need also like secrets like credentials, like like what if I want my application needs
like an access token to somewhere, right? So I cannot really use that because I can never
for access token you need to kind of access to the plain text secret and you can never get it through
the trusted key. Another problem like what we see in Cloud for Rostris keys is actually management
of extra state, so how would you generate a wrapped key or trusted key, so you have random
non-generator, get your key and you wrap it to your TPM, right? And now we're unlike thousands
of services in production, yeah and what I wanted to say, the extra state is even though
this kind of works this is a stateless system, so if you want your key to be preserved upon
machine we start, you need to store it somewhere, right? But if you build like secure production
system you sometimes want to have like a mutable infrastructure with only file system,
your disk is break, you have to replace them, and the point of here is like, yeah your secrets are
secure, but if you lose it and crypt it below, you lose your secret, right? Our S3 team wants
in a while accidentally wipes a machine as well. Yeah, and another problem is with like scalability
of this extra state, so you have like this works well for one application when you have like
many many services and everyone wants their unique key, so you start in having this like extra
state that you really need to care about, if you don't want to lose these keys you have to
manage these bulbs somehow back to them up and others. So what if we do a different approach?
What if we do a derived key? So TPM has this like secure seed inside which is guaranteed by
the standard, right? And you can never extract it and it has a KDF implementation, yeah and it's
all part of the standard, so every TPM has that. What if we just like provide some public
I call it a mixing or application specific metadata, a unique identifier to that, to just generate
an application specific key application unique key for that particular system. And it's now
it's like perfectly scalable because like every service on your system can have their own keys,
but this kind of state is totally disposable, right? Because the whole thing is full reproducible,
it works in like immutable systems, you don't have to store it anywhere, you can just recover all
your keys every bullet. And this kind of approach is submitted like I think more than a year of the
first iteration of to do like the right keys in the Linux kernel, I received not very action number
feedback. I did receive even though I would argue that I'm not actually inventing anything
you here, so if you ever worked with any Tras Zone implementations, I are proprietary or OPTI,
you have this concept of hardware unique key which kind of does exactly the same thing but for
your trusted applications, right? It's just like I want to bring that concept up or level, so like
I want to just bring that concept to any application that runs on the Linux system. And there is
like a prison, so we kind of already use that approach, but in user space, like 10 years ago I
developed this tool which we kind of like tested this approach in production and it's still being
used for 10 years and it's very scalable in terms of managing secrets for a wide
for a big infrastructure. If you're interested, I have a talk from 2017 on SR Recon, so
it kind of brings you an interesting properties how you can quickly rotate keys across like
a thousand of service with one configuration line change and you can target like either you can
rotate your OLSSH keys, you can rotate all your disk encryption keys, you can rotate all your
machine keys, so like you can have like any slicing and dicing granularity as you wish.
But there was some also useful feedback, so I since then implemented the key derivation approach,
I simplified it much, so I initially used TPM based Hmark from the seed, but I migrated it
to a simple AS 256 counter, random number generator. James luckily he raised a question on my original
patch set, like how do you protect confidentiality when the data transfers on the bus between the
kernel and the TPM, but he since then fixed it by implementing encrypted sessions, so I
added support there, so now the communication between the kernel and the TPM is encrypted as well,
and now it kind of most of it is compiles as a module, so if you still don't like it, you don't
need to enable it, you don't have it, or even like you want to compile it, but not law it, so it's
worked fine, so the way how it works now, so you have a TPM, we use a special command create primary
and create an IS counter key, and to create applications specifically, I have the create primary
may accept an optional unique parameter, so like how to make this key unique for a particular
purpose, and we use some like different kernel metadata identifiers for a particular process,
which allows users to kind of slice and dice how unique that key should be, should be unique
per application per user per user running the application, and you have implemented all these options,
I will just show them, so there are basically all these data as user requests that is being
hashed and provided as a unique value to the create primary, so TPM would create primary key for
AS counter, and we just like extract the cipher stream as our secret for a particular application.
So now I'll show you a demo, which is was copy pasted from a real terminal, but it never fails,
because I tested it, so like here is a simple example, imagine this is my code, but I'm using
the command line bash tool to just show how we call it, so we request to is a wrapper around
request key system call on Linux for Linux course key store, so I'm requesting at the right key,
and I'm saying like I want a key of size 32 bytes, and I want a key unique to my user ID, right,
so, and then I can see the key, and now I'll show like the process is fully reproducible, so I delete
the key now, and we request it again, and you can see like the blue numbers are serial numbers,
it's a unique number of each key in the kernel key store, and you see there are different,
so from a kernel perspective, there are different objects, so the key was destroyed and recreated,
the different key, but the key bytes are the same, so this is how the process is fully reproducible,
and basically any user, anything running as my user ID can recover that key on my system, and the key
is specific to my user ID and specific to the system, right, yeah, and to show it works,
if I run the same corner with pseudo, so I'm changing my user ID to root, root gets a different
key, so technical I cannot steal a key from rule, and the other identifier I can use is a file
system pass, so let's say my tool is located as slash user bin, I get one key, I move my tool
to a different pass, and the same tool I run, I get a different key, and this is how you can bind
your unique key to a particular file system pass, the original feedback that the pass are not really
good at like for security, right, so you can trick those with user namespaces and other stuff,
but based on the latest LPC, so like many other solutions are still preferring to give passes an option
for different policies, because they're kind of like really well understood with users more than ever,
and basically to protect, for example, from unprovedly user namespaces, you can combine user
ID and pass, so user ID in this context means user ID in the initial user namespace, so even if you
bind your key to pass, but somebody creates a unprovedly container and then creates a mount point to
kind of circumvent that pass, they will still get a different key, because in this case, their
user ID in the initial namespace would be like unprovedly, so you technically cannot steal the keys like that.
If you are very concerned, still, and you find pass unreliable, I decided to add it like and
let's say you're managing an immutable infrastructure or embedded device with a read-only file system,
you can technically bind instead to a file system, you ID, and I know number, so in this case,
your key is unique to a particular file system instance, right, no matter where it's mounted,
and you can kind of like rely on that. One thing is also like another interesting option is
application checksum, so what checksum means is the key is unique to a particular binary code
that requested it, so like when you request key in this way, the hash of the key CTL itself will be
hashed by the system or taken from iMASUPSystem, and it will be mixed into the unique key, so in this case,
we can say if somebody accidentally patches your code, right, so I know key CTL has a string
bad message, and I want key patch it to bad massage. If I rerun the comment, I will get a different
key because even though I changed one byte in my requesting application, the key is not not available,
so it's kind of you. It's not very convenient for if you keep updating your code, but like if you
don't update, you feel you don't need to update your code, you can have a unique key specific
to your application binary. James also pointed out that what if the application needs to rotate keys
and you suspect it compromised, for example, so in this case I also added like a free form label
parameter, and instead of applications instead of having one unique key, they can have a whole
key space, right, and they can slice it disinternally, so for example, you can introduce
a simple versioning scheme like label V1, if that key or secret was considered compromised,
you can kind of reconfigure your infrastructure to generate a V2 key and you will have a different
key for each application. The other like useful feedback was that all these mixing values are well
known, so technically, yeah, I can do all the things, but like if I have direct access to TPM,
I can use users by tools to craft the same commands and kind of steal those keys for a particular
machine, like even like a privileged application. So this is where kind of I think still
integrating it within the kernel will be useful because the trick I explored is the fact that
even if you use encrypted sessions which you should, here is how the TPM create primary command
look, so you have like a header and so-called handle area, you have a so-called sensitive area where
if you use an encrypted session, both the request and response will be probably encrypted,
and then you have the public area which is never encrypted, even though you use encrypted session.
So yeah, this is encrypted and the nice thing about that the unique parameters that we use
is in the public area. So we can actually separate within the kernel,
to instead of just using shot to 506 of our mixing application identifiers directly, we can
propend it with the kernel string, for example, or any arbitrary string, right? And then in the driver,
we can look for the in the TPM driver itself which is responsible for passing commands from user space
to the TPM, we can monitor for this command for this string. So for example, in my test
implementation, here I'm using the user space tool to create primary and I'm sending ABC as a unique
parameter so it works, but I can monitor if a unique parameter starts with the kernel, the driver can
be modified to return permission denied. So you only allow the kernel itself to initiate the
great primary command from the kernel space and not allow the user space to kind of seal those keys.
So yeah, I wanted to actually submit the patch set before this talk, but just like the last week,
I found I have a kind of vulnerability that I need to fix. It doesn't change the concept, but like
some minor things that I need to fix before patches, but what else is missing for easy adoptions
of TPM and Linux software, any other make sense, you would find useful, should we support
better separation between generic secrets, the secrets that we are allowed to read as keybites
and like actual cryptographic keys should we make the keys non-exportable, so you can only
like use the keys, but not the C, see the plain text material. If we're using TPM, it would be nice,
somehow to piggyback on the advanced TPM capabilities if I use this way to generate a key or a
secret that I also have like a cryptographic pool which I can send to remote system that this key
was generated by the TPM so I can build like easy provisioning systems for those keys.
Yeah, but the current patch set in this tree, if you're interested and basically that's it,
a little bit of shameless self-promotion, so cloud for a pledge to hire like
1000, 100, 111 interns this year, so we have a big internship program if you're interested in
and all the teams are looking for interns, so if you're interested in internship in any
place in the software stack, so please reach out. Thank you very much, and I think so have time
very close. Do you have any kind of TPM policy on the keys? Because you know, you like one
and one of your last slides, you kind of suggest that you want to do the access control to the
keys by doing the kernel, string pre-fixing and things like this, but that's software access control,
right? Like a bit TPM and the policy language, and this is really important, right? Like because
if I understand this correctly and I boot a kernel that I compile myself, right? Like I get access
to all your keys because I can submit everything I want regardless of the kernel than itself
blocks us to use this place. So what's your like, is there no TPM policy, hook up whatsoever,
and you just have the keys completely accessible or what's the model there?
I try to make it simpler. I think you're right, like TPM policy can be a non-trolantive
thing to restrict, and you're totally right, I consider the case that like, yeah, we rely on the
patch driver, right? And now if I just replace it with the custom kernel, I can still do whatever I want,
but I think it's kind of different areas like, well, if you're concerned about that, you still need
to have secure boot or basically some system integrity. So if you're really concerned about this
case, like you should solve it with something else, right? You shouldn't allow just people to
easily replace it. So just like in my view of the world, right? Like the, you always want to
tie like a verified bootstuff to this, right? Like so that the keys that you have are only unmarkable
on the systems, you actually think they're, they're stressful. That's, I mean, I was just
surprised about the complete absence of TPM policy. Well, they're just more complicated to
understand for users. I'm coming from an application perspective, but maybe you're right, maybe
we should have both options. So I, I just prefer to give more, not have a policy in the kernel,
what is right or wrong, but like give more choices to user, and they can like select what they want.
Yes. So, um, it's a little bit early maybe, but do you have any plans in the future to integrate
these with higher level cryptographic software, like open it's a sale or new TLS?
Well, it could be. Yeah. So like, I know open SSL. I think had a provider to use a kernel key store.
So I don't see why not. Like, here is a question. Yeah.
Hi. Yeah. So again, I think this is in that secure boot chain. What I have seen is that,
usually in the boot loader or somewhere that you use this TPM as a firmware, a talk that the
the boot loader talk to the firmware, and then says that whether it is a secure boot or not,
but when you boot into the Linux kernel system, it generally some verification with some
SSL or some checks or something, right? So do you think that during the boot process or something,
this this could be used, this library could be used as a part of some standard demand,
some system, people will see or something? I think like Lennon, this is better, right?
But I can today transfer this question once you have actually nice ideas about that.
Okay. He didn't pay attention and we're out of time and for thank you.