Thanks everyone. Just in case people are curious, we do have like a democratic voting process
for how we pick the talks that get accepted to the devrim, they get anonymized. So all
of the names and organizations get removed from the proposals. So me being selected to
talk was completely not to do with anything being a maintainer or anything. It's all
we do a lot of stuff to make sure that we're not biasing who we pick to speak in the room.
That being said, thank you for coming along to listen to me talk about attestations. Something
which I'm sure other people in the room know way better than I do or at the very least
can define better than I do and hopefully I will do justice on the definitions that I will
bring. I want to particularly at the top of my talk, thank the Open Home Foundation,
which I recently joined the team, the Design and Product team. They make home assistant
and other smart home opens a smart home technology. And I look after the product design
of something called Music Assistant in ESP home among other things. And I want to thank them
for supporting me to be here actually to talk to you all about a previous project that I did
at a previous organization. So this attestations work was part of my previous work at a design
num profit called Super Bloom. You can find out more about that organization that I
no longer work at Super Bloom.design. What we're going to talk about, we're going to talk
about essentially the structure of this project. How did this project come to be? What happened
during a typical kind of short term design project? I'll go over the timelines later. And
also just some of the aspects of what we learned, how we approached UI and UX for this, what
were the goals of the project and also some of the foundational aspects of open source transparency
around doing a design project with the team. And I'll talk a little bit more about the team
that we did this project with later. But we're going to talk a little bit about package registry
pages as well. And what we learned about those more broadly when we were thinking and asking developers
and people that look at open source packages on registry pages will learn a little bit more
not just about the security of that supply chain. Those supply chains of open source packages.
But also how they think and regard package registry pages more generally in this work.
So what is actually I'd love to get an idea of the room, who has heard of the term attestation
as far as like a cool. Okay. Cool. Who absolutely no idea ready to learn, ready to be
interested, ready to nerd out about it, no idea. Okay, cool. So good, good amount. Awesome.
Um, I may ask some of the people that raised our hands first if you would like to add anything
or change anything about my definitions of attestations because you may have additional details or
information that you want to offer. But an attestation, I took this definition from this year
I'll hear from the docs at Docker. Um, so an attestation is a sign statement provides verify,
verifiable information about an image around Docker or a package or some some kind of art of
that. Um, as to how it was built, what's inside and what security checks is passed.
So attestations are typically signed using six store tooling, such as co-sign or other things
making them tamper, evident in cryptographic verifiable. Now I realized this is from one source.
So I went to another source. Um, so again software attestations, they're a trust mechanism.
Um, they allow a verifier a person that wants to consume the package to independently validate
the integrity of those packages or something that's asserted by an authority. Um, which could be
the vendor, so it could be the package registry, um, like brand or identity. Um, so these are
also often automatically machine readable and what we wanted to explore in this attestations work
that we were doing is when people, when we want people to better understand their stations,
when they come to interrogate the information, not just rely on like the machine readable aspects of
attestation information that kind of gets processed and sent back to them like how, how do humans,
which developers are, how do humans understand the process of verifying an attestation,
do they go through a process of verifying an attestation? What kind of steps did they go through
to understand that and what makes sense to them? Um, so this is what we were at one of the
challenges that we were trying to understand better in this project. Um, would anybody
that racer hand about attestations like to add any essential, you think is essential information
about attestations that you think is missed? Okay, but if people, um, might do you want it?
More complicated than this thing.
Might for the recording, might said it's more complicated than you think. Um, and if I can
expand on that and attestation is not always just about verifying the supply chain security,
it can be an attestation about any aspect of the package, is essentially a statement of verifier
bonus. Um, so, I mean, the tricky thing here is we're trying to improve the UI UX, the
usability of something that is an emergent understanding and emergent term and emergent topic,
which is challenging in of itself, but also a fantastic time to be doing usability work.
Um, so a little bit about why checking open source software package, um, package security matters.
Something that I learned a lot about while I was working with the team, um, at Open SSF, the
securing software repositories working group. Um, I learned about typoscotting, which is when a
URL is very similar to a package URL, but it looks so similar to the, you can accidentally
go ahead and, and use something, um, and that the build supply chain of dependencies, they could
have a link that's also malicious or affected in some way. Um, you want to avoid, um, so understanding
attestation, these are the things that why it matters to understand attestations as a person that's
consuming open source packages. You want to avoid critical dependencies being compromised and ensure
that pre-built packages are built using secure practices as well. Um, uh, I hope you like some
of my illustrations actually all open source so you can use some of these illustrations afterwards.
So, should you, should you want to, this is Inspector Package for those that watched the
cartoon of Inspector Guy. Um, so without an attestation developers and users of open source
software packages, because they're not always developers, you're not, it's not always developers
that are consuming open source software packages. Sometimes it's researchers, somebody that might
identify, say, a researcher in an institution, documentary, and any number of people that are
working in open source can be consumers of open source software packages. Um, but they're forced,
without attestations, they're forced to implicitly trust information that they can find around that
package. Um, so without that, it's very, without an attestation, it's very difficult to validate,
unless you know the process of validation and that's part of your role or part of what you do,
in your day-to-day development work or your day-to-day work, to really be sure that this is safe.
So, unless you're a person that's really like knows how to go in and double check the cryptographic
signing of things, knowing that it kind of matches certain user names, or you have certain
processes around that, you have to go on other information around packages to trust whether it
is legitimate. So, this work was done with the securing software repositories working group,
at the open SSF. This was paid for by the working group, which is actually all transparent and
valuable on the various issues and the project board. So, you can see, you can see the person
submitted the fund request, how much they submitted it for, and what they wanted to do around a UI
UX and usability around these attestation issues. Um, but the things that the Open SSF
securing software repositories working group wanted to look at is they, they wanted human
consumers of attestations. Um, they, they knew that they contained a lot of information,
which users were largely unfamiliar with, depending on your context, and they want to understand
how useful our attestations, as well as how essential they are. So, there's an understanding that
attestations are essential for this kind of verifyability, but how useful up really are they in a
day-to-day life of somebody consuming Open SSF packages. Um, so what is the second one is
what information is being shown to users, and what is the maybe minimum viable information to
show to users to better meaningful, meaningfully communicate attestations, and why they're useful,
and why they can be trusted, and why you can rely on them, and trustworthy. Um, and how can we
start to build, and this was a large aspect of the work across the different package registry pages,
and package registries, is how can we make UI and UX consistent across different package registry pages,
different ecosystems of packages, because attestations and security information need to be identified
kind of familiar across these ecosystems, so they can be trusted. So, if there's very different
looking, seeming information, about security, and safety, on say, um, NPM versus say PIPI,
how is any developer that might be working, or any person that's what consuming packages across
these ecosystems? How are they using knowledge that they know from one ecosystem around like,
okay, I'm looking for this information, I trust this information, oh, it suddenly looks really
different over here for this other package registry page, is that the same, and there's spending
extra time trying to understand whether the same information is as relevant, or even looking for the
same information in different places, so how can we, I don't want to use the word standard eyes,
but how can we make it easier to find that information, and commonly, and commonly, looked for
places across, um, people that might be across the ecosystems. Um, if you want to find out more about
the working group, there is the link to the, um, securing software, repository's working group,
lovely folks, um, I hang out with them past this project, as well on the, on the, on the
team course, lots of fun. Um, I want to talk a little bit about the user research approach,
so we had a few designers on the team, and we started with the user research and usability
testing and user testing. Um, to better understand what adaptations were in the context of open
software, um, we actually also spent a good amount of time as designers trying to really understand
what adaptations really were. We were like, okay, we need to make this, this, um, concept more
understandable, really we need to understand it ourselves and the very least basic terms at first.
Actually, this is, this is why one of the illustrations here is around apples. Um, um, like the
concept of an attestation could be granny from granny's farm makes granny's apples, and then
our package and send and then to a consumer often we think about attestations are the label of
granny's apples. You know that it's come from granny's apples because it has a, an attested to label
that this is granny's apples. Um, anyway. Um, so the user research, uh, so we, we formed some
user personas, uh, of just to give us an understanding of who we wanted to talk to, give us a broad
kind of idea of, okay, these are the kinds of people that we think might be consuming outfits or
sat, uh, software packages, how can we segment them so that we can find enough of each kind of
person to talk to about how they use the, the packages. Um, we then, after forming personas,
we then, uh, conducted, uh, semi-structured, uh, interviews together and, and understanding of how
really how security and safety and the liberty of a package was understood by people consuming them.
Um, after the interview of insights, we then developed some visuals and first kind of concepts
of how do we communicate attestations better on these package registry pages. We then tested those,
again, with the same people and some new people, like, how does this improve how you understand these,
these concepts in terms of attestations? Um, there was, uh, a lot of development, a developed,
we developed a lot of support documentation because we found out throughout the process that,
hey, documentation is really important and understandable, human understandable documentation is
really important alongside anything that's like visually representative as well. And often a lot
of what we were doing with the UI and UX was making sure that there were really clear links to
relevant documentation next to key terms that might be new or might be evolving terms as well.
And then we, um, built and published a style guide, a UI style guide of this is how we suggest
this get implemented on specific package registry pages, but that could be extended,
worked on, contributed to change and also rolled out to other, um, package registry
ecosystem, too. How am I doing for time, gang, that is, ideally?
Okay, I'm going to go faster. Um, so we focused on NPM, uh, PIPI and uh, Ruby Gems for this work, um,
there's a variety of information on hair effects, um, just boiled down my longer list of what
I would talk about. Um, the typically users when we did the user research here, they typically,
spend more time, uh, trusting information like who are the maintainers, who are the maintainers
affiliated with? Are those recognisable, um, avatars icons, um, and names? Also, how many times
there's a package been downloaded and used? And it was a lot of this like, uh, web of trust information,
or this, uh, like social proof information that a lot of people that were consuming packet,
and consuming open source packages that were relying on. But there is a lot of information in here
in these package registry pages, um, and actually a lot of people, uh, that we did the user testing
with, we'll go straight to the repository and actually don't spend a lot of time interrogating
the information on this page, uh, and it's validity. So here's just a bit more detail about how
some of the, um, more detailed, uh, information about provenance, uh, provenance is quite
closely linked at its stations, but if you ask me to define the difference between the two,
I will not be able to do that for you, um, but somebody else will be able to do that.
Um, the three personas that we had, and I'm not going to go into detail here,
please take a picture if you're interested. We had a security architect. This is somebody that is
really deep in the world of security at our stations and espons, read all the salsa, documentations,
deep in it. For whatever reasons that may be, maybe they have a really complex ecosystem,
maybe they're working on hardware that they actually can't get to, so they actually have to be
really robust. So if it breaks, they actually can't physically get to that hardware, like stuff like
them that might be in space, or in a very, like, in the desert, um, so it has to be really important.
Um, and also people that are working on anything finance related, so some people that we talk
to about Web3 crypto and working on wallet, they really had to be really careful about, um, the
security of their packages. The pragmatic developer, um, there's more of these than there are
security, our architects, but they, and they've heard of these terms, but they really struggle to
define why it's important, it's actually not terribly important in their role unless they dedicate
time to understanding attestations and espons and things like that. And then we've got an incidental
consumer, there's lots more of these, um, and people, these are the people that go straight to the
download when they see like a million downloads. They're like, great, a million downloads, this is
safe, I'm going to download it, great. Um, they might not even be familiar with things like type of
squatting, they're definitely not keeping up to date with all the things that are coming and,
uh, emerging about security of open source software packages. These are some of the things we learned
about user research, I'm going to pick them out so that it can go through the visuals a little bit more
detail. Um, so yeah, all the personas factored in social proof, even the security architects
were interested in social proof, but they also had detailed steps that they would go to to verify
things like the char, um, cryptographic keys, the hashes, all of those things. Um, the thing that
I found the most interesting and maybe the team also found the most interesting was the all of the
people that we interviewed were fine with critical information about security being repeated multiple
times. So there were a couple of designs that we tested that had, um, like hashes and shars,
repeated numerous times. And that wasn't a deterrent. Like in design, we often think we don't
want to ever repeat information because it becomes less, um, trustworthy if there's repeated
information, like, oh, is this a error above? But with security information and these users,
they said, oh, if something's repeated in multiple places, they definitely want me to see it,
and they definitely want this to be important. So repeating information actually wasn't like
didn't decrease trust, it increased, oh, I'm going to see this more readily. That's actually,
it went against a lot of my background training as a designer to repeat information across different
sections, um, but yeah, they, they, they, they wanted that. And yeah, there was a lot of emphasis
as I'll go through on the visuals about, um, badges, symbols and icons, um, and then the key thing
which I'll show in the visuals is a dedicated security page was really needed across package registry
pages, um, for detailed information, um, about all these concepts, but also the quick reference
in side bars. Um, so here's some of the first visual explorations where we were trying to
boil down the concepts of a concept of an attestation to an icon. We came up with, uh, or like a symbol
of some description, um, we were trying to combine, build with the bricks, package with the box
and signing, um, as like a, as imagery and we kind, we got to the bottom one here as, as actually
an icon for attestations that we ended up using in some of the final designs. Yeah, done.
Okay, I'm going to go through some of these designs really quickly and then, uh, go for questions.
Uh, here's some of the, um, different orientations of information and badges that we ended up
going with for the visuals, uh, and some of the essential information that was tested, but needed to be
in these UI UX improvements on the package registry pages. You can see things like swore syrup build
conference and then the attestations. Um, the style guide is all open. You can find it out there,
on the repository, uh, in PDF form, um, but we suggested three different approaches. We had a
light medium and heavy. Heavy was the dedicated security page. Medium was some slightly bigger
modules with more information and light was very small, um, box sections of the central
information, um, which you can kind of see here is the heavy version where we have sidebar information
and then a full security page, uh, uh, the example here being the MPM adding a security page in there.
Um, one of the last things I'll say, the words confirmed and verified be so careful using
this across your documentation. As soon as a user sees the words verified confirms, they're like
great. This is safe, fantastic. I don't have to further interrogate it. Um, so we were, we had to be
so careful with the language we were using in small UI modules around this because some words
instantly made users go and download something. Uh, we, I also did a bit of work on inclusion and
localization because things are different, different places like orientation of, uh, words and
longer words and we need to consider those. Um, and the last thing I'll say is, we, this is essentially
I can ball this down to one statement. We need to invite more designers into these complex open
source software, like, uh, practices, security practices, because there's so much that we can do
as designers to improve the usability, understandability and the UI UX of these kinds of things,
like package registry pages. So I urge designers to really think, oh, S-bombs, do I really
want to go there? Yes, you do, because S-bombs need your help, security of open source needs your
help. So, and with that, I'll say, done.
Thank you, Ariel. I think there will be a time for one of these questions.
Do you have fun? Uh, yes. Thanks for helping out as of these, I mean, it's really important.
The question is, did you think about the things that have not been attached to,
because it's not that there's multiple things that have not been attached to, yeah.
It's like, you know, it came too, and the most important thing for you is missing,
whether you need a liking for that as well. So the question was around, is there a design template
and have we considered when there is no attestation for something? So what's the design?
We did designs for when there is an attestation, but what should, um, what should pages,
what should information look like when there is, uh, absent of attestations? So the projects
wasn't scoped to include that, though the conversation did move to that shortly after the project
wrapped up, actually, or was wrapping up the design phase. And yeah, I actually think that what I'd
love design contributors to do with this work, um, now that it's kind of out of the paid, um,
paid project side of things, is okay. So what do we need to show when there is no attestation?
Do we need to show no attestation? What does that mean when a user sees, oh, there's no
attestation for this, but also what does that mean for the community building, the understanding
around attestations? Um, because does that lower or affect the trust in attestations, or the
trust in the pages? So I think the balance there needs to not just be, oh, this is what the UI
would look like if there's an empty stay. It doesn't, I think it, empty state would be bad.
We need to really consider what's showing information around security and the absence of that
security detail, does to users understanding.