So, welcome everyone. So, throughout the day, we have been talking about performances and databases
service and not over the places. One thing that we haven't talked about is on the clients,
specifically on the web app. So, we are going to talk about that today. My name is
Abair. I work there software engineer at Mattermost. Mattermost is an open source collaboration
software used by companies who want to keep their data secure, sell first aid and all
of that. So, before we get in, this is the structure of the application, when you
are similar to whatever new chat application that you are familiar with. So, let us start talking
about performance. So, what is performing the scale mean specifically for Mattermost.
We are going to talk about that. First, we are going to talk about and then first, let us define
quantitatively what does the performance metrics value for something like a friend-day
and application. So, this is something that you have been seeing today and now we are going
to focus on client side. This is what this talk is about. So, let us start understanding
first what we use the C's in the application and then when we start seeing that as the
result and then we try to quantify these metrics and next is actually improving them
to see if they have been any improvement or not. So, something like how does the typing
look like when you are typing in the text editor does its flow or is it lags all of that.
And what happens when you open the application and you have other applications or anything
in the background to the hang up or to the freeze on all that. So, this is what the user
actually sees when they are trying to use the application. So, next thing is again specific
to Mattermost is how quickly you can switch channels, how quickly you can switch teams on
of them. So, all of this basically is what user perceives performance. So, this is now we have
absolute performances which we can measure and we have something called perceived performance
with the user is different because the clients which on which your application is running
is very much different. You might have different OS, you might have different RAM all
of that. So, the performance is also is different because you have all the applications
connected to the same server on Cb database but you have various types of clients which
are actually running on application. So, why does no next question is why does it matter
to actually measure performance skill. Now, this is a simple application you are running
on your laptop right. Why does it matter that you have 10,000 users running in the same
application. But yours is a very isolated one because you are running on your own local
machine. So, that is a challenge with a real time application. So, real time applications
have web socket connections connected to the server which brings in the information when
there has been a change or a date in the information. And this new data which comes in has
to again goes to your local store, see which part of the app got changed and the library
has to see this thing has to change. So, let us update the particular note. So, that
the reason real time applications have performance issues especially the client facing ones.
Now, let us not confine these. Let us first start with metrics for all of our familiar
with that is memory profile which is obviously you have open that in your developer tools to
understand where your memory leaks are going, why is something occupying a lot of memory
and all of that. Then, you know, obviously you can also attack CPUs inches while your app
is running. And then, we have a really cool to actually see how how lagging or how smooth
your app is by actually seeing the frame limit of the application while you are doing some
interaction with it. Then, again, you can also see how many are you using are you sending
you know, unnecessary, are you requesting any unnecessary questions or not all of that you can
track. Now, this is something has been mostly in this standard where we have a set of
metrics defined. I think you are familiar with how much time they take when the first bite
arrives on your client, how much time they take for new paint complete or largest paint
of the operator of your page to complete all of its standard metrics. Then, this is not
a standard metric but this is something you can actually access on the app with the help
of chrome dev tools. You can actually understand what is the GS heap size, how many
events listen to are there, we need those events listen to not all of that. Now, you
might also need your own specific metrics because of the different. So, at least for
mathematics we need to understand how much time does it take to sit channels, which
teams, how quickly the threads pop up all of that. So, these are all what metrics that
we are going to first define and then measure all of this, then let us start improving that.
So, let us start looking at some real-world examples of what we actually did.
At mathematics, I think you are all familiar with so many for example, lazy loading,
code splitting all of that is fine. So, for mathematics, one of the things that we did is,
this is again the PR is already reading, you can see what actually work was then. So,
basically the work was to simplify the redux state tree. And so, having instead of having
the jealousy to go into that nested objects and then mapping them or iterating with them,
we actually reduced little bit of the complexity with the, not relative flat data structure of
the store. Then we have all the we have a stocket connection, they are places where we cannot
use the stocket connections. One of the such cases one status is in IDs. So, the only solution
the problem was when this user, when you have found use connector on your scene, same channel
and they are all standard same messages and then the app has to also fetch in the status of the
particular user also other details like name full name and all of that. So, we found pooling to be
a better method over here. We actually tried to convert this to the web socket connection,
but it actually degraded the performance at the server lot. So, we dropped that idea and we just
moved to polling away here. And many times what we usually do is we put everything
we try to make the database on the client side. So, it might not be required to put everything
on your state machine. So, just be careful on what you want to put because whatever states
get those in the application react rights to monitor that state and see if they are trained
so that requires lot of work. So, be careful on what you want to put in your local state
especially on your store. And as I was saying you can also try lazy loading for example,
breaking down the bundles of your heavy components over here what we did is we tried to
break down the initial component a bit more. And so, what that basically did is it increased
caching on the client side. So, though even though there was no change on the code or the
particular initial loading and we were able to just use it while able to just download it
and it helps in at least in low bandwidth conditions. There is another one is there were
the change in react which actually wants us to load library not on dorm loaded but on on dorm
content load event. Now, again there are several more it is an open source library you can
also please look at. Now, one thing what we have understood while doing this is no matter how much
performance improvements we do on the client side network list and see we will eat all of that.
If it is not you know control. So, even if you have like improved 100 percent of that and
80 percent of the effort goes up in that release network latency that means you know at least
use them will not see oh has the application improved or not. So, they will not see the effect.
And client side performance has lot of interdependent factors within as I was saying what I
for machine here using what is architecture machine how much time do you have and all of that.
So, even though if we try to have dashboard all the metrics of what users are using that
and metrics might have so many other spice because we do not expect all the users were using that
to have the same desktop or laptop. So, in as in any performance study comparison is a key
you need to understand what what was A and then what was B and have something change in
and we also have to analyze beforehand which performance improvement might give us a
maximum return not all performance gains are same. So, we have to first analyze if doing this
something is worth the effort or not. So, now that we have analyzed the metrics that we wanted to
do and we dissolve few of the things as as we did over here. And the next question is can we
automate this process so the user so the user even before the application goes to the user can
we rectify it and improve that. So, that is when that is with load testing terms and
load testing we usually do for server side or database. So, on the client side not many companies
do this client side load testing for us it requires because of which real time publication. So,
it makes sense for us to understand what happens when 50,000 users logs in on the same channel
and communicating over some topic. So, just to just to before over you of the existing load
testing architecture we just we can divide that into 3. First is architecture and next is load
load basically how many users are actually might on might not be using your in inside the load
testing architecture and then monitoring how many errors find errors are we getting and all of
that memory managing and memory are all of that. So, as you know we can use this to identify
equations and you know bring the changes even before the client sees them. So, this is what the
load testing dashboard looks like that in matter most we have trying to see which are
top APIs consuming the memory or how much is the API latency and all of that. Now, we have this
already open source. Now, we had to add the client side capabilities to the existing tool.
So, that is where if you are going to go now. So, before that let us talk about the architecture
a little bit more so that you can understand all of this work. So, we have we have the
matter most app on various app instances they are connecting third part. So, this is like job
servers like such on all of that all of this is behind a proxy and then you have the load testing
tool which has something or agents that I will come to agents later on and these but all
all of these agents have been controlled by something or coordinate.
Now, coordinator now let us also add in the metrics or observability of a load test. So,
we add the permit here and see that to Gafana to visualize and all of that and all of this forms
a loop. Now, let us try to expand each of them a little bit more. So, to deploy this we have
the JSON file which can was in a data form library and then the data form skip to deploy it
on the AWS instances. So, as I was saying let us talk back let us get back to the coordinator
which we are talking about. So, the coordinator as you can see agents are basically users over here
these users currently are just using the application not by actually browser but they are actually
simulating the API response. So, example log it. So, it just sends a log in API and that is how
it is mimicking the user behavior and the function of the coordinator over here is to basically say I
need 10 users or I need 20 users based upon something or feedback. Now, this feedback is if the
application is we have started the application and applications has maybe like 95 percent of 95
is not there a rate. So, coordinator will say oh you want there are no errors. So, let me
try to increase the user a little bit more. So, that is how the feedback loop works. So,
yes something this is the feed this is these are all the metric constraints that we have for
example. I want to see by utilization to be around this much and based upon that coordinator
until this particular limit is reached coordinator will try to increase the users.
Now, a little bit more inside what the agents are. Ines are as I told that they actually mock
a user and so, user over here at least before this. So, users are they are just sending the API
responses like that they are told about log in or they are just switching all of that. So,
they are not actually users but they are we are just simulating them through API endpoints.
Now, this is what I was saying we just run that particular function which is underneath which
is just a new API call. Now, that is fine now we need to add an actual browser to it so,
that we can actually see how the mathematics is performing, mathematics user are going to perform
that. So, to do that what we did is along with the agents we added something called a browser agent.
Now, this browser agent is nothing but playwright script running on loop. Now, this playwright
played this loop could be something like user log in, success channels, write some posts all of
that. So, this particular simulation is been done by the users. So, this is the browser control
run that in headless environment, it runs on chromium and the web app also has the metrics in
build up them so, that whenever play starts running those those particular metrics are exposed
to permit here and we can see them in graphana here. So, this is the graphana board of the
after the simulation after the load test ends and we can see we have defined some metrics that
we have already defined last time, how much is the chance which are just all of that. So,
why did we do this instead of just collecting the user data user usage data of the app.
One thing is uniformity as there was also early on this talk is having a constant
environment where you can actually run your client and which is similar in all the aspects,
same browser, same variable as instance all of that. So, that removes the uncertainty factor
from your client set performance and then you can actually see what the performance over here is,
which is what we actually trying to achieve over here. So, this one was the video of actually
using the tool to deploy that. So, how did we come to this journey? So, this is how we did,
we wanted to have 100k users under the same instance of the matter most. So, naturally the
naturally what we wanted to do is improve the server set performance, we did that. Then we thought
okay we need something more to actually actually simulate the real users. So, that is where we added
the client site, give a little load test. And yeah that actually gave us the continue
app to improve in cycle and with that you know we identify and catch the client site errors
or performing issues the way before they use actually see them. And it's open source. So, yeah
anywhere into that. And that's the end of my session. Thank you so much.
Thank you very much.
Hi, can I hear me? In terms of resources to simulate
the 100,000 clients. How many know the CPU memory are we talking about for
this simulation? So, the question is how many what is the CPU, what is the
resource that you want us to do this? How does it look like this stuff? Yes. So, what we did is
again it's all configurable in the tool that you can actually define if you want
tenions is or 20 inches. It's all configurable on it's on a preview. But what we have seen is
having five browser agents in a single instance gives you the same result of running
tenions. So, that's why we have we commend running five browser agents in one single instance.
So, but to answer your other question is yes there is no restriction on what resources.
It's all dependent on because this is running on your
alibless account. So, you can just configure it to whatever
configuration you have. It's all applied to telephone.
Are there any other questions?
Okay, thank you for the talk. Yeah, thank you so much.
Would you like to see? If you see some trash near your place, take it like an empty.