WEBVTT 00:00.000 --> 00:07.760 Hello everyone, thank you for attending this session. 00:07.760 --> 00:14.040 This will be a bit interactive and very useful for anyone who is working on containers, 00:14.040 --> 00:18.120 regardless of local or some other sources available. 00:18.120 --> 00:20.280 So I will get to start it. 00:20.280 --> 00:24.320 Very short interval about me, my name is Usman, I work at your final in the 00:24.320 --> 00:25.320 airplane team. 00:25.320 --> 00:28.960 I am actually working very closely with the open source community on their 00:28.960 --> 00:34.160 final or other areas on GitHub or Slack and yeah, I do a lot of open source 00:34.160 --> 00:35.160 stuff. 00:35.160 --> 00:42.600 So you can find more about me also if you scan this QR code or so. 00:42.600 --> 00:47.440 And yeah, I am actually a Linux user and while working with the communities for 00:47.440 --> 00:51.840 your final end doctor, I learned a lot of things and I am just going to explain one topic 00:51.840 --> 00:56.200 which I really like, not life. 00:56.200 --> 01:02.200 So by we are today, so let's go back in the history. 01:02.200 --> 01:07.440 So back in the days, we have like this simple architecture of application, like you 01:07.440 --> 01:12.760 have one server and everything is there and people use to access through it. 01:12.760 --> 01:21.160 We have devices like laptop, PC and then smartphone and for you or the customer or the 01:21.160 --> 01:27.040 client, you name it like it works fine and they were happy and everyone was happy and 01:27.040 --> 01:31.080 they are like, can go home and relax, right? 01:31.080 --> 01:36.800 But then they were problems and they identified that they need a better solution because 01:36.800 --> 01:43.480 putting everything on a single page is not good and high on resources as well like back 01:43.480 --> 01:45.560 in the days we used virtual machine. 01:45.560 --> 01:52.680 So the evolution of containerization started and Docker was the pioneer for this and they 01:52.680 --> 01:59.640 solved this problem like if you have multiple services running, you can use like Docker or 01:59.640 --> 02:05.800 Docker Compos or Docker run or Docker Compos to do this is kind of a stuff and they are all 02:05.880 --> 02:13.240 isolated but you can still make your web application up and running without having too much 02:13.240 --> 02:22.680 problems and then this micro services application ideology also follows in Kubernetes as 02:22.680 --> 02:29.680 well because deep down like Kubernetes big thing but deep down there is the part of 02:29.680 --> 02:35.840 containerization so you have to use like a container based image and the concepts also 02:35.840 --> 02:37.240 flow there. 02:37.240 --> 02:43.360 So then the question is like how do you observe your applications to there, right? 02:43.360 --> 02:48.160 So you say like, hey I'm using Docker in my organization, me and my team we are happy, 02:48.160 --> 02:55.600 we have four applications replicas they are up and running and then if something goes 02:55.600 --> 03:02.800 bad or the service is degraded it's broken but it's active so it's degraded then what you do 03:02.800 --> 03:09.520 or how you know if thing goes wrong you say okay application for is not working why not 03:09.520 --> 03:18.000 spin up more applications right that will solve the problem yes it will but I used still 03:18.000 --> 03:25.120 having the complete view maybe not right those application will also fall at some time 03:25.760 --> 03:33.840 and you were like hmm what should we do? How do we find these problems because then the thing 03:33.840 --> 03:41.200 get out of hand that your customers start screaming or your team isn't in the firing mode 03:41.200 --> 03:46.560 and you're trying to solve the problem without knowing what the problems are you don't know 03:46.560 --> 03:54.480 the root cause and yeah you start screaming in your building or office or from the customers 03:54.560 --> 04:01.280 and then you finally like oh my god we need to do something right so what is the solution? 04:02.800 --> 04:08.720 Let's talk about a little bit about the foundations of observability it's a fancy word but 04:08.720 --> 04:16.560 it is very important to understand so I will explain this that it's basically a journey not a destination 04:16.560 --> 04:24.080 because whether you create applications in any environment you make it on the production it will 04:24.160 --> 04:29.520 break at some time because you will do a customization of code also so it will break there is this 04:29.520 --> 04:35.600 big chance but you will fix it and you will try to solve or find out what the root cause is 04:35.600 --> 04:40.560 doing the root cause analysis and after that it will happen again but it's like a never ending 04:40.560 --> 04:48.400 cycle and you need understanding of observability and the three plus are like logs, 04:48.480 --> 04:55.280 metrics and traces so this is very important to know at least on the ground level what you are 04:55.280 --> 05:01.440 looking into it for your application just simply looking one or two of it may or may not solve all 05:01.440 --> 05:07.440 the problems so you need better understanding like how they are working together and then the 05:07.440 --> 05:13.600 question come is like finding a complete observability stacks so let's talk a little bit about 05:13.600 --> 05:21.120 graphana so I would say graphana use case in the market in general is known as a visualization tool 05:21.120 --> 05:26.160 it does not solve the problem it's a tool which gives you that visualization experience right 05:26.160 --> 05:31.600 it is available on all platforms and you can connect the source of data to graphana and it will 05:31.600 --> 05:39.840 give you that in a nice visual format and the question is like how you can make your own observability 05:39.840 --> 05:43.920 stack right because okay you know graphana raise your hand if you have used graphana 05:44.800 --> 05:51.520 very nice a lot of it so you probably know that this visualization layer this is the dashboard 05:51.520 --> 05:58.320 which is connected to your data sources of any kind right and that is coming from your environment 05:58.800 --> 06:05.680 but here at the bottom we have this composable open source stack and you can use any of your 06:06.640 --> 06:14.320 we call it like LGTM looks good to me and you can read like it's we have for logs visualization 06:14.320 --> 06:20.880 the main graphana tracing and metric is for either memory or promise yes but you can use like 06:20.880 --> 06:28.000 open telemetry nowadays or you can use any tool of your GIS it will still work and with that you can 06:28.000 --> 06:33.840 do more and more you can observe the area which you are looking for either is a performance testing 06:34.000 --> 06:41.680 or infrastructure observability and so on then since we are talking about container so the next 06:41.680 --> 06:49.840 tool is known as C-adviler how many of know about C-adviler okay a few nice very nice so basically 06:49.840 --> 06:58.080 C-adviler known as container advisor basically it's main job is to give you that observability 06:58.160 --> 07:04.160 or monitoring experience for your container because the common mistake which everyone thinks like 07:04.160 --> 07:11.840 hey we can get the metrics via Docker engine but Docker engine give you the the information only 07:11.840 --> 07:17.920 of the Docker itself not the container itself while the C-adviler gives you much more deep down 07:17.920 --> 07:22.800 what's going on in that particular container how many process the number of memory they are 07:22.880 --> 07:28.160 using and so on so this is a big difference and this difference will come handy and we will have 07:28.160 --> 07:40.320 a live demo of it as well let's go in metrics or demo path so I think this is big enough for now 07:42.000 --> 07:47.760 so one thing which we can also do is that Docker has a built in command called Docker steps 07:47.760 --> 07:57.840 and okay yeah so it can give you also that live information of the container how they are working 07:57.840 --> 08:03.680 internally and this is like a runtime but the problem is that this is only visible on your screen 08:03.680 --> 08:09.360 but you cannot share it with others so there's this limitation within the core Docker path so 08:10.080 --> 08:16.400 to overcome this limitation we can use C-adviler so C-adviler comes with this integration 08:17.280 --> 08:26.320 integrated web UI so if we do very quickly so this is the C-adviler default web UI and as you can 08:26.320 --> 08:32.320 see here it has a Docker container path and if I just say like I just want to even visualize 08:32.320 --> 08:41.200 how the gyrphana container is doing so I can get all these live data here and view it and 08:41.440 --> 08:49.600 understand how the performance is going on and the one limitation here is as well because since 08:49.600 --> 08:59.040 it is limited so it does not store anything right so you need a solution and the best way here is 08:59.040 --> 09:05.600 to use the prometheus so if you hook up the C-adviler data into prometheus and then visualize in 09:05.600 --> 09:11.040 gyrphana then you can have the data which is stored in a particular place you can view the 09:11.040 --> 09:16.560 historical data as well and the live data as well so in gyrphana this is how it will look like 09:24.800 --> 09:31.040 so now you have a complete overview of all your containers so you can see here that they are 09:31.040 --> 09:39.120 telling you all the information and this data is coming from C-adviler so you can see here for example 09:39.120 --> 09:46.720 this graph tells you like how much currently I believe this is the CPU usage is it's done so currently 09:46.720 --> 09:54.640 C-adviler is using us bit and as the other services which are listed here so this is the very 09:55.120 --> 10:02.160 useful and a productive way to get more information about your containers because you will have 10:02.160 --> 10:08.000 some application which may behave and you do not know what's going on so you can get this 10:09.200 --> 10:20.480 visualization layer then the next question is about logs obviously logs is very important as 10:21.440 --> 10:26.560 someone who is also been in the Linux world I love you viewing logs and they tell you the more 10:26.560 --> 10:32.880 information like what happened something crash or out of memory and so on and yeah in the IP world 10:32.880 --> 10:40.720 we say like customer may live but logs don't live so logs are very important and and for logs 10:40.720 --> 10:46.080 I think that for me the default choice is to go for Loki it's again an open source but the cool thing 10:46.080 --> 10:52.640 about Loki is that it can accept from everything so you can use Loki if you want to visualize 10:52.640 --> 10:59.760 log for Linux for database for Kubernetes something on Windows which I'm not very fond of but yes there 10:59.760 --> 11:08.160 it is so yeah and it has also an API so you can use Loki with a called functions with API and 11:08.160 --> 11:16.320 you it in a JSON protocol format and in Loki you need an agent an agent can be of your choice 11:16.960 --> 11:22.880 independent your own freedom so you can use like for example the promptail is or Defana 11:22.880 --> 11:28.400 Alois from Defana but you can use fluent bed logist as and you have probably heard of like 11:28.400 --> 11:35.680 ELK stacks or Splunk or Elastic and they also use Loki as a top layer for the visualization so 11:35.680 --> 11:44.560 you can build your own choice of tool with Loki there is no problem and Loki what it does it does 11:44.560 --> 11:52.560 the indexing of the timestamp not the data or the information in the log line so it also saved 11:52.560 --> 11:58.320 a lot of this space which is very vital when it comes to logs on the cloud style they are big so 11:59.280 --> 12:06.720 it has like one key advantage that is good to know and to view the demo so 12:09.120 --> 12:18.240 here I have logs as well so now if I so I'm using Loki with promptail with Defana Alois and 12:18.240 --> 12:23.200 I can view all the logs so you can see like here we have a very nice visualization layer and it's 12:23.200 --> 12:29.760 already shows like something was bad so it is highlighted in red good our ingredients unknowns are 12:29.760 --> 12:44.320 always in like gray so you can get all the information in a single place so there's also one part 12:44.320 --> 12:52.480 which is tracing you can use tracing as well to get this since it is due to time limitation 12:52.480 --> 12:57.360 I'm just sending this link here you can take a screenshot or at the end you will get all the 12:57.360 --> 13:04.320 content I will share the code so but with this demo which is I link here what you can do basically 13:04.880 --> 13:11.760 yeah so you can get the whole experience so not only you can just use for logs and matrix you can 13:11.760 --> 13:17.840 use tracing and profiling as well so once you have an application you can get all of this data in a 13:17.920 --> 13:23.040 single place and you can again choose the tool of your choice you can use promethias or 13:23.040 --> 13:30.880 your ELK stack whatever and just use your fun as a visualization layer it works fine so this is 13:30.880 --> 13:40.000 what the whole picture looks like and the final take of this for me will be like the OSS part 13:40.000 --> 13:46.080 because I love working on OSS and since there are many tools available on the open source platforms 13:46.080 --> 13:53.840 so they're try to use it community is a very backbone because most of these tools are 13:53.840 --> 14:00.960 come from the community side and also the new features which they're where we get up also 14:01.680 --> 14:09.680 and yeah and the last but most important is like you want a layer a complete stack of everything 14:09.680 --> 14:13.920 you don't want like too so many application which are scattered around and you have to connect one 14:14.000 --> 14:20.400 place to another so you try to use a tool which can give you all thing in a single place so that is 14:20.400 --> 14:30.000 very important and will be saving you a lot of time this is the QR code you can scan it because 14:30.000 --> 14:37.520 I link all the documentation links and also that MLTP repo which you can use at home it's basically 14:37.680 --> 14:45.360 a fun way to test out how doctor works so yeah and yeah that's pretty much it I have 14:45.360 --> 14:53.120 us also just want to say that we have a griffana booth here as well in the cable so ask questions 14:53.120 --> 14:59.440 get some swag or just chat or what you are working on griffana we will love to know your feedback 14:59.440 --> 15:02.160 so yeah any questions 15:07.520 --> 15:20.560 okay then thank you thank you