Embedded Files
In this lab, we will be performing the following:
Import a sample log file into Kibana
Use Log Categorization to find unusual log entries
Step 1: Import Sample Log Data into Kibana
Step 1: Import Sample Log Data into Kibana
Download the sample log file from:
Go to Machine Learning > Data Visualiser.
Click on Select file.
4. Upload the “it_ops_app_logs.json” file from Step 1 above.
5. Accept the default mapping and click on Import at the bottom.
6. Name the index “lab2a_it_logs”.
7. Click Import.
8. Once the import is done, right-click on View index in discover, Discover and ML to open new tabs.
9. Navigate to the tab for Discover.
10. In Discover, explore what the raw data looks like. Note that the “message” and “message_text” fields contain the log message entry.
11. Navigate to Machine Learning > Anomaly Detection > Jobs, and then Create Job (or continue from the previously opened tab).
12. Now, let’s create a ML job using the index; labs2a_it_logs, to detect unusual log entries.
13. Select the Categorization job wizard.
14. Click Use full data.
15. Click Next.
16. Select "message" for the Categorization field.
17. Click Next.
18. Name the job “lab2b_unusual_log_entries” and place it in “mylabs” group and click Next.
19. Click Next to progress after Job Validation.
20. Review the final job configuration and click Create Job to start the job.
21. Click View Results.
22. We can see that the ML job flagged out abnormal log entries.
Typically these categories of log messages only appear once in the time bucket (default 15 mins) but the count went up to 49 & 50 during that time period. Hence ML has flagged that out as an anomaly.
Page updated
Report abuse