A Security Incident and Event Monitoring (SIEM) system is responsible for collecting, monitoring, analyzing, and generating various security alerts for any suspicious activity in the cluster. SIEM systems usually collect the various system logs, network logs, and application logs to identify these security incidents and events. Hadoop itself can be used to perform the analysis and correlation of these security events in a batch mode.
The first step in any SIEM system is to collect the various system logs and identify corresponding events. The following are the events that need to be monitored in a Hadoop cluster to detect any security incidents:
