YARN application logging

With the execution of an application over a YARN cluster, logs are generated for activities in the different components. These logs are broadly classified as follows:

Services logs

The ResourceManager and NodeManager daemons run 24 x 7 on the cluster nodes. These services keep a track of the activities on the cluster and coordinate with the other processes such as ApplicationMaster and container. The YARN service logs are created under the logs directory in the HADOOP_PREFIX directory. You can refer to the managing service logs section in the previous chapter.

Application logs

The ApplicationMaster, as well as the containers running in the cluster, generate the application logs. Logging allows the debugging and analyzing of applications. By default, these logs are generated under the user_logs directory of logs in the Hadoop installation folder. You can configure the location of the directory by using the following property:

<property>
    <name>yarn.nodemanager...