Diagnostic data can be used to identify problems with a hosted service. The ability to view the data from several sources and across different instances eases the task of identifying a problem.
Diagnostic data can be used to identify when service capacity is either too high or too low for the expected workload. This can guide capacity decisions such as whether to scale up or down the number of instances.
The configuration of Windows Azure Diagnostics is performed at the instance level. The code to do that configuration is at the role level, but the diagnostics configuration for each instance is stored in individual blobs in a container named wad-control-container located in the storage service account configured for Windows Azure Diagnostics.
Read more:
Windows Azure Diagnostics: Initializing the Configuration and Using a Configuration File
There is no need for application data and diagnostics data to be located in the same storage service account. Indeed, a best practice from both security and performance perspectives would be to host application data and diagnostic data in separate storage service accounts.
The configuration of Windows Azure Diagnostics is centered on the concept of data buffers with each data buffer representing a specific type of diagnostic information. Some of the data buffers have associated data sources which represent a further refining of the data captured and persisted. For example, the performance counter data buffer has individual data sources for each configured performance counter. Windows Azure Diagnostics supports record-based data buffers that are persisted to Windows Azure tables and file-based data buffers that are persisted to Windows Azure blobs. In the Accessing data persisted to Windows Azure Storage recipe we see that we can access the diagnostic data in the same way we access other data in Windows Azure storage.
Windows Azure Diagnostics supports the following record-based data buffers:
Windows Azure basic logs
Performance counters
Windows Event Logs
Windows Azure Diagnostic infrastructure logs
The Windows Azure basic logs data buffer captures information written to a Windows Azure trace listener. In the Using the Windows Azure Diagnostics trace listener recipe, we see how to configure and use the basic logs data buffer. The performance counters data buffer captures the data of any configured performance counters. The Windows Event Logs data buffer captures the events form any configured Windows Event Log. The Windows Azure Diagnostic infrastructure logs data buffer captures diagnostic data produced by the Windows Azure Diagnostics process.
Windows Azure Diagnostics supports the following file-based data sources for the Directories data buffer:
IIS logs
IIS Failed Request Logs
Crash dumps
Custom directories
The Directories data buffer copies new files in a specified directory to blobs in a specified container in the Windows Azure Blob Service. The data captured by IIS Logs, IIS Failed Request Logs, and crash dumps is self-evident. With the custom directories data source, Windows Azure Diagnostics supports the association of any directory on the instance with a specified container in Windows Azure storage. This allows for the coherent integration of third-party logs into Windows Azure Diagnostics. We see how to do this in the Implementing custom logging recipe.
The implementation of Windows Azure Diagnostics was changed in Windows Azure SDK v1.3 and it is now one of the pluggable modules that have to be explicitly imported into a role in the service definition file. As Windows Azure Diagnostics persists both its configuration and data to Windows Azure storage, it is necessary to specify a storage service account for diagnostics in the service configuration file.
The default configuration for Windows Azure Diagnostics captures some data but does not persist it. Consequently, the diagnostics configuration should be modified at role startup. In the Initializing the configuration of Windows Azure Diagnostics recipe, we see how to do this programmatically, which is the normal way to do it. In the Using a configuration file with Windows Azure Diagnostics recipe, we see how to use a configuration file to do this, which is necessary in a VM role.
In normal use, diagnostics data is captured all the time and is then persisted to the storage service according to some schedule. In the event of a problem, it may be necessary to persist diagnostics data before the next scheduled transfer time. We see how to do this in the Performing an on-demand transfer recipe.
Both Microsoft and Cerebrata have released PowerShell cmdlets that facilitate the remote administration of Windows Azure Diagnostics. We see how to do this in the Using the Windows Azure Platform PowerShell cmdlets to configure Windows Azure Diagnostics recipe.
There are times, especially early in the development process, when non-intrusive diagnostics monitoring is not sufficient. In the Using IntelliTrace to Diagnose Problems with a Hosted Service recipe, we see the benefits of intrusive monitoring of a Windows Azure role instance.
Using the Windows Azure Diagnostics trace listener
Windows Azure Diagnostics supports the use of Trace to log messages. The Windows Azure SDK provides the DiagnosticMonitorTraceListener trace listener to capture the messages. The Windows Azure Diagnostics basic logs data buffer is used to configure their persistence to the Windows Azure Table Service.
The trace listener must be added to the Listeners collection for the Windows Azure hosted service. This is typically done through configuration in the appropriate app.config or web.config file, but it can also be done in code. When it creates a worker or web role, the Windows Azure tooling for Visual Studio adds the DiagnosticMonitorTraceListener to the list of trace listeners specified in the Configuration section of the relevant configuration file.
Methods of the System.Diagnostics.Trace class can be used to write error, warning and informational messages. When persisting the messages to the storage service, the Diagnostics Agent can filter the messages if a LogLevel filter is configured for the BasicLogsBufferConfiguration.
The Compute Emulator in the development environment adds an additional trace listener, so that trace messages can be displayed in the Compute Emulator UI.
In this recipe, we will learn how to trace messages using the Windows Azure trace listener.
How to do it...
We are going to see how to use the trace listener provided in the Windows Azure SDK to trace messages and persist them to the storage service. We do this as follows:
Ensure that the DiagnosticMonitorTraceListener has been added to the appropriate configuration file: app.config for a worker role and web.config for a web role.
If necessary, add the following to the Configuration section of app.config or web.config file:
<system.diagnostics>
<trace>
<listeners>
<add type="Microsoft.WindowsAzure.Diagnostics.
DiagnosticMonitorTraceListener,
Microsoft.WindowsAzure.Diagnostics, Version=1.0.0.0,
Culture=neutral, PublicKeyToken=31bf3856ad364e35"
name="AzureDiagnostics">
<filter type="" />
</add>
</listeners>
</trace>
</system.diagnostics>
Use the following to write an informational message:
System.Diagnostics.Trace.TraceInformation("Information");
Use the following to write a warning message:
System.Diagnostics.Trace.Warning("Warning ");
Use the following to write an error message:
System.Diagnostics.Trace.TraceError("Error");
Ensure that the DiagnosticMonitorConfiguration.Logs property is configured with an appropriate ScheduledTransferPeriod and ScheduledTransferLogLevelFilter when DiagnosticMonitor.Start() is invoked.
How it works...
In steps 1 and 2, we ensure that the DiagnosticMonitorTraceListener is added to the collection of trace listeners for the web role or worker role.
In steps 3 through 5, we see how to write messages to the trace listener.
In step 6, we ensure that the Diagnostic Agent has been configured to persist the messages to the storage service. Note that they can also be persisted through an on-demand transfer. This configuration is described in the recipe Initializing the configuration of Windows Azure Diagnostics.
There's more...
The Windows Azure SDK v1.3 introduced full IIS in place of the hosted web core used previously for web roles. With full IIS, the web role entry point and IIS are hosted in separate processes. Consequently, the trace listener must be configured separately for each process. The configuration using web.config configures the trace listener for IIS, not the web role entry point. Note that Windows Azure Diagnostics needs to be configured only once in each role, even though the trace listener is configured separately in both the web role entry point and in IIS.
The web role entry point runs under a process named WaIISHost.exe. Consequently, one solution is to create a special configuration file for this process named WaIISHost.exe.config and add the trace listener configuration to it.
A more convenient solution is to add the DiagnosticMonitorTraceListener trace listener programmatically to the list of trace listeners for the web role entry point. The following demonstrates an overridden OnStart() method in a web role entry point modified to add the trace listener and write an informational message:
public override bool OnStart()
{
System.Diagnostics.Trace.Listeners.Add(new Microsoft.
WindowsAzure.Diagnostics.DiagnosticMonitorTraceListener());
System.Diagnostics.Trace.AutoFlush = true;
System.Diagnostics.Trace.TraceInformation("Information");
return base.OnStart();
}
The AutoFlush property is set to true to indicate that messages should be flushed through the trace listener as soon as they are written.
Performing an on-demand transfer
The Windows Azure Diagnostics configuration file specifies a schedule in which the various data buffers are persisted to the Windows Azure Storage Service. The on-demand transfer capability in Windows Azure Diagnostics allows a transfer to be requested outside this schedule. This is useful if a problem occurs with an instance and it becomes necessary to look at the captured logs before the next scheduled transfer.
An on-demand transfer is requested for a specific data buffer in a specific instance. This request is inserted into the diagnostics configuration for the instance stored in a blob in wad-control-container. This is an asynchronous operation whose completion is indicated by the insertion of a message in a specified notification queue. The on-demand transfer is configured using an OnDemandTransferOptions instance that specifies the DateTime range for the transfer, a LogLevelFilter that filters the data to be transferred, and the name of the notification queue. The RoleInstanceDiagnosticeManager.BeginOnDemandTransfer() method is used to request the on-demand transfer with the configured options for the specified data buffer.
Following the completion of an on-demand transfer, the request must be removed from the diagnostics configuration for the instance by using the RoleInstanceDiagnosticManager.EndOnDemandTransfer() method. The completion message in the notification queue should also be removed. The GetActiveTransfers() and CancelOnDemandTransfers() methods of the RoleInstanceDiagnosticManager class can be used to enumerate and cancel active on-demand transfers. Note that it is not possible to modify the diagnostics configuration for the instance if there is a current request for an on-demand transfer, even if the transfer has completed.
Note that requesting an on-demand transfer does not require a direct connection with the hosted service. The request merely modifies the diagnostic configuration for the instance. This change is then picked up when the Diagnostic Agent on the instance next polls the diagnostic configuration for the instance. The default value for this polling interval is 1 minute. This means that a request for an on-demand transfer needs to be authenticated only against the storage service account containing the diagnostic configuration for the hosted service.
In this recipe, we will learn how to request an on-demand transfer and clean up after it completes.
How to do it...
We are going to see how to request an on-demand transfer and clean up after it completes. We do this as follows:
Use Visual Studio to create a WPF project.
Add the following assembly references to the project:
Microsoft.WindowsAzure.Diagnostics.dll
Microsoft.WindowsAzure.ServiceRuntime.dll
Microsoft.WindowsAzure.StorageClient.dll
System.configuration.dll
Add a class named OnDemandTransferExample to the project.
Add the following using statements to the class:
using Microsoft.WindowsAzure;
using Microsoft.WindowsAzure.Diagnostics;
using Microsoft.WindowsAzure.Diagnostics.Management;
using Microsoft.WindowsAzure.ServiceRuntime;
using Microsoft.WindowsAzure.StorageClient;
using System.Configuration;
Add the following private member to the class:
String wadNotificationQueueName = "wad-transfer-queue";
Add the following method, requesting an on-demand transfer, to the class:
public void RequestOnDemandTransfer(
String deploymentId, String roleName, String roleInstanceId)
{
CloudStorageAccount cloudStorageAccount =
CloudStorageAccount.Parse(
ConfigurationManager.AppSettings[
"DiagnosticsConnectionString"]);
OnDemandTransferOptions onDemandTransferOptions =
new OnDemandTransferOptions()
{
From = DateTime.UtcNow.AddHours(-1),
To = DateTime.UtcNow,
LogLevelFilter =
Microsoft.WindowsAzure.Diagnostics.LogLevel.Verbose,
NotificationQueueName = wadNotificationQueueName
};
RoleInstanceDiagnosticManager ridm =
cloudStorageAccount.CreateRoleInstanceDiagnosticManager(
deploymentId, roleName, roleInstanceId);
IDictionary<DataBufferName, OnDemandTransferInfo>
activeTransfers = ridm.GetActiveTransfers();
if (activeTransfers.Count == 0)
{
Guid onDemandTransferId = ridm.BeginOnDemandTransfer(
DataBufferName.PerformanceCounters,
onDemandTransferOptions);
}
}
Add the following method, cleaning up after an on-demand transfer, to the class:
public void CleanupOnDemandTransfers()
{
CloudStorageAccount cloudStorageAccount =
CloudStorageAccount.Parse(
ConfigurationManager.AppSettings[
"DiagnosticsConnectionString"]);
CloudQueueClient cloudQueueClient =
cloudStorageAccount.CreateCloudQueueClient();
CloudQueue cloudQueue = cloudQueueClient.GetQueueReference(
wadNotificationQueueName);
CloudQueueMessage cloudQueueMessage;
while ((cloudQueueMessage = cloudQueue.GetMessage())
!= null)
{
OnDemandTransferInfo onDemandTransferInfo =
OnDemandTransferInfo.FromQueueMessage(
cloudQueueMessage);
String deploymentId = onDemandTransferInfo.DeploymentId;
String roleName = onDemandTransferInfo.RoleName;
String roleInstanceId =
onDemandTransferInfo.RoleInstanceId;
Guid requestId = onDemandTransferInfo.RequestId;
RoleInstanceDiagnosticManager ridm =
cloudStorageAccount.CreateRoleInstanceDiagnosticManager(
deploymentId, roleName, roleInstanceId);
Boolean result = ridm.EndOnDemandTransfer(requestId);
cloudQueue.DeleteMessage(cloudQueueMessage);
}
}
Add the following Grid declaration to the Window element of MainWindow.xaml:
<Grid>
<Label Content="DeploymentId:" Height="28"
HorizontalAlignment="Left"
VerticalAlignment="Top" Margin="30,60,0,0"
Name="label1" />
<Label Content="Role name:" Height="28"
HorizontalAlignment="Left" VerticalAlignment="Top"
Margin="30,110,0,0" Name="label2" />
<Label Content="Instance Id:" Height="28"
HorizontalAlignment="Left" VerticalAlignment="Top"
Margin="30,160,0,0" Name="label3" />
<TextBox HorizontalAlignment="Left" VerticalAlignment="Top"
Margin="120,60,0,0" Name="DeploymentId" Height="23"
Width="120" Text="24447326eed3475ca58d01c223efb778" />
<TextBox HorizontalAlignment="Left" VerticalAlignment="Top"
Margin="120,110,0,0" Width="120" Name="RoleName"
Text="WebRole1" />
<TextBox Height="23" HorizontalAlignment="Left"
VerticalAlignment="Top" Margin="120,160,0,0" Width="120"
Name="InstanceId" Text="WebRole1_IN_0" />
<Button Content="Request On-Demand Transfer" Height="23"
HorizontalAlignment="Left" VerticalAlignment="Top"
Margin="60,220,0,0" Width="175" Name="RequestTransfer"
Click="RequestTransfer_Click" />
<Button Content="Cleanup On-Demand Transfers" Height="23"
HorizontalAlignment="Left" VerticalAlignment="Top"
Margin="300,220,0,0" Width="175" Name="CleanupTransfers"
Click="CleanupTransfers_Click" />
</Grid>
Add the following event handler to MainWindow.xaml.cs:
private void RequestTransfer_Click(
object sender, RoutedEventArgs e)
{
String deploymentId = DeploymentId.Text;
String roleName = RoleName.Text;
String roleInstanceId = InstanceId.Text;
OnDemandTransferExample example =
new OnDemandTransferExample();
example.RequestOnDemandTransfer(
deploymentId, roleName, roleInstanceId);
}
Add the following event handler to MainWindow.xaml.cs:
private void CleanupTransfers_Click(
object sender, RoutedEventArgs e)
{
OnDemandTransferExample example =
new OnDemandTransferExample();
example.CleanupOnDemandTransfers();
}
Add the following to the configuration element of app.config:
<appSettings>
<add key="DiagnosticsConnectionString"
value="DefaultEndpointsProtocol=https;AccountName={
ACCOUNT_NAME};AccountKey={ACCESS_KEY}"/>
</appSettings>
How it works...
We create a WPF project in step 1 and add the required assembly references in step 2.
We set up the OnDemandTransferExample class in steps 3 and 4. We add a private member to hold the name of the Windows Azure Diagnostics notification queue in step 5.
In step 6, we add a method requesting an on-demand transfer. We create an OnDemandTransferOptions object configuring an on-demand transfer for data captured in the last hour. We provide the name of the notification queue Windows Azure Diagnostics inserts a message indicating the completion of the transfer. We use the deployment information captured in the UI to create a RoleInstanceDiagnosticManager instance. If there are no active on-demand transfers, then we request an on-demand transfer for the performance counters data buffer.
In step 7, we add a method cleaning up after an on-demand transfer. We create a CloudStorageAccount object that we use to create the CloudQueueClient object with which we access to the notification queue. We then retrieve the transfer-completion messages in the notification queue. For each transfer-completion message found, we create an OnDemandTransferInfo object describing the deploymentID, roleName, instanceId, and requestId of a completed on-demand transfer. We use the requestId to end the transfer and remove it from the diagnostics configuration for the instance allowing on-demand transfers to be requested. Finally, we remove the notification message from the notification queue.
In step 8, we add the UI used to capture the deployment ID, role name, and instance ID used to request the on-demand transfer. We can get this information from the Windows Azure Portal or the Compute Emulator UI. This information is not needed for cleaning up on-demand transfers, which uses the transfer-completion messages in the notification queue.
In steps 9 and 10, we add the event handlers for the Request On-Demand Transfer and Cleanup On-Demand Transfers buttons in the UI. These methods forward the requests to the methods we added in steps 6 and 7.
In step 11, we add the DiagnosticsConnectionString to the app.config file. This contains the connection string used to interact with the Windows Azure Diagnostics configuration. We must replace {ACCOUNT_NAME} and {ACCESS_KEY} with the storage service account name and access key for the storage account in which the Windows Azure Diagnostics configuration is located.
Read more
United States
Great Britain
India
Germany
France
Canada
Russia
Spain
Brazil
Australia
Singapore
Canary Islands
Hungary
Ukraine
Luxembourg
Estonia
Lithuania
South Korea
Turkey
Switzerland
Colombia
Taiwan
Chile
Norway
Ecuador
Indonesia
New Zealand
Cyprus
Denmark
Finland
Poland
Malta
Czechia
Austria
Sweden
Italy
Egypt
Belgium
Portugal
Slovenia
Ireland
Romania
Greece
Argentina
Netherlands
Bulgaria
Latvia
South Africa
Malaysia
Japan
Slovakia
Philippines
Mexico
Thailand
