Chapter 8. Building Worker Roles

In this chapter, we're going to start looking at Microsoft Azure Cloud services and building our production order processor worker role. By the end, we'll have a scalable worker role that consumes order messages from the Service Bus topic subscription we created previously. Orders will be stored in the production database, and periodic tasks will create batch schedules to manufacture the products and allocate the stock.

We'll cover the following topics:

• Introduction to cloud services

• Creating a worker role

• Running a worker role locally

• Publishing a worker role

• Building the production order processor

• Creating a scheduled work activity

• Testing the production order processor

• Deleting idle cloud services

Along with looking at worker roles, we'll be revisiting Service Bus topics and building databases. We'll also be introducing scheduled tasks and storage queues, which we've not looked at yet.

Introducing cloud services

Cloud services are PaaS Azure services that allow us to create highly available applications (99.95 percent monthly SLA) and deploy them to dedicated virtual machines of various image sizes, which can be scaled out to meet the demands of the system.

There are two types of cloud services, worker roles and web roles. Worker roles are similar to Windows Services; they are unattended applications with no user interface which that long-running tasks. Web roles are websites deployed in IIS and run on dedicated cloud service VMs; they are very similar to normal Azure websites, with the additional configuration and diagnostic capabilities of a cloud service.

Cloud services come with production and staging deployment slots that run on dedicated VM instances so that staged deployments don't impact the performance of the production deployment (unlike staging slots in websites). Cloud services can be debugged remotely (with IntelliTrace), run start-up tasks, and allow remote...

Exploring worker roles

To begin with, we're going to create a boiler-plate worker role in Visual Studio, run it locally, then publish it, and run it on Azure. It's possible to create a cloud service in the portal, then publish a project to it or upload a package.

It's also possible to automatically provision a cloud service during the first publish from Visual Studio, which is the approach we'll take.

Creating a worker role

We'll create the role using the following procedure:

1. I've created a blank solution called AzureBakery.Production, which is the start of our production business unit solution for the worker role, production data model, WPF management application, and Web API service.

2. To create a worker role, right-click on the solution root in Visual Studio and go to Add | New Project, then select Windows Azure Cloud Service under the Cloud templates, choose Name, and click on OK:

   

3. Select Worker Role (notice that there is a template for a Worker Role with Service Bus Queue option, which is a...

Running locally

We know that we can easily run Azure websites locally while we're developing them because they're normal websites and will run in IIS and IIS Express without any difficulty. Cloud services aren't normal executables, which can run on Windows, but they can be run locally on the Windows Azure compute emulator and storage emulator. We have a full and express emulator available to us in the SDK; we'll start with the default express version and look at the full version when we need it later.

To get started, put some break points in the application in the OnStart and Run methods, one at the start and one inside the while loop. Run the debugger in Visual Studio and we'll see a progress dialog that notifies us of the compute emulator, storage emulator, and worker role initialization progress:

Once this is completed, the worker role should break on the OnStart method, and we notice that we have a new Windows icon in the system tray; if we right-click on it, we have a number of options...

Publishing a worker role

To test the worker role that just runs the boiler-plate code on Azure, we'll publish it using the following procedure:

1. Right-click on the cloud service in the Solution Explorer window and select Publish... from the context-sensitive menu:

   

2. Choose Subscription and click on Next; if you've not got any worker roles configured in the portal, you'll get a dialog that prompts you to create one with a storage account.

3. Choose Name and select Region or Affinity Group. The Enable Geo-Replication option is for the storage account. Click on Create to complete.

4. Choose a Cloud Service option to publish to Azure, select the Environment option (I chose Staging, so we can test the service before swopping to production), and leave Build configuration as Release and Service configuration as Cloud:

   

5. Check Enable Remote Desktop for all roles (this will allows us to access the virtual machine via remote desktop and take a look around later). Enter your login credentials (the Account expiration...

Building the production order processor

We're going to start working on the production business domain order processor worker role, which first subscribes to the OrderStatus.New status order messages on the topic, then adds the orders to its own database, creates batch schedules, and allocates stock for the products requested in the orders. The worker role will be structured with separate tasks and business logic to perform the individual messaging and business domain activities.

Adding an entity model

We've already built a data model in a fair amount of detail, so we'll go through the process pretty quickly for the production order processor. The entity model for the production system looks like this:

Get the AzureBakery.Production.Model project, which contains all the entities from code, and add it to the solution, and then perform the following procedure to implement it in the worker role, which will manage the database migrations:

1. Add a Model project reference to the OrderProcessorRole project...

Creating a scheduled work activity

We've got most of the production worker role completed with orders safely being inserted into the production database, but there's a bit missing, and we have a slight problem with it.

The production order processor must create product batch schedules and allocate stock for the production management application. We don't want to perform this operation every time an order message comes in, as it will put too much load on the system. Also, when we have multiple roles running as we scale out, we don't want to get database contention from multiple instances trying to work on the same records in the database. Pessimistic and optimistic concurrency will not even help us here as we need to create single new batches when existing batches are full as well as adding to existing incomplete batches.

To solve this problem, we need a way of only one role performing these activities at any one time and a way of requesting the scheduled activities. Azure gives us a number...

Completing the worker role

We now have all the tasks we need, so we can add them to the role:

using Microsoft.WindowsAzure.ServiceRuntime;
using OrderProcessorRole.Messaging;
using System;
using System.Diagnostics;
using System.Threading.Tasks;

namespace OrderProcessorRole
{
    public class WorkerRole : RoleEntryPoint
    {
        private OrderTopicProcessor _orderTopicProcessor = null;
        private BatchQueueProcessor _batchQueueProcessor = null;
        private StockQueueProcessor _stockQueueProcessor = null;
               
        public override void Run()
        {
            Trace.TraceInformation("OrderProcessorRole.WorkerRole.Run - Begin");

            try
            {
                // WaitAll tasks will block Run until they have all completed, otherwise the role will recycle itself
                Task.WaitAll(
                    Task.Run(() => this._orderTopicProcessor.ProcessSubscriptionAsync()),
                    Task.Run(() => this._batchQueueProcessor.ProcessQueueAsync...

Testing the production order processor

Now that we've completed our worker role, we need to test whether it is behaving as expected, as a single instance to start with, and then multiple instances once we're happy with its behavior.

If the worker role cannot run in a scaled out configuration with multiple instances, it won't be able to cope with a large volume of work and will not run in a resilient configuration.

Testing a single instance

We can start testing locally, and once we're happy, publish and test on Azure. Running as a single instance, we need to check the following:

• The worker role starts and runs

• It doesn't recycle on its own

• There are no exceptions being logged

• Orders and order items are being inserted by the order processor tasks

• Batches are created and assigned to order items

• Order items have stock allocated

• Batches are not fragmented (there must never be more than one incomplete batch per product)

The following T-SQL script can be used to help you do the fragmentation checks on our...

Deleting idle cloud services

When we're developing systems, we may have a number of cloud services that we've finished working on and may be sitting idle for a period of time, so it's a good idea to delete them, as we're not paying for them to just sit there doing nothing.

We can delete a cloud service from the toolbar of a cloud service's workspace; however, we need to be careful to delete instances and not the full service, as we would lose our entire cloud service including the DNS name, which would require a completely new publishing profile and may cause configuration issues with other systems, relying on the allocated DNS name if someone else takes it, and we cannot use it again:

Summary

In this chapter, we've gone from examining and running a boiler-plate cloud service to building our production order processor worker role, which runs multiple concurrent tasks to process orders, create batches and assign order items to them, and allocate stock for new items.

There are two other worker roles for the sales and dispatch business domains in the system, which we're not going to look at in this book as they're less complex and cover fewer features than the production worker role, but you can get them from the code files of this chapter.

In the next chapter, we're going to take a deeper dive into cloud service debugging and diagnostics to help us develop and maintain our worker and web roles.

Questions

1. What are the two types of cloud services?

2. What is the difference between a website and a web role?

3. What is the base class of a worker role?

4. Name the three worker role life cycle methods.

5. What happens if the Run method is not blocked?

6. How can we find the status of the storage emulator?

7. What method must we call on a BrokeredMessage object to delete it from a subscription when we use PeekLock ReceiveMode?

8. When we use async tasks in a worker role, why do we need to implement Task.Wait or Task.WaitAll in the Run method?

9. Which NuGet package do we need to interact with a storage queue and what command do we use to install it?

10. How do we remove a storage queue message when we're finished with it?

11. What extra steps do we need to take to run multiple role instances locally?

12. When is it a good idea to delete a cloud service and why must we only delete the instances?

Answers

1. Web role and worker role.

2. Web roles run on dedicated virtual machines, which can be debugged remotely using IntelliTrace and allow advanced configuration and debugging with remote desktop and start-up tasks available on the virtual machines.

3. RoleEntryPoint.

4. OnStart, Run, and OnStop.

5. The role will recycle.

6. Use the WAStorageEmulator.exe status command.

7. Complete or CompleteAsync.

8. As the tasks are asynchronous, they are queued onto the thread pool and not called immediately, so the calling method continues and is not blocked. We need to use Task.Wait or Task.WaitAll to block the thread while the tasks(s) complete; otherwise, the role will recycle.

9. WindowsAzure.Storage and Install-package WindowsAzure.Storage.

10. Call the CloudQueue.Delete method.

11. Set the role instances in the role config and switch to using the full emulator in the project config.

12. If we're not currently using a role instance during development, it's a good idea to delete it during development as we will be incurring charges for...