Scheduling WebJobs, Batch, and Worker Roles?

All three choices perform background processing and can be scheduled, but the scalability, features and built-in infrastructure vary. Before scheduling one of these background processing mechanisms, let’s compare the options.

Worker Roles

Worker Roles were the first out of the gate from Microsoft, but require a certain amount of DIY plumbing — reading a message queue, checking for undeliverable “poison” messages, removing processed messages and so on. Need to work with Azure blob storage? Write some more code first. The code snippet below is from the Get Started with Azure Cloud Services and ASP.NET sample downloadable here. You might use a Worker Role if you needed closer control over the environment, for example, to run code with elevated privileges.

Note: I’m cherry-picking this demo code to exaggerate the point that Worker Roles require building your own infrastructure.

WebJobs

WebJobs are a bit like a command-line for the cloud. Setup is straight-forward and commonly used features are built-in. For example, the Visual Studio Azure WebJob template generates a sample function ProcessQueueMessage (see below) that is triggered by new entries in the queue. This declarative style keeps your code focused on the job, not the infrastructure. WebJobs can be on-demand, triggered from blob or queue input and, of course, scheduled.

Azure Batch

Azure Batch is designed for large-scale, resource and CPU intensive jobs that can run in parallel (see Running Resource Intensive Jobs Using Azure Batch). Azure Batch has its own queueing, dispatching, monitoring and scheduling facilities. While Azure Batch does abstract infrastructure from the actual work, it can require more code than a WebJob to setup.

Azure Scheduler

Azure Scheduler runs jobs at a specific time and can kick off HTTP/HTTPS endpoints, queue messages or service bus options. You can create scheduled jobs in the Azure Portal or using Visual Studio.

To keep us oriented, we can picture the Azure Scheduler organization like this:

  • Scheduler Job Collection
  •     Scheduled Job
  •         Action Settings: Your choice of Http/Https/queue/service bus. Depending on your selection, additional settings display.
  •         Schedule: The frequency and scheduling constraints of the job.

In the portal, Scheduler Job Collection defines the resource group and pricing tier. This example is patterned off the configuration for a WebJob published from Visual Studio.

The job Action settings define what to run while the Schedule sets the frequency.

In this example, I want the scheduler to call a WebJob HTTPS endpoint. Because the WebJob methods have parameters, I’ll choose Post for the Method setting. The Url for a WebJob follows this pattern:

https://<service name>.scm.azurewebsites.net/api/triggeredjobs/<webjob name>/run

Note: If you select a new Action from the drop-down list, watch out for the authentication settings. The Authentication settings appear to stay the same, but the password is erased. A good thing, but there’s no visual indicator.

The Schedule allows the job to run once at the Start date and time or recur every month/week/day/hour/minute. The End settings help keep a lid on expenses by ending the job on a given date and time or after a certain number of occurrences.

Once the action settings and schedule are defined, you can complete creating the scheduler job. The Portal’s Scheduler Job “blade” summarizes job state and metrics. The toolbar lets you temporarily disable the job and the Run now button is helpful during initial testing.

Scheduling WebJobs from Visual Studio

If you’re creating WebJobs, Visual Studio does most of this work for you during publishing. The Visual Studio project’s Publish as Azure WebJob… context menu shows the Add Azure WebJob dialog to help you through the process. In this scenario I’ll choose Run on a Schedule from the WebJob run mode drop down.

The job schedule can run once or recur. I’ll make this a recurring job that runs every 15 minutes.

Note: This first step creates a file called webjob-publish-settings.json in the Properties folder. If you ever want to reset the scheduling, delete the file and republish. The json file is pretty easy to edit, so you could tweak that by hand instead.

In the Profile area, you choose the Microsoft Azure App Service, either by creating a new service or choosing from a list of services you’ve already created.

The Connection tab shows up next to summarize key information about the job. When I hit the Publish button, all this info is saved to the project’s Properties/PublishProfiles directory.

Note: The Settings tab simply lets you choose between Release and Debug configurations.

Azure Batch Scheduling

Azure Batch has its own API to handle scheduling. The custom SubmitJobAsync() method from the Running Resource Intensive Jobs Using Azure Batch blog can be modified to create a scheduled job instead. That example used the batch client JobOperations object to create a job. Now we will use the batch client JobScheduleOperations to create a CloudJobSchedule object. Here are the objects I’ll need to feed BatchClient.CreateJobSchedule():

  • PoolInformation describes the makeup and behavior of compute resources that will run the job.
  • Schedule defines the frequency and scheduling constraints of the job run.
  • JobManagerTask describes the task to be executed, including the command line and display name.
  • JobSpecification defines the overall environment that the job runs in. The example below only assigns the PoolInformation and JobManagerTask. You can go further by assigning environment settings for all tasks in jobs created by the schedule, tasks to run on a compute node before and after the main task execution, constraints (maximum retries, maximum run time), metadata (a list of name-value pairs), display name and priority.

The example creates these objects and passes them as parameters to CreateJobSchedule(). The call to CommitAsync() sends the information to the Azure Batch service.

Conclusion

The Azure Scheduler can execute recurring actions against HTTP/HTTPS endpoints, storage queues and service bus options. The HTTP/HTTPS actions work nicely with WebJob URLs, and Visual Studio’s Publish as Azure WebJob helps with the configuration details. Azure batch has its own scheduling mechanism that you can configure in code.

The following two tabs change content below.