One week ago we released the first beta of our Fremantle release, which includes Ehcache 2.4, Terracotta Enterprise Suite 3.5 and at last but not least Quartz 2.0. You have been able to cluster your Quartz Scheduler instances using Terracotta for a while already. Yet, as with a JDBC backed storage, you had no control over what node your job would be executed on. The only guarantee was that your job would be executed once within the cluster. Quartz Where aims at addressing exactly that, and is one of the many new features that are part of this new major release of our product line.
A popular demand from clustered Quartz Scheduler users was to be able to specify where a Job would be executed: because data for the job is known to be present on some machine (like using NFS-like file sharing) or because the Job requires much processing and memory. Controlling the locality of execution is now feasible. We have tried to make this a seamless addition to Quartz: you can configure jobs to be dispatched to node groups using a simple configuration file; or programmatically schedule
LocalityTrigger instances. Let’s first cover the configuration based approach, which doesn’t require any code changes to an existing Quartz 2.0 application.
Configuration based locality of execution
Before getting started with this new feature, you will have to configure your Quartz scheduler to use the Terracotta Enterprise Store by setting the property
org.terracotta.quartz.EnterpriseTerracottaJobStore. If you were not using Terracotta to cluster Quartz, you will also have to set the
org.quartz.jobStore.tcConfigUrl property to point to the Terracotta server. Here is a small example of a quartz.properties
org.quartz.scheduler.instanceName = QuartzWhereScheduler org.quartz.scheduler.instanceId = AUTO org.quartz.scheduler.instanceIdGenerator.class = org.terracotta.quartz.demo.locality.SystemPropertyIdGenerator org.quartz.threadPool.class = org.quartz.simpl.SimpleThreadPool org.quartz.threadPool.threadCount = 10 org.quartz.threadPool.threadPriority = 5 org.quartz.threadPool.threadsInheritContextClassLoaderOfInitializingThread = true org.quartz.jobStore.class = org.terracotta.quartz.EnterpriseTerracottaJobStore org.quartz.jobStore.tcConfigUrl = localhost:9510
quartzLocality.properties configuration file, you can define node groups. A node group is composed of one or more Quartz Scheduler instance nodes (generally one per machine within your cluster). You define them as such:
org.quartz.locality.nodeGroup.slowNodes = tortoise, snail org.quartz.locality.nodeGroup.fastNodes = hare, leopard org.quartz.locality.nodeGroup.linuxNodes = tortoise
We have now defined three groups: the slowNodes, fastNodes and linuxNodes. We can now use the node groups to have jobs or triggers being executed by them, depending on their
group. Quartz Jobs and Triggers were and still are uniquely identified by a name and group pair. We can now have all jobs (or triggers) of a certain group only get executed on a node of a given group through the same configuration file:
org.quartz.locality.nodeGroup.fastNodes.triggerGroups = bigJobGroup org.quartz.locality.nodeGroup.linuxNodes.triggerGroups = reporting
Now all triggers from the group
bigJobGroup will be executed by a Scheduler from the group
fastNodes, either the
leopard scheduler. These scheduler nodes receive unique ids as before by providing an
org.quartz.spi.InstanceIdGenerator implementation to the scheduler at configuration time (don’t mix this with the instanceName, which needs to be the same for all nodes from them to be a single clustered Scheduler). Triggers from the group
reporting will always be executed on tortoise, as this is the only scheduler in the
Programmatic locality of execution
Using the new locality API for Quartz that is part of our Terracotta Enterprise Suite 3.5 you can achieve even finer grained control and express more complex constraints about where a job should be executed. The example below uses the new DSL like builder API introduced with Quartz 2.0. Let’s see how that looks:
LocalityJobDetail jobDetail = localJob( newJob(ImportantJob.class) .withIdentity("importantJob") .build()) .where( node() .is(partOfNodeGroup("fastNodes"))) .build();
On line 3, we create a new
JobDetail for the
ImportantJob. We then wrap it on line 2 as a localJob that needs to be executed on a node that is part of the group
fastNodes. You might have noticed that creating the
JobDetail is pretty straight forward with the new API. Adding the locality information isn’t much more work neither. You can be much more precise on where the job should be executed though. Let’s have a look at this example:
scheduler.scheduleJob( localTrigger( newTrigger() .forJob("importantJob")) .where(node() .has(atLeastAvailable(512, MemoryConstraint.Unit.MB) .is(OsConstraint.LINUX))) .build());
Here we schedule an immediate trigger for the
importantJob we’ve registered in the previous example. Line 2 is creating the locality aware trigger, defining it to require a node that is running Linux and has at least 512 MB of heap available. Using these constraints, here memory and OS, you can be much more explicit about what the characteristics of the node executing the Job should be.
The new Terracotta clustered JobStore we’ve introduced will evaluate the constraint expressed on a Trigger and/or a Job to decide where to dispatch the Job for execution. We plan on providing implementation for expressing constraints on the CPU, Memory and Operating System characteristics of the node. I am still heavily working on these, but what is being shipped as part of this first beta should give you a good feel for where we are headed. To test it yourself today, go fetch the Fremantle beta 1 from the Terracotta website now!