The Apache Sling Discovery API quietly made its way into Adobe Experience Manager 5.6.1. In conjunction with the initial release of the Discovery API, Apache Sling event handling got a complete makeover. That is because eventing and the Discovery API work together to bring Apache Sling and AEM a great feature. A job is a special type of event that is processed only once. And jobs can be queued to guaranteed their execution at a later time if they cannot be processed immediately. The Discovery API allows jobs to be distributed among a group of servers. Each job is run once, and only once, by a single server.
Within AEM, workflows use jobs for execution. The Discovery API allows workflow execution to be distributed among the group of servers. The workflows can be distributed evenly to equalize the burden on servers, or processor intensive workflows can all be concentrated on a single server.
The term, cluster, is used quite a bit in the documentation for the Discovery API. But the Discovery API is not clustering in the normal sense of the word. Within AEM, the Discovery API is used mostly with servers in a cluster, it does not have to be. The Discovery API is a way to distribute the job events, it does not synchronize the data in servers with one another. Running Discovery within an environment that already synchronizes makes the burden of data transport easier. A cluster is that sort of environment.
By default, the Discovery API job sharing between servers is disabled within AEM. This can be frustrating if you are not aware of that and you try to leverage the Discovery API. Jayan Kandathil recently wrote a blog post, describing how to enable it.