More Cluster-Aware Features In Diffusion 6.4 and 6.5

When your application delivers real-time data, high availability is important. In production, you will probably want to take advantage of Diffusion™’s clustering capabilities. By clustering multiple Diffusion servers, you can add resilience to your application and keep your data flowing even through a hardware failure.

What is cluster awareness?

In earlier versions of Diffusion, not every Diffusion feature was cluster-aware. In other words, a client attached to a server could only perform operations on that individual server, rather than the whole cluster.

For example, a client can use the client control feature to close a session on the server it is connected to, but in Diffusion 6.4 and earlier, the operation can’t affect sessions on another server in the cluster.

Many Diffusion features are already cluster-aware. An example is topic replication. With topic replication enabled, a topic update is applied to all servers in the cluster, so that the topic tree is mirrored in each server. This allows a load balancer to direct a client to any server in the cluster and deliver the same stream of topic updates.

Expanded cluster awareness in Diffusion 6.4

In newer versions of Diffusion, we’ve expanded the number of features which are cluster-aware.

In Diffusion 6.4, we added cluster awareness to control authentication handlers (used to authenticate client connections or restricted actions) and missing topic notifications.

What does cluster awareness mean for these features? If an authentication handler or missing topic notification handler is not registered on the server that receives the request, it will be forwarded to another server in the same cluster (if it has one registered). So if you have a three-server cluster, instead of needing three handlers each registered on a different server, you only need one for the whole cluster.

6.5 brings cluster awareness to even more features

In 6.5 we will also be making request-response messaging and many client control operations cluster-aware.

Sending a request to a request-response handler benefits from cluster awareness in the same way as authentication and missing topic notification.

For client control operations, if a client specifies a session ID, the target session need not be connected to the same server. The command will be routed to the server in the cluster that hosts that specified session. Also, when specifying a session filter, the command will be applied to all matching sessions across the cluster, rather than only to those which the sending session is connected.

The client control operations that now work across a cluster are:

• changeRoles
• close
• setConflated
• setSessionProperties
• getSessionProperties

If a session identified by a session ID is not found on the server that the calling session is connected to, it will be routed to a server in the cluster that the target session is connected to.

In addition, version 6.5 has added the ability to use close and setConflated with a session filter instead of a session ID, enabling batch operations. A command that identifies a set of sessions using a session filter will be executed on all servers.

When to use cluster awareness

Note that the performance of local operations will always be optimal because they do not require inter-server communication. However, taking advantage of cluster awareness provides three distinct advantages:

1. Handlers which are called infrequently can be available on only a subset of your servers.
2. In the event that a local handler or client is unavailable, cluster awareness
provides further resilience.
3. Servers can be dynamically added to horizontally scale a cluster without needing to create and connect extra clients.

In summary, increased cluster awareness gives you more options to architect your Diffusion-powered application, while making it more reliable and easier to manage.