Non-sticky request/response handlers

In addition to pub/sub, Diffusion has the ability to do request/response messaging. This is where a client sends in a message to a named endpoint (which looks a lot like a topic path, but is not part of the topic tree) and a previously registered handler processes the message and sends a response back to the client.

An example of this is where a client wants to place an order. It sends a request with the order as a payload to a well-known endpoint (e.g. “orders”) and then be asynchronously notified when it had been processed.

For high availability and scalability, more than one request handler can be registered to an endpoint. Diffusion round-robins requests between them, but treats them as sticky, so that the client is always routed to the same handler. The rationale behind this is that the handler might have multiple requests from the same client, and if it wishes, set up its own state, such as loading in a customer profile from a database.

However, there are a couple of cases where stickiness causes a problem.

Adding extra handlers to manage increased load

If you find that there are too many requests for your handler to process, and request processing is taking too long then you may choose to start up another handler – and this will work, but only half of the new clients will be redirected to the new handler. The existing sticky sessions will remain on the existing, already loaded handler. Due to the round-robin distribution, half of all new clients will be directed to the new handler and the other half will go to the original handler!

Handler disconnects & reconnects

Now consider that you already have sufficient handlers to manage the requests, but then one of those handlers goes down and is restarted. What happens to the connected clients? What you’ll see is the sessions that had been sticky to that handler will get redistributed when they make their next request, and the newly-restarted server will be underutilised, very similar to the previous diagram.

Turning off session stickiness

For a very large number of use cases, we don’t expect that the order handlers will actually be storing state; they’re more likely to be used in the same manner as REST microservices. That is, they do not retain context between calls.

To change the default behaviour, we have introduced a system property, diffusion.disable.sticky.messaging, which can be set to either true or false (if you don’t set the property, it is false by default).

By setting this to true, we can force Diffusion to iterate through handlers for every request.

Now, you can add as many order handlers as necessary, and they will all be used equally.