Time Series Topic Enhancements in Diffusion 6.6

For Diffusion 6.6, we are evolving the time series topic APIs to deliver a streamlined development experience. This post will summarize some of the improvements coming to 6.6.

All these improvements are also available in Diffusion 6.6 Preview 1, so you can try them right now!

Time series topics: a refresher

Time series topics are a Diffusion topic type which store a record of previous values as an ordered series of events.

This makes them ideal for applications like storing a transaction history, historical price data or a series of chat messages.

A Diffusion client can automatically receive a sequence of past events when it subscribes to a time series topic. A client can also request a range of events, and with the right security permissions even make non-destructive edits to past events.

The metadata provided with a time series value are the author, a timestamp, and a sequence number. These allow the time series to be queried based on timestamp or ordering of stored events.

In previous releases, when using a time series topic, you had to perform some actions using a different API feature than for other topic types. Some functionality was not available at all for a time series topic.

In 6.6 Preview 1, we’ve made time series topics work more like other topics, simplifying development and making them even more useful.

Enhancements to updating time series topics

In previous releases, you had to interact with a time series topic using the TimeSeries feature which provides append, edit, and range query operations. This is distinct from the TopicUpdate feature provided to interact with other topic types, such as string or JSON.

We have extended TopicUpdate to be compatible with time series topics. This allows a time series topic to be used as though it were a standard topic that had the same same event type as the time series.

This API provides a number of distinct advantages:

1. 1. It supports update constraints which allow an update to be applied only if the condition is met.
   2. It provides an addAndSet method which allows a topic to be created with an initial value.
   3. Update streams provide the capability to send a series of events to a topic. These streams support data optimization provided by delta updates which provide significant bandwidth savings.

These enhancements mean you can use features that were previously unavailable to time series topics. We’ve linked to the Java SDK docs above, but all the SDKs support these improvements (apart from the beta Python SDK).

Custom timestamps

You can now set the timestamp when you append an event instead of using the one generated by the server, provided it is later than any existing timestamp for other events in the topic.

This makes time series topics a lot easier to work with. For example, you can preload historical data into a new time series topic, or test your application with captured data.

Enhancements to interacting with time series topics as a subscriber

Subscription to time series topics works in the same way as all other topics. You use a topic selector to define the topics you want to receive.

In order to process events, you need to configure a stream for the topic. In previous releases, time series event streams had to be registered with a separate mechanism than you would use for other topic types, in order to receive event metadata.

In some cases, you won’t be interested in the event metadata. So we now allow a ValueStream with a compatible topic type to receive values from time series topics. This allows you to register a single value stream for all topics, regardless of whether or not they are a time series.

In 6.6 Preview 1, this improvement is only available in the Java SDK. Support for other SDKs will be added in future releases.

Enhancements to the management console

The Diffusion management console is always under development. Diffusion 6.6 enhances the presentation of time series topics in the management console. If they use a numerical value, you will now see a graph like this:

These are only the first of our planned enhancements to time series topics. We believe they reinforce a powerful Diffusion feature, while giving you the flexibility as a developer to deliver your vision.

Remember, you can try out all these improvements now in Diffusion 6.6 Preview 1 or Diffusion Cloud. We would love to hear your feedback, and any feature requests you have for the future of time series topics.