Security Changes in Diffusion 6.5
July 10, 2020 | Philip Aston
Controlling access to data
Diffusion™ excels at delivering real-time data, but it’s important to have control over exactly where your data gets delivered.
In a typical deployment, your Diffusion server accepts connections over the Internet from multiple remote users and applications. For most applications, you trust some users more than others, so you need to place restrictions on the data each user can access or modify.
Diffusion provides exceptional control over access to the data stored in a server’s topic tree. Each user can be granted separate rights to read, write, or modify individual topics. Data can be divided across many topics, giving you precise, granular access control. If necessary, a server can host millions of topics. Topic views can be used to subdivide and rearrange data into alternative topic structures, which can have separate access control rules.
In response to the needs of our enterprise customers, we’ve made a number of improvements to Diffusion 6.5 to make the permissions system more scalable. In 6.5, you can have millions of permission rules, and apply them dynamically to hundreds of thousands of connected sessions.
To achieve this:
- we simplified the permission configuration language to make it straightforward to combine different permission rules
- we implemented continuous evaluation of subscriptions, so permission changes affecting read access are applied instantly
- we have made extensive code optimizations to improve performance
Roles and permissions in Diffusion security
This section is a quick overview of Diffusion’s security model. If you’re already familiar with how this works, you can skip to the next section.
Diffusion security uses role-based authorization. Each client session has a set of roles through which it gains rights to read, write, or modify data, and to perform other server functions.
The set of roles assigned to a session is determined when a session is created. A client application provides a principal (a name that uniquely identifies a user or a system component) and credentials when it first connects to the server. If authentication is successful, the server creates a session for the client, and grants the session appropriate roles based on the principal.
A session’s roles can be changed if it re-authenticates with the server, or if a control session changes the session’s roles.
Each role is configured to have a set of permissions. The association between roles and permissions is stored by the server, and can be updated by a suitably privileged user using the administration console or the client API.
As an administrative convenience, roles can include other roles. For example, in the default configuration used when Diffusion is first installed, the OPERATOR role has permissions that grant access to view the server configuration and hosted sessions. The ADMINISTRATOR role has permissions that grant access to control the server and modify the security configuration. The ADMINISTRATOR role includes the OPERATOR role, so sessions that are assigned only the ADMINISTRATOR role are granted the permissions of both roles.
Path permissions secure access to resources that have paths and belong to a hierarchy, such as topics in the topic tree. Path permissions allow separate permission assignments for different branches of the hierarchy. There are three classes of resources with paths: topics; messaging paths; and session locks. Topics are the most common, and we’ll use them in the following examples.
A session can update a topic if the most specific path permission it has that matches the topic’s path grants it the UPDATE_TOPIC permission. Similarly, a session can read the data from the topic if the most specific path permission grants it READ_TOPIC permission.
The following are example path permission rules, written using the Diffusion security language.
set "STOCK_CONTROL_NW" path "stock" permissions [READ_TOPIC] set "STOCK_CONTROL_NW" path "stock/regions/northwest" permissions [READ_TOPIC UPDATE_TOPIC]
In this example, there are two path permission rules. both for the role STOCK_CONTROL_NW. For a topic with a given path, the most specific rule is chosen; that is, the rule for the longest prefix of the path. A session assigned the STOCK_CONTROL_NW role can read any topic with a path that starts stock, and read and update any topic with a path that starts stock/regions/northwest. A single rule applies for a given topic and role, so the READ_TOPIC permission has to be repeated in the stock/regions/northwest rule.
Diffusion 6.5: simplification of path permissions
In Diffusion 6.5, a significant change was made to the way path permissions for different roles are evaluated. In previous releases, path permission rules from different roles were merged before evaluation. Consequently, creating a permission rule for a particular path would mask rules for parent paths from other roles. From Diffusion 6.5, path permissions are evaluated independently for each role, making it much easier to design role rule sets that can be composed together usefully. Now there is a consistently applied rule:
A session has a permission if one its assigned roles, or a role included by an assigned role, has the permission..
To illustrate this rule, here’s an example with two roles.
set "READ_STOCK" path "stock" permissions [READ_TOPIC] set "STOCK_CONTROL_NW" path "stock/regions/northwest" permissions [UPDATE_TOPIC] set "STOCK_CONTROL_NW" includes ["READ_STOCK"]
Since the STOCK_CONTROL_NW role includes the READ_STOCK role, a session assigned the STOCK_CONTROL_NW role also has the READ_STOCK role. In earlier versions of Diffusion, such a session would not have READ_TOPIC permissions for topics with paths that start stock/regions/northwest – to fix this, it is necessary to also give the STOCK_CONTROL_NW role READ_TOPIC permission for that path. In Diffusion 6.5, the session inherits READ_TOPIC permission for all topics with paths that start stock, regardless of the permissions rules for other roles.
Diffusion 6.5: the isolate path statement
Sometimes you might want to administer a branch of the topic tree independently. For example, different branches hold topics for separate applications, and you wish to avoid inheriting permissions from parent paths. The sole advantage of the previous scheme was that it made this easy to do: creating a path permission rule for a role prevented all inheritance from any rules for parent paths.
To support this need, Diffusion 6.5 provides a new security language statement, isolate path. In the following example, the path stock/administration is isolated. Sessions assigned the READ_STOCK role can read topics with paths beginning stock except for those beginning stock/administration which require the session to be assigned the STOCK_ADMINISTRATOR role.
set "READ_STOCK" path "stock" permissions [READ_TOPIC] isolate path "stock/administration" set "STOCK_ADMINISTRATOR" path "stock/administration" [READ_TOPIC UPDATE_TOPIC]
There is a simple way to rewrite Diffusion 6.4 security configuration to preserve behavior when upgrading to Diffusion 6.5. Simply add an isolate path statement for every path permission. Diffusion will perform this transformation automatically if you start a Diffusion 6.5 server with a Diffusion 6.4 security store, either by installing Diffusion 6.5 over the top of Diffusion 6.4 or by copying the file to persistence/Security.store. For example, the following subset of a Diffusion 6.4 security configuration:
set "CLIENT" default path permissions [ SELECT_TOPIC READ_TOPIC SEND_TO_MESSAGE_HANDLER ] set "CONTROL" default path permissions [ UPDATE_TOPIC MODIFY_TOPIC SEND_TO_SESSION EDIT_TIME_SERIES_EVENTS ACQUIRE_LOCK ] set "STOCK_CONTROL_NW" path "stock" permissions [ READ_TOPIC ] set "STOCK_CONTROL_NW" path "stock/regions/northwest" permissions [ READ_TOPIC UPDATE_TOPIC ] set "CONTROL" includes [ "CLIENT" ]
will be rewritten as follows if it is used with a Diffusion 6.5 server:
language version 2 set "CLIENT" default path permissions [ SELECT_TOPIC READ_TOPIC SEND_TO_MESSAGE_HANDLER ] set "CONTROL" default path permissions [ UPDATE_TOPIC MODIFY_TOPIC SEND_TO_SESSION EDIT_TIME_SERIES_EVENTS ACQUIRE_LOCK ] set "STOCK_CONTROL_NW" path "stock" permissions [ READ_TOPIC ] set "STOCK_CONTROL_NW" path "stock/regions/northwest" permissions [ READ_TOPIC UPDATE_TOPIC ] set "CONTROL" includes [ "CLIENT" ] isolate path "stock" isolate path "stock/regions/northwest"
The rewrite has made two changes. The language version 2 statement marks the new configuration as having been transformed for Diffusion 6.5. The isolate path statements preserve the previous behavior of the Diffusion 6.4 configuration so, for example, a session assigned the CLIENT role but not the STOCK_CONTROL_NW role will not have READ_TOPIC permission for topics with paths starting stock. An INFO message will also be logged: Upgraded security store from language version 1 to version 2.
Applying the READ_TOPIC permission to subscriptions
In Diffusion, sessions do not subscribe directly to topics. Instead, sessions request subscriptions to topic paths using topic selectors. The server persists the set of topic selectors for each session, and dynamically joins selectors against its topics to resolve subscriptions. The dynamic join between topic selectors and topics is unique to Diffusion and is a powerful way to link client applications with a changing data model. The set of topic selectors defines the view of the data model the client is interested in. The server keeps each session up-to-date with the available data that matches the provided topic selectors. Let’s look at how this works.
When a session subscribes with a topic selector, the server will create subscriptions to all topics with paths matching the topic selector. The session will be notified of the created subscriptions and the current value of each topic that has one. The server will further notify the session of topic value changes as they occur. A session can request a subscription to a topic path for which there is no topic. If a topic is created for the path at a later time, the server will create the subscription and notify the session. For example, if a topic stock/regions/northwest/widgets is added, all sessions that have previously subscribed using the topic selector ?stock/regions/northwest/ will be subscribed to the topic. The server will notify the subscribing sessions of the new subscriptions.
If the topic is removed, any subscriptions will be removed, and the previously subscribed sessions will be notified of the unsubscription.
Diffusion 6.5: continuous evaluation of READ_TOPIC permissions
In versions of Diffusion prior to Diffusion 6.5, the READ_TOPIC permission is applied before a subscription is created for a session. That is, when a topic is added that matches the session’s topic selectors, or when the session adds a topic selector that matches an existing topic. If the session does not have READ_TOPIC permission for the topic path at this time, the subscription will be not be created. If the session has READ_TOPIC permission, the subscription will be created and the session will remain subscribed until a later unsubscription operation, even if the security configuration changes to remove the READ_TOPIC permission.
From Diffusion 6.5, READ_TOPIC permission is evaluated continuously. Changes to path permission assignments in the security configuration are applied immediately to existing sessions and can cause subscriptions to be created or removed. This capability is scalable so that changes are applied instantly, even if you have hundreds of thousands of connected sessions.
If a session has a topic selector matching a topic but does not have READ_TOPIC permission for the topic path, and then a change to the security configuration grants it READ_TOPIC permission for the path, a subscription will be created. Similarly, if a session is subscribed to a topic, and the security configuration changes so it no longer has READ_TOPIC permission for the topic path, the subscription will be removed. Effectively, the server now performs a three-way join between a session’s topic selectors, the existing topics, and READ_TOPIC permissions.
Other security changes in Diffusion 6.5
In previous versions of Diffusion, the API types and methods that represent path permissions have names that are based on the phrase “topic permission”. This is confusing because they equally apply to all types of path permissions. In Diffusion 6.5, these types and methods have been deprecated, and replaced with a new set of types and methods that use the more precise phrase “path permission”.
Summary
Diffusion 6.5 builds on Diffusion’s existing security features, making it easier than ever to manage access to data, even if your requirements are complex. The changes we’ve made to our security model allow access to be as fine-grained as you need, with changes applied in real time, even at scale with hundreds of thousands of end users.
Whether you’re building real-time applications for a growing start-up or a huge enterprise, you can be confident that Diffusion’s security model will be able to meet your needs, now and in the future.
Further reading
BLOG
Unlocking the Value of ISO 27001 Certification: A Journey of Security and Continuous Improvement
March 25, 2024
BLOG
Exploring Generative AI: Opportunity or Potential Headache?
March 25, 2024
Read More about Exploring Generative AI: Opportunity or Potential Headache?/span>
BLOG
100 million updates per second - Landmark Diffusion cluster performance
July 02, 2024
Read More about 100 million updates per second - Landmark Diffusion cluster performance/span>
