How to write good APIs

The secret to writing a successful API, is to model what the product does and not how the product works.

A successful API needs to be customer-centric, developer-friendly, and maintain the same level of functionality that existing customers expect — but also present its functionality in a clear, concise way that gradually introduces its complexities in order to lower the entry barrier for developers new to the product.

To achieve this, we need to start with a top-down view of all concepts and the method that establishes the relations between them and is organized in several bounded contexts according to the use cases.

Each bounded context has a single, ubiquitous objective: including subscribing, publishing, mapping, session management and server configuration, among others.

This modelling is done with subject matter experts (SMEs) from the beginning to ensure that the names for the concepts and methods clearly reflect the same language used in the use cases.

The methodology described above is called Domain-driven design (DDD).

The most fundamental use cases for the Diffusion Client reflect the Publishing / Subscribing (Pub/Sub) capabilities that Diffusion offers.

The first use case reflects a user who wants to publish a JSON value to a topic path. The typical thought process of a developer writing the code would be:

1. I want the topic at topic_path which I know it’s a JSON.
2. In this topic, I want to publish the JSON value.

This is the use case, written in JavaScript, using the new API:

await session
    .topics("<topic_path>", "JSON")
    .publish({"hello": "world"});

Let’s compare with the same use case, also written in JavaScript, using the current API:

const jsonDataType = diffusion.datatypes.json();

const topicValue = jsonDataType.fromJsonString(
    '{"hello": "world"}’
);

await session.topicUpdate.set(
    "<topic_path>",
    jsonDataType,
    topicValue
);

We’ve reduced not only the lines of code required for the same use case, but we’ve also reduced the number of concepts and processes that a developer needs to know to effectively use Diffusion.

The developer no longer needs to know that the method for publishing to a topic is in the topicUpdate feature and it’s called set, nor the existence of, and methods in diffusion.datatypes.json() that convert a JSON object into a value readable by Diffusion.

The second use case reflects a user that wants to subscribe to a given topic path which they know contains JSON data. The typical thought process of a developer writing the code would be:

1. I want the topic at topic_path which I know it’s a JSON.
2. I want to subscribe to it, passing the event value handlers for when a value is received or when the subscription state changes.
3. The act of subscribing should return a subscription which I can close to terminate it.

This is the use case, written in JavaScript, using the new API:

const subscription = await session
    .topics("<topic_path>", "JSON")
    .subscribe(function (event) {
        // ...
    });
// ...
await subscription.close();

Let’s compare with the same use case, also written in JavaScript, using the current API:

const valueStream = session.addStream(
    "<topic_path>",
    diffusion.datatypes.json()
);

valueStream.on(
    "value",
    function (topic, specification, newValue, oldValue) {
        // ...
    }
);

await session.select("<topic_path>");
// ...
await session.unsubscribe("<topic_path>");

Much like the publishing use case, we’ve reduced the lines of code and amount of concepts and operations that a user would need to know before writing the code, such as setting up a JSON value stream for the topic_path, using the method select for subscribing, and the unsubscribe method for terminating a subscription.

As you can see, both use cases start with the same topics method, with the purpose of retrieving one or more topic paths in the Diffusion Topic Tree for extended functionality.

As part of an internal effort to apply this methodology, we’ve begun a series of Domain-Driven, Design-for-Diffusion Marathons, where the Client SDK team, along with several SMEs, discuss use cases and the reasons behind certain features in order to define the domains and concepts of what Diffusion does from a user-centric perspective.

From the first marathon, we calculated that only 30% of the totality of the API was covered, but it was more than enough to start refining a limited scope around publishing and subscribing (Pub/Sub) which became this new JavaScript API.

This API allows the introduction of simple use cases, like retrieving a topic from the topic tree, and extending it with use cases of increasing complexity, for example publishing and subscribing to said topic.

From now on, DDD will be at the core of Diffusion, providing a single set of concepts and methods of how Diffusion solves the customers problems, with a clean, unambiguous and consistent language across all APIs as well as extending into documentation and other product related areas.

Further domain-driven design marathons will be scheduled for 2022 with the aim of modelling the entire API and its bonded contexts.

The JavaScript API is first iteration of the DDD methodology and is going to be available soon via NPM.