Multi-Destination-Cast

Multiple-Destination-Cast (MDC) is a capability that allows Aeron to deliver data to multiple simultaneous destinations from a single Publication. Multiple-Destination-Cast is an advanced feature of Aeron, and this guide touches on the bare basics of how to develop a simple sample.

MDC Publications¶

Note

MDC Publications are able to provide roughly equivalent behavior to UDP multicast in environments in which there is no support for UDP multicast. Note that it is not true multicast - data is sent individually to each subscription, however, flow control capabilities are supported.

MDC Publications can run in either a dynamic and manual mode. With dynamic MDC Publications, a Subscription adds itself to the Publication dynamically at runtime. With manual MDC Publications, the Subscription must be explicitly added to the Publication.

Dynamic MDC Publications work much like standard Aeron Publication, although the configuration is inverted.

Type	Publication	Subscription
Standard	Channel points to Subscription	Channel/port on localhost
Dynamic Multi-Destination-Cast Publication	Channel/port on localhost	Channel points to Publication

Sample Dynamic MDC Publication¶

A full, multi-host sample can be found on GitHub. This sample creates a single publisher with a dynamic MDC Publication, along with two clients which subscribe to the MDC Publication. Each process is in a dedicated Docker container.

Media Driver Configuration¶

The Media Driver requires no specific configuration, however, you can optionally enable spiesSimulateConnection if you require Publications to be able to publish data without any Subscriptions connected. The sample code does this as follows:

final var mediaDriverContext = new MediaDriver.Context()
    .spiesSimulateConnection(true)
    .errorHandler(this::errorHandler)
    .threadingMode(ThreadingMode.SHARED)
    .sharedIdleStrategy(new SleepingMillisIdleStrategy())
    .dirDeleteOnStart(true);

Publication¶

The MDC dynamic Publication uses a specific channel configuration, which at minimum contains:

aeron:udp?control-mode=dynamic|control=MDC_HOST:MDC_CONTROL_PORT

The control-mode=dynamic tells Aeron that this is a Dynamic MDC Publication. The MDC_HOST in the sample above must be the host on which the Publication is running. Behind the scenes, Subscriptions connect to this host using the control port to construct the runtime flow, so the control port must be a port known to the clients.

The sample code constructs the Publication channel as follows:

final var publicationChannel = "aeron:udp?control-mode=dynamic|control=" + host
            + ":" + controlChannelPort;
final var publication = aeron.addExclusivePublication(publicationChannel, 100);

In addition to the basic configuration, the flow control strategy can also be configured via the Channel. This includes flow control strategies with group based limits, where a minimum number of nodes must be connected before the Publication will move to connected status.

Subscription¶

Connecting an MDC Subscription to a dynamic Publication is simple - the channel configuration must include the MDC Publication host and a local endpoint to which the remote host will publish. The minimal configuration is:

aeron:udp?endpoint=LOCALHOST:0|control=MDC_HOST:MDC_CONTROL_PORT|control-mode=dynamic

As with the Publication, the control-mode=dynamic tells Aeron that this is a dynamic MDC connection. The endpoint=LOCALHOST:0 refers to an ephemeral port on the localhost to which the MDC Dynamic Publication will publish data - this can be a predefined port as well. Finally, the control=MDC_HOST:MDC_CONTROL_PORT tells Aeron how to connect to the remote dynamic MDC Publication.

The sample code constructs the Subscription channel as follows:

final mdcSubscription = aeron.addSubscription("aeron:udp?endpoint=" + host
    + ":0|control=" + mdcHost + ":" + mdcControlPort + "|control-mode=dynamic", 100);

The sample runs two instances of the clients, each in a dedicated Docker host.

Sample output¶

The following is the lightly edited output of a sample run via a docker-compose up:

mdc-subscriber-2_1  | 16:47:00.945 [main] INFO MultiDestinationSubscriberAgent - launching media driver
mdc-publisher_1     | 16:47:00.946 [main] INFO MultiDestinationPublisherAgent - launching media driver
mdc-subscriber-1_1  | 16:47:01.066 [main] INFO MultiDestinationSubscriberAgent - launching media driver
mdc-subscriber-2_1  | 16:47:01.082 [main] INFO MultiDestinationSubscriberAgent - connecting aeron; media driver directory /dev/shm/aeron-root
mdc-publisher_1     | 16:47:01.083 [main] INFO MultiDestinationPublisherAgent - launching aeron
mdc-subscriber-2_1  | 16:47:01.093 [main] INFO MultiDestinationSubscriberAgent - adding the subscription
mdc-subscriber-2_1  | 16:47:01.093 [main] INFO MultiDestinationSubscriberAgent - detected ip4 address as 10.1.0.3
mdc-publisher_1     | 16:47:01.093 [main] INFO MultiDestinationPublisherAgent - Media Driver directory is /dev/shm/aeron-root
mdc-publisher_1     | 16:47:01.095 [main] INFO MultiDestinationPublisherAgent - detected ip4 address as 10.1.0.2
mdc-publisher_1     | 16:47:01.107 [main] INFO MultiDestinationPublisherAgent - creating publication
mdc-subscriber-2_1  | 16:47:01.128 [mdc-subscriber] INFO MultiDestinationSubscriberAgent - starting
mdc-subscriber-1_1  | 16:47:01.162 [main] INFO MultiDestinationSubscriberAgent - connecting aeron; media driver directory /dev/shm/aeron-root
mdc-publisher_1     | 16:47:01.165 [mdc-publisher] INFO MultiDestinationPublisherAgent - Starting up
mdc-publisher_1     | 16:47:01.166 [mdc-publisher] INFO MultiDestinationPublisherAgent - appended 1
mdc-subscriber-1_1  | 16:47:01.170 [main] INFO MultiDestinationSubscriberAgent - adding the subscription
mdc-subscriber-1_1  | 16:47:01.170 [main] INFO MultiDestinationSubscriberAgent - detected ip4 address as 10.1.0.4
mdc-subscriber-1_1  | 16:47:01.201 [mdc-subscriber] INFO MultiDestinationSubscriberAgent - starting
mdc-publisher_1     | 16:47:03.167 [mdc-publisher] INFO MultiDestinationPublisherAgent - appended 2
mdc-subscriber-2_1  | 16:47:03.170 [mdc-subscriber] INFO MultiDestinationSubscriberFragmentHandler - received 2
mdc-subscriber-1_1  | 16:47:03.171 [mdc-subscriber] INFO MultiDestinationSubscriberFragmentHandler - received 2
mdc-publisher_1     | 16:47:05.167 [mdc-publisher] INFO MultiDestinationPublisherAgent - appended 3
mdc-subscriber-1_1  | 16:47:05.171 [mdc-subscriber] INFO MultiDestinationSubscriberFragmentHandler - received 3
mdc-subscriber-2_1  | 16:47:05.170 [mdc-subscriber] INFO MultiDestinationSubscriberFragmentHandler - received 3
...

Notes:

the samples make use of a relaxed IdleStrategy configuration, which explains why there is a 3 to 4 millisecond gap between the append and receive.
the mdc-publisher_1 has a log entry where it appended 1, but the two clients do not show a corresponding received 1 log. This is because the mdc-publisher_1 Media Driver has spiesSimulateConnection set to true, and the mdc-subscriber-* processes were not yet connected. They do receive 2 onwards.

Flow Control¶

The example above raises a question: what should the mdc-publisher do if mdc-subscriber-1 starts to fall behind mdc-subscriber-2? This behavior is configurable in Aeron by way of flow control. Flow control can be configured to a default value within the Media Driver and then customized per channel.

Flow Control Types¶

Type	Description
max	Publication will be limited by the fastest Subscription. Slow consumers may lose data packets. This is the default in Aeron.
min	Publication will be limited by the slowest Subscription.
tagged	Publication will be limited by the slowest tagged Subscription within a group

If the Publication is producing data at a pace beyond the level as defined by the flow control strategy, the Publication will be subjected to back pressure.

Max Flow Control¶

Configure max flow control by setting fc=max on the Channel configuration, for example:

aeron:udp?control-mode=dynamic|control=MDC_HOST:MDC_CONTROL_PORT|fc=max

Or set the default with the Media Driver Context (note that multicastFlowControlSupplier is used for both UDP multicast and Multi-Destination-Cast):

final var mediaDriverContext = new MediaDriver.Context()
...
    .multicastFlowControlSupplier(new MaxMulticastFlowControlSupplier())
...

Min Flow Control¶

Configure min flow control by setting fc=min on the Channel configuration, for example:

aeron:udp?control-mode=dynamic|control=MDC_HOST:MDC_CONTROL_PORT|fc=min

Or set the default with the Media Driver Context:

final var mediaDriverContext = new MediaDriver.Context()
...
    .multicastFlowControlSupplier(new MinMulticastFlowControlSupplier())
...

Additional options that can be set include group size. The example below sets a group size of 5 with g:/5. Group size controls the connected state of the Publication — for example, with a group size of 5, the Publication will only be considered connected if 5 Subscriptions are connected. You can then have the system functional with at least 5 connected Subscriptions, the slowest of which sets the pace for all connected Subscriptions.

aeron:udp?control-mode=dynamic|control=MDC_HOST:MDC_CONTROL_PORT|fc=min,g:/5

This can also be defaulted in the Media Driver Context (here setting group size to 5):

final var mediaDriverContext = new MediaDriver.Context()
...
    .multicastFlowControlSupplier(new MinMulticastFlowControlSupplier())
    .flowControlGroupMinSize(5)
...

Tagged Flow Control¶

Sometimes you need a finer level of control within Multi-Destination-Cast — for example, you may have a group of subscribers which should not lose data packets, but you may also have other subscribers connected that are not impacted by data loss and you do not want these loss accepting subscribers to hold all Subscriptions back with a min flow control strategy. The tagged flow control strategy enables this.

The following configuration sets the flow control to tagged and the group to 101 for a Publication channel.

aeron:udp?control-mode=dynamic|control=MDC_HOST:MDC_CONTROL_PORT|fc=tagged,g:101

For a Subscription to join the tagged flow control with the 101 tag (and thus be subject to flow control, much like the min strategy), it would need gtag set as follows:

aeron:udp?endpoint=LOCALHOST:0|control=MDC_HOST:MDC_CONTROL_PORT|control-mode=dynamic|gtag=101

If the Subscription needs to join the same MDC Publication, but is not concerned with data loss (i.e. is not subject to flow control), then the gtag can be removed:

aeron:udp?endpoint=LOCALHOST:0|control=MDC_HOST:MDC_CONTROL_PORT|control-mode=dynamic

The following is the equivalent for setting the Media Driver defaults:

final var mediaDriverContext = new MediaDriver.Context()
...
    .multicastFlowControlSupplier(new TaggedMulticastFlowControlSupplier())
    .flowControlGroupTag(101)
...

As with the min flow control strategy, a minimum group size can be set for the tagged flow control. In this Publication channel example, the tag is set to 101, and the group size is set to 5:

aeron:udp?control-mode=dynamic|control=MDC_HOST:MDC_CONTROL_PORT|fc=tagged,g:101/5