Skip to content

Basic Sample

Full example can be found online at GitHub. Also see the RPC Server which shows SBE being used for a simple RPC server.

Preparing for SBE

First, take a copy of the sbe.xsd schema from the SBE GitHub project, and place it in a folder

Setting up Gradle

Next, include the sbe-tool as a dependency:

...
compile "uk.co.real-logic:sbe-tool:${SBE_VERSION}"
...

Then add a task to invoke the SBE tool to process the message definitions. The snippet below assumes you placed the sbe.xsd under src/main/resources/sbe/sbe.xsd. It further assumes you will define your messages in a src/main/resources/messages.xml.

...
  task generateMessages(type: JavaExec) {
        main = 'uk.co.real_logic.sbe.SbeTool'
        classpath = sourceSets.main.runtimeClasspath
        systemProperties('sbe.output.dir': generatedDir,
                'sbe.target.language': 'Java',
                'sbe.validation.stop.on.error': 'true',
                'sbe.validation.xsd': 'src/main/resources/sbe/sbe.xsd')
        args = [
                'src/main/resources/messages.xml'
        ]
    }
...    

Defining a simple set of messages

Start with a base message schema. In this one below, the message header, variable length string varStringEncoding and group size encoder are pre defined. The group size encoder holds the dimensions within a repeating group, much like FIX. The message header is used during decoding to ensure that the message decoder in use is the correct one for the incoming byte array.

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<sbe:messageSchema xmlns:sbe="http://fixprotocol.io/2016/sbe"
                   package="com.aeroncookbook.sbe"
                   id="688"
                   version="1"
                   semanticVersion="0.1"
                   description="Sample SBE Messages"
                   byteOrder="littleEndian">
    <types>
        <composite name="messageHeader" description="Message identifiers and length of message root">
            <type name="blockLength" primitiveType="uint16"/>
            <type name="templateId" primitiveType="uint16"/>
            <type name="schemaId" primitiveType="uint16"/>
            <type name="version" primitiveType="uint16"/>
        </composite>
        <composite name="varStringEncoding">
            <type name="length" primitiveType="uint32" maxValue="1073741824"/>
            <type name="varData" primitiveType="uint8" length="0" characterEncoding="UTF-8"/>
        </composite>
        <composite name="groupSizeEncoding" description="Repeating group dimensions.">
            <type name="blockLength" primitiveType="uint16"/>
            <type name="numInGroup" primitiveType="uint16"/>
        </composite>
    </types>
</sbe:messageSchema>

Then, create any composite types you may need, although note that they are not well suited for use groups and cannot be used within other composite types. Any structure that is more than 1 layer deep would typically need to be flattened out.

<composite name="composite" description="Sample Composite type">
    <type name="field1" primitiveType="uint16"/>
    <type name="field2" primitiveType="uint16"/>
</composite>

Then, create any enumerations you may want specifically within the SBE message. If you're using SBE just a serializer, enumerations may be overkill - just hold an encoded value.

<enum name="SampleEnum" encodingType="int32">
    <validValue name="VALUE_1">1</validValue>
    <validValue name="VALUE_2">2</validValue>
    <validValue name="VALUE_3">3</validValue>
</enum>

Next, create any strongly typed common values. This makes it simpler to change the underlying primative type and to ensure consistency across fields. Note that no specific code is generated for these items that is to say, they are not [tiny types](https://techbeacon.com/app-dev-testing/big-benefits-tiny-types-how-make-your-codes-domain-concepts-explicit).

<types>
    <type name="Sequence" primitiveType="int64"/>
    <type name="Timestamp" primitiveType="int64"/>
</types>

Warning

SBE's XML schema is strict and enforced. Within messages, the layout must always be fields followed by groups and then variable length data items, such as variable length strings.

id values are scoped. No sbe:message can share an id, and no field/data/group within a single message can shared an id. And within a group, no item may share an id. They do not need to be sequential.

With all the basics in place, one can start by creating a sample message. This message, SampleA, includes 3 fields: sequence, enumField and message. Note that the string is made up of a variable length string, and must be within the data field types, which must be declared last within the fields.

<sbe:message name="SampleSimple" id="1" description="Simple sample">
   <field name="sequence" id="1" type="Sequence"/>
   <field name="enumField" id="2" type="SampleEnum"/>
   <data name="message" id="3" type="varStringEncoding"/>
</sbe:message>

And here's a sample with a repeating group.

<sbe:message name="SampleGroup" id="2" description="Sample with group">
    <field name="timestamp" id="1" type="Timestamp"/>
    <group name="group" id="10" dimensionType="groupSizeEncoding">
        <field name="groupField1" id="11" type="uint16"/>
        <field name="groupField2" id="12" type="uint16"/>
        <data name="groupField3" id="13" type="varStringEncoding"/>
    </group>
    <data name="message" id="2" type="varStringEncoding"/>
</sbe:message>

Using the API

Warning

When reading and writing using SBE decoders and encoders, all fields MUST be read and set in the exact same order as the XML definition, or data corruption is likely. See Warnings for more detail.

Encoding

Encoding comprises four steps:

  • Build the message header and actual header objects. This can be done once and reused if desired;
  • Allocate memory for the encoded data to be written to;
  • Wrap the buffer, and apply the header;
  • Set the data fields as desired.
//Construct the two encoders required
final SampleAEncoder encoder = new SampleAEncoder();
final MessageHeaderEncoder messageHeaderEncoder = new MessageHeaderEncoder();

//Allocate memory and an unsafe buffer to act as a destination for the data
final ByteBuffer byteBuffer = ByteBuffer.allocateDirect(128);
final UnsafeBuffer directBuffer = new UnsafeBuffer(byteBuffer);

//write the encoded output to the direct buffer
encoder.wrapAndApplyHeader(directBuffer, 0, messageHeaderEncoder);

//set the fields to desired values
encoder.sequence(123);
encoder.enumField(SampleEnum.VALUE_1);
encoder.message("a message");

int encodedLength = MessageHeaderEncoder.ENCODED_LENGTH + encoder.encodedLength();

Decoding

Decoding involves the following steps:

  • Construct the decoder and message header decoder
  • The header decoder wraps the inbound buffer, and reads the header
  • A safety check can be done at this point to verify that the correct decoder is being used
  • Next, wrap the inbound buffer.
final SampleADecoder decoder = new SampleADecoder();
final MessageHeaderDecoder headerDecoder = new MessageHeaderDecoder();
int bufferOffset = 0;
headerDecoder.wrap(directBuffer, bufferOffset);

// Lookup the applicable flyweight to decode this type of message based on templateId and version.
final int templateId = headerDecoder.templateId();
if (templateId != SampleADecoder.TEMPLATE_ID)
{
    throw new IllegalStateException("Template ids do not match");
}

final int actingBlockLength = headerDecoder.blockLength();
final int actingVersion = headerDecoder.version();

bufferOffset += headerDecoder.encodedLength();
decoder.wrap(directBuffer, bufferOffset, actingBlockLength, actingVersion);

Assertions.assertEquals(123, decoder.sequence());
Assertions.assertEquals(SampleEnum.VALUE_1, decoder.enumField());
Assertions.assertEquals("a message", decoder.message());