How to run Aeron Cluster on Kubernetes

Problem

You want to run a multi-node cluster of Aeron Cluster on Kubernetes

Discussion

Running on Kubernetes is achieved via the use of a Kubernetes stateful set and cluster scoped ClusterIps. Aeron Cluster configuration - using the sample ClusterConfig object - requires a reference to the index of CLUSTER_ADDRESSES in order to properly configure a node. This can be seen with the -Dnode.id=${POD_NAME##*-} argument below.

Note that DNS can be slow within Kubernetes, and as a result, the application should await proper DNS resolution. In the sample, this is found here

Example Cluster Controller

---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: aeron-io-sample-cluster
  namespace: aeron-io-sample-cluster
  labels:
    app.kubernetes.io/name: aeron-io-sample-cluster
    app.kubernetes.io/instance: aeron-io-sample-cluster
spec:
  replicas: 3
  podManagementPolicy: Parallel
  serviceName: aeron-io-sample-cluster
  selector:
    matchLabels:
      app.kubernetes.io/name: aeron-io-sample-cluster
      app.kubernetes.io/instance: aeron-io-sample-cluster
  template:
    metadata:
      annotations:
        {}
      labels:
        app.kubernetes.io/name: aeron-io-sample-cluster
        app.kubernetes.io/instance: aeron-io-sample-cluster
    spec:
      serviceAccountName: aeron-io-sample-cluster
      volumes:
        - emptyDir:
            medium: Memory
            sizeLimit: 1Gi
          name: shm
      containers:
        - name: aeron-io-sample-cluster
          image: "cluster:latest"
          imagePullPolicy: IfNotPresent
          command:
            - /bin/bash
          args:
            - -c
            - /root/jar/entrypoint.sh -Dnode.id=${POD_NAME##*-}
          env:
            - name: DNS_DELAY
              value: "true"
            - name: CLUSTER_ADDRESSES
              value: aeron-io-sample-cluster-0.aeron-io-sample-cluster.aeron-io-sample-cluster.svc.cluster.local.,aeron-io-sample-cluster-1.aeron-io-sample-cluster.aeron-io-sample-cluster.svc.cluster.local.,aeron-io-sample-cluster-2.aeron-io-sample-cluster.aeron-io-sample-cluster.svc.cluster.local.
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
          volumeMounts:
            - mountPath: /dev/shm
              name: shm

Example Cluster Service

---
apiVersion: v1
kind: Service
metadata:
  name: aeron-io-sample-cluster
  namespace: aeron-io-sample-cluster
  labels:
    app.kubernetes.io/name: aeron-io-sample-cluster
    app.kubernetes.io/instance: aeron-io-sample-cluster
spec:
  type: ClusterIP
  clusterIP: None
  selector:
    app.kubernetes.io/name: aeron-io-sample-cluster
    app.kubernetes.io/instance: aeron-io-sample-cluster

The full example is available on GitHub: Aeron Sample - Kubernetes