My New Favorite Feature of Cass Operator

I have been working with Cass Operator a lot as part of my work on K8ssandra. I have a new favorite feature that I want to discuss. That feature is the podTemplateSpec property of CassandraDatacenter.

The podTemplateSpec property is for advanced use cases. It is essentially a hook to customize the underlying StatefulSets that Cass Operator creates.

Let’s look at an example to illustrate how it can be put to use.

apiVersion: cassandra.datastax.com/v1beta1
kind: CassandraDatacenter
metadata:
name: dc1
spec:
clusterName: demo
serverType: cassandra
serverVersion: 3.11.7
managementApiAuth:
insecure: {}
size: 3
podTemplateSpec:
spec:
initContainers:
- name: echo-username
image: busybox
imagePullPolicy: IfNotPresent
env:
- name: USERNAME
value: jsanda
args:
- /bin/sh
- -c
- echo -n "USERNAME = $USERNAME"
containers: []

storageConfig:
cassandraDataVolumeClaimSpec:
storageClassName: standard
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 5Gi

I declared an initContainer that simply echoes the value of the USERNAME environment variable.

Now let’s look at the .spec.template.spec property of the StatefulSet created by Cass Operator:

$ kubectl -n deploy-reaper-test get sts demo-dc1-default-sts -o yamlapiVersion: apps/v1
kind: StatefulSet
metadata:
annotations:
cassandra.datastax.com/resource-hash: soyJEGWaLENAD2p9wVn/+9AxakV5DGs8vk7y/JPrHWI=
creationTimestamp: "2020-11-26T03:42:06Z"
generation: 2
labels:
app.kubernetes.io/managed-by: cass-operator
cassandra.datastax.com/cluster: demo
cassandra.datastax.com/datacenter: dc1
cassandra.datastax.com/rack: default
name: demo-dc1-default-sts
namespace: deploy-reaper-test
ownerReferences:
- apiVersion: cassandra.datastax.com/v1beta1
blockOwnerDeletion: true
controller: true
kind: CassandraDatacenter
name: dc1
uid: 3f6b8600-148b-48f9-8ae8-a2166a0b6b0a
resourceVersion: "17320734"
selfLink: /apis/apps/v1/namespaces/deploy-reaper-test/statefulsets/demo-dc1-default-sts
uid: 0679475f-357b-40d1-bf37-92c6af96e093
spec:
podManagementPolicy: Parallel
replicas: 3
revisionHistoryLimit: 10
selector:
matchLabels:
cassandra.datastax.com/cluster: demo
cassandra.datastax.com/datacenter: dc1
cassandra.datastax.com/rack: default
serviceName: demo-dc1-service
template:
metadata:
creationTimestamp: null
labels:
app.kubernetes.io/managed-by: cass-operator
cassandra.datastax.com/cluster: demo
cassandra.datastax.com/datacenter: dc1
cassandra.datastax.com/node-state: Ready-to-Start
cassandra.datastax.com/rack: default
spec:
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: cassandra.datastax.com/cluster
operator: Exists
- key: cassandra.datastax.com/datacenter
operator: Exists
- key: cassandra.datastax.com/rack
operator: Exists
topologyKey: kubernetes.io/hostname
containers:
- env:
- name: DS_LICENSE
value: accept
- name: DSE_AUTO_CONF_OFF
value: all
- name: USE_MGMT_API
value: "true"
- name: MGMT_API_EXPLICIT_START
value: "true"
- name: DSE_MGMT_EXPLICIT_START
value: "true"
image: datastax/cassandra-mgmtapi-3_11_7:v0.1.13
imagePullPolicy: IfNotPresent
lifecycle:
preStop:
exec:
command:
- wget
- --output-document
- /dev/null
- --no-check-certificate
- --post-data=''
- http://localhost:8080/api/v0/ops/node/drain
livenessProbe:
failureThreshold: 3
httpGet:
path: /api/v0/probes/liveness
port: 8080
scheme: HTTP
initialDelaySeconds: 15
periodSeconds: 15
successThreshold: 1
timeoutSeconds: 1
name: cassandra
ports:
- containerPort: 9042
name: native
protocol: TCP
- containerPort: 9142
name: tls-native
protocol: TCP
- containerPort: 7000
name: internode
protocol: TCP
- containerPort: 7001
name: tls-internode
protocol: TCP
- containerPort: 7199
name: jmx
protocol: TCP
- containerPort: 8080
name: mgmt-api-http
protocol: TCP
- containerPort: 9103
name: prometheus
protocol: TCP
- containerPort: 9160
name: thrift
protocol: TCP
readinessProbe:
failureThreshold: 3
httpGet:
path: /api/v0/probes/readiness
port: 8080
scheme: HTTP
initialDelaySeconds: 20
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /config
name: server-config
- mountPath: /var/log/cassandra
name: server-logs
- mountPath: /var/lib/cassandra
name: server-data
- mountPath: /etc/encryption/
name: encryption-cred-storage
- args:
- /bin/sh
- -c
- tail -n+1 -F /var/log/cassandra/system.log
image: busybox
imagePullPolicy: Always
name: server-system-logger
resources:
limits:
cpu: 100m
memory: 64M
requests:
cpu: 100m
memory: 64M
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /var/log/cassandra
name: server-logs
dnsPolicy: ClusterFirst
initContainers:
- env:
- name: CONFIG_FILE_DATA
value: '{"cassandra-yaml":{},"cluster-info":{"name":"demo","seeds":"demo-seed-service"},"datacenter-info":{"graph-enabled":0,"name":"dc1","solr-enabled":0,"spark-enabled":0}}'
- name: POD_IP
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: status.podIP
- name: HOST_IP
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: status.hostIP
- name: USE_HOST_IP_FOR_BROADCAST
value: "false"
- name: RACK_NAME
value: default
- name: PRODUCT_VERSION
value: 3.11.7
- name: PRODUCT_NAME
value: cassandra
- name: DSE_VERSION
value: 3.11.7
image: datastax/cass-config-builder:1.0.3
imagePullPolicy: IfNotPresent
name: server-config-init
resources:
limits:
cpu: "1"
memory: 256M
requests:
cpu: "1"
memory: 256M
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /config
name: server-config
- args:
- /bin/sh
- -c
- echo -n "USERNAME = $USERNAME"
env:
- name: USERNAME
value: jsanda
image: busybox
imagePullPolicy: IfNotPresent
name: echo
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File

restartPolicy: Always
schedulerName: default-scheduler
securityContext: {}
serviceAccount: default
serviceAccountName: default
terminationGracePeriodSeconds: 120
volumes:
- emptyDir: {}
name: server-config
- emptyDir: {}
name: server-logs
- name: encryption-cred-storage
secret:
defaultMode: 420
secretName: dc1-keystore
updateStrategy:
rollingUpdate:
partition: 0
type: RollingUpdate
volumeClaimTemplates:
- apiVersion: v1
kind: PersistentVolumeClaim
metadata:
creationTimestamp: null
labels:
app.kubernetes.io/managed-by: cass-operator
cassandra.datastax.com/cluster: demo
cassandra.datastax.com/datacenter: dc1
cassandra.datastax.com/rack: default
name: server-data
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 5Gi
storageClassName: standard
volumeMode: Filesystem
status:
phase: Pending

My echo initContainer is included in the output. Every StatefulSet that Cass Operator creates will include the following containers:

  • server-config-init initContainer
  • cassandra container
  • server-system-logger container

The podTemplateSpec property allowed me to modify the StatefulSet to include the echo initContainer. Pretty cool!

The echo initContainer is a bit contrived, but it provides a blueprint for a way to apply advanced configurations with Cass Operator. In my next post I will go through a real world example. Stay tuned!

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store