Kubernetes 部署 canal 集群

本文为K8s集群部署阿里云canal集群，canal信息可见GitHub – alibaba/canal: 阿里巴巴 MySQL binlog 增量订阅&消费组件，canal分为canal-admin和canal-sever两个组件。

初始化数据库

canal需要两个库，一个用来存放canal-admin的数据，需要使用sql进行初始化操作；

另一个用来存放canal-server工作时instance监听的库的数据结构信息，无需初始化

初始化sql位置在canal_manager.sql

因Canal集群需要依赖zk

部署zookeeper

kind: Deployment
apiVersion: apps/v1
metadata:
  name: zk-1
  namespace: ops
  labels:
    app: zk-1
  annotations:
    deployment.kubernetes.io/revision: '4'
    kubesphere.io/creator: admin
spec:
  replicas: 1
  selector:
    matchLabels:
      app: zk-1
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: zk-1
      annotations:
        logging.kubesphere.io/logsidecar-config: '{}'
    spec:
      volumes:
        - name: volume-ek2m55  ##持久化卷
          persistentVolumeClaim:
            claimName: zk1-data
      containers:
        - name: zk-1
          image: 'zookeeper:3.8.0'
          ports:
            - name: tcp-2181
              containerPort: 2181
              protocol: TCP
            - name: tcp-2888
              containerPort: 2888
              protocol: TCP
            - name: tcp-3888
              containerPort: 3888
              protocol: TCP
          env:
            - name: ZOO_MY_ID
              value: '1'
            - name: ZOO_SERVERS
              value: >-
                server.1=0.0.0.0:2888:3888;2181  ## 这个是3.5版本以上需要配置区别处
                server.2=zk-2.ops.svc.cluster.local:2888:3888;2181
                server.3=zk-3.ops.svc.cluster.local:2888:3888;2181
          resources:
            limits:
              cpu: '1'
              memory: 2Gi
            requests:
              cpu: 500m
              memory: 1Gi
          volumeMounts:
            - name: volume-ek2m55  ##持久化卷
              mountPath: /data
          terminationMessagePath: /dev/termination-log
          terminationMessagePolicy: File
          imagePullPolicy: IfNotPresent
      restartPolicy: Always
      terminationGracePeriodSeconds: 30
      dnsPolicy: ClusterFirst
      serviceAccountName: default
      serviceAccount: default
      securityContext: {}
      schedulerName: default-scheduler
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 25%
      maxSurge: 25%
  revisionHistoryLimit: 10
  progressDeadlineSeconds: 600

kind: Service
apiVersion: v1
metadata:
  name: zk-1
  namespace: ops
  labels:
    app: zk-1
  annotations:
    kubesphere.io/creator: admin
spec:
  ports:
    - name: tcp-2181
      protocol: TCP
      port: 2181
      targetPort: 2181
    - name: tcp-2888
      protocol: TCP
      port: 2888
      targetPort: 2888
    - name: tcp-3888
      protocol: TCP
      port: 3888
      targetPort: 3888
  selector:
    app: zk-1
  clusterIP: 10.233.16.219
  clusterIPs:
    - 10.233.16.219
  type: ClusterIP
  sessionAffinity: None
  ipFamilies:
    - IPv4
  ipFamilyPolicy: SingleStack

部署canal-admin组件

apiVersion: v1
kind: ConfigMap
metadata:
  name: canal-admin
  namespace: ops
data:
  admin_user: "admin"
  admin_password: "123456"
  datasource_address: "xxxxx:3306"
  datasource_database: "canal_manager"
  datasource_username: "canal"
  datasource_password: "xxxxxxxx"

kind: Deployment
apiVersion: apps/v1
metadata:
  name: canal-admin
  namespace: ops
  labels:
    app.kubernetes.io/name: canal-admin
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: canal-admin
  template:
    metadata:
      name: canal-admin
      labels:
        app.kubernetes.io/name: canal-admin
    spec:
      imagePullSecrets:
      - name: secret
      containers:
        - name: canal-admin
          image: 'canal/canal-admin:v1.1.5'
          imagePullPolicy: IfNotPresent
          ports:
            - name: web
              containerPort: 8089
              protocol: TCP
          env:
            - name: server.port
              value: '8089'
            - name: canal.adminUser
              valueFrom:
                configMapKeyRef:
                  name: canal-admin
                  key: admin_user
            - name: canal.adminPasswd
              valueFrom:
                configMapKeyRef:
                  name: canal-admin
                  key: admin_password
            - name: spring.datasource.address
              valueFrom:
                configMapKeyRef:
                  name: canal-admin
                  key: datasource_address
            - name: spring.datasource.database
              valueFrom:
                configMapKeyRef:
                  name: canal-admin
                  key: datasource_database
            - name: spring.datasource.username
              valueFrom:
                configMapKeyRef:
                  name: canal-admin
                  key: datasource_username
            - name: spring.datasource.password
              valueFrom:
                configMapKeyRef:
                  name: canal-admin
                  key: datasource_password
          resources:
            requests:
              cpu: 250m
              memory: 256Mi
          livenessProbe:
            httpGet:
              path: /
              port: 8089
            initialDelaySeconds: 10
            timeoutSeconds: 5
            periodSeconds: 30
          readinessProbe:
            httpGet:
              path: /
              port: 8089
            initialDelaySeconds: 10
            timeoutSeconds: 5
            periodSeconds: 30
      restartPolicy: Always
      nodeSelector: {}
      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
            - weight: 1
              podAffinityTerm:
                labelSelector:
                  matchLabels:
                    app.kubernetes.io/name: canal-admin
                namespaces:
                  - ops
                topologyKey: kubernetes.io/hostname
  revisionHistoryLimit: 10

kind: Service
apiVersion: v1
metadata:
  name: canal-admin
  namespace: ops
spec:
  ports:
    - protocol: TCP
      port: 8089
      targetPort: 8089
  selector:
    app.kubernetes.io/name: canal-admin

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: canal-admin
  namespace: ops
spec:
  tls: []
  rules:
  - host: canal-admin.xxx.xyz        # ingress 域名
    http:
      paths:
      - backend:
          serviceName: canal-admin   # 对应服务名
          servicePort: 8089

部署canal-server

在源库中创建同步数据账号

部署canal-server 在源库中创建同步数据账号

mysql > GRANT SELECT, REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO 'canal'@'%';
mysql > FLUSH PRIVILEGES;

apiVersion: v1
kind: ConfigMap
metadata:
  name: canal-server
  namespace: ops
data:
  admin_manager: "canal-admin.ops:8089"
  admin_port: "11110"
  admin_user: "admin"
  admin_password: "6BB4837EB74329105EE4568DDA7DC67ED2CA2AD9"   # 123456加密
  admin_register_cluster: "local"  ##集群名称
  admin_register_auto: "true"
  admin_register_name: "dev-canal-server"  ##canal-server 自动注册名称

kind: StatefulSet    ##canal-server 采用StatefulSet 方式部署
apiVersion: apps/v1
metadata:
  name: canal-server
  namespace: ops
  labels:
    app.kubernetes.io/name: canal-server
    app: canal-server
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: canal-server
      app: canal-server
  template:
    metadata:
      name: canal-server
      labels:
        app.kubernetes.io/name: canal-server
        app: canal-server
    spec:
      imagePullSecrets:
      - name: secret
      containers:
        - name: canal-server
          image: 'canal/canal-server:v1.1.5'
          imagePullPolicy: IfNotPresent
          ports:
            - name: tcp
              containerPort: 11111
              protocol: TCP
          env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  apiVersion: v1
                  fieldPath: metadata.name
            - name: SERVICE_NAME
              valueFrom:
                fieldRef:
                  apiVersion: v1
                  fieldPath: 'metadata.labels[''app'']'
            - name: STS_NAMESPACE
              valueFrom:
                fieldRef:
                  apiVersion: v1
                  fieldPath: metadata.namespace
            - name: canal.register.ip
              value: $(POD_NAME).$(SERVICE_NAME).$(STS_NAMESPACE)  ##  canal-server采用实例名，防止每次重启导致的IP不同，Instance实例选择时的变化
            - name: canal.admin.manager
              valueFrom:
                configMapKeyRef:
                  name: canal-server
                  key: admin_manager
            - name: canal.admin.port
              valueFrom:
                configMapKeyRef:
                  name: canal-server
                  key: admin_port
            - name: canal.admin.user
              valueFrom:
                configMapKeyRef:
                  name: canal-server
                  key: admin_user
            - name: canal.admin.passwd
              valueFrom:
                configMapKeyRef:
                  name: canal-server
                  key: admin_password
            - name: canal.admin.register.cluster
              valueFrom:
                configMapKeyRef:
                  name: canal-server
                  key: admin_register_cluster
            - name: canal.admin.register.auto
              valueFrom:
                configMapKeyRef:
                  name: canal-server
                  key: admin_register_auto
            - name: canal.admin.register.name
              valueFrom:
                configMapKeyRef:
                  name: canal-server
                  key: admin_register_name
          resources:
            requests:
              cpu: 250m
              memory: 256Mi
          livenessProbe:
            tcpSocket:
              port: 11112
            initialDelaySeconds: 10
            timeoutSeconds: 5
            periodSeconds: 30
          readinessProbe:
            tcpSocket:
              port: 11112
            initialDelaySeconds: 10
            timeoutSeconds: 5
            periodSeconds: 30
      restartPolicy: Always
      nodeSelector: {}
      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
            - weight: 1
              podAffinityTerm:
                labelSelector:
                  matchLabels:
                    app.kubernetes.io/name: canal-server
                namespaces:
                  - ops
                topologyKey: kubernetes.io/hostname
  serviceName: canal-server
  revisionHistoryLimit: 10

kind: Service
apiVersion: v1
metadata:
  name: canal-server
  namespace: ops
spec:
  ports:
    - protocol: TCP
      port: 11110
      targetPort: 11110
  type: ClusterIP
  selector:
    app.kubernetes.io/name: canal-server
    app: canal-server

文章转载请说明出处：八零岁月 » Kubernetes 部署 canal 集群