flink-kubernetes-operator学习

安装flink-kubernetes-operator,目前flink operator的稳定版本是1.11

1
2


helm repo add flink-operator-repo https://downloads.apache.org/flink/flink-kubernetes-operator-1.11.0/
helm install flink-kubernetes-operator flink-operator-repo/flink-kubernetes-operator

目前flink-k8s-op使用的image是ghcr.io/apache/flink-kubernetes-operator:cc9c6cb 这里可能需要考虑帮忙下载这个镜像

1

skopeo copy docker://ghcr.io/apache/flink-kubernetes-operator:cc9c6cb docker://10.7.20.12:5000/apache/flink-kubernetes-operator:cc9c6cb --insecure-policy --dest-tls-verify=false

目前遇到一个问题flinkdeployment卡主了

1
2
3


 kubectl get flinkdeployment
NAME                 JOB STATUS    RECONCILIATION STATUS
basic-example-read   RECONCILING   DEPLOYED

无法删除掉，解决办法

1
2
3


1. 移除此终结器。
    
    `finalizers:   - flinkdeployments.flink.apache.org/finalizer`

这样即可成功删除。再次查看crd是否成功创建开始创建flinkdeployment

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19


 cat basic-session-deployment-only.yaml
apiVersion: flink.apache.org/v1beta1
kind: FlinkDeployment
metadata:
  name: basic-session-deployment-only-example
spec:
  image: flink:1.20
  flinkVersion: v1_20
  flinkConfiguration:
    taskmanager.numberOfTaskSlots: "1"
  serviceAccount: flink
  jobManager:
    resource:
      memory: "1048m"
      cpu: 1
  taskManager:
    resource:
      memory: "1048m"
      cpu: 1

接着创建flink的node service

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15


apiVersion: v1
kind: Service
metadata:
  name: basic-session-deployment-only-example-rest-nodeport
spec:
  type: NodePort
  ports:
    - port: 8081
      targetPort: 8081
      nodePort: 30081
      name: result
  selector:
    app: basic-session-deployment-only-example
    component: jobmanager
    type: flink-native-kubernetes

接下来可以访问界面看看我们可以看的出来，这个时候是没有taskmanger的。接下来就是要创建job了，直接上配置

 1
 2
 3
 4
 5
 6
 7
 8
 9
10


apiVersion: flink.apache.org/v1beta1
kind: FlinkSessionJob
metadata:
  name: basic-session-job-only-example
spec:
  deploymentName: basic-session-deployment-only-example
  job:
    jarURI: https://repo1.maven.org/maven2/org/apache/flink/flink-examples-streaming_2.12/1.17.2/flink-examples-streaming_2.12-1.17.2-TopSpeedWindowing.jar
    parallelism: 4
    upgradeMode: stateless

注意一下，taskmanger的image是根据flinkdeployment的镜像一样的。再taskmanger界面看到如下即可：成功运行：