XiaodaBlogSource/source/_posts/Docker-Compose.md

title, date, tags

title	date	tags
Docker-Compose	2024-04-01 11:02:56

集群搭建

Flink 集群

1. 首先启动flink 容器 JobManager、TaskManager 两个容器将配置文件复制出来方便挂载

   docker run \
     -itd \
     --name=jobmanager \
     --publish 8081:8081 \
     --network flink-network \
     --env FLINK_PROPERTIES="jobmanager.rpc.address: jobmanager" \
     flink:scala_2.12-java8 jobmanager 
   
   docker run \
     -itd \
     --name=taskmanager \
     --network flink-network \
     --env FLINK_PROPERTIES="jobmanager.rpc.address: jobmanager" \
     flink:scala_2.12-java8 taskmanager 
   
   
   

2. 创建本地卷挂载目录，拷贝文件

    mkdir -p /usr/local/flink-docker/jobmanager
    mkdir -p /usr/local/flink-docker/taskmanager
   docker cp jobmanager:/opt/flink/lib /usr/local/flink-docker/jobmanager
   docker cp jobmanager:/opt/flink/log /usr/local/flink-docker/jobmanager
   docker cp jobmanager:/opt/flink/conf /usr/local/flink-docker/jobmanager
   
   docker cp taskmanager:/opt/flink/lib /usr/local/flink-docker/taskmanager
   docker cp taskmanager:/opt/flink/log /usr/local/flink-docker/taskmanager
   docker cp taskmanager:/opt/flink/conf /usr/local/flink-docker/taskmanager
   

3. 搭建集群

   version: "2.2"
   services:
     jobmanager:
       image: flink:scala_2.12-java8
       container_name: jobmanager-1
       expose:
         - "6123"
       ports:
         - "8081:8081"
       command: jobmanager
       volumes:
         - /usr/local/flink-docker/jobmanager/conf:/opt/flink/conf
         - /usr/local/flink-docker/jobmanager/lib:/opt/flink/lib
         - /usr/local/flink-docker/jobmanager/log:/opt/flink/log
       environment:
         - |
           FLINK_PROPERTIES=
           jobmanager.rpc.address: jobmanager
           parallelism.default: 2
             #web.upload.dir: /opt/flink/target
       networks:
         - flink-network
     taskmanager:
       image: flink:scala_2.12-java8
       container_name: taskmanager-1
       depends_on:
         - jobmanager
       command: taskmanager
       scale: 1
       volumes:
         - /usr/local/flink-docker/taskmanager/conf:/opt/flink/conf
         - /usr/local/flink-docker/taskmanager/lib:/opt/flink/lib
         - /usr/local/flink-docker/taskmanager/log:/opt/flink/log
       environment:
         - |
           FLINK_PROPERTIES=
           jobmanager.rpc.address: jobmanager
           taskmanager.numberOfTaskSlots: 2
           parallelism.default: 2
       networks:
         - flink-network
   networks:
     flink-network:
       external: true