Блог

моя работа для flink на yarn ,часто закрывается,

журналы(часть)

 2021-08-20 14:36:28,996 WARN org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: Exit code from container container_e04_1629116893859_0031_01_000001 is : 163
2021-08-20 14:36:28,996 WARN org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: Exception from container-launch with container ID: container_e04_1629116893859_0031_01_000001 and exit code: 163
ExitCodeException exitCode=163: 
    at org.apache.hadoop.util.Shell.runCommand(Shell.java:585)
    at org.apache.hadoop.util.Shell.run(Shell.java:482)
    at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:776)
    at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:212)
    at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
    at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
2021-08-20 14:36:28,997 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: Exception from container-launch.
2021-08-20 14:36:28,997 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: Container id: container_e04_1629116893859_0031_01_000001
2021-08-20 14:36:28,997 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: Exit code: 163
2021-08-20 14:36:28,997 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: Stack trace: ExitCodeException exitCode=163: 
2021-08-20 14:36:28,997 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:   at org.apache.hadoop.util.Shell.runCommand(Shell.java:585)
2021-08-20 14:36:28,997 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:   at org.apache.hadoop.util.Shell.run(Shell.java:482)
2021-08-20 14:36:28,997 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:   at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:776)
2021-08-20 14:36:28,997 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:   at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:212)
2021-08-20 14:36:28,997 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:   at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
2021-08-20 14:36:28,997 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:   at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
2021-08-20 14:36:28,997 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
2021-08-20 14:36:28,997 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
2021-08-20 14:36:28,997 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
2021-08-20 14:36:28,997 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:   at java.lang.Thread.run(Thread.java:748)
2021-08-20 14:36:28,997 WARN org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch: Container exited with a non-zero exit code 163
2021-08-20 14:36:28,997 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl: Container container_e04_1629116893859_0031_01_000001 transitioned from RUNNING to EXITED_WITH_FAILURE
2021-08-20 14:36:28,997 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch: Cleaning up container container_e04_1629116893859_0031_01_000001
 

мой yarn-site.xml (часть)

          <!-- 关闭yarn内存检查 -->
         <property>
                 <name>yarn.nodemanager.pmem-check-enabled</name>
                 <value>false</value>
         </property>

         <property>
                 <name>yarn.nodemanager.vmem-check-enabled</name>
                 <value>false</value>
         </property>
        <property>
                <name>yarn.nodemanager.vmem-pmem-ratio</name>
                <value>5</value>
        </property>

        <property>
                <name>yarn.scheduler.minimum-allocation-mb</name>
                <value>1024</value>
        </property>

                <property>
                <name>yarn.scheduler.maximum-allocation-mb</name>
                <value>4096</value>
        </property>

                <property>
                <name>yarn.scheduler.maximun-allocation-vcores</name>
                <value>4</value>
        </property>

                <property>
                <name>yarn.scheduler.minimun-allocation-vcores</name>
                <value>1</value>
        </property>

 

more details:
the flink on yarn job is often shutdown , i dont known ,if i have new logs information ,i will push at once, if know please help me!
If you have the corresponding flink task startup parameters, such as -tm -jm- -s -p and other reasonable configuration, you can tell me, or the yarn-site.xml configuration, etc. can let me refer to it, thank you