当前位置 : 主页 > 网络编程 > PHP >

报错解决:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.yarn.exceptions.InvalidResourceRequestE

来源:互联网 收集:自由互联 发布时间:2023-09-07
Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.yarn.exceptions.InvalidResourceRequestException): Invalid resource request! Cannot allocate containers as requested resource is greater than maximum allowed allocation. Requ


Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.yarn.exceptions.InvalidResourceRequestException): Invalid resource request! Cannot allocate containers as requested resource is greater than maximum allowed allocation. Requested resource type=[memory-mb], Requested resource=<memory:1536, vCores:1>, maximum allowed allocation=<memory:1024, vCores:2>, please note that maximum allowed allocation is calculated by scheduler based on maximum resource of registered NodeManagers, which might be less than configured maximum allocation=<memory:8192, vCores:4>

给NodeManager配置的内存是1024MB,而默认MapReduce作业所需的最小内存是1536,所以会报出“Invalid resource request! Cannot allocate containers as requested resource is greater than maximum allowed allocation.”的错误!

解决:设置任务的MapReduce作业所需的最小内存设置为1024M。

在 ​​mapred-site.xml​​ 配置文件中添加配置即可:

<property>
<name>yarn.app.mapreduce.am.resource.mb</name>
<value>1024</value>
</property>
<property>
<name>yarn.app.mapreduce.am.resource.cpu-vcores</name>
<value>1</value>
</property>


【文章转自台湾大带宽服务器 http://www.558idc.com/tw.html提供,感恩】
网友评论