节点规划表:
新项目开始前先把原来的进行备份
搭建步骤:
1.(Hadoop01和Hadoop02相互免秘钥)
要想对别人免秘钥,上面三步是前提,必须要做
做完后就可以把公钥给hadoop01分发
在hadoop01中追加
验证免秘钥是否成功
2.这个时候统一配置hadoop02 hadoop03 hadoop04的环境变量:
(1)配置hdfs-site.xml
dfs.nameservices
mycluster
dfs.ha.namenodes.mycluster
nn1,nn2
dfs.namenode.rpc-address.mycluster.nn1
hadoop01:8020
dfs.namenode.rpc-address.mycluster.nn2
hadoop02:8020
dfs.namenode.http-address.mycluster.nn1
hadoop01:50070
dfs.namenode.http-address.mycluster.nn2
hadoop02:50070
dfs.namenode.shared.edits.dir
qjournal://hadoop01:8485;hadoop02:8485;hadoop03:8485/mycluster
dfs.journalnode.edits.dir
/var/hadoop/ha/jn
dfs.client.failover.proxy.provider.mycluster
org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider
dfs.ha.fencing.methods
sshfence
dfs.ha.fencing.ssh.private-key-files
/root/.ssh/id_dsa
dfs.ha.automatic-failover.enabled
true
(2)配置core-site.xml
fs.defaultFS
hdfs://mycluster
hadoop.tmp.dir
/var/hadoop/zk
ha.zookeeper.quorum
hadoop02:2181,hadoop03:2181,hadoop04:2181
(3)修改完这两个文件然后分发给hadoop02-04
3.hadoop02-04安装并启动zookeeper
(1)将解压后的文件移动
(2)配置环境变量:
(3)配置好后,需要修改zoo_sample.cfg文件,先备份。
(4)配置zoo.cfg文件:
默认的保存目录不安全所以修改:
配置三台zookeeper节点
(5)给zookeeper的myid设置不同的数值
(6)把部署好的zookeeper进行对hadoop03-04两个zookeeper节点分发
(7)对hadoop03-04设置myid
(8)hadoop03-hadoop04环境变量配置
重新加载
(9)启动hadoop02zookeeper
(10)再启动hadoop03和hadoop04,发现hadoop03是leader而hadoop02-04是follower
4.启动JN(hadoop01-03)
使用jps命令查看
可以看到JournalNode进程已启动
5.第一台NN:
hdfs namenode –format
hadoop-daemon.sh start namenode
另一台NN:
hdfs namenode -bootstrapStandby
hadoop01:
hadoop02:
6.格式化ZK
hdfs zkfc -formatZK
7.启动集群
hadoop01:
8.浏览器验证 hadoop01:50070
hadoop02:50070
在hadoop01节点创建目录/user/root目录
上传文件,并自定义block块为1MB
9.验证高可用
(1)杀死主NameNode
一刷新hadoop01:50070:
再刷新hadoop02:50070:
【文章原创作者:站群服务器 http://www.558idc.com/mggfzq.html 欢迎留下您的宝贵建议】