ES集群7.3.2版本在线扩容Data节点

背景

接到生产业务需求，需要在线扩容ES集群且不能影响数据安全性，经过对Elasticsearch角色的分析，发现直接扩容Data节点最简单而且能满足需求。备注：原来ES集群3节点dim角色默认都开启了。

原节点信息

[root@188_33_centos ~]# curl 192.168.188.33:9200/_cat/nodes 192.168.188.33 51 98 3 0.23 0.20 0.17 dim - es-33 192.168.188.39 33 97 2 0.01 0.06 0.11 dim * es-39 192.168.188.40 24 97 2 0.30 0.16 0.15 dim - es-40 [root@188_33_centos ~]# curl 192.168.188.33:9200 { "name" : "es-33", "cluster_name" : "escluster", "cluster_uuid" : "mc5KtwBYSh-OHX5VdW3D8g", "version" : { "number" : "7.3.2", "build_flavor" : "default", "build_type" : "rpm", "build_hash" : "1c1faf1", "build_date" : "2019-09-06T14:40:30.409026Z", "build_snapshot" : false, "lucene_version" : "8.1.0", "minimum_wire_compatibility_version" : "6.8.0", "minimum_index_compatibility_version" : "6.0.0-beta1" }, "tagline" : "You Know, for Search" }

在线扩容节点

备注：分别在新增节点es-87与es-135上执行如下操作。

1.设置基本依赖Java环境

[root@188-87-centos ~]# java -version java version "1.8.0_171" Java(TM) SE Runtime Environment (build 1.8.0_171-b11) Java HotSpot(TM) 64-Bit Server VM (build 25.171-b11, mixed mode)

2.调整hosts信息

[root@188-87-centos ~]# cat /etc/hosts 192.168.188.33 es-33 192.168.188.39 es-39 192.168.188.40 es-40 192.168.188.87 es-87 192.168.188.135 es-135

3.安装elasticsearch

[root@188-87-centos ~]# rpm -ivh elasticsearch-7.3.2-x86_64.rpm warning: elasticsearch-7.3.2-x86_64.rpm: Header V4 RSA/SHA512 Signature, key ID d88e42b4: NOKEY Preparing... ################################# [100%] Creating elasticsearch group... OK Creating elasticsearch user... OK Updating / installing... 1:elasticsearch-0:7.3.2-1 ################################# [100%] ### NOT starting on installation, please execute the following statements to configure elasticsearch service to start automatically using systemd sudo systemctl daemon-reload sudo systemctl enable elasticsearch.service ### You can start elasticsearch service by executing sudo systemctl start elasticsearch.service future versions of Elasticsearch will require Java 11; your Java version from [/usr/local/jdk/jre] does not meet this requirement Created elasticsearch keystore in /etc/elasticsearch

4.修改Node节点配置文件

参考之前节点配置文件，这里只需要修改node.name、network.host即可，其他保持不变。

[root@188-87-centos elasticsearch]# cat elasticsearch.yml cluster.name: escluster node.name: es-87 node.master: false node.data: true path.data: /data1/esdata path.logs: /data1/eslog network.host: 192.168.188.87 http.port: 9200 discovery.zen.ping.unicast.hosts: ["192.168.188.33:9300","192.168.188.39:9300", "192.168.188.40:9300"] cluster.initial_master_nodes: ["es-33","es-39","es-40"] http.cors.enabled: true http.cors.allow-origin: "*"

5.设置目录权限

[root@188_87_centos elasticsearch]# mkdir /data1/{esdata,eslog} [root@188_87_centos elasticsearch]# chown elasticsearch:elasticsearch /data1/esdata [root@188_87_centos elasticsearch]# chown elasticsearch:elasticsearch /data1/eslog [root@188_87_centos elasticsearch]# ls -lh /data1/ total 8.0K drwxr-xr-x 2 elasticsearch elasticsearch 4.0K Jun 5 10:22 esdata drwxr-xr-x 2 elasticsearch elasticsearch 4.0K Jun 5 10:22 eslog

6.启动Node节点

[root@188-87-centos elasticsearch]# systemctl enable elasticsearch.service Created symlink from /etc/systemd/system/multi-user.target.wants/elasticsearch.service to /usr/lib/systemd/system/elasticsearch.service. [root@188-87-centos elasticsearch]# systemctl start elasticsearch [root@188-87-centos elasticsearch]# systemctl status elasticsearch ● elasticsearch.service - Elasticsearch Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; enabled; vendor preset: disabled) Active: active (running) since Fri 2020-06-05 10:25:13 CST; 5s ago Docs: http://www.elastic.co Main PID: 17417 (java) CGroup: /system.slice/elasticsearch.service ├─17417 /usr/share/elasticsearch/jdk/bin/java -Xms3g -Xmx3g -XX:+UseConcMarkSweepG... └─17513 /usr/share/elasticsearch/modules/x-pack-ml/platform/linux-x86_64/bin/contr... Jun 05 10:25:13 188-87-centos systemd[1]: Started Elasticsearch. Jun 05 10:25:13 188-87-centos systemd[1]: Starting Elasticsearch... Jun 05 10:25:14 188-87-centos elasticsearch[17417]: OpenJDK 64-Bit Server VM warning: Opti...e. Hint: Some lines were ellipsized, use -l to show in full.

7.检查集群情况

[root@188_33_centos elasticsearch]# curl 192.168.188.33:9200/_cat/nodes 192.168.188.40 23 98 4 0.40 0.19 0.15 dim - es-40 192.168.188.87 9 98 5 0.89 0.44 0.20 di - es-87 192.168.188.33 57 98 7 0.17 0.22 0.21 dim - es-33 192.168.188.39 31 98 3 0.29 0.17 0.15 dim * es-39 192.168.188.135 9 63 0 0.27 0.10 0.07 di - es-135 [root@188_33_centos elasticsearch]# curl 192.168.188.33:9200/_cluster/health?pretty { "cluster_name" : "escluster", "status" : "green", "timed_out" : false, "number_of_nodes" : 5, "number_of_data_nodes" : 5, "active_primary_shards" : 45, "active_shards" : 90, "relocating_shards" : 2, "initializing_shards" : 0, "unassigned_shards" : 0, "delayed_unassigned_shards" : 0, "number_of_pending_tasks" : 0, "number_of_in_flight_fetch" : 0, "task_max_waiting_in_queue_millis" : 0, "active_shards_percent_as_number" : 100.0 }

总结

弄清Elasticsearch节点类型，这里主要用到Master节点、 Data节点、 Ingest节点。主节点负责管理整个集群。它管理所有节点的状态，并周期性地将集群状态同步到集群中的所有其他节点，通知大家有什么新节点加入了集群，有什么节点脱离了集群。主节点会定期向所有其他节点发送ping消息，以此判断它们是否正常存活（别的节点也会向主节点发送ping消息）。主节点的重要任务之一是配置管理。它管理着全部元数据，以及集群中所有索引的映射。数据节点负责保存数据、段合并和执行查询。数据节点是集群中真正承担工作任务的地方，因此服务器的配置应该比集群中的其他节点高。数据处理管道由一到多个ingest节点组成，由ingest节点负责每个环节的处理。依ingest节点要处理的任务不同，它们可能会需要很多资源，因此有时候需要在集群中指定专用的ingest节点。