Etcd 集群搭建

1、集群

主机名 ip 配置 etcd-01 192.168.1.61 8C/16G/100G etcd-02 192.168.1.62  8C/16G/100G etcd-03 192.168.1.63  8C/16G/100G

2、配置etcd集群

1> hosts文件

[root@k01 ~]# cat /etc/hosts

127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 192.168.1.61 k01 192.168.1.62 k02 192.168.1.63 k03 192.168.1.71 node01 192.168.1.72 node02

2> 创建 ca证书

mkdir -pv etcdssl
cd etcdssl

创建ca-config、ca-csr 

cat ca-config.json << EOF { "signing": { "default": { "expiry": "87600h" }, "profiles": { "www": { "expiry": "87600h", "usages": [ "signing", "key encipherment", "server auth", "client auth" ] } } } } EOF cat ca-csr.json << EOF { "CN": "etcd CA", "key": { "algo": "rsa", "size": 2048 }, "names": [ { "C": "CN", "L": "Beijing", "ST": "Beijing" } ] } EOF

生成证书

[root@k01 etcdssl]# cfssl gencert -initca ca-csr.json | cfssljson -bare ca - 2021/08/20 21:05:48 [INFO] generating a new CA key and certificate from CSR 2021/08/20 21:05:48 [INFO] generate received request 2021/08/20 21:05:48 [INFO] received CSR 2021/08/20 21:05:48 [INFO] generating key: rsa-2048 2021/08/20 21:05:48 [INFO] encoded CSR 2021/08/20 21:05:48 [INFO] signed certificate with serial number 437xxxxxxxxxxxxxx394879734494661358352

查看

[root@k01 etcdssl]# ls ca-config.json ca.csr ca-csr.json ca-key.pem ca.pem

3> 使用ca签发etcd证书

创建etcd证书签名请求文件

[root@k01 etcdssl]# cat server-csr.json { "CN": "etcd", "hosts": [ "192.168.1.61", "192.168.1.62", "192.168.1.63", "192.168.1.64" ], "key": { "algo": "rsa", "size": 2048 }, "names": [ { "C": "CN", "L": "BeiJing", "ST": "BeiJing" } ] }

生成证书

[root@k01 etcdssl]# cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=www server-csr.json | cfssljson -bare server 2021/08/20 21:07:05 [INFO] generate received request 2021/08/20 21:07:05 [INFO] received CSR 2021/08/20 21:07:05 [INFO] generating key: rsa-2048 2021/08/20 21:07:06 [INFO] encoded CSR 2021/08/20 21:07:06 [INFO] signed certificate with serial number 5579849624423xxxxxxxxxxxxxxxxx91627608515054481 2021/08/20 21:07:06 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for websites. For more information see the Baseline Requirements for the Issuance and Management of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org); specifically, section 10.2.3 ("Information Requirements").

证书分发至其它节点

创建目录

[root@k01 etcdssl]# ansible etcd -m shell -a "mkdir -p /opt/etcd/ssl"

创建分发脚本

vim scpetcdssl.sh #！/bin/bash for node in k01 k02 k03 do ssh root@${node} "mkdir -pv /opt/etcd/ssl" scp ./ca*.pem root@${node}:/opt/etcd/ssl scp ./server*.pem root@${node}:/opt/etcd/ssl done

三、部署Etcd

解压创建配置文件

tar xzvf etcd-v3.4.16-linux-amd64.tar.gz cp -a etcd-v3.4.16-linux-amd64/etcd* /usr/local/bin/ ansible etcd -m shell -a "mkdir -pv /opt/etcd/cfg" cat >/opt/etcd/cfg/etcd.conf <<EOF #[Member] ETCD_NAME="etcd-01" ETCD_DATA_DIR="/var/lib/etcd/default.etcd" ETCD_LISTEN_PEER_URLS="https://192.168.1.61:2380" ETCD_LISTEN_CLIENT_URLS="https://192.168.1.61:2379,http://127.0.0.1:2379" #[Clustering] ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.1.61:2380" ETCD_ADVERTISE_CLIENT_URLS="https://192.168.1.61:2379" ETCD_INITIAL_CLUSTER="etcd-01=https://192.168.1.61:2380,etcd-02=https://192.168.1.62:2380,etcd-03=https://192.168.1.63:2380" ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster" ETCD_INITIAL_CLUSTER_STATE="new" EOF [root@k01 ~]# cat /usr/lib/systemd/system/etcd.service << EOF [Unit] Description=Etcd Server After=network.target After=network-online.target Wants=network-online.target [Service] Type=notify EnvironmentFile=/opt/etcd/cfg/etcd.conf ExecStart=/usr/local/bin/etcd \ --cert-file=/opt/etcd/ssl/server.pem \ --key-file=/opt/etcd/ssl/server-key.pem \ --peer-cert-file=/opt/etcd/ssl/server.pem \ --peer-key-file=/opt/etcd/ssl/server-key.pem \ --trusted-ca-file=/opt/etcd/ssl/ca.pem \ --peer-trusted-ca-file=/opt/etcd/ssl/ca.pem \ --logger=zap Restart=on-failure LimitNOFILE=65536 [Install] WantedBy=multi-user.target EOF

注：
ETCD_NAME：节点名称，集群中唯一
ETCD_DATA_DIR：数据目录
ETCD_LISTEN_PEER_URLS：集群通信监听地址
ETCD_LISTEN_CLIENT_URLS：客户端访问监听地址
ETCD_INITIAL_ADVERTISE_PEER_URLS：集群通告地址
ETCD_ADVERTISE_CLIENT_URLS：客户端通告地址
ETCD_INITIAL_CLUSTER：集群节点地址
ETCD_INITIAL_CLUSTER_TOKEN：集群Token
ETCD_INITIAL_CLUSTER_STATE：加入集群的当前状态，new是新集群，existing表示加入已有集群

启动服务

systemctl start etcd.service systemctl enable etcd.service [root@k01 ~]# systemctl status etcd.service ● etcd.service - Etcd Server Loaded: loaded (/usr/lib/systemd/system/etcd.service; enabled; vendor preset: disabled) Active: active (running) since Sun 2021-08-22 18:26:28 CST; 5h 11min ago Main PID: 17312 (etcd) Tasks: 15 Memory: 54.3M CGroup: /system.slice/etcd.service └─17312 /usr/local/bin/etcd --cert-file=/opt/etcd/ssl/server.pem --key-file=/opt/etcd/ssl/server-key.pem --peer-cert-file=/opt/etcd/ssl/server.pem --peer-key-file=/opt/etcd/ssl/server-key.pem --tr... Aug 22 23:36:04 k01 etcd[17312]: {"level":"warn","ts":"2021-08-22T23:36:04.194+0800","caller":"rafthttp/probing_status.go:86","msg":"prober found high clock drift","round-tripper-name":"ROUND_...":"30.14026ms"}

健康状态检查

[root@k01 ~]# etcdctl --cacert=/opt/etcd/ssl/ca.pem --cert=/opt/etcd/ssl/server.pem --key=/opt/etcd/ssl/server-key.pem --endpoints="https://192.168.1.61:2379,https://192.168.1.62:2379,https://192.168.1.63:2379" endpoint health https://192.168.1.61:2379 is healthy: successfully committed proposal: took = 57.708307ms https://192.168.1.62:2379 is healthy: successfully committed proposal: took = 59.868422ms https://192.168.1.63:2379 is healthy: successfully committed proposal: took = 94.302948ms