本文将为您提供关于MysqlMGR+Consul+Consul-template+Haproxy搭建mysql高可用集群的详细介绍，我们还将为您解释二的相关知识，同时，我们还将为您提供关于032：基于
本文将为您提供关于Mysql MGR + Consul + Consul-template + Haproxy 搭建mysql 高可用集群 的详细介绍，我们还将为您解释二的相关知识，同时，我们还将为您提供关于032：基于 Consul 和 MGR 的 MySQL 高可用架构、Consul Template、consul-template + nginx部署高可用负载均衡、consul命令行查看服务_Consul集群高可用测试的实用信息。
本文目录一览：- Mysql MGR + Consul + Consul-template + Haproxy 搭建mysql 高可用集群 （二）（mysql高可用集群怎么搭建）
- 032：基于 Consul 和 MGR 的 MySQL 高可用架构
- Consul Template
- consul-template + nginx部署高可用负载均衡
- consul命令行查看服务_Consul集群高可用测试
Mysql MGR + Consul + Consul-template + Haproxy 搭建mysql 高可用集群 （二）（mysql高可用集群怎么搭建）
Mysql MGR 的搭建：
下载新版的mysql 8.0.13
wget https://dev.mysql.com/get/Downloads/MySQL-8.0/mysql-8.0.13-el7-x86_64.tar
使用uuidgen命令生成个uuid：
uuidgen
cc5e2627-2285-451f-86e6-0be21581539f
三台物理机部署：
修改hosts文件：
vi /etc/hosts
172.16.0.90 swarm1
172.16.0.76 swarm2
172.16.0.175 swarm3
创建工作目录：
mkdir /data/mysql_root/base_phony/40010
mkdir /data/mysql_root/data/40010
#将解压后的mysql软体放到/data/mysql_root/base_phony/40010下
chown -R mysql:mysql /data/mysql_root/base_phony/40010
chown -R mysql:mysql /data/mysql_root/data/40010
创建my.cnf文件：已swarm1的为例
（swarm1的server-id为1，swarm1的server-id为2，swarm1的server-id为3；group_replication_local_address修改为本机hostname或ip）
cd /data/mysql_root/data/40010
vi my.cnf
[client]
port = 40010
socket = /tmp/mysql_40010.sock
[mysqld]
port = 40010
socket = /tmp/mysql_40010.sock
datadir = /data/mysql_root/data/40010
slow_query_log_file = /data1/mysql_root/data/40010/slow_query.log
log-error = /data/mysql_root/data/40010/error.log
server-id = 1
#for MGR
gtid_mode=on
enforce-gtid-consistency=on
binlog_checksum=NONE
loose-group_replication_ip_whitelist="swarm1,swarm2,swarm3"
loose-group_replication_group_name="cc5e2627-2285-451f-86e6-0be21581539f"
loose-group_replication_local_address="swarm1:2525"
loose-group_replication_group_seeds="swarm1:2525,swarm22525,swarm3:2525"参照官方文档，mysql8中一些默认参数不用为MGR修改。
初始化mysql：
cd /data/mysql_root/base_phony/40010/bin/
./mysqld --defaults-file=/data/mysql_root/data/40010/my.cnf --basedir=/data/mysql_root/base_phony/40010 --initialize-insecure --user=mysql启动mysql：
nohup /data/mysql_root/base_phony/40010/bin/mysqld --defaults-file=/data/mysql_root/data/40010/my.cnf --basedir=/data/mysql_root/base_phony/40010 --user=mysql &安装MGR插件：
mysql -uroot -S /home/mysql/mysql40010/mysql40010.sock
mysql> install plugin group_replication soname ''group_replication.so'';
至此，三台MGR实例初始化完毕，接下来配置启动MGR
登录swarm1的mysql：
set sql_log_bin=0;
CREATE USER repl@''%'' IDENTIFIED WITH ''mysql_native_password'' BY ''repl'';
GRANT REPLICATION SLAVE ON *.* TO ''repl''@''%'';
FLUSH PRIVILEGES;
set sql_log_bin=1;
CHANGE MASTER TO MASTER_USER=''repl'',MASTER_PASSWORD=''repl'' FOR CHANNEL ''group_replication_recovery'';
# group_replication_bootstrap_group参数仅在第一台启动的MGR实例上开启，其他实例不要操作
set global group_replication_bootstrap_group=ON;
START GROUP_REPLICATION;
set global group_replication_bootstrap_group=OFF;
SELECT * FROM performance_schema.replication_group_members;登录swarm2、swarm3的mysql：
set sql_log_bin=0;
CREATE USER repl@''%'' IDENTIFIED WITH ''mysql_native_password'' BY ''repl'';
GRANT REPLICATION SLAVE ON *.* TO ''repl''@''%'';
FLUSH PRIVILEGES;
set sql_log_bin=1;
CHANGE MASTER TO MASTER_USER=''repl'',MASTER_PASSWORD=''repl'' FOR CHANNEL ''group_replication_recovery'';
START GROUP_REPLICATION;
SELECT * FROM performance_schema.replication_group_members;至此，一个三节点的MGR集群搭建完毕，可以实现MGR的特性了。
+---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+
| CHANNEL_NAME | MEMBER_ID | MEMBER_HOST | MEMBER_PORT | MEMBER_STATE | MEMBER_ROLE | MEMBER_VERSION |
+---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+
| group_replication_applier | bbdb8668-1866-11e9-89a5-525400447342 | swarm1 | 40010 | ONLINE | PRIMARY | 8.0.13 |
| group_replication_applier | c433a5af-1866-11e9-96b6-525400cd1c31 | swarm2 | 40010 | ONLINE | SECONDARY | 8.0.13 |
| group_replication_applier | eabb341d-1895-11e9-90dd-525400dd19e5 | swarm3 | 40010 | ONLINE | SECONDARY | 8.0.13 |
+---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+
遇到的问题：
发现节点重启后经常性的无法加入集群，检查报错日志如下：
2019-01-16T15:38:02.952875Z 651 [ERROR] [MY-010584] [Repl] Slave I/O for channel ''group_replication_recovery'': error connecting to master ''repl@swarm3:40010'' - retry-time: 60 retries: 1, Error_code: MY-002061反复检查复制用户的账号密码，未发现错误，开始使用的创建语句是：
CREATE USER repl@''%'' IDENTIFIED BY ''repl'';修改为：
CREATE USER repl@''%'' IDENTIFIED WITH ''mysql_native_password'' BY ''repl'';此后节点间间接未发生问题，应该是和mysql8修改了默认加密方式有关。
032：基于 Consul 和 MGR 的 MySQL 高可用架构
032：基于 Consul 和 MGR 的 MySQL 高可用架构
[TOC]
一、Consul
1、Consul 简介
Consul 是 HashiCorp 公司推出的一款开源工具， 基于 go 语言开发，用于实现分布式系统的服务发现与配置。
官网： https://www.consul.io/
Consul 的优势：
- 多数据中心服务发现（DNS + HTTP）
- 支持健康检查.
- 单数据中心微服务
- 内置 Web UI，用于编辑 K/V 和查看健康检查状态
- 支持热配置，不用重启服务
- 支持编排任务
基础架构： 
-
Consul Cluster 由部署和运行了
Consul Agent的节点组成。 在 Cluster 中有两种角色:Server和Client。 -
Server 和 Client 的角色和 Consul Cluster 上运行的应用服务无关，是基于 Consul 层面的一种角色划分.
-
Consul Server: 用于维护 Consul Cluster 的状态信息， 实现数据一致性， 响应 RPC 请求。
官方建议是：至少要运行 3 个或者 3 个以上的 Consul Server。 多个 server 之中需要选举一个 leader, 这个选举过程 Consul 基于 Raft 协议实现。多个 Server 节点上的 Consul 数据信息保持强一致性。 在局域网内与本地客户端通讯，通过广域网与其他数据中心通讯。
-
Consul Client: 只维护自身的状态，并将 HTTP 和 DNS 接口请求转发给服务端。
-
Consul 支持多数据中心， 多个数据中心要求每个数据中心都要安装一组 Consul cluster，多个数据中心间基于 gossip protocol 协议来通讯， 使用
Raft算法实现一致性
2、准备环境
| hosts | ip | software | consul.json | service_json_01 | service_json_02 | script01 | script02 |
|---|---|---|---|---|---|---|---|
| node01 | 192.168.222.171 | mysql | Client.json | mysql_mgr_write_3306.json | mysql_mgr_read_3306.json | check_mysql_mgr_master.sh | check_mysql_mgr_slave.sh |
| node02 | 192.168.222.172 | mysql | Client.json | mysql_mgr_write_3306.json | mysql_mgr_read_3306.json | check_mysql_mgr_master.sh | check_mysql_mgr_slave.sh |
| node03 | 192.168.222.173 | mysql | Client.json | mysql_mgr_write_3306.json | mysql_mgr_read_3306.json | check_mysql_mgr_master.sh | check_mysql_mgr_slave.sh |
| node04 | 192.168.222.174 | mysql | Client.json | mysql_mgr_write_3306.json | mysql_mgr_read_3306.json | check_mysql_mgr_master.sh | check_mysql_mgr_slave.sh |
| node05 | 192.168.222.175 | mysql | |||||
| node06 | 192.168.222.176 | mysql | |||||
| node09 | 192.168.222.179 | server.json | |||||
| node10 | 192.168.222.180 | server.json | |||||
| node11 | 192.168.222.181 | server.json | |||||
| node12 | 192.168.222.182 | porxysql | Client.json | proxysql.json | proxysql_check.sh | ||
| audit01 | 192.168.222.199 | porxysql | Client.json | proxysql.json | proxysql_check.sh |
- node05、node06 部署成单机多实例 MGR，端口分别（33061~33065）
3、Consul 安装
这里采用 ansible 安装，不累述了
- 步骤如下：
--每台都安装consul
cd /software
wget https://releases.hashicorp.com/consul/1.4.0/consul_1.4.0_linux_amd64.zip
unzip /software/consul_1.4.0_linux_amd64.zip
cp /software/consul /usr/local/bin/
consul --version
--所有主机都创建目录/etc/consul.d/scripts/和/data/consul/，分别是放配置文件、存放数据的、mysql的健康检查脚本。
mkdir -p /etc/consul.d/scripts && mkdir -p /data/consul/
- 目录结构
[root@node12 consul.d]# tree /etc/consul.d
/etc/consul.d
├── client.json
├── proxysql.json
└── scripts
└── proxysql_check.sh
1 directory, 3 files
[root@node12 consul.d]# tree /data/consul/ --初始/data/consul/是空目录
/data/consul/
├── checkpoint-signature
├── consul.log
├── node-id
├── proxy
│ └── snapshot.json
├── serf
│ └── local.snapshot
└── services
3 directories, 5 files
[root@node12 consul.d]#
4、Consul 配置文件
-
consul server 配置文件
advertise_addr（主机 IP）和node_name(主机名),（node09、node10、node11）三个主机修改对应信息"domain":"gczheng", 所以后缀是 servers.gczheng
[root@node09 ~]# cat /etc/consul.d/server.json
{
"addresses":{
"http":"0.0.0.0",
"dns":"0.0.0.0"
},
"bind_addr":"0.0.0.0",
"advertise_addr":"192.168.222.179",
"bootstrap_expect":3,
"datacenter":"dc1",
"data_dir":"/data/consul",
"dns_config":{
"allow_stale":true,
"max_stale":"87600h",
"node_ttl":"0s",
"service_ttl":{
"*":"0s"
}
},
"domain":"gczheng",
"enable_syslog":false,
"leave_on_terminate":false,
"log_level":"info",
"node_name":"node09.test.com",
"node_meta":{
"location":"gczheng"
},
"performance":{
"raft_multiplier":1
},
"ports":{
"http":8500,
"dns":53
},
"reconnect_timeout":"72h",
"recursors":[
"192.168.222.179",
"192.168.222.180",
"192.168.222.181"
],
"retry_join":[
"192.168.222.179",
"192.168.222.180",
"192.168.222.181"
],
"retry_interval":"10s",
"server":true,
"skip_leave_on_interrupt":true,
"ui":true
}
[root@node09 ~]#
-
consul client 配置文件
bind_addr修改为成对应的主机 ip (node01、node02、node03、node04、node05、node06、node12、audit01)
[root@node01 consul.d]# cat /etc/consul.d/client.json
{
"data_dir": "/data/consul",
"enable_script_checks": true,
"bind_addr": "192.168.222.171",
"retry_join": ["192.168.222.179"],
"retry_interval": "30s",
"rejoin_after_leave": true,
"start_join": ["192.168.222.179"],
"datacenter": "dc1"
}
- proxysql 配置文件
[root@audit01 consul.d]# cat proxysql.json
{
"services":
[{
"id": "proxy1",
"name": "proxysql",
"address": "",
"tags": ["mysql_proxy"],
"port": 6033,
"check": {
"args": ["/etc/consul.d/scripts/proxysql_check.sh", "192.168.222.199"],
"interval": "5s"
},
"id": "proxy2",
"name": "proxysql",
"address": "",
"tags": ["mysql_proxy"],
"port": 6033,
"check":
{
"args": ["/etc/consul.d/scripts/proxysql_check.sh", "192.168.222.182"],
"interval": "5s"
}
}]
}
- mysql_mgr_write_3306.json
[root@node02 consul.d]# cat mysql_mgr_write_3306.json
{
"services": [{
"id": "mysql_write",
"name": "mysql_3306_w",
"address": "",
"port": 3306,
"enable_tag_override": false,
"checks": [{
"id": "mysql_w_c_01",
"name": "MySQL Write Check",
"args": ["/etc/consul.d/scripts/check_mysql_mgr_master.sh", "3306"],
"interval": "15s",
"timeout": "1s",
"service_id": "mysql_w"
}]
}]
}
[root@node02 consul.d]#
- mysql_mgr_read_3306.json
[root@node02 consul.d]# cat mysql_mgr_read_3306.json
{
"services": [{
"id": "mysql_read",
"name": "mysql_3306_r",
"address": "",
"port": 3306,
"enable_tag_override": false,
"checks": [{
"id": "mysql_r_c_01",
"name": "MySQL read Check",
"args": ["/etc/consul.d/scripts/check_mysql_mgr_slave.sh", "3306"],
"interval": "15s",
"timeout": "1s",
"service_id": "mysql_r"
}]
}]
}
5、Consul 服务检查脚本
- check_mysql_mgr_master.sh
#!/bin/bash
port=$1
user="gcdb"
passwod="iforgot"
comm="/usr/local/mysql/bin/mysql -u$user -h127.0.0.1 -P $port -p$passwod"
value=`$comm -Nse "select 1"`
primary_member=`$comm -Nse "select variable_value from performance_schema.global_status WHERE VARIABLE_NAME= ''group_replication_primary_member''"`
server_uuid=`$comm -Nse "select variable_value from performance_schema.global_variables where VARIABLE_NAME=''server_uuid'';"`
# 判断MySQL是否存活
if [ -z $value ]
then
echo "mysql $port is down....."
exit 2
fi
# 判断节点状态,是否存活
node_state=`$comm -Nse "select MEMBER_STATE from performance_schema.replication_group_members where MEMBER_ID=''$server_uuid''"`
if [ $node_state != "ONLINE" ]
then
echo "MySQL $port state is not online...."
exit 2
fi
# 先判断MGR模式，在判断是否主节点
if [ -z "$primary_member" ]
then
echo "primary_member is empty,MGR is multi-primary mode"
exit 0
else
if [[ $server_uuid == $primary_member ]]
then
echo "MySQL $port Instance is master ........"
exit 0
else
echo "MySQL $port Instance is slave ........"
exit 2
fi
fi
# 判断是不是主节点
if [[ $server_uuid == $primary_member ]]
then
echo "MySQL $port Instance is master ........"
exit 0
else
echo "MySQL $port Instance is slave ........"
exit 2
fi
- check_mysql_mgr_slave.sh
#!/bin/bash
port=$1
user="gcdb"
passwod="iforgot"
comm="/usr/local/mysql/bin/mysql -u$user -h127.0.0.1 -P $port -p$passwod"
value=`$comm -Nse "select 1"`
primary_member=`$comm -Nse "select variable_value from performance_schema.global_status WHERE VARIABLE_NAME= ''group_replication_primary_member''"`
server_uuid=`$comm -Nse "select variable_value from performance_schema.global_variables where VARIABLE_NAME=''server_uuid'';"`
# 判断mysql是否存活
if [ -z $value ]
then
echo "mysql $port is down....."
exit 2
fi
# 判断节点状态
node_state=`$comm -Nse "select MEMBER_STATE from performance_schema.replication_group_members where MEMBER_ID=''$server_uuid''"`
if [ $node_state != "ONLINE" ]
then
echo "MySQL $port state is not online...."
exit 2
fi
# 先判断MGR模式，再判断是否主节点
if [ -z "$primary_member" ]
then
echo "primary_member is empty,MGR is multi-primary mode"
exit 2
else
# 判断是不是主节点
if [[ $server_uuid != $primary_member ]]
then
echo "MySQL $port Instance is slave ........"
exit 0
else
node_num=`$comm -Nse "select count(*) from performance_schema.replication_group_members"`
# 判断如果没有任何从节点，主节点也注册从角色服务。
if [ $node_num -eq 1 ]
then
echo "MySQL $port Instance is slave ........"
exit 0
else
echo "MySQL $port Instance is master ........"
exit 2
fi
fi
fi
两个脚本放在 /etc/consul.d/scripts/ 路径下授权执行权限。
[root@node01 ~]# chmod 755 /etc/consul.d/scripts/*
[root@node01 ~]# ll /etc/consul.d/scripts/ |grep check_mysql_mgr
-rwxr-xr-x 1 root root 1331 Dec 4 15:05 check_mysql_mgr_master.sh
-rwxr-xr-x 1 root root 1489 Dec 4 15:05 check_mysql_mgr_slave.sh
- proxysql_check.sh
#!/bin/bash
host=$1
user="gcdb"
passwod="iforgot"
port=6033
alive_nums=`/usr/local/mysql/bin/mysqladmin ping -h$host -P$port -u$user -p$passwod |grep "mysqld is alive" |wc -l`
# 判断是不是从库
if [ $alive_nums -eq 1 ]
then
echo "$host mysqld is alive........"
exit 0
else
echo "$host mysqld is failed........"
exit 2
fi
6、Consul 启动
* consul server 启动（node09、node10、node11）、clients 也是用下面命令启动 consul
nohup consul agent -config-dir=/etc/consul.d > /data/consul/consul.log &
node09 启动 consul join
--启动三个server节点后，node09 推举为主
[root@node09 ~]# consul operator raft list-peers
Node ID Address State Voter RaftProtocol
node09.test.com fcca17ba-8ddf-ec76-4931-5f5bfa043e05 192.168.222.179:8300 leader true 3
node10.test.com c95847a4-a305-ac21-4ae5-21fd16bb39c2 192.168.222.180:8300 follower true 3
node11.test.com 8d8f6757-0ef1-79a7-d3ed-4b7e04430f88 192.168.222.181:8300 follower true 3
[root@node09 ~]#
--启动所有clients后，会自动加入集群，标记为client
[root@node09 ~]# consul members
Node Address Status Type Build Protocol DC Segment
node09.test.com 192.168.222.179:8301 alive server 1.4.0 2 dc1 <all>
node10.test.com 192.168.222.180:8301 alive server 1.4.0 2 dc1 <all>
node11.test.com 192.168.222.181:8301 alive server 1.4.0 2 dc1 <all>
node01.test.com 192.168.222.171:8301 alive client 1.4.0 2 dc1 <default>
node02.test.com 192.168.222.172:8301 alive client 1.4.0 2 dc1 <default>
node03.test.com 192.168.222.173:8301 alive client 1.4.0 2 dc1 <default>
node04.test.com 192.168.222.174:8301 alive client 1.4.0 2 dc1 <default>
node05.test.com 192.168.222.175:8301 alive client 1.4.0 2 dc1 <default>
[root@node09 ~]#
[root@node11 ~]# cat /data/consul/consul.log
bootstrap_expect > 0: expecting 3 servers
==> Starting Consul agent...
==> Consul agent running!
Version: ''v1.4.0''
Node ID: ''a1b27430-1a44-5d63-3af8-ec4a616803cc''
Node name: ''node11.test.com''
Datacenter: ''dc1'' (Segment: ''<all>'')
Server: true (Bootstrap: false)
Client Addr: [127.0.0.1] (HTTP: 8500, HTTPS: -1, gRPC: -1, DNS: 53)
Cluster Addr: 192.168.222.181 (LAN: 8301, WAN: 8302)
Encrypt: Gossip: false, TLS-Outgoing: false, TLS-Incoming: false
==> Log data will now stream in as it occurs:
2018/12/03 16:27:39 [WARN] agent: Node name "node11.test.com" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes.
2018/12/03 16:27:39 [INFO] raft: Initial configuration (index=0): []
2018/12/03 16:27:39 [INFO] raft: Node at 192.168.222.181:8300 [Follower] entering Follower state (Leader: "")
2018/12/03 16:27:39 [INFO] serf: EventMemberJoin: node11.test.com.dc1 192.168.222.181
2018/12/03 16:27:39 [INFO] serf: EventMemberJoin: node11.test.com 192.168.222.181
2018/12/03 16:27:39 [INFO] consul: Adding LAN server node11.test.com (Addr: tcp/192.168.222.181:8300) (DC: dc1)
2018/12/03 16:27:39 [INFO] agent: Started DNS server 0.0.0.0:53 (udp)
2018/12/03 16:27:39 [WARN] agent/proxy: running as root, will not start managed proxies
2018/12/03 16:27:39 [INFO] consul: Handled member-join event for server "node11.test.com.dc1" in area "wan"
2018/12/03 16:27:39 [INFO] agent: Started DNS server 0.0.0.0:53 (tcp)
2018/12/03 16:27:39 [INFO] agent: Started HTTP server on [::]:8500 (tcp)
2018/12/03 16:27:39 [INFO] agent: started state syncer
2018/12/03 16:27:39 [INFO] agent: Retry join LAN is supported for: aliyun aws azure digitalocean gce k8s os packet scaleway softlayer triton vsphere
2018/12/03 16:27:39 [INFO] agent: Joining LAN cluster...
2018/12/03 16:27:39 [INFO] agent: (LAN) joining: [192.168.222.179 192.168.222.180 192.168.222.181]
2018/12/03 16:27:39 [INFO] serf: EventMemberJoin: node10.test.com 192.168.222.180
2018/12/03 16:27:39 [WARN] memberlist: Refuting a suspect message (from: node11.test.com)
2018/12/03 16:27:39 [INFO] consul: Adding LAN server node10.test.com (Addr: tcp/192.168.222.180:8300) (DC: dc1)
- consul client 启动（node01、node02、node03、node04、node05、node06）
--主机分别执行启动
shell> nohup consul agent -config-dir=/etc/consul.d > /data/consul/consul.log &
--node01的启动过程
[root@node01 ~]# cat /data/consul/consul.log |egrep -v ERR /data/consul/consul.log |egrep -v ''ERR|WARN''
==> Starting Consul agent...
==> Joining cluster...
Join completed. Synced with 1 initial agents
==> Consul agent running!
Version: ''v1.4.0''
Node ID: ''feb4c253-f861-fbe0-10c8-50ff7bbbbf17''
Node name: ''node01.test.com''
Datacenter: ''dc1'' (Segment: '''')
Server: false (Bootstrap: false)
Client Addr: [127.0.0.1] (HTTP: 8500, HTTPS: -1, gRPC: -1, DNS: 8600)
Cluster Addr: 192.168.222.171 (LAN: 8301, WAN: 8302)
Encrypt: Gossip: false, TLS-Outgoing: false, TLS-Incoming: false
==> Log data will now stream in as it occurs:
2018/12/03 15:30:04 [INFO] serf: EventMemberJoin: node01.test.com 192.168.222.171
2018/12/03 15:30:04 [INFO] serf: Attempting re-join to previously known node: node04.test.com: 192.168.222.174:8301
2018/12/03 15:30:04 [INFO] agent: Started DNS server 127.0.0.1:8600 (udp)
2018/12/03 15:30:04 [INFO] agent: Started DNS server 127.0.0.1:8600 (tcp)
2018/12/03 15:30:04 [INFO] agent: Started HTTP server on 127.0.0.1:8500 (tcp)
2018/12/03 15:30:04 [INFO] agent: (LAN) joining: [192.168.222.179]
2018/12/03 15:30:04 [INFO] agent: Retry join LAN is supported for: aliyun aws azure digitalocean gce k8s os packet scaleway softlayer triton vsphere
2018/12/03 15:30:04 [INFO] agent: Joining LAN cluster...
2018/12/03 15:30:04 [INFO] agent: (LAN) joining: [192.168.222.179]
2018/12/03 15:30:04 [INFO] serf: EventMemberJoin: node09.test.com 192.168.222.179
2018/12/03 15:30:04 [INFO] serf: EventMemberJoin: node11.test.com 192.168.222.181
2018/12/03 15:30:04 [INFO] serf: EventMemberJoin: node10.test.com 192.168.222.180
2018/12/03 15:30:04 [INFO] serf: EventMemberJoin: node04.test.com 192.168.222.174
2018/12/03 15:30:04 [INFO] consul: adding server node09.test.com (Addr: tcp/192.168.222.179:8300) (DC: dc1)
2018/12/03 15:30:04 [INFO] consul: adding server node11.test.com (Addr: tcp/192.168.222.181:8300) (DC: dc1)
2018/12/03 15:30:04 [INFO] consul: adding server node10.test.com (Addr: tcp/192.168.222.180:8300) (DC: dc1)
2018/12/03 15:30:04 [INFO] serf: EventMemberJoin: node05.test.com 192.168.222.175
2018/12/03 15:30:04 [INFO] serf: Re-joined to previously known node: node04.test.com: 192.168.222.174:8301
2018/12/03 15:30:04 [INFO] serf: EventMemberJoin: node03.test.com 192.168.222.173
2018/12/03 15:30:04 [INFO] serf: EventMemberJoin: node02.test.com 192.168.222.172
2018/12/03 15:30:04 [INFO] agent: (LAN) joined: 1 Err: <nil>
2018/12/03 15:30:04 [INFO] agent: started state syncer
2018/12/03 15:30:04 [INFO] agent: (LAN) joined: 1 Err: <nil>
2018/12/03 15:30:04 [INFO] agent: Join LAN completed. Synced with 1 initial agents
2018/12/03 15:30:04 [INFO] agent: Synced node info
2018/12/03 15:30:05 [INFO] agent: Caught signal: hangup
2018/12/03 15:30:05 [INFO] agent: Reloading configuration...
[root@node01 ~]# consul members
Node Address Status Type Build Protocol DC Segment
node09.test.com 192.168.222.179:8301 alive server 1.4.0 2 dc1 <all>
node10.test.com 192.168.222.180:8301 alive server 1.4.0 2 dc1 <all>
node11.test.com 192.168.222.181:8301 alive server 1.4.0 2 dc1 <all>
node01.test.com 192.168.222.171:8301 alive client 1.4.0 2 dc1 <default>
node02.test.com 192.168.222.172:8301 alive client 1.4.0 2 dc1 <default>
node03.test.com 192.168.222.173:8301 alive client 1.4.0 2 dc1 <default>
node04.test.com 192.168.222.174:8301 alive client 1.4.0 2 dc1 <default>
node05.test.com 192.168.222.175:8301 alive client 1.4.0 2 dc1 <default>
[root@node01 ~]#
更新 consul 配置执行刷新
--刷新consul
[root@node01 ~]# consul reload
Configuration reload triggered
[root@node01 ~]#
二、MGR 搭建
1、MGR 配置
-
直接使用杨建荣的快速搭建 mgr 脚本，配置多实例（node05）（一主四从）
- 脚本地址：Github
-
手动搭建 MGR（node01、node02、node03、node04）（多主模式）
- 请自行搭建
-
MGR 默认账号
GRANT REPLICATION SLAVE ON *.* TO rpl_user@''%'' IDENTIFIED BY ''rpl_pass'';GRANT ALL ON *.* TO ''gcdb''@''%'' IDENTIFIED BY ''iforgot'';
2、MGR 查看
（node01、node02、node03、node04）（多主模式）MGR 启动完如下：
mysql> SELECT * FROM performance_schema.replication_group_members;
+---------------------------+--------------------------------------+-----------------+-------------+--------------+
| CHANNEL_NAME | MEMBER_ID | MEMBER_HOST | MEMBER_PORT | MEMBER_STATE |
+---------------------------+--------------------------------------+-----------------+-------------+--------------+
| group_replication_applier | 30955199-ebdc-11e8-ac1f-005056ab820e | node04.test.com | 3306 | ONLINE |
| group_replication_applier | 8770cc17-ebdb-11e8-981b-005056ab9fa5 | node01.test.com | 3306 | ONLINE |
| group_replication_applier | c38416c3-ebd4-11e8-9b2c-005056aba7b5 | node03.test.com | 3306 | ONLINE |
| group_replication_applier | e5dba457-f6d5-11e8-97ca-005056abaf95 | node02.test.com | 3306 | ONLINE |
+---------------------------+--------------------------------------+-----------------+-------------+--------------+
4 rows in set (0.00 sec)
mysql> SELECT member_id,member_host,member_port,member_state,IF (global_status.variable_name IS NOT NULL,''primary'',''secondary'') AS member_role FROM PERFORMANCE_SCHEMA.replication_group_members LEFT JOIN PERFORMANCE_SCHEMA.global_status ON global_status.variable_name=''group_replication_primary_member'' AND global_status.variable_value=replication_group_members.member_id;
+--------------------------------------+-----------------+-------------+--------------+-------------+
| member_id | member_host | member_port | member_state | member_role |
+--------------------------------------+-----------------+-------------+--------------+-------------+
| 30955199-ebdc-11e8-ac1f-005056ab820e | node04.test.com | 3306 | ONLINE | secondary |
| 8770cc17-ebdb-11e8-981b-005056ab9fa5 | node01.test.com | 3306 | ONLINE | secondary |
| c38416c3-ebd4-11e8-9b2c-005056aba7b5 | node03.test.com | 3306 | ONLINE | secondary |
| e5dba457-f6d5-11e8-97ca-005056abaf95 | node02.test.com | 3306 | ONLINE | secondary |
+--------------------------------------+-----------------+-------------+--------------+-------------+
4 rows in set (0.00 sec)
mysql>
（node05）（一主四从）MGR 启动完如下：
mysql> SELECT * FROM performance_schema.replication_group_members;
+---------------------------+--------------------------------------+-----------------+-------------+--------------+
| CHANNEL_NAME | MEMBER_ID | MEMBER_HOST | MEMBER_PORT | MEMBER_STATE |
+---------------------------+--------------------------------------+-----------------+-------------+--------------+
| group_replication_applier | 0ea53bea-ec72-11e8-913c-005056ab74f8 | node05.test.com | 33062 | ONLINE |
| group_replication_applier | 331664a5-ec72-11e8-a6f0-005056ab74f8 | node05.test.com | 33063 | ONLINE |
| group_replication_applier | 54f7be9e-ec72-11e8-ba6c-005056ab74f8 | node05.test.com | 33064 | ONLINE |
| group_replication_applier | 779b4473-ec72-11e8-8f08-005056ab74f8 | node05.test.com | 33065 | ONLINE |
| group_replication_applier | ee039eb5-ec71-11e8-bd33-005056ab74f8 | node05.test.com | 33061 | ONLINE |
+---------------------------+--------------------------------------+-----------------+-------------+--------------+
5 rows in set (0.00 sec)
mysql> select member_id,member_host,member_port,member_state,if (global_status.variable_name is not null,''primary'',''secondary'') as member_role from performance_schema.replication_group_members left join performance_schema.global_status on global_status.variable_name=''group_replication_primary_member'' and global_status.variable_value=replication_group_members.member_id;
+--------------------------------------+-----------------+-------------+--------------+-------------+
| member_id | member_host | member_port | member_state | member_role |
+--------------------------------------+-----------------+-------------+--------------+-------------+
| 331664a5-ec72-11e8-a6f0-005056ab74f8 | node05.test.com | 33063 | ONLINE | primary |
| 0ea53bea-ec72-11e8-913c-005056ab74f8 | node05.test.com | 33062 | ONLINE | secondary |
| 54f7be9e-ec72-11e8-ba6c-005056ab74f8 | node05.test.com | 33064 | ONLINE | secondary |
| 779b4473-ec72-11e8-8f08-005056ab74f8 | node05.test.com | 33065 | ONLINE | secondary |
| ee039eb5-ec71-11e8-bd33-005056ab74f8 | node05.test.com | 33061 | ONLINE | secondary |
+--------------------------------------+-----------------+-------------+--------------+-------------+
5 rows in set (0.00 sec)
mysql> create database dbtest;
Query OK, 1 row affected (0.16 sec)
mysql> create table dbtest.t2(id int(4)primary key not null auto_increment,nums int(20) not null);
Query OK, 0 rows affected (0.17 sec)
mysql> insert into dbtest.t2(nums) values(1),(2),(3),(4),(5);
Query OK, 5 rows affected (0.05 sec)
Records: 5 Duplicates: 0 Warnings: 0
mysql> update dbtest.t2 set nums=100 where id =3; --id自增为7，这条语句执行没结果
Query OK, 0 rows affected (0.00 sec)
Rows matched: 0 Changed: 0 Warnings: 0
mysql> delete from dbtest.t2 where id =4; --id自增为7，这条语句执行没结果
Query OK, 0 rows affected (0.00 sec)
mysql> select * from dbtest.t2;
+----+------+
| id | nums |
+----+------+
| 7 | 1 | --group_replication_auto_increment_increment=7的缘故， --id自增为7
| 14 | 2 |
| 21 | 3 |
| 28 | 4 |
| 35 | 5 |
+----+------+
5 rows in set (0.00 sec)
mysql> create database test;
Query OK, 1 row affected (0.09 sec)
mysql> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| dbtest |
| mysql |
| performance_schema |
| sys |
| test |
+--------------------+
6 rows in set (0.00 sec)
mysql>
三 、Consul 测试
1、MGR（多主模式）+ Consul 模式
1.1 、Consul UI 界面
地址：http://192.168.49.181:8500/ui/
检查到 3 个服务，consul 、mysql_3306_w、mysql_3306_r
- consul
- mysql_3306_r
- mysql_3306_w
1.2、Consul 检查 DNS 解析
- 域名转发
域名转发官方说明
这里用 Dnsmasq 来做域名转发（访问 MGR 的客户端），域名： mysql_3306_w.service.gczheng
[root@node05 consul.d]# echo "server=/gczheng/127.0.0.1#53" > /etc/dnsmasq.d/10-consul
[root@node05 consul.d]# service dnsmasq restart
Redirecting to /bin/systemctl restart dnsmasq.service
[root@node05 consul.d]# cat /etc/resolv.conf
# Generated by NetworkManager
#nameserver 192.168.10.247
nameserver 127.0.0.1
[root@node05 consul.d]# consul reload
Configuration reload triggered
[root@node05 consul.d]# service dnsmasq restart
Redirecting to /bin/systemctl restart dnsmasq.service
[root@node05 consul.d]# ping mysql_3306_w.service.gczheng
PING mysql_3306_w.service.gczheng (192.168.222.173) 56(84) bytes of data.
64 bytes from node03.test.com (192.168.222.173): icmp_seq=1 ttl=64 time=0.283 ms --测试ok
64 bytes from node03.test.com (192.168.222.173): icmp_seq=2 ttl=64 time=0.266 ms
64 bytes from node03.test.com (192.168.222.173): icmp_seq=3 ttl=64 time=0.274 ms
dig 解析域名 mysql_3306_w.service.gczheng
[root@node11 ~]# dig @127.0.0.1 -p 53 mysql_3306_w.service.gczheng ANY
; <<>> DiG 9.9.4-RedHat-9.9.4-51.el7_4.2 <<>> @127.0.0.1 -p 53 mysql_3306_w.service.gczheng ANY
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 7004
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 8, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;mysql_3306_w.service.gczheng. IN ANY
;; ANSWER SECTION:
mysql_3306_w.service.gczheng. 0 IN A 192.168.222.174
mysql_3306_w.service.gczheng. 0 IN TXT "consul-network-segment="
mysql_3306_w.service.gczheng. 0 IN A 192.168.222.172
mysql_3306_w.service.gczheng. 0 IN TXT "consul-network-segment="
mysql_3306_w.service.gczheng. 0 IN A 192.168.222.173
mysql_3306_w.service.gczheng. 0 IN TXT "consul-network-segment="
mysql_3306_w.service.gczheng. 0 IN A 192.168.222.171
mysql_3306_w.service.gczheng. 0 IN TXT "consul-network-segment="
;; Query time: 12 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Wed Dec 05 09:32:20 CST 2018
;; MSG SIZE rcvd: 265
[root@node11 ~]#
[root@node11 ~]# dig @127.0.0.1 -p 53 mysql_3306_w.service.gczheng SRV
; <<>> DiG 9.9.4-RedHat-9.9.4-51.el7_4.2 <<>> @127.0.0.1 -p 53 mysql_3306_w.service.gczheng SRV
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 450
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 9
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;mysql_3306_w.service.gczheng. IN SRV
;; ANSWER SECTION:
mysql_3306_w.service.gczheng. 0 IN SRV 1 1 3306 node04.test.com.node.dc1.gczheng.
mysql_3306_w.service.gczheng. 0 IN SRV 1 1 3306 node03.test.com.node.dc1.gczheng.
mysql_3306_w.service.gczheng. 0 IN SRV 1 1 3306 node01.test.com.node.dc1.gczheng.
mysql_3306_w.service.gczheng. 0 IN SRV 1 1 3306 node02.test.com.node.dc1.gczheng.
;; ADDITIONAL SECTION:
node04.test.com.node.dc1.gczheng. 0 IN A 192.168.222.174
node04.test.com.node.dc1.gczheng. 0 IN TXT "consul-network-segment="
node03.test.com.node.dc1.gczheng. 0 IN A 192.168.222.173
node03.test.com.node.dc1.gczheng. 0 IN TXT "consul-network-segment="
node01.test.com.node.dc1.gczheng. 0 IN A 192.168.222.171
node01.test.com.node.dc1.gczheng. 0 IN TXT "consul-network-segment="
node02.test.com.node.dc1.gczheng. 0 IN A 192.168.222.172
node02.test.com.node.dc1.gczheng. 0 IN TXT "consul-network-segment="
;; Query time: 228 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Wed Dec 05 09:32:11 CST 2018
;; MSG SIZE rcvd: 473
[root@node11 ~]# ping mysql_3306_w.service.gczheng
PING mysql_3306_w.service.gczheng (192.168.222.171) 56(84) bytes of data.
64 bytes from node01.test.com (192.168.222.171): icmp_seq=1 ttl=64 time=0.174 ms
64 bytes from node01.test.com (192.168.222.171): icmp_seq=2 ttl=64 time=0.373 ms
[root@node11 ~]# ping mysql_3306_w.service.gczheng
PING mysql_3306_w.service.gczheng (192.168.222.172) 56(84) bytes of data.
64 bytes from node02.test.com (192.168.222.172): icmp_seq=1 ttl=64 time=0.258 ms
64 bytes from node02.test.com (192.168.222.172): icmp_seq=2 ttl=64 time=0.251 ms
[root@node11 ~]# ping mysql_3306_w.service.gczheng
PING mysql_3306_w.service.gczheng (192.168.222.173) 56(84) bytes of data.
64 bytes from node03.test.com (192.168.222.173): icmp_seq=1 ttl=64 time=0.184 ms
64 bytes from node03.test.com (192.168.222.173): icmp_seq=2 ttl=64 time=0.189 ms
[root@node11 ~]# ping mysql_3306_w.service.gczheng
PING mysql_3306_w.service.gczheng (192.168.222.174) 56(84) bytes of data.
64 bytes from node04.test.com (192.168.222.174): icmp_seq=1 ttl=64 time=0.576 ms
64 bytes from node04.test.com (192.168.222.174): icmp_seq=2 ttl=64 time=1.03 ms
[root@node11 ~]# mysql -uxxxx -pxxxx -h''mysql_3306_w.service.gczheng'' -P3306
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 12730
Server version: 5.7.18-log MySQL Community Server (GPL)
Type ''help;'' or ''\h'' for help. Type ''\c'' to clear the current input statement.
mysql> exit
Bye
1.3、切换测试
1、关闭 node04 节点
[root@node04 consul.d]# mysqladmin -uroot -p -P3306 shutdown
Enter password:
[root@node04 consul.d]#
- 查看 mgr 状态
mysql> SELECT member_id,member_host,member_port,member_state,IF (global_status.variable_name IS NOT NULL,''primary'',''secondary'') AS member_role FROM PERFORMANCE_SCHEMA.replication_group_members LEFT JOIN PERFORMANCE_SCHEMA.global_status ON global_status.variable_name=''group_replication_primary_member'' AND global_status.variable_value=replication_group_members.member_id;
+--------------------------------------+-----------------+-------------+--------------+-------------+
| member_id | member_host | member_port | member_state | member_role |
+--------------------------------------+-----------------+-------------+--------------+-------------+
| 8770cc17-ebdb-11e8-981b-005056ab9fa5 | node01.test.com | 3306 | ONLINE | secondary | --剩下三个节点
| c38416c3-ebd4-11e8-9b2c-005056aba7b5 | node03.test.com | 3306 | ONLINE | secondary |
| e5dba457-f6d5-11e8-97ca-005056abaf95 | node02.test.com | 3306 | ONLINE | secondary |
+--------------------------------------+-----------------+-------------+--------------+-------------+
3 rows in set (0.00 sec)
mysql>
-
查看 consul 状态
解析域名（mysql_3306_w）
[root@node11 ~]# dig @127.0.0.1 -p 53 mysql_3306_w.service.gczheng SRV
; <<>> DiG 9.9.4-RedHat-9.9.4-51.el7_4.2 <<>> @127.0.0.1 -p 53 mysql_3306_w.service.gczheng SRV
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 19419
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 7
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;mysql_3306_w.service.gczheng. IN SRV
;; ANSWER SECTION:
mysql_3306_w.service.gczheng. 0 IN SRV 1 1 3306 node01.test.com.node.dc1.gczheng. --三个节点
mysql_3306_w.service.gczheng. 0 IN SRV 1 1 3306 node02.test.com.node.dc1.gczheng.
mysql_3306_w.service.gczheng. 0 IN SRV 1 1 3306 node03.test.com.node.dc1.gczheng.
;; ADDITIONAL SECTION:
node01.test.com.node.dc1.gczheng. 0 IN A 192.168.222.171
node01.test.com.node.dc1.gczheng. 0 IN TXT "consul-network-segment="
node02.test.com.node.dc1.gczheng. 0 IN A 192.168.222.172
node02.test.com.node.dc1.gczheng. 0 IN TXT "consul-network-segment="
node03.test.com.node.dc1.gczheng. 0 IN A 192.168.222.173
node03.test.com.node.dc1.gczheng. 0 IN TXT "consul-network-segment="
;; Query time: 0 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Wed Dec 05 11:47:15 CST 2018
;; MSG SIZE rcvd: 369
[root@node11 ~]#
- Consul 界面查看
mysql_3306_w 服务，剩下三个写服务（node01、node02、node03）
2、恢复 node04 节点
如果 mysql 故障无法写的情况，先把 consul 服务停止，在启动 mysql 服务加入 MGR 集群，最后启动 consul 服务
--注释掉服务
[root@node04 consul.d]# consul services deregister mysql_mgr_read_3306.json
Deregistered service: mysql_read
[root@node04 consul.d]# consul services deregister mysql_mgr_write_3306.json
Deregistered service: mysql_write
[root@node04 consul.d]# systemctl start mysql
[root@node04 consul.d]# mysql -uroot -piforgot
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 4
Server version: 5.7.18-log MySQL Community Server (GPL)
Copyright (c) 2009-2017 Percona LLC and/or its affiliates
Copyright (c) 2000, 2017, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type ''help;'' or ''\h'' for help. Type ''\c'' to clear the current input statement.
(root@localhost) 11:58:08 [(none)]> SELECT * FROM performance_schema.replication_group_members;
+---------------------------+-----------+-------------+-------------+--------------+
| CHANNEL_NAME | MEMBER_ID | MEMBER_HOST | MEMBER_PORT | MEMBER_STATE |
+---------------------------+-----------+-------------+-------------+--------------+
| group_replication_applier | | | NULL | OFFLINE |
+---------------------------+-----------+-------------+-------------+--------------+
1 row in set (0.00 sec)
(root@localhost) 11:58:23 [(none)]> start group_replication; --重新加入集群
Query OK, 0 rows affected (2.63 sec)
(root@localhost) 11:58:35 [(none)]> SELECT * FROM performance_schema.replication_group_members;
+---------------------------+--------------------------------------+-----------------+-------------+--------------+
| CHANNEL_NAME | MEMBER_ID | MEMBER_HOST | MEMBER_PORT | MEMBER_STATE |
+---------------------------+--------------------------------------+-----------------+-------------+--------------+
| group_replication_applier | 30955199-ebdc-11e8-ac1f-005056ab820e | node04.test.com | 3306 | ONLINE |
| group_replication_applier | 8770cc17-ebdb-11e8-981b-005056ab9fa5 | node01.test.com | 3306 | ONLINE |
| group_replication_applier | c38416c3-ebd4-11e8-9b2c-005056aba7b5 | node03.test.com | 3306 | ONLINE |
| group_replication_applier | e5dba457-f6d5-11e8-97ca-005056abaf95 | node02.test.com | 3306 | ONLINE |
+---------------------------+--------------------------------------+-----------------+-------------+--------------+
4 rows in set (0.00 sec)
(root@localhost) 11:58:40 [(none)]> exit
Bye
[root@node04 consul.d]# consul reload --重新读取配置
Configuration reload triggered
[root@node04 consul.d]#
- mysql_3306_w 服务
- 查看 consul 状态
解析域名（mysql_3306_w）
[root@node11 ~]# dig @127.0.0.1 -p 53 mysql_3306_w.service.gczheng SRV
; <<>> DiG 9.9.4-RedHat-9.9.4-51.el7_4.2 <<>> @127.0.0.1 -p 53 mysql_3306_w.service.gczheng SRV
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 11299
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 9
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;mysql_3306_w.service.gczheng. IN SRV
;; ANSWER SECTION:
mysql_3306_w.service.gczheng. 0 IN SRV 1 1 3306 node02.test.com.node.dc1.gczheng.
mysql_3306_w.service.gczheng. 0 IN SRV 1 1 3306 node01.test.com.node.dc1.gczheng.
mysql_3306_w.service.gczheng. 0 IN SRV 1 1 3306 node03.test.com.node.dc1.gczheng.
mysql_3306_w.service.gczheng. 0 IN SRV 1 1 3306 node04.test.com.node.dc1.gczheng.
;; ADDITIONAL SECTION:
node02.test.com.node.dc1.gczheng. 0 IN A 192.168.222.172
node02.test.com.node.dc1.gczheng. 0 IN TXT "consul-network-segment="
node01.test.com.node.dc1.gczheng. 0 IN A 192.168.222.171
node01.test.com.node.dc1.gczheng. 0 IN TXT "consul-network-segment="
node03.test.com.node.dc1.gczheng. 0 IN A 192.168.222.173
node03.test.com.node.dc1.gczheng. 0 IN TXT "consul-network-segment="
node04.test.com.node.dc1.gczheng. 0 IN A 192.168.222.174
node04.test.com.node.dc1.gczheng. 0 IN TXT "consul-network-segment="
;; Query time: 9 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Wed Dec 05 11:48:51 CST 2018
;; MSG SIZE rcvd: 473
[root@node11 ~]#
2、MGR（单主模式）+ Consul 模式 + PorxySQL
2.1、PorxySQL 配置
-
安装配置步骤
- 安装请自行参考官方文档安装指南
- 修改 proxysql.cnf 配置文件，配置后端 mysql 连接
- 修改 proxysql.json 配置文件
- 启动 consul 服务读取 proxysql.json 配置
- 配置域名转发（访问 proxysql 的客户端），指向 consul 服务器
[root@audit01 consul.d]# egrep -v "^#|^$" /etc/proxysql.cnf
datadir="/var/lib/proxysql"
admin_variables=
{
admin_credentials="admin:admin" #更改账号
mysql_ifaces="127.0.0.1:6032;/tmp/proxysql_admin.sock" #限制本地访问
}
mysql_variables=
{
threads=4
max_connections=2048
default_query_delay=0
default_query_timeout=36000000
have_compress=true
poll_timeout=2000
interfaces="0.0.0.0:6033"
default_schema="information_schema"
stacksize=1048576
server_version="5.5.30"
connect_timeout_server=3000
monitor_username="rpl_user" #后端mysql监控账号
monitor_password="rpl_pass" #后端mysql监控密码
monitor_history=600000
monitor_connect_interval=60000
monitor_ping_interval=10000
monitor_read_only_interval=1500
monitor_read_only_timeout=500
ping_interval_server_msec=120000
ping_timeout_server=500
commands_stats=true
sessions_sort=true
connect_retries_on_failure=10
}
mysql_servers =
(
)
mysql_users:
(
)
mysql_query_rules:
(
)
scheduler=
(
)
mysql_replication_hostgroups=
(
)
[root@audit01 consul.d]#
[root@audit01 consul.d]# service proxysql restart
Shutting down ProxySQL: DONE!
Starting ProxySQL: 2018-12-05 17:58:53 [INFO] Using config file /etc/proxysql.cnf
DONE!
[root@audit01 consul.d]# service proxysql status
ProxySQL is running (3408).
[root@audit01 consul.d]#
-
配置后端 mysql
登录管理端（mysql -P6032 -h127.0.0.1 -uadmin -padmin）配置后端 mysql（audit01,node12）
INSERT INTO mysql_replication_hostgroups VALUES (20,21,''Standard Replication Groups'');
INSERT INTO mysql_servers (hostname,hostgroup_id,port,weight) VALUES (''192.168.222.175'',21,33061,1000),(''192.168.222.175'',21,33062,1000),(''192.168.222.175'',21,33063,1000),(''192.168.222.175'',21,33064,1000),(''192.168.222.175'',21,33065,1000);
LOAD MYSQL SERVERS TO RUNTIME; SAVE MYSQL SERVERS TO DISK;
INSERT INTO mysql_query_rules (active, match_pattern, destination_hostgroup, cache_ttl, apply) VALUES (1, ''^SELECT .* FOR UPDATE'', 20, NULL, 1);
INSERT INTO mysql_query_rules (active, match_pattern, destination_hostgroup, cache_ttl, apply) VALUES (1, ''^SELECT .*'', 21, NULL, 1);
LOAD MYSQL QUERY RULES TO RUNTIME; SAVE MYSQL QUERY RULES TO DISK;
INSERT INTO mysql_users (username,password,active,default_hostgroup,default_schema) VALUES (''gcdb'',''iforgot'',1,20,''test'');
LOAD MYSQL USERS TO RUNTIME; SAVE MYSQL USERS TO DISK;
use monitor;
UPDATE global_variables SET variable_value=''rpl_user'' WHERE variable_name=''mysql-monitor_username'';
UPDATE global_variables SET variable_value=''rpl_pass'' WHERE variable_name=''mysql-monitor_password'';
LOAD MYSQL VARIABLES TO RUNTIME; SAVE MYSQL VARIABLES TO DISK;
select * from mysql_servers;
select * from mysql_query_rules;
select * from mysql_users;
select * from global_variables where variable_name like "mysql-monitor%";
- 确认配置没问题，两个主机启动 consul
[root@audit01 consul.d]# pwd
/etc/consul.d
[root@audit01 consul.d]# ll
total 8
-rw-r--r-- 1 root root 237 Dec 5 14:59 client.json
-rw-r--r-- 1 root root 542 Dec 5 16:48 proxysql.json
drwxr-xr-x 2 root root 6 Dec 5 14:57 scripts
#由于consul是通过ansible启动，这里就确认是否已启动
[root@audit01 consul.d]# ps -ef|grep consul |grep -v grep
root 1636 1 0 15:00 ? 00:00:49 consul agent -config-dir=/etc/consul.d
[root@audit01 consul.d]# consul catalog nodes
Node ID Address DC
audit01.test.com 8f638db7 192.168.222.199 dc1 --node节点已经加入
node01.test.com feb4c253 192.168.222.171 dc1
node02.test.com 2993cd81 192.168.222.172 dc1
node03.test.com 305293a0 192.168.222.173 dc1
node04.test.com 06c47aea 192.168.222.174 dc1
node05.test.com c8e917fb 192.168.222.175 dc1
node06.test.com 0bd03c71 192.168.222.176 dc1
node09.test.com fcca17ba 192.168.222.179 dc1
node10.test.com c95847a4 192.168.222.180 dc1
node11.test.com 8d8f6757 192.168.222.181 dc1
node12.test.com 3a0f5fdd 192.168.222.182 dc1 --node节点已经加入
[root@audit01 consul.d]# consul catalog services
consul
mysql_3306_r
mysql_3306_w
proxysql --可以看到服务已经加入，在页面也可以到
[root@audit01 consul.d]#
2.2 、查看页面
显示是四个心跳节点
2.3、检查 DNS 解析
配置 DNS 转发的客户端找一台测试
[root@node05 consul.d]# dig @127.0.0.1 -p 53 proxysql.service.gczheng SRV
; <<>> DiG 9.9.4-RedHat-9.9.4-61.el7_5.1 <<>> @127.0.0.1 -p 53 proxysql.service.gczheng SRV
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 4960
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 9
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;proxysql.service.gczheng. IN SRV
;; ANSWER SECTION:
proxysql.service.gczheng. 0 IN SRV 1 1 6033 c0a8dec7.addr.dc1.gczheng.
proxysql.service.gczheng. 0 IN SRV 1 1 6033 audit01.test.com.node.dc1.gczheng.
proxysql.service.gczheng. 0 IN SRV 1 1 6033 c0a8deb6.addr.dc1.gczheng.
proxysql.service.gczheng. 0 IN SRV 1 1 6033 node12.test.com.node.dc1.gczheng.
;; ADDITIONAL SECTION:
c0a8dec7.addr.dc1.gczheng. 0 IN A 192.168.222.199
node12.test.com.node.dc1.gczheng. 0 IN TXT "consul-network-segment="
audit01.test.com.node.dc1.gczheng. 0 IN A 192.168.222.199
audit01.test.com.node.dc1.gczheng. 0 IN TXT "consul-network-segment="
c0a8deb6.addr.dc1.gczheng. 0 IN A 192.168.222.182
audit01.test.com.node.dc1.gczheng. 0 IN TXT "consul-network-segment="
node12.test.com.node.dc1.gczheng. 0 IN A 192.168.222.182
node12.test.com.node.dc1.gczheng. 0 IN TXT "consul-network-segment="
;; Query time: 3 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Wed Dec 05 18:14:41 CST 2018
;; MSG SIZE rcvd: 456
[root@node05 consul.d]# ping proxysql.service.gczheng
PING proxysql.service.gczheng (192.168.222.182) 56(84) bytes of data.
64 bytes from node12.test.com (192.168.222.182): icmp_seq=1 ttl=64 time=0.221 ms
64 bytes from node12.test.com (192.168.222.182): icmp_seq=2 ttl=64 time=0.282 ms
2.4、切换测试
1、测试 proxysql 负载均衡
[root@node05 consul.d]# mysql -ugcdb -piforgot -P6033 -hproxysql.service.gczheng -e "select @@port as 端口"
mysql: [Warning] Using a password on the command line interface can be insecure.
+--------+
| 端口 |
+--------+
| 33061 | --根据延时随机连接后端mysql
+--------+
[root@node05 consul.d]# mysql -ugcdb -piforgot -P6033 -hproxysql.service.gczheng -e "select @@port as 端口"
mysql: [Warning] Using a password on the command line interface can be insecure.
+--------+
| 端口 |
+--------+
| 33062 |
+--------+
[root@node05 consul.d]# mysql -ugcdb -piforgot -P6033 -hproxysql.service.gczheng -e "select @@port as 端口"
mysql: [Warning] Using a password on the command line interface can be insecure.
+--------+
| 端口 |
+--------+
| 33064 |
+--------+
[root@node05 consul.d]# mysql -ugcdb -piforgot -P6033 -hproxysql.service.gczheng -e "select @@port as 端口"
mysql: [Warning] Using a password on the command line interface can be insecure.
+--------+
| 端口 |
+--------+
| 33065 |
+--------+
[root@node05 consul.d]# mysql -ugcdb -piforgot -P6033 -hproxysql.service.gczheng -e "select @@port as 端口"
mysql: [Warning] Using a password on the command line interface can be insecure.
+--------+
| 端口 |
+--------+
| 33063 |
+--------+
2、测试 proxysql 宕机或者 roxysql 服务不可用
[root@node05 mgr_scripts]# dig @127.0.0.1 -p 53 proxysql.service.gczheng
; <<>> DiG 9.9.4-RedHat-9.9.4-61.el7_5.1 <<>> @127.0.0.1 -p 53 proxysql.service.gczheng
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 10214
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 3
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;proxysql.service.gczheng. IN A
;; ANSWER SECTION:
proxysql.service.gczheng. 0 IN A 192.168.222.199
proxysql.service.gczheng. 0 IN A 192.168.222.182
;; ADDITIONAL SECTION:
proxysql.service.gczheng. 0 IN TXT "consul-network-segment="
proxysql.service.gczheng. 0 IN TXT "consul-network-segment="
;; Query time: 1 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Thu Dec 06 14:19:18 CST 2018
;; MSG SIZE rcvd: 157
[root@node05 mgr_scripts]#
- node12 ifdown ens224 网口
[root@node12 ~]# ifconfig ens224
ens224: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.222.182 netmask 255.255.255.0 broadcast 192.168.222.255
inet6 fe80::250:56ff:feab:bc9d prefixlen 64 scopeid 0x20<link>
ether 00:50:56:ab:bc:9d txqueuelen 1000 (Ethernet)
RX packets 1059557 bytes 178058970 (169.8 MiB)
RX errors 0 dropped 57 overruns 0 frame 0
TX packets 1022717 bytes 534301681 (509.5 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
[root@node12 ~]# ifdown ens224
Device ''ens224'' successfully disconnected.
[root@node12 ~]#
- audit 日志显示如下
[root@audit01 consul.d]# tail -f /data/consul/consul.log
2018/12/06 16:40:41 [INFO] agent: Synced service "proxy"
2018/12/06 16:40:41 [INFO] agent: Synced service "proxy1"
2018/12/06 16:40:41 [INFO] agent: Synced service "proxy2"
2018/12/06 16:40:45 [INFO] agent: Synced check "mysql_proxy_02"
2018/12/06 16:40:50 [INFO] agent: Synced check "mysql_proxy_01"
2018/12/06 16:42:43 [INFO] memberlist: Marking node12.test.com as failed, suspect timeout reached (2 peer confirmations)
2018/12/06 16:42:43 [INFO] serf: EventMemberFailed: node12.test.com 192.168.222.182
2018/12/06 16:42:46 [WARN] agent: Check "mysql_proxy_02": Timed out (1s) running check
2018/12/06 16:42:46 [INFO] agent: Synced check "mysql_proxy_02"
2018/12/06 16:43:02 [WARN] agent: Check "mysql_proxy_02": Timed out (1s) running check
2018/12/06 16:43:18 [WARN] agent: Check "mysql_proxy_02": Timed out (1s) running check
2018/12/06 16:43:34 [WARN] agent: Check "mysql_proxy_02": Timed out (1s) running check
2018/12/06 16:43:50 [WARN] agent: Check "mysql_proxy_02": Timed out (1s) running check
2018/12/06 16:44:06 [WARN] agent: Check "mysql_proxy_02": Timed out (1s) running check
2018/12/06 16:44:22 [WARN] agent: Check "mysql_proxy_02": Timed out (1s) running check
2018/12/06 16:44:38 [WARN] agent: Check "mysql_proxy_02": Timed out (1s) running check
2018/12/06 16:44:54 [WARN] agent: Check "mysql_proxy_02": Timed out (1s) running check
2018/12/06 16:45:10 [WARN] agent: Check "mysql_proxy_02": Timed out (1s) running check
2018/12/06 16:45:26 [WARN] agent: Check "mysql_proxy_02": Timed out (1s) running check
2018/12/06 16:45:42 [WARN] agent: Check "mysql_proxy_02": Timed out (1s) running check
2018/12/06 16:45:58 [WARN] agent: Check "mysql_proxy_02": Timed out (1s) running check
2018/12/06 16:46:14 [WARN] agent: Check "mysql_proxy_02": Timed out (1s) running check
2018/12/06 16:46:30 [WARN] agent: Check "mysql_proxy_02": Timed out (1s) running check
2018/12/06 16:46:46 [WARN] agent: Check "mysql_proxy_02": Timed out (1s) running check
2018/12/06 16:47:02 [WARN] agent: Check "mysql_proxy_02": Timed out (1s) running check
2018/12/06 16:47:18 [WARN] agent: Check "mysql_proxy_02": Timed out (1s) running check
2018/12/06 16:47:34 [WARN] agent: Check "mysql_proxy_02": Timed out (1s) running check
2018/12/06 16:47:50 [WARN] agent: Check "mysql_proxy_02": Timed out (1s) running check
2018/12/06 16:48:06 [WARN] agent: Check "mysql_proxy_02": Timed out (1s) running check
2018/12/06 16:48:22 [WARN] agent: Check "mysql_proxy_02": Timed out (1s) running check
2018/12/06 16:48:38 [WARN] agent: Check "mysql_proxy_02": Timed out (1s) running check
2018/12/06 16:48:54 [WARN] agent: Check "mysql_proxy_02": Timed out (1s) running check
2018/12/06 16:49:10 [WARN] agent: Check "mysql_proxy_02": Timed out (1s) running check
2018/12/06 16:49:11 [INFO] serf: attempting reconnect to node12.test.com 192.168.222.182:8301
2018/12/06 16:49:26 [WARN] agent: Check "mysql_proxy_02": Timed out (1s) running check
2018/12/06 16:49:42 [WARN] agent: Check "mysql_proxy_02": Timed out (1s) running check
2018/12/06 16:49:58 [WARN] agent: Check "mysql_proxy_02": Timed out (1s) running check
2018/12/06 16:50:14 [WARN] agent: Check "mysql_proxy_02": Timed out (1s) running check
2018/12/06 16:50:30 [WARN] agent: Check "mysql_proxy_02": Timed out (1s) running check
2018/12/06 16:50:46 [WARN] agent: Check "mysql_proxy_02": Timed out (1s) running check
2018/12/06 16:51:02 [WARN] agent: Check "mysql_proxy_02": Timed out (1s) running check
2018/12/06 16:51:18 [WARN] agent: Check "mysql_proxy_02": Timed out (1s) running check
2018/12/06 16:51:34 [WARN] agent: Check "mysql_proxy_02": Timed out (1s) running check
2018/12/06 16:51:42 [INFO] serf: attempting reconnect to node12.test.com 192.168.222.182:8301
2018/12/06 16:51:50 [WARN] agent: Check "mysql_proxy_02": Timed out (1s) running check
2018/12/06 16:52:06 [WARN] agent: Check "mysql_proxy_02": Timed out (1s) running check
2018/12/06 16:52:22 [WARN] agent: Check "mysql_proxy_02": Timed out (1s) running check
2018/12/06 16:52:24 [INFO] serf: EventMemberJoin: node12.test.com 192.168.222.182
2018/12/06 16:52:30 [INFO] agent: Caught signal: hangup
2018/12/06 16:52:30 [INFO] agent: Reloading configuration...
2018/12/06 16:52:30 [INFO] agent: Deregistered service "proxy"
2018/12/06 16:52:30 [INFO] agent: Synced service "proxy1"
2018/12/06 16:52:30 [INFO] agent: Synced service "proxy2"
2018/12/06 16:52:30 [INFO] agent: Deregistered check "mysql_proxy_02"
2018/12/06 16:52:30 [INFO] agent: Deregistered check "mysql_proxy_01"
2018/12/06 16:52:33 [INFO] agent: Synced check "service:proxy2"
2018/12/06 16:54:00 [INFO] agent: Caught signal: hangup
2018/12/06 16:54:00 [INFO] agent: Reloading configuration...
2018/12/06 16:54:00 [INFO] agent: Synced service "proxy2"
2018/12/06 16:54:00 [INFO] agent: Synced service "proxy1"
2018/12/06 16:55:26 [INFO] memberlist: Suspect node12.test.com has failed, no acks received
2018/12/06 16:55:30 [INFO] memberlist: Marking node12.test.com as failed, suspect timeout reached (2 peer confirmations)
2018/12/06 16:55:30 [INFO] serf: EventMemberFailed: node12.test.com 192.168.222.182
2018/12/06 16:55:58 [WARN] agent: Check "service:proxy2": Timed out (30s) running check
2018/12/06 16:55:58 [INFO] agent: Synced check "service:proxy2"
2018/12/06 16:56:21 [WARN] agent: Check "service:proxy2" is now critical
2018/12/06 16:56:29 [WARN] agent: Check "service:proxy2" is now critical
2018/12/06 16:56:37 [WARN] agent: Check "service:proxy2" is now critical
2018/12/06 16:56:45 [WARN] agent: Check "service:proxy2" is now critical
2018/12/06 16:56:53 [WARN] agent: Check "service:proxy2" is now critical
2018/12/06 16:57:01 [WARN] agent: Check "service:proxy2" is now critical
2018/12/06 16:57:09 [WARN] agent: Check "service:proxy2" is now critical
2018/12/06 16:57:17 [WARN] agent: Check "service:proxy2" is now critical
2018/12/06 16:57:25 [WARN] agent: Check "service:proxy2" is now critical
2018/12/06 16:57:33 [WARN] agent: Check "service:proxy2" is now critical
- 域名解析
[root@node05 consul.d]# dig @127.0.0.1 -p 53 proxysql.service.gczheng
; <<>> DiG 9.9.4-RedHat-9.9.4-61.el7_5.1 <<>> @127.0.0.1 -p 53 proxysql.service.gczheng
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 53883
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 3
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;proxysql.service.gczheng. IN A
;; ANSWER SECTION:
proxysql.service.gczheng. 0 IN A 192.168.222.182 --199和182主机应答
proxysql.service.gczheng. 0 IN A 192.168.222.199
;; ADDITIONAL SECTION:
proxysql.service.gczheng. 0 IN TXT "consul-network-segment="
proxysql.service.gczheng. 0 IN TXT "consul-network-segment="
;; Query time: 1 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Thu Dec 06 16:55:03 CST 2018
;; MSG SIZE rcvd: 157
[root@node05 consul.d]# dig @127.0.0.1 -p 53 proxysql.service.gczheng
; <<>> DiG 9.9.4-RedHat-9.9.4-61.el7_5.1 <<>> @127.0.0.1 -p 53 proxysql.service.gczheng
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 37713
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 2
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;proxysql.service.gczheng. IN A
;; ANSWER SECTION:
proxysql.service.gczheng. 0 IN A 192.168.222.199 --只有199 应答
;; ADDITIONAL SECTION:
proxysql.service.gczheng. 0 IN TXT "consul-network-segment="
;; Query time: 1 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Thu Dec 06 16:56:16 CST 2018
;; MSG SIZE rcvd: 105
- ping
[root@node05 ~]# ping proxysql.service.gczheng
PING proxysql.service.gczheng (192.168.222.182) 56(84) bytes of data.
64 bytes from node12.test.com (192.168.222.182): icmp_seq=1 ttl=64 time=0.197 ms
64 bytes from node12.test.com (192.168.222.182): icmp_seq=2 ttl=64 time=0.291 ms
64 bytes from node12.test.com (192.168.222.182): icmp_seq=3 ttl=64 time=0.288 ms
64 bytes from node12.test.com (192.168.222.182): icmp_seq=4 ttl=64 time=0.235 ms
64 bytes from node12.test.com (192.168.222.182): icmp_seq=5 ttl=64 time=0.261 ms
64 bytes from node12.test.com (192.168.222.182): icmp_seq=6 ttl=64 time=0.313 ms
64 bytes from node12.test.com (192.168.222.182): icmp_seq=7 ttl=64 time=0.283 ms
64 bytes from node12.test.com (192.168.222.182): icmp_seq=8 ttl=64 time=0.337 ms
64 bytes from node12.test.com (192.168.222.182): icmp_seq=9 ttl=64 time=0.141 ms
64 bytes from node12.test.com (192.168.222.182): icmp_seq=10 ttl=64 time=0.337 ms
64 bytes from node12.test.com (192.168.222.182): icmp_seq=11 ttl=64 time=0.273 ms
64 bytes from node12.test.com (192.168.222.182): icmp_seq=12 ttl=64 time=0.258 ms
From node05.test.com (192.168.222.175) icmp_seq=60 Destination Host Unreachable
From node05.test.com (192.168.222.175) icmp_seq=61 Destination Host Unreachable
From node05.test.com (192.168.222.175) icmp_seq=62 Destination Host Unreachable
From node05.test.com (192.168.222.175) icmp_seq=63 Destination Host Unreachable
From node05.test.com (192.168.222.175) icmp_seq=64 Destination Host Unreachable
From node05.test.com (192.168.222.175) icmp_seq=65 Destination Host Unreachable
From node05.test.com (192.168.222.175) icmp_seq=66 Destination Host Unreachable
From node05.test.com (192.168.222.175) icmp_seq=67 Destination Host Unreachable
From node05.test.com (192.168.222.175) icmp_seq=68 Destination Host Unreachable
From node05.test.com (192.168.222.175) icmp_seq=69 Destination Host Unreachable
From node05.test.com (192.168.222.175) icmp_seq=70 Destination Host Unreachable
From node05.test.com (192.168.222.175) icmp_seq=71 Destination Host Unreachable
^C
--- proxysql.service.gczheng ping statistics ---
72 packets transmitted, 12 received, +12 errors, 83% packet loss, time 71004ms
rtt min/avg/max/mdev = 0.141/0.267/0.337/0.058 ms, pipe 4
[root@node05 ~]# ping proxysql.service.gczheng --重新连接后跳到199
PING proxysql.service.gczheng (192.168.222.199) 56(84) bytes of data.
64 bytes from 192.168.222.199 (192.168.222.199): icmp_seq=1 ttl=64 time=0.233 ms
64 bytes from 192.168.222.199 (192.168.222.199): icmp_seq=2 ttl=64 time=0.220 ms
64 bytes from 192.168.222.199 (192.168.222.199): icmp_seq=3 ttl=64 time=0.259 ms
64 bytes from 192.168.222.199 (192.168.222.199): icmp_seq=4 ttl=64 time=0.249 ms
64 bytes from 192.168.222.199 (192.168.222.199): icmp_seq=5 ttl=64 time=0.254 ms
- node12 ifup ens224 网口
[root@node12 consul.d]# ifup ens224
Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/7)
-
域名解析
等待 5s 时间，域名解析到 182
[root@node05 consul.d]# dig @127.0.0.1 -p 53 proxysql.service.gczheng
; <<>> DiG 9.9.4-RedHat-9.9.4-61.el7_5.1 <<>> @127.0.0.1 -p 53 proxysql.service.gczheng
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 29882
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 2
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;proxysql.service.gczheng. IN A
;; ANSWER SECTION:
proxysql.service.gczheng. 0 IN A 192.168.222.199
;; ADDITIONAL SECTION:
proxysql.service.gczheng. 0 IN TXT "consul-network-segment="
;; Query time: 1 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Thu Dec 06 17:04:33 CST 2018
;; MSG SIZE rcvd: 105
[root@node05 consul.d]# dig @127.0.0.1 -p 53 proxysql.service.gczheng
; <<>> DiG 9.9.4-RedHat-9.9.4-61.el7_5.1 <<>> @127.0.0.1 -p 53 proxysql.service.gczheng
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 65219
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 3
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;proxysql.service.gczheng. IN A
;; ANSWER SECTION:
proxysql.service.gczheng. 0 IN A 192.168.222.182
proxysql.service.gczheng. 0 IN A 192.168.222.199
;; ADDITIONAL SECTION:
proxysql.service.gczheng. 0 IN TXT "consul-network-segment="
proxysql.service.gczheng. 0 IN TXT "consul-network-segment="
;; Query time: 1 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Thu Dec 06 17:04:33 CST 2018
;; MSG SIZE rcvd: 157
- pin
[root@node05 ~]# ping proxysql.service.gczheng
PING proxysql.service.gczheng (192.168.222.199) 56(84) bytes of data.
64 bytes from 192.168.222.199 (192.168.222.199): icmp_seq=1 ttl=64 time=0.185 ms
64 bytes from 192.168.222.199 (192.168.222.199): icmp_seq=2 ttl=64 time=0.285 ms
64 bytes from 192.168.222.199 (192.168.222.199): icmp_seq=3 ttl=64 time=0.238 ms
64 bytes from 192.168.222.199 (192.168.222.199): icmp_seq=4 ttl=64 time=0.300 ms
64 bytes from 192.168.222.199 (192.168.222.199): icmp_seq=5 ttl=64 time=0.285 ms
64 bytes from 192.168.222.199 (192.168.222.199): icmp_seq=6 ttl=64 time=0.182 ms
64 bytes from 192.168.222.199 (192.168.222.199): icmp_seq=7 ttl=64 time=0.209 ms
64 bytes from 192.168.222.199 (192.168.222.199): icmp_seq=8 ttl=64 time=0.271 ms
64 bytes from 192.168.222.199 (192.168.222.199): icmp_seq=9 ttl=64 time=0.845 ms
64 bytes from 192.168.222.199 (192.168.222.199): icmp_seq=10 ttl=64 time=0.222 ms
64 bytes from 192.168.222.199 (192.168.222.199): icmp_seq=11 ttl=64 time=0.251 ms
64 bytes from 192.168.222.199 (192.168.222.199): icmp_seq=12 ttl=64 time=0.276 ms
64 bytes from 192.168.222.199 (192.168.222.199): icmp_seq=13 ttl=64 time=0.237 ms
64 bytes from 192.168.222.199 (192.168.222.199): icmp_seq=14 ttl=64 time=0.266 ms
64 bytes from 192.168.222.199 (192.168.222.199): icmp_seq=15 ttl=64 time=0.267 ms
64 bytes from 192.168.222.199 (192.168.222.199): icmp_seq=16 ttl=64 time=0.251 ms
64 bytes from 192.168.222.199 (192.168.222.199): icmp_seq=17 ttl=64 time=0.162 ms
64 bytes from 192.168.222.199 (192.168.222.199): icmp_seq=18 ttl=64 time=0.297 ms
64 bytes from 192.168.222.199 (192.168.222.199): icmp_seq=19 ttl=64 time=0.244 ms
^C
--- proxysql.service.gczheng ping statistics ---
19 packets transmitted, 19 received, 0% packet loss, time 18002ms
rtt min/avg/max/mdev = 0.162/0.277/0.845/0.140 ms
[root@node05 ~]# ping proxysql.service.gczheng
PING proxysql.service.gczheng (192.168.222.182) 56(84) bytes of data.
64 bytes from node12.test.com (192.168.222.182): icmp_seq=1 ttl=64 time=0.227 ms
64 bytes from node12.test.com (192.168.222.182): icmp_seq=2 ttl=64 time=0.320 ms
3、注意事项
- 恢复节点：
需要确认是否只是 mysql 节点故障，还是整个节点故障，避免 consul 服务恢复，mysql 节点未加入集群的情况。
- 域名解析：
客户端节点做本地域名解析时要指向三个服务器节点
Consul Template
在 consul-template 没出现之前，大家构建服务发现系统，大多采用的是 zookeeper、etcd+confd 这样类似的系统，之前写过一篇 consul+confd 的文，讲的是如何动态生成配置文件的，如今 consul 官方推出了自己的模板系统，就是 consul-template，这样的话动态的配置系统可以分化为 etcd+confd 和 consul+consul-template 两大阵营。consul 是一个和 etcd 类似但又强于 etcd 的系统，关于 etcd 和 consul 可以翻阅以前的文章，consul-template 的定位就和 confd 差不多一样了，confd 的后端可以是 etcd 或者 consul，相信 consul 搭配 consul-template 能发挥更大的效果。consul-template 提供了一个便捷的方式从 consul 中获取存储的值，consul-template 守护进程会查询 consul 实例，来更新系统上指定的任何模板，当更新完成后，模板可以选择运行一些任意的命令。
consul template 的使用场景：consul template 可以查询 consul 中的服务目录、key、key-values 等。这种强大的抽象功能和查询语言模板可以使 consul template 特别适合动态的创建配置文件。例如：创建 apache/nginx proxy balancers、haproxy backends、varnish servers、application configurations。
consul template 的特性：
quiescence：consul template内制静止平衡功能，可以智能的发现consul实例中的更改信息。这个功能可以防止频繁的更新模板而引起系统的波动。
dry mode：不确定当前架构的状态？担心模板的变化会破坏子系统？无须担心，因为consul template还有-dry模式。在dry模式，consul template会将结果呈现在STDOUT，所以操作员可以检查输出是否正常，以决定更换模板是否安全
CLI and Config：如果你喜欢在命令行上指定一切，consul template都可以hold住。随着内置HCL的支持，consul template接收一个配置文件，命令行参数，或者两者的混合。通过这种方式你可以继续使用你现在已有的配置管理工具和consul template来配合。
verbose debugging：即使每件事你都做的近乎完美，但是有时候还是会有失败发生。consul template可以提供更详细的debug日志信息。下载 consul-template
继承了 consul 的风格，consul-template 下载下来解压后就是一个二进制文件，没有其他多余的内容，download
查看帮助
执行 consul-template -h 即可看到 consul-temple 的使用参数
-auth=<user[:pass]> 设置基本的认证用户名和密码
-consul=<address> 设置Consul实例的地址
-max-stale=<duration> 查询过期的最大频率，默认是1s
-dedup 启用重复数据删除，当许多consul template实例渲染一个模板的时候可以降低consul的负载
-ssl 使用https连接Consul使用SSL
-ssl-verify 通过SSL连接的时候检查证书
-ssl-cert SSL客户端证书发送给服务器
-ssl-key 客户端认证时使用的SSL/TLS私钥
-ssl-ca-cert 验证服务器的CA证书列表
-token=<token> 设置Consul API的token
-syslog 把标准输出和标准错误重定向到syslog，syslog的默认级别是local0。
-syslog-facility=<f> 设置syslog级别，默认是local0，必须和-syslog配合使用
-template=<template> 增加一个需要监控的模板，格式是：''templatePath:outputPath(:command)''，多个模板则可以设置多次
-wait=<duration> 当呈现一个新的模板到系统和触发一个命令的时候，等待的最大最小时间。如果最大值被忽略，默认是最小值的4倍。
-retry=<duration> 当在和consul api交互的返回值是error的时候，等待的时间，默认是5s。
-config=<path> 配置文件或者配置目录的路径
-pid-file=<path> PID文件的路径
-log-level=<level> 设置日志级别，可以是"debug","info", "warn" (default), and "err"
-dry Dump生成的模板到标准输出，不会生成到磁盘
-once 运行consul-template一次后退出，不以守护进程运行
-reap 子进程自动收割下面看一些例子：
consul 实例：demo.consul.io
模板：/tmp/template.ctmpl
模板输出路径：/tmp/result
1 运行 consul-temple 作为一个服务
consul-template \
-consul demo.consul.io \
-template "/tmp/template.ctmpl:/tmp/result"2 查询本地 consl 实例，生成模板后重启 nginx，如果 consul 不可用，如果 api 故障则每 30s 尝试检测一次值，consul-template 运行一次后退出
consul-template \
-consul 127.0.0.1:8500 \
-template "/tmp/template.ctmpl:/var/www/nginx.conf:service nginx restart" \
-retry 30s \
-once3 查询一个实例，渲染多个模板，然后重启相关服务
consul-template \
-consul my.consul.internal:6124 \
-template "/tmp/nginx.ctmpl:/var/nginx/nginx.conf:service nginx restart" \
-template "/tmp/redis.ctmpl:/var/redis/redis.conf:service redis restart" \
-template "/tmp/haproxy.ctmpl:/var/haproxy/haproxy.conf"4 查询一个实例，dump 模板到标准输出，参数中的 - template 则会被忽略
consul-template \
-consul my.consul.internal:6124 \
-template "/tmp/template.ctmpl:/tmp/result:service nginx restart"
-dry以上参数除了在命令行使用，也可以直接配置在文件中，下面看看 Consul-Template 的配置文件，简称 HCL (HashiCorp Configuration Language)，它是和 JSON 兼容的，下面看个例子：
consul = "127.0.0.1:8500"
token = "abcd1234"
retry = "10s"
max_stale = "10m"
log_level = "warn"
pid_file = "/path/to/pid"
wait = "5s:10s"
vault {
address = "https://vault.service.consul:8200"
token = "abcd1234"
renew = true
ssl {
// ...
}
}
auth {
enabled = true
username = "test"
password = "test"
}
ssl {
enabled = true
verify = false
cert = "/path/to/client/cert"
key = "/path/to/client/key"
ca_cert = "/path/to/ca"
}
syslog {
enabled = true
facility = "LOCAL5"
}
deduplicate {
enabled = true
prefix = "consul-template/dedup/"
}
template {
source = "/path/on/disk/to/template.ctmpl"
destination = "/path/on/disk/where/template/will/render.txt"
command = "restart service foo"
command_timeout = "60s"
perms = 0600
backup = true
left_delimiter = "{{"
right_delimiter = "}}"
wait = "2s:6s"
}以上并不是所有的 fields 都需要，比如 Vault 你可能就不需要，所以你就不需要指定 Vault 配置。以上就是配置文件。
下面看看配置模板到底怎么写，模板文件的语法和 Go template 的格式一样，confd 也是遵循 Go template 的。
先看看 API 功能语法：
datacenters：在consul目录中查询所有的datacenters，{{datacenters}}
file：读取并输出本地磁盘上的文件，如果无法读取，则报错，{{file "/path/to/local/file"}}
key：查询consul中该key的值，如果无法转换成一个类字符串的值，则会报错，{{key "service/redis/maxconns@east-aws"}} east-aws指定的是数据中心，{{key "service/redis/maxconns"}}
key_or_default：查询consul中该key的值，如果key不存在，则使用指定的值代替，{{key_or_default "service/redis/maxconns@east-aws" "5"}}
ls：在consul中查询给定前缀的key的顶级域值，{{range ls "service/redis@east-aws"}} {{.Key}} {{.Value}}{{end}}
node：查询consul目录中的单个node，如果不指定node，则是当前agent的，{{node "node1"}}
nodes：查询consul目录中的所有nodes，你也可以指定datacenter，{{nodes "@east-aws"}}
service：查询consul中匹配的service组，{{service "release.web@east-aws"}}或者{{service "web"}}，也可以返回一组HealthService服务{{range service "web@datacenter"}} server {{.Name}} {{.Address}}:{{.Port}}{{end}}，默认值返回健康的服务，如果你想返回所有服务，则{{service "web" "any"}}
services：查询consul目录中的所有services，{{services}}，也可以指定datacenter：{{services "@east-aws"}}
tree：查询consul中给定前缀的所有K/V值，{{range tree "service/redis@east-aws"}} {{.Key}} {{.Value}}{{end}}再看看辅助函数语法：
byKey、byTag、contains、env、explode、in、loop、trimSpace、join、parseBool、parseFloat、parseInt、parseJSON、parseUint、regexMatch、regexReplaceAll、replaceAll、split、timestamp、toJSON 等函数可以使用，具体用法看官文
consul-template + nginx部署高可用负载均衡
一、Consul-Template简介
Consul-Template是基于Consul的自动替换配置文件的应用。在Consul-Template没出现之前，大家构建服务发现系统大多采用的是Zookeeper、Etcd+Confd这样类似的系统。
Consul官方推出了自己的模板系统Consul-Template后，动态的配置系统可以分化为Etcd+Confd和Consul+Consul-Template两大阵营。Consul-Template的定位和Confd差不多，Confd的后端可以是Etcd或者Consul。
Consul-Template提供了一个便捷的方式从Consul中获取存储的值，Consul-Template守护进程会查询Consul实例来更新系统上指定的任何模板。当更新完成后，模板还可以选择运行一些任意的命令。
Consul-Template的使用场景
Consul-Template可以查询Consul中的服务目录、Key、Key-values等。这种强大的抽象功能和查询语言模板可以使Consul-Template特别适合动态的创建配置文件。例如：创建Apache/Nginx Proxy Balancers、Haproxy Backends、Varnish Servers、Application Configurations等。
Consul-Template特性
- Quiescence：Consul-Template内置静止平衡功能，可以智能的发现Consul实例中的更改信息。这个功能可以防止频繁的更新模板而引起系统的波动。
- Dry Mode：不确定当前架构的状态，担心模板的变化会破坏子系统？无须担心。因为Consul-Template还有Dry模式。在Dry模式，Consul-Template会将结果呈现在STDOUT，所以操作员可以检查输出是否正常，以决定更换模板是否安全。
- CLI and Config：Consul-Template同时支持命令行和配置文件。
- Verbose Debugging：即使每件事你都做的近乎完美，但是有时候还是会有失败发生。Consul-Template可以提供更详细的Debug日志信息。
项目地址：https://github.com/hashicorp/consul-template
二、Consul-Template安装
Consul-Template和Consul一样，也是用Golang实现。因此具有天然可移植性(支持 Linux、windows 和macOS)。安装包仅包含一个可执行文件。Consul-Template安装非常简单，只需要下载对应系统的软件包并解压后就可使用。
只需要下载可执行文件：https://releases.hashicorp.com/consul-template/
将执行文件解压放到/usr/local/bin/下即可，如下：
我下载的是：consul-template_0.20.0_linux_amd64.tgz
[root@localhost consul-template]# tar -xvf consul-template_0.20.0_linux_amd64.tgz
consul-template
[root@localhost consul-template]# ll
总用量 12696
-rwxr-xr-x. 1 501 games 9451232 2月 20 08:39 consul-template
-rw-r--r--. 1 root root 3543379 3月 22 09:57 consul-template_0.20.0_linux_amd64.tgz
[root@localhost consul-template]# cd consul-template
-bash: cd: consul-template: 不是目录
[root@localhost consul-template]# cp consul-template /usr/local/bin
[root@localhost consul-template]# consul-template -v
consul-template v0.20.0 (b709612c)
[root@localhost consul-template]#三、Consul-Template使用帮助
查看帮助
执行consul-template -h即可看到consul-temple的使用参数
-auth=<user[:pass]> 设置基本的认证用户名和密码
-consul=<address> 设置Consul实例的地址
-max-stale=<duration> 查询过期的最大频率，默认是1s -dedup 启用重复数据删除，当许多consul template实例渲染一个模板的时候可以降低consul的负载 -ssl 使用https连接Consul使用SSL -ssl-verify 通过SSL连接的时候检查证书 -ssl-cert SSL客户端证书发送给服务器 -ssl-key 客户端认证时使用的SSL/TLS私钥 -ssl-ca-cert 验证服务器的CA证书列表 -token=<token> 设置Consul API的token -syslog 把标准输出和标准错误重定向到syslog，syslog的默认级别是local0。 -syslog-facility=<f> 设置syslog级别，默认是local0，必须和-syslog配合使用 -template=<template> 增加一个需要监控的模板，格式是：''templatePath:outputPath(:command)''，多个模板则可以设置多次 -wait=<duration> 当呈现一个新的模板到系统和触发一个命令的时候，等待的最大最小时间。如果最大值被忽略，默认是最小值的4倍。 -retry=<duration> 当在和consul api交互的返回值是error的时候，等待的时间，默认是5s。 -config=<path> 配置文件或者配置目录的路径 -pid-file=<path> PID文件的路径 -log-level=<level> 设置日志级别，可以是"debug","info", "warn" (default), and "err" -dry Dump生成的模板到标准输出，不会生成到磁盘 -once 运行consul-template一次后退出，不以守护进程运行 -reap 子进程自动收割Consul-Template模版语法
Consul-Template模板文件的语法和Go Template的格式一样，Confd也是遵循Go Template的。
下面看看配置模板到底怎么写，模板文件的语法和Go template的格式一样，confd也是遵循Go template的。
先看看API 功能语法：
datacenters：在consul目录中查询所有的datacenters，{{datacenters}}
file：读取并输出本地磁盘上的文件，如果无法读取，则报错，{{file "/path/to/local/file"}}
key：查询consul中该key的值，如果无法转换成一个类字符串的值，则会报错，{{key "service/redis/maxconns@east-aws"}} east-aws指定的是数据中心，{{key "service/redis/maxconns"}}
key_or_default：查询consul中该key的值，如果key不存在，则使用指定的值代替，{{key_or_default "service/redis/maxconns@east-aws" "5"}} ls：在consul中查询给定前缀的key的顶级域值，{{range ls "service/redis@east-aws"}} {{.Key}} {{.Value}}{{end}} node：查询consul目录中的单个node，如果不指定node，则是当前agent的，{{node "node1"}} nodes：查询consul目录中的所有nodes，你也可以指定datacenter，{{nodes "@east-aws"}} service：查询consul中匹配的service组，{{service "release.web@east-aws"}}或者{{service "web"}}，也可以返回一组HealthService服务{{range service "web@datacenter"}} server {{.Name}} {{.Address}}:{{.Port}}{{end}}，默认值返回健康的服务，如果你想返回所有服务，则{{service "web" "any"}} services：查询consul目录中的所有services，{{services}}，也可以指定datacenter：{{services "@east-aws"}} tree：查询consul中给定前缀的所有K/V值，{{range tree "service/redis@east-aws"}} {{.Key}} {{.Value}}{{end}}再看看辅助函数语法：
byKey、byTag、contains、env、explode、in、loop、trimSpace、join、parseBool、parseFloat、parseInt、parseJSON、parseUint、regexMatch、regexReplaceAll、replaceAll、split、timestamp、toJSON等函数可以使用，具体用法看官文
四、Consul-Template使用实例
在进行以下测试前，你首先得有一个Consul集群。如果你还没有，可参考「consul分布式集群搭建」 一文搭建一个。当然单节点Consul环境也是可以的，如果你只想做一下简单的测试也可以参考「Consul入门」一文先快速搭建一个单节点环境。
4.1、命令行方式
1、输出已注册服务的名称和Tags。
接着《consul分布式集群搭建》文章后面的service-front示例，通过命令行方式输出已经注册服务的名称和Tags，如下：
[root@localhost consul-template]# curl http://10.200.110.90:8500/v1/catalog/service/service-front
[{"ID":"d1b05900-4f8f-b956-5ba6-5a3c798d93d3","Node":"10.200.110.91","Address":"10.200.110.91","Datacenter":"shenzhen","TaggedAddresses":{"lan":"10.200.110.91","wan":"10.200.110.91"},"NodeMeta":{"consul-network-segment":""},"ServiceKind":"","ServiceID":"service-front-10-200-110-100-8001","ServiceName":"service-front","ServiceTags":["front-dev,这是个前置应用，网关层","duan"],"ServiceAddress":"10.200.110.100","ServiceWeights":{"Passing":1,"Warning":1},"ServiceMeta":{},"ServicePort":8001,"ServiceEnableTagOverride":false,"ServiceProxyDestination":"","ServiceProxy":{},"ServiceConnect":{},"CreateIndex":11382,"ModifyIndex":11382},{"ID":"382f88c2-4482-e1f7-1453-28f94ff65108","Node":"10.200.110.97","Address":"10.200.110.97","Datacenter":"shenzhen","TaggedAddresses":{"lan":"10.200.110.97","wan":"10.200.110.97"},"NodeMeta":{"consul-network-segment":""},"ServiceKind":"","ServiceID":"front1","ServiceName":"service-front","ServiceTags":["local-dev"],"ServiceAddress":"","ServiceWeights":{"Passing":1,"Warning":1},"ServiceMeta":{},"ServicePort":8001,"ServiceEnableTagOverride":false,"ServiceProxyDestination":"","ServiceProxy":{},"ServiceConnect":{},"CreateIndex":11976,"ModifyIndex":11976}][root@localhost consul-template]#2、通过consul-template生成nginx配置文件
2.1、创建模板文件
[root@localhost config]# vi tmpltest.ctmpl
[root@localhost config]# cat tmpltest.ctmpl
{{range services}}
{{.Name}}
{{range .Tags}}
{{.}}{{end}}
{{end}}
[root@localhost config]
2.2、调用模板文件生成查询结果
[root@localhost config]# consul-template -consul-addr 10.200.110.90:8500 -template "tmpltest.ctmpl:result" -once
[root@localhost config]# 命令说明：
-consul-addr:指定Consul的API接口 ，默认是8500端口。
-template：模板参数，第一个参数是模板文件位置，第二个参数是结果输出位置。
-once：只运行一次就退出。
查看模板渲染的结果
[root@localhost config]# ll
总用量 8
-rw-r--r--. 1 root root 143 3月 22 11:08 result
-rw-r--r--. 1 root root 66 3月 22 11:05 tmpltest.ctmpl
[root@localhost config]# cat result
consul
service-consumer
service-demo
jar
service-front
duan
front-dev,这是个前置应用，网关层
local-dev
service-producter
[root@localhost config]#输出结果说明：
- consul是系统自带的服务；
- service-front是通过consul的配置方式注册的服务,其Tags为【duan front-dev,这是个前置应用，网关层 local-dev】；
2.3、根据已注册的服务动态生成Nginx配置文件
新建Nginx配置模板文件
[root@localhost config]# vi nginx.conf.ctmpl
[root@localhost config]# cat nginx.conf.ctmpl
{{range services}} {{$name := .Name}} {{$service := service .Name}}
upstream {{$name}} {
zone upstream-{{$name}} 64k;
{{range $service}}server {{.Address}}:{{.Port}} max_fails=3 fail_timeout=60 weight=1;
{{else}}server 127.0.0.1:65535; # force a 502{{end}}
} {{end}}
server {
listen 80 default_server;
location / {
root /usr/share/nginx/html/;
index index.html;
}
location /stub_status {
stub_status;
}
{{range services}} {{$name := .Name}}
location /{{$name}} {
proxy_pass http://{{$name}};
}
{{end}}
}
[root@localhost config]#调用模板文件生成Nginx配置文件
[root@localhost config]# consul-template -consul-addr 10.200.110.90:8500 -template="nginx.conf.ctmpl:default.conf" -once
[root@localhost config]# cat default.conf
upstream consul {
zone upstream-consul 64k;
server 10.200.110.90:8300 max_fails=3 fail_timeout=60 weight=1;
server 10.200.110.91:8300 max_fails=3 fail_timeout=60 weight=1;
server 10.200.110.93:8300 max_fails=3 fail_timeout=60 weight=1;
}
upstream service-consumer {
zone upstream-service-consumer 64k;
server 10.200.110.89:8091 max_fails=3 fail_timeout=60 weight=1;
server 10.200.110.90:8091 max_fails=3 fail_timeout=60 weight=1;
}
upstream service-demo {
zone upstream-service-demo 64k;
server 10.200.110.97:8071 max_fails=3 fail_timeout=60 weight=1;
}
upstream service-front {
zone upstream-service-front 64k;
server 10.200.110.97:8001 max_fails=3 fail_timeout=60 weight=1;
}
upstream service-producter {
zone upstream-service-producter 64k;
server 10.200.110.95:8081 max_fails=3 fail_timeout=60 weight=1;
}
server {
listen 80 default_server;
location / {
root /usr/share/nginx/html/;
index index.html;
}
location /stub_status {
stub_status;
}
location /consul {
proxy_pass http://consul;
}
location /service-consumer {
proxy_pass http://service-consumer;
}
location /service-demo {
proxy_pass http://service-demo;
}
location /service-front {
proxy_pass http://service-front;
}
location /service-producter {
proxy_pass http://service-producter;
}
}
[root@localhost config]#如果想生成Nginx配置文件后自动加载配置，可以这样：
[root@localhost config]# consul-template -consul-addr 10.200.110.90:8500 -template="nginx.conf.ctmpl:/usr/local/nginx/conf/conf.d/default.conf:service nginx reload" -once
[root@localhost config]#生成的nginx配置如下：
[root@localhost config]# cat /usr/local/nginx/conf/conf.d/default.conf
upstream consul {
zone upstream-consul 64k;
server 10.200.110.90:8300 max_fails=3 fail_timeout=60 weight=1;
server 10.200.110.91:8300 max_fails=3 fail_timeout=60 weight=1;
server 10.200.110.93:8300 max_fails=3 fail_timeout=60 weight=1;
}
upstream service-consumer {
zone upstream-service-consumer 64k;
server 10.200.110.89:8091 max_fails=3 fail_timeout=60 weight=1;
server 10.200.110.90:8091 max_fails=3 fail_timeout=60 weight=1;
}
upstream service-demo {
zone upstream-service-demo 64k;
server 10.200.110.97:8071 max_fails=3 fail_timeout=60 weight=1;
}
upstream service-front {
zone upstream-service-front 64k;
server 10.200.110.97:8001 max_fails=3 fail_timeout=60 weight=1;
}
upstream service-producter {
zone upstream-service-producter 64k;
server 10.200.110.95:8081 max_fails=3 fail_timeout=60 weight=1;
}
server {
listen 80 default_server;
location / {
root /usr/share/nginx/html/;
index index.html;
}
location /stub_status {
stub_status;
}
location /consul {
proxy_pass http://consul;
}
location /service-consumer {
proxy_pass http://service-consumer;
}
location /service-demo {
proxy_pass http://service-demo;
}
location /service-front {
proxy_pass http://service-front;
}
location /service-producter {
proxy_pass http://service-producter;
}
}
[root@localhost config]#;
}
location /service-demo {
proxy_pass http://service-demo;
}
location /service-front {
proxy_pass http://service-front;
}
location /service-producter {
proxy_pass http://service-producter;
}
}
[root@localhost config]#注意，/usr/local/nginx/conf/nginx.conf的配置里增加一行，加载conf.d目录下的其它动态生成的配置文件：
[root@localhost sbin]# vi ../conf/nginx.conf
worker_processes 1;
#error_log /var/log/nginx/error.log warn;
#pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
#include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main ''$remote_addr - $remote_user [$time_local] "$request" ''
''$status $body_bytes_sent "$http_referer" ''
''"$http_user_agent" "$http_x_forwarded_for"'';
#access_log /var/log/nginx/access.log main;
sendfile on;
#tcp_nopush on;
server_tokens off;
underscores_in_headers on;
keepalive_timeout 65;
gzip on;
gzip_min_length 1024;
gzip_buffers 4 1k;
#gzip_http_version 1.0;
gzip_comp_level 8;
gzip_types text/plain application/json application/javascript application/x-javascript text/css application/xml text/javascript image/jpeg image/gif image/png;
gzip_vary off;
gzip_disable "MSIE [1-6]\.";
include /usr/local/nginx/conf/conf.d/*.conf;
}consul-template的模板如下：
[root@localhost config]# cat tmplconsumer.ctmpl
## Settings for a TLS enabled server.
upstream frontend {
{{range service "service-consumer" }} {{$name := .Name}}
##upstream {{$name}}
##zone upstream-{{$name}} 64k;
server {{.Address}}:{{.Port}} max_fails=3 fail_timeout=60 weight=1;
{{else}}server 127.0.0.1:65535; # force a 502
{{end}}
}
server {
listen 80;
server_name localhost; #appci.mydomain.net;
#location /h5/ {
#index index.html
#root /usr/share/nginx/html/h5;
#proxy_pass http://xxxxxoms/;
#}
location / {
#if ($request_method ~ ^(HEAD)$) {
# access_log off;
#}
add_header ''Access-Control-Allow-Origin'' ''*'';
add_header Access-Control-Allow-Methods ''GET, POST, OPTIONS'';
add_header Access-Control-Allow-Headers ''DNT,X-Mx-ReqToken,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Authorization'';
if ($request_method = ''OPTIONS'') {
return 204;
}
proxy_pass http://frontend/;
#Proxy Settings
proxy_set_header Host $host;
proxy_set_header Referer $http_referer;
proxy_set_header Cookie $http_cookie;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_connect_timeout 90;
proxy_send_timeout 90;
proxy_read_timeout 90;
proxy_buffer_size 4k;
proxy_buffers 4 32k;
proxy_busy_buffers_size 64k;
proxy_temp_file_write_size 64k;
}
error_page 404 /404.html;
location = /40x.html {
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
}
}
[root@localhost config]#执行
[root@localhost config]# consul-template -consul-addr 10.200.110.90:8500 -template="tmplconsumer.ctmpl:/usr/local/nginx/conf/conf.d/default.conf:service nginx reload" -once生成的nginx配置文件如下：
## Settings for a TLS enabled server.
upstream frontend {
##upstream service-consumer
##zone upstream-service-consumer 64k;
server 10.200.110.89:8091 max_fails=3 fail_timeout=60 weight=1;
##upstream service-consumer
##zone upstream-service-consumer 64k;
server 10.200.110.90:8091 max_fails=3 fail_timeout=60 weight=1;
}
server {
listen 80;
server_name localhost; #appci.mydomain.net;
#location /h5/ {
#index index.html
#root /usr/share/nginx/html/h5;
#proxy_pass http://xxxxxoms/;
#}
location / {
#if ($request_method ~ ^(HEAD)$) {
# access_log off;
#}
add_header ''Access-Control-Allow-Origin'' ''*'';
add_header Access-Control-Allow-Methods ''GET, POST, OPTIONS'';
add_header Access-Control-Allow-Headers ''DNT,X-Mx-ReqToken,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Authorization'';
if ($request_method = ''OPTIONS'') {
return 204;
}
proxy_pass http://frontend/;
#Proxy Settings
proxy_set_header Host $host;
proxy_set_header Referer $http_referer;
proxy_set_header Cookie $http_cookie;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_connect_timeout 90;
proxy_send_timeout 90;
proxy_read_timeout 90;
proxy_buffer_size 4k;
proxy_buffers 4 32k;
proxy_busy_buffers_size 64k;
proxy_temp_file_write_size 64k;
}用浏览器访问测试：http://10.200.110.90/consumer3/showbook6/1
如果不生效，可以如下排查：
首先，检查生成的nginx配置文件中是否有了对应的服务的配置信息。
再检查，nginx是否重加载成功，可以[root@localhost config]# /usr/local/nginx/sbin/nginx -s reload试试。
多台nginx+consul-template
上面的示例中的nginx是单节点的，有单点故障的问题。那么再+nginx时，是怎样的呢？
先放一个图：
将consul-template的虚拟机克隆下，新ip为99，在启动consul-template，consul关联的还是90（consul-template和consul可以不再同一台机器上，但consul-template和nginx必须同一台机器）
consul-template -consul-addr 10.200.110.90:8500 -template="tmplconsumer.ctmpl:/usr/local/nginx/conf/conf.d/default.conf:service nginx reload" -once访问如下，也是成功的。
consul-template以服务方式运行
上面两个例子都是consul-template运行一次后就退出了，如果想运行为一个服务可以这样：
consul-template -consul-addr=10.200.110.90:8500 -template "tmpltest.ctmpl:test.out"查询Consul实例同时渲染多个模板，然后重启相关服务。如果API故障则每30s尝试检测一次值，consul-template运行一次后退出。
consul-template \
-consul-addr=10.200.110.90:8500 \
-retry 30s \
-once \
-template "nginx.ctmpl:/etc/nginx/nginx.conf:service nginx restart" \
-template "redis.ctmpl:/etc/redis/redis.conf:service redis restart" \
-template "haproxy.ctmpl:/etc/haproxy/haproxy.conf:service haproxy restart"查询一个实例，Dump模板到标准输出。主要用作测试模板输出。
4.2、配置文件方式
以上参数除了在命令行使用也可以直接配置在文件中，下面看看Consul-Template的配置文件，简称HCL(HashiCorp Configuration Language)，它是和JSON兼容的。下面先看看官方给的配置文件格式：
consul {
auth {
enabled = true
username = "test"
password = "test"
}
address = "192.168.2.210:8500"
token = "abcd1234"
retry {
enabled = true
attempts = 5
backoff = "250ms"
}
ssl {
enabled = true
verify = false
cert = "/path/to/client/cert"
key = "/path/to/client/key"
ca_cert = "/path/to/ca"
ca_path = "path/to/certs/"
server_name = "my-server.com"
}
}
reload_signal = "SIGHUP"
dump_signal = "SIGQUIT"
kill_signal = "SIGINT"
max_stale = "10m"
log_level = "warn"
pid_file = "/path/to/pid"
wait {
min = "5s"
max = "10s"
}
vault {
address = "https://vault.service.consul:8200"
token = "abcd1234"
unwrap_token = true
renew_token = true
retry {
# ...
}
ssl {
# ...
}
}
syslog {
enabled = true
facility = "LOCAL5"
}
deduplicate {
enabled = true
prefix = "consul-template/dedup/"
}
exec {
command = "/usr/bin/app"
splay = "5s"
env {
pristine = false
custom = ["PATH=$PATH:/etc/myapp/bin"]
whitelist = ["CONSUL_*"]
blacklist = ["VAULT_*"]
}
reload_signal = ""
kill_signal = "SIGINT"
kill_timeout = "2s"
}
template {
source = "/path/on/disk/to/template.ctmpl"
destination = "/path/on/disk/where/template/will/render.txt"
contents = "{{ keyOrDefault \"service/redis/maxconns@east-aws\" \"5\" }}"
command = "restart service foo"
command_timeout = "60s"
perms = 0600
backup = true
left_delimiter = "{{"
right_delimiter = "}}"
wait {
min = "2s"
max = "10s"
}
}更多详细的参数可以参考这里： https://github.com/hashicorp/consul-template#configuration-file-format
注意: 上面的选项不都是必选的，可根据实际情况调整。为了更加安全，token也可以从环境变量里读取，使用CONSUL_TOKEN和VAULT_TOKEN。强烈建议你不要把token放到未加密的文本配置文件中。
接下来我们看一个简单的实例，生成Nginx配置文件并重新加载：
vim nginx.hcl
consul {
address = "10.200.110.90:8500"
}
template {
source = "nginx.conf.ctmpl"
destination = "/usr/local/nginx/conf/conf.d/default.conf"
command = "service nginx reload"
}注：nginx.conf.ctmpl模板文件内容还是和前面的一样。
执行以下命令就可渲染文件了：
consul-template -config "nginx.hcl"更多官方例子可参考：https://github.com/hashicorp/consul-template/tree/master/examples
===============================================================================================
五、Consul-Template的redis读写分离的一个应用（转载，未验证）
此处假定一个高可用redis集群读写分离的场景，存在一个业务服务BusinessService.Sample，通过对配置文件XXX.BusinessService.Sample.Redis.json中Redis的集群信息的读取来维持Redis客户端的正常读写，配置文件中的数据发生变化时。Redis客户端会进行更新（假定场景- -只表达大致意思，勿喷）。
XXX.BusinessService.Sample.Redis.json中的配置信息假定为
{
"redis":{
"servers": ["127.0.0.1:6378""127.0.0.1:6379"],
"master": "127.0.0.1:6378"
}
}
注册Redis健康监测及配置中心维护Redis的Master节点信息
将Redis的所有节点注册到Consul，并应用Consul进行健康监测。
{
"services": [{
"id":"redis1",
"name":"redis",
"tags":["redis1"],
"address": "127.0.0.1",
"port":6378,
"checks": [
{
"Tcp": "127.0.0.1:6378",
"interval": "3s"
}
]
},{
"id":"redis2",
"name":"redis",
"tags":["redis2"],
"address": "127.0.0.1",
"port":6379,
"checks": [
{
"Tcp": "127.0.0.1:6379",
"interval": "3s"
}
]
}
]
}
Redis正常运行的状态如下：
对于Master的节点信息，这里采用配置中心进程存储
到这里基本工作算是完成了
创建Consul-template模板并注册
consul-template会通过Http请求从Consul中读取集群中的数据，数据发生变更时 consul-template会触发更新指定配置文件的操作。此处根据XXX.BusinessService.Sample.Redis.json的数据格式创建.ctmpl模板文件，内容如下（配置格式参考 Consul-template文档）：
{
"redis":{
"servers": [{{range service "redis" "passing"}}"{{.Address}}:{{.Port}}"{{end}}],
"master": "{{key "service/redis/master"}}"
}
}
然后进行注册(此处以windows作为实例，Linux操作一样)
consul-template.exe -consul "127.0.0.1:8500" -template="C:\Users\admin\Desktop\consul064\consul-temp\templates\XXX.BusinessService.Sample.Redis.json.ctmpl:C:\Users\admin\Desktop\BusinessService.Sample\BusinessService.Sample\BusinessService.Sample\XXX.BusinessService.Sample.Redis.json">> C:\Users\admin\Desktop\consul064\consul-temp\templates\logs\consul.log 2>&1
-consul后是consul的webui接口 ，用web管理consul就用的8500端口。
-template 后面是模板参数 第一个是模板地址 。冒号后的第二个参数是输出位置。
>> 后为日志输出路径
注册完成后即可动态触发XXX.BusinessService.Sample.Redis.json文件的动态修改了，比如关闭一个Redis实例或者Master节点发生变更时即可及时完成更新。
Redis的Master节点更新
master节点的更新此处可以通过监听redis哨兵的+switch-master事件监听，触发修改Consul配置中心中指定Redis Key完成。
consul命令行查看服务_Consul集群高可用测试
通过Consul集群的failover测试，我们可以了解到它的高可用工作机制。
consul节点IP角色consul版本日志路径consul01192.168.101.11server0.9.3/var/log/consulconsul02192.168.101.12server0.9.3/var/log/consulconsul03192.168.101.13server(leader)0.9.3/var/log/consul
注：我们是验证consul cluster的高可用性，而非验证consul自带的Geo failover(注册服务的故障转移功能)。
当前集群初始状态(client节点忽略)
[root@consul01 consul]# ./consul operator raft list-peersNode ID Address State Voter RaftProtocolconsul02 192.168.101.12:8300 192.168.101.12:8300 follower true 2consul03 192.168.101.13:8300 192.168.101.13:8300 leader true 2consul01 192.168.101.11:8300 192.168.101.11:8300 follower true 2[root@consul01 consul]# ./consul membersNode Address Status Type Build Protocol DC Segmentconsul01 192.168.101.11:8301 alive server 0.9.3 2 dc consul02 192.168.101.12:8301 alive server 0.9.3 2 dc consul03 192.168.101.13:8301 alive server 0.9.3 2 dc ...TSOP域的consul为3个server节点集群，理论上支持最多一个server节点故障，因此我们测试一个server节点故障是否影响集群
consul cluster 模拟故障测试
§1. 停掉一个follower server节点
此处以consul01节点为例
[root@consul01 consul]# systemctl stop consul其它节点日志信息
[root@consul02 ~]# tail -100 /var/log/consul 2019/02/12 02:30:38 [INFO] serf: EventMemberFailed: consul01.dc 192.168.101.11 2019/02/12 02:30:38 [INFO] consul: Handled member-failed event for server "consul01.dc" in area "wan" 2019/02/12 02:30:39 [INFO] serf: EventMemberLeave: consul01 192.168.101.11在其它节点查看集群信息
[root@consul03 consul]# ./consul operator raft list-peersNode ID Address State Voter RaftProtocolconsul02 192.168.101.12:8300 192.168.101.12:8300 follower true 2consul03 192.168.101.13:8300 192.168.101.13:8300 leader true 2[root@consul03 consul]# ./consul membersNode Address Status Type Build Protocol DC Segmentconsul01 192.168.101.11:8301 left server 0.9.3 2 dc consul02 192.168.101.12:8301 alive server 0.9.3 2 dc consul03 192.168.101.13:8301 alive server 0.9.3 2 dc ...查看集群是否正常(查询注册服务，若没有服务可以通过consul api手动创建一个)
[root@consul03 consul]# ./consul catalog servicesconsultest-csdemo-v0-snapshottest-zuul-v0-snapshot可以看到，被停止的server节点处于left状态，但集群仍正常可用
因此，
server节点，不会影响consul集群服务```。
恢复此server节点```shell[root@consul03 consul]# ./consul operator raft list-peersNode ID Address State Voter RaftProtocolconsul02 192.168.101.12:8300 192.168.101.12:8300 follower true 2consul03 192.168.101.13:8300 192.168.101.13:8300 leader true 2consul01 192.168.101.11:8300 192.168.101.11:8300 follower true 2[root@consul03 consul]# ./consul membersNode Address Status Type Build Protocol DC Segmentconsul01 192.168.101.11:8301 alive server 0.9.3 2 dc consul02 192.168.101.12:8301 alive server 0.9.3 2 dc consul03 192.168.101.13:8301 alive server 0.9.3 2 dc ...其它节点已检测并将此节点加入
[root@consul02 ~]# tail -100 /var/log/consul 2019/02/12 02:43:51 [INFO] serf: EventMemberJoin: consul01.dc 192.168.101.11 2019/02/12 02:43:51 [INFO] consul: Handled member-join event for server "consul01.dc" in area "wan" 2019/02/12 02:43:51 [INFO] serf: EventMemberJoin: consul01 192.168.101.11 2019/02/12 02:43:51 [INFO] consul: Adding LAN server consul01 (Addr: tcp/192.168.101.11:8300) (DC: dc)§1. 停掉leader server节点
此处以consul03节点为例
[root@consul03 consul]# systemctl stop consul其它follow server节点检测到leader节点下线，并重新选举leader
[root@consul02 ~]# tail -100 /var/log/consul 2019/02/12 02:48:27 [INFO] serf: EventMemberLeave: consul03.dc 192.168.101.13 2019/02/12 02:48:27 [INFO] consul: Handled member-leave event for server "consul03.dc" in area "wan" 2019/02/12 02:48:28 [INFO] serf: EventMemberLeave: consul03 192.168.101.13 2019/02/12 02:48:28 [INFO] consul: Removing LAN server consul03 (Addr: tcp/192.168.101.13:8300) (DC: dc) 2019/02/12 02:48:37 [WARN] raft: Rejecting vote request from 192.168.101.11:8300 since we have a leader: 192.168.101.13:8300 2019/02/12 02:48:39 [ERR] agent: Coordinate update error: No cluster leader 2019/02/12 02:48:39 [WARN] raft: Heartbeat timeout from "192.168.101.13:8300" reached, starting election 2019/02/12 02:48:39 [INFO] raft: Node at 192.168.101.12:8300 [Candidate] entering Candidate state in term 5 2019/02/12 02:48:43 [ERR] http: Request GET /v1/catalog/services, error: No cluster leader from=127.0.0.1:44370 2019/02/12 02:48:43 [ERR] http: Request GET /v1/catalog/nodes, error: No cluster leader from=127.0.0.1:36548 2019/02/12 02:48:44 [INFO] raft: Node at 192.168.101.12:8300 [Follower] entering Follower state (Leader: "") 2019/02/12 02:48:44 [INFO] consul: New leader elected: consul01
可以从日志中看到，consul01已被选为新的leader，查看集群信息
[root@consul02 consul]# ./consul operator raft list-peersNode ID Address State Voter RaftProtocolconsul02 192.168.101.12:8300 192.168.101.12:8300 follower true 2consul01 192.168.101.11:8300 192.168.101.11:8300 leader true 2[root@consul02 consul]# ./consul membersNode Address Status Type Build Protocol DC Segmentconsul01 192.168.101.11:8301 alive server 0.9.3 2 dc consul02 192.168.101.12:8301 alive server 0.9.3 2 dc consul03 192.168.101.13:8301 left server 0.9.3 2 dc 可以看到，被停止的leader server节点处于left状态，但集群仍正常可用，查询服务验证
[root@consul02 consul]# ./consul catalog servicesconsultest-csdemo-v0-snapshottest-zuul-v0-snapshot因此，
server节点，也不会影响consul集群服务。
我们再恢复此节点到consul集群中```shell[root@consul03 consul]# systemctl start consul
可以通过它的日志看到它现在变成了follower server
[root@consul03 ~]# tail -f /var/log/consul 2019/02/12 03:01:33 [INFO] raft: Node at 192.168.101.13:8300 [Follower] entering Follower state (Leader: "") 2019/02/12 03:01:33 [INFO] serf: Ignoring previous leave in snapshot 2019/02/12 03:01:33 [INFO] agent: Retry join LAN is supported for: aws azure gce softlayer 2019/02/12 03:01:33 [INFO] agent: Joining LAN cluster... 2019/02/12 03:01:33 [INFO] agent: (LAN) joining: [consul01 consul02 consul03]新的leader和follower上也会更新节点信息
[root@consul01 ~]# tail -f /var/log/consul 2019/02/12 03:01:33 [INFO] serf: EventMemberJoin: consul03.dc 192.168.101.13 2019/02/12 03:01:33 [INFO] consul: Handled member-join event for server "consul03.dc" in area "wan" 2019/02/12 03:01:33 [INFO] serf: EventMemberJoin: consul03 192.168.101.13 2019/02/12 03:01:33 [INFO] consul: Adding LAN server consul03 (Addr: tcp/192.168.101.13:8300) (DC: dc) 2019/02/12 03:01:33 [INFO] raft: Updating configuration with AddStaging (192.168.101.13:8300, 192.168.101.13:8300) to [{Suffrage:Voter ID:192.168.101.12:8300 Address:192.168.101.12:8300} {Suffrage:Voter ID:192.168.101.11:8300 Address:192.168.101.11:8300} {Suffrage:Voter ID:192.168.101.13:8300 Address:192.168.101.13:8300}] 2019/02/12 03:01:33 [INFO] raft: Added peer 192.168.101.13:8300, starting replication 2019/02/12 03:01:33 [WARN] raft: AppendEntries to {Voter 192.168.101.13:8300 192.168.101.13:8300} rejected, sending older logs (next: 394016) 2019/02/12 03:01:33 [INFO] consul: member ''consul03'' joined, marking health alive 2019/02/12 03:01:33 [INFO] raft: pipelining replication to peer {Voter 192.168.101.13:8300 192.168.101.13:8300}再次查看集群信息
[root@consul01 consul]# ./consul operator raft list-peersNode ID Address State Voter RaftProtocolconsul02 192.168.101.12:8300 192.168.101.12:8300 follower true 2consul01 192.168.101.11:8300 192.168.101.11:8300 leader true 2consul03 192.168.101.13:8300 192.168.101.13:8300 follower true 2[root@consul01 consul]# ./consul membersNode Address Status Type Build Protocol DC Segmentconsul01 192.168.101.11:8301 alive server 0.9.3 2 dc consul02 192.168.101.12:8301 alive server 0.9.3 2 dc consul03 192.168.101.13:8301 alive server 0.9.3 2 dc 集群各节点均已正常，此时仅leader角色改变，不影响集群对外提供服务。
关于Mysql MGR + Consul + Consul-template + Haproxy 搭建mysql 高可用集群 和二的介绍现已完结，谢谢您的耐心阅读，如果想了解更多关于032：基于 Consul 和 MGR 的 MySQL 高可用架构、Consul Template、consul-template + nginx部署高可用负载均衡、consul命令行查看服务_Consul集群高可用测试的相关知识，请在本站寻找。
本文标签：
![[转帖]Ubuntu 安装 Wine方法（ubuntu如何安装wine）](https://www.gvkun.com/zb_users/cache/thumbs/4c83df0e2303284d68480d1b1378581d-180-120-1.jpg)
