LVS高可用（七）LVS+keepalived双主

发表于2016-05-10 作者 admin

在 <a href="https://www.361way.com/lvs-keepalived-dr-master-backup/5221.html" target="_blank" rel="noopener">LVS高可用（六）LVS+keepalived主从</a> 篇中提到了LVS的高可用及后端的LB，不过一台LVS提供服务，另一台只做为backup，显然是种浪费，实际现网应用中比较多的用法是，两台LVS都作为MASTER节点，互相做为另一台的backup 。具体架构如下：



<img src="https://www.361way.com/wp-content/uploads/2016/05/keepalived-two-master.png" title="lvs keepalived master master" alt="lvs keepalived master master" width="627" height="231" />



注：与master-backup 架构相比， master-master架构需要多加一个VIP地址。

一、IP及规划

<table class="" style="width:226px;" cellspacing="0" cellpadding="0" border="1">
    <tbody>
        <tr>
            <td rowspan="2" class="xl65" width="96" height="38">
                realserver
            </td>
            <td class="xl65" width="130">
                192.168.122.10
            </td>
        </tr>
        <tr>
            <td class="xl65" width="130" height="19">
                192.168.122.20
            </td>
        </tr>
        <tr>
            <td rowspan="2" class="xl65" width="96" height="38">
                director
            </td>
            <td class="xl65" width="130">
                192.168.122.30
            </td>
        </tr>
        <tr>
            <td class="xl65" width="130" height="19">
                192.168.122.40
            </td>
        </tr>
        <tr>
            <td rowspan="2" class="xl66" width="96" height="38">
                VIP
            </td>
            <td class="xl65" width="130">
                192.168.122.100
            </td>
        </tr>
        <tr>
            <td class="xl65" width="130" height="19">
                192.168.122.110
            </td>
        </tr>
    </tbody>
</table>



同上一篇，两台realserver还是安装httpd ，两台director安装ipvsadm、keepalived 。realserver的回环地址上会配置两个VIP的地址。



另外操作前，需要先将几台主机的时间同步掉。

二、director主机配置

<strong>MASTER HostA主机的配置如下</strong>：



<br />

[root@lvs-dr ~]# cat /etc/keepalived/keepalived.conf
vrrp_instance bl_one {
    state MASTER             #指定Keepalived的角色，MASTER为主服务器，BACKUP为备用服务器
    interface eth0           #指定HA监测的接口
    lvs_sync_daemon_interface eth0
    virtual_router_id 38     #虚拟路由标识(1-255)，在一个VRRP实例中主备服务器ID必须一样
    priority 150             #优先级，数字越大越优先，主服务器优先级必须高于备服务器
    advert_int 3             #设置主备之间同步检查时间间隔，单位秒
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress { #定义虚拟IP地址
      192.168.122.100
    }
}
vrrp_instance bl_two {
    state BACKUP
    interface eth0
    lvs_sync_daemon_interface eth0
    virtual_router_id 48
    priority 120
    advert_int 3
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
      192.168.122.110
    }
}
virtual_server 192.168.122.100 80 {
    delay_loop 3                    #设置健康状态检查时间
    lb_algo rr                      #设置负载调度算法
    lb_kind DR                      #设置LVS实现负载均衡的机制
    persistence_timeout 50          #会话保持时间
    protocol TCP
    real_server 192.168.122.10 80 {
        weight 1
        TCP_CHECK {
            connect_timeout 10       #设置响应超时时间
            nb_get_retry 3           #设置超时重试次数
            delay_before_retry 3     #设置超时重试间隔
            connect_port 80
        }
    }
    real_server 192.168.122.20 80 {
        weight 1
        TCP_CHECK {
            connect_timeout 10
            nb_get_retry 3
            delay_before_retry 3
            connect_port 80
        }
    }
}
virtual_server 192.168.122.110 80 {
    delay_loop 3
    lb_algo rr
    lb_kind DR
    persistence_timeout 50
    protocol TCP
    real_server 192.168.122.10 80 {
        weight 1
        HTTP_GET {
            url {
                 path /index.html
                 status_code 200
              }
            connect_timeout 10
            nb_get_retry 3
            delay_before_retry 3
            connect_port 80
        }
    }
    real_server 192.168.122.20 80 {
        weight 1
        HTTP_GET {
            url {
                 path /index.html
                 status_code 200
              }
            connect_timeout 10
            nb_get_retry 3
            delay_before_retry 3
            connect_port 80
        }
    }
}

<strong>MASTER HostB主机的配置如下</strong>：



<br />

[root@lvs-dr2 ~]# cat /etc/keepalived/keepalived.conf
vrrp_instance bl_one {
    state BACKUP
    interface eth0
    lvs_sync_daemon_interface eth0
    virtual_router_id 38
    priority 120
    advert_int 3
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
      192.168.122.100
    }
}
vrrp_instance bl_two {
    state MASTER
    interface eth0
    lvs_sync_daemon_interface eth0
    virtual_router_id 48
    priority 150
    advert_int 3
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
      192.168.122.110
    }
}
virtual_server 192.168.122.100 80 {
    delay_loop 3
    lb_algo rr
    lb_kind DR
    persistence_timeout 50
    protocol TCP
    real_server 192.168.122.10 80 {
        weight 1
        TCP_CHECK {
            connect_timeout 10
            nb_get_retry 3
            delay_before_retry 3
            connect_port 80
        }
    }
    real_server 192.168.122.20 80 {
        weight 1
        TCP_CHECK {
            connect_timeout 10
            nb_get_retry 3
            delay_before_retry 3
            connect_port 80
        }
    }
}
virtual_server 192.168.122.110 80 {
    delay_loop 3
    lb_algo rr
    lb_kind DR
    persistence_timeout 50
    protocol TCP
    real_server 192.168.122.10 80 {
        weight 1
        HTTP_GET {
            url {
                 path /index.html
                 status_code 200
              }
            connect_timeout 10
            nb_get_retry 3
            delay_before_retry 3
            connect_port 80
        }
    }
    real_server 192.168.122.20 80 {
        weight 1
        HTTP_GET {
            url {
                 path /index.html
                 status_code 200
              }
            connect_timeout 10
            nb_get_retry 3
            delay_before_retry 3
            connect_port 80
        }
    }
}

以上创建了两个实例lb_one、lb_two ，其中A主机做为VIP1地址的master，B主机做为VIP2地址的master，A、B主机互为backup 。另外还有一个技巧点，在master-backup架构中也可以配置多个VIP地址，只需要在实例的VIP配置中增加地址、并在后面指定对应的虚地址后面的realserver即可。在主备脚架中配置多个VIP的方法如下：



<br />

virtual_ipaddress {
  192.168.122.100
  192.168.122.110
  ………………
}

<br />



<br />

三、realserver主机配置

<br />



两台realserver主机的配置启停脚本如下：



<br />

[root@lvs01 ~]# cat dr_client.sh
#!/bin/bash
VIP1=192.168.122.100
VIP2=192.168.122.110
#vip's broadcast
BROADCAST=192.168.122.255
. /etc/rc.d/init.d/functions
case "$1" in
 start)
  echo "reparing for Real Server"
    echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
    echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
    echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
    echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
    ifconfig lo:0 $VIP1 netmask 255.255.255.255 broadcast $BROADCAST up
    ifconfig lo:1 $VIP2 netmask 255.255.255.255 broadcast $BROADCAST up
     /sbin/route add -host $VIP1 dev lo:0
     /sbin/route add -host $VIP2 dev lo:0
     ;;
 stop)
     ifconfig lo:0 down
     ifconfig lo:1 down
    echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore
    echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce
    echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore
    echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce
     ;;
 *)
     echo "Usage: lvs {start|stop}"
     exit 1
esac

四、验证

启动服务后，查看两台director和realserver主机的地址信息如下：



master hostA主机的IP信息如下：



<br />

[root@lvs-dr ~]# ip add show
1: lo:  mtu 65536 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0:  mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 52:54:00:39:42:be brd ff:ff:ff:ff:ff:ff
    inet 192.168.122.30/24 brd 192.168.122.255 scope global eth0
    inet 192.168.122.100/32 scope global eth0
    inet6 fe80::5054:ff:fe39:42be/64 scope link
       valid_lft forever preferred_lft forever

<br />



master hostB主机的IP信息如下：



<br />

[root@lvs-dr2 ~]# ip add show
1: lo:  mtu 65536 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0:  mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 52:54:00:7b:f2:1a brd ff:ff:ff:ff:ff:ff
    inet 192.168.122.40/24 brd 192.168.122.255 scope global eth0
    inet 192.168.122.110/32 scope global eth0
    inet6 fe80::5054:ff:fe7b:f21a/64 scope link
       valid_lft forever preferred_lft forever

realserver主机的IP信息如下：



<br />

[root@lvs01 ~]# ip add show
1: lo:  mtu 65536 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet 192.168.122.100/32 brd 192.168.122.255 scope global lo:0
    inet 192.168.122.110/32 brd 192.168.122.255 scope global lo:1
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0:  mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 52:54:00:a9:3b:6a brd ff:ff:ff:ff:ff:ff
    inet 192.168.122.10/24 brd 192.168.122.255 scope global eth0
    inet6 fe80::5054:ff:fea9:3b6a/64 scope link
       valid_lft forever preferred_lft forever

以上可以看到，VIP在master节点上可以看到，另一个VIP同样只在该实例的master上可以看到，在两台realserver上的回环地址上有两个VIP地址。



分别到各VIP的主节点上通过ipvadm查看，可以获取到对的连接信息，如下：



<br />

[root@lvs-dr ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.122.100:80 rr persistent 1
  -> 192.168.122.10:80            Route   1      0          12
  -> 192.168.122.20:80            Route   1      0          11
TCP  192.168.122.110:80 rr persistent 1
  -> 192.168.122.10:80            Route   1      0          0
  -> 192.168.122.20:80            Route   1      0          0

更详细的测试步骤略过 。

发表回复取消回复