一、helm 安装 redis
通过 helm 安装redis的方法如下:
[root@testcce-92497 ~]# helm install my-release oci://registry-1.docker.io/bitnamicharts/redis --set global.storageClass=csi-disk
[root@testcce-92497 ~]# export REDIS_PASSWORD=$(kubectl get secret --namespace default my-release-redis -o jsonpath="{.data.redis-password}" | base64 -d)
[root@testcce-92497 ~]# echo $REDIS_PASSWORD
ejL6JhgNNs
[root@testcce-92497 ~]# kubectl get pods
NAME READY STATUS RESTARTS AGE
my-release-redis-master-0 1/1 Running 0 2m41s
my-release-redis-replicas-0 1/1 Running 0 2m41s
my-release-redis-replicas-1 1/1 Running 0 100s
my-release-redis-replicas-2 1/1 Running 0 62s
[root@testcce-92497 ~]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.247.0.1 <none> 443/TCP 8d
my-release-redis-headless ClusterIP None <none> 6379/TCP 13m
my-release-redis-master ClusterIP 10.247.153.87 <none> 6379/TCP 13m
my-release-redis-replicas ClusterIP 10.247.3.245 <none> 6379/TCP 13m
这里默认安装的是 1 master 3 replicas 架构,这个可以从 artifacthub 网站上查到。安装完成后,可以通过如下命令进行连接:
REDISCLI_AUTH="$REDIS_PASSWORD" redis-cli -h my-release-redis-master
REDISCLI_AUTH="$REDIS_PASSWORD" redis-cli -h my-release-redis-replicas
二、helm 安装 redis cluster
bitnami 除了提供便捷的 redis 安装外,还提供了 redis cluster的 helm 安装包,具体安装命令如下:
[root@testcce-92497 ~]# helm install my-release oci://registry-1.docker.io/bitnamicharts/redis-cluster --set global.storageClass=csi-disk
[root@testcce-92497 ~]# export REDIS_PASSWORD=$(kubectl get secret --namespace "default" my-release-redis-cluster -o jsonpath="{.data.redis-password}" | base64 -d)
[root@testcce-92497 ~]# echo $REDIS_PASSWORD
iYcK8ISfoF
[root@testcce-92497 ~]# kubectl get pods
NAME READY STATUS RESTARTS AGE
my-release-redis-cluster-0 1/1 Running 0 87s
my-release-redis-cluster-1 1/1 Running 0 87s
my-release-redis-cluster-2 1/1 Running 0 87s
my-release-redis-cluster-3 1/1 Running 0 87s
my-release-redis-cluster-4 1/1 Running 0 87s
my-release-redis-cluster-5 1/1 Running 0 87s
[root@testcce-92497 ~]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.247.0.1 <none> 443/TCP 8d
my-release-redis-cluster ClusterIP 10.247.1.249 <none> 6379/TCP 12m
my-release-redis-cluster-headless ClusterIP None <none> 6379/TCP,16379/TCP 12m
[root@testcce-92497 ~]# kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
redis-data-my-release-redis-cluster-0 Bound pvc-55a55e1d-66ac-4c38-9d57-e3407de450e9 8Gi RWO csi-disk 15s
redis-data-my-release-redis-cluster-1 Bound pvc-78fcbc7c-7909-4462-834d-38342cae9dea 8Gi RWO csi-disk 15s
redis-data-my-release-redis-cluster-2 Bound pvc-ad7ffaee-a6fe-4273-b03d-648b5bf10072 8Gi RWO csi-disk 15s
redis-data-my-release-redis-cluster-3 Bound pvc-5cb0e3f7-57a9-4694-9350-c1325f93cef9 8Gi RWO csi-disk 15s
redis-data-my-release-redis-cluster-4 Bound pvc-c1b16f6a-bce1-4e87-a016-c573e77008f3 8Gi RWO csi-disk 15s
redis-data-my-release-redis-cluster-5 Bound pvc-5c14c80b-197e-412b-af74-674d5cd9ed1f 8Gi RWO csi-disk 15
执行连接测试:
[root@testcce-92497 ~]# kubectl exec -it my-release-redis-cluster-0 -- /bin/bash
I have no name!@my-release-redis-cluster-0:/$ redis-cli -c -h my-release-redis-cluster -a $REDIS_PASSWORD
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
my-release-redis-cluster:6379> set site www.361way.com
-> Redirected to slot [9421] located at 10.0.0.10:6379
OK
10.0.0.10:6379> get site
"www.361way.com"
10.0.0.10:6379>
查看其对应的配置文件内容如下:
my-release-redis-cluster-0:/etc$ cat /opt/bitnami/redis/etc/redis.conf |grep -v ^#|grep -v ^$
bind 0.0.0.0 ::
bind 0.0.0.0 ::
bind 0.0.0.0 ::
bind 0.0.0.0 ::
protected-mode yes
port 6379
tcp-backlog 511
timeout 0
tcp-keepalive 300
port 6379
daemonize no
pidfile /opt/bitnami/redis/tmp/redis_6379.pid
loglevel notice
logfile ""
databases 16
always-show-logo yes
set-proc-title yes
proc-title-template "{title} {listen-addr} {server-mode}"
save 900 1
save 300 10
save 60 10000
stop-writes-on-bgsave-error yes
rdbcompression yes
rdbchecksum yes
dbfilename dump.rdb
rdb-del-sync-files no
dir /bitnami/redis/data
replica-serve-stale-data yes
replica-read-only yes
repl-diskless-sync no
repl-diskless-sync-delay 5
repl-diskless-sync-max-replicas 0
repl-diskless-load disabled
repl-disable-tcp-nodelay no
replica-priority 100
acllog-max-len 128
requirepass iYcK8ISfoF
lazyfree-lazy-eviction no
lazyfree-lazy-expire no
lazyfree-lazy-server-del no
replica-lazy-flush no
lazyfree-lazy-user-del no
lazyfree-lazy-user-flush no
oom-score-adj no
oom-score-adj-values 0 200 800
disable-thp yes
appendonly yes
appendfilename "appendonly.aof"
appenddirname "appendonlydir"
appendfsync everysec
no-appendfsync-on-rewrite no
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
aof-load-truncated yes
aof-use-rdb-preamble yes
aof-timestamp-enabled no
lua-time-limit 5000
cluster-enabled yes
cluster-config-file /bitnami/redis/data/nodes.conf
cluster-preferred-endpoint-type ip
cluster-announce-ip 10.0.0.12
slowlog-log-slower-than 10000
slowlog-max-len 128
latency-monitor-threshold 0
notify-keyspace-events ""
hash-max-listpack-entries 512
hash-max-listpack-value 64
list-max-listpack-size -2
list-compress-depth 0
set-max-intset-entries 512
zset-max-listpack-entries 128
zset-max-listpack-value 64
hll-sparse-max-bytes 3000
stream-node-max-bytes 4096
stream-node-max-entries 100
activerehashing yes
client-output-buffer-limit normal 0 0 0
client-output-buffer-limit replica 256mb 64mb 60
client-output-buffer-limit pubsub 32mb 8mb 60
hz 10
dynamic-hz yes
aof-rewrite-incremental-fsync yes
rdb-save-incremental-fsync yes
jemalloc-bg-thread yes
save ""
三、两者之间的区别
部署 Redis 集群有两种不同的方法,即使用Redis Helm chart 或 Redis cluster Helm chart 。这两种解决方案都提供了一种在生产环境中运行 Redis 的简单可靠的方法。
- Redis cluster Helm chart 默认配置了一个6个节点的集群,多个写入点(三个master)和三个slave节点。Redis Helm Chart 默认部署 4 个节点,其中只有 1 个写入点(1 个主节点)和 3 个副本节点(从节点)。
- Redis Cluster Helm Chart 将部署带有分片的 Redis Cluster 拓扑,Redis Cluster 使用Redis Sentinel部署主从集群。
- Redis Cluster 仅支持一个数据库(如果您有一个大数据集则表明),而 Redis 支持多个数据库。
- Redis Cluster客户端必须使用重定向,而用于Redis的客户端则不需要。
- Redis Cluster Helm chart 允许用户从外部和内部访问集群,并且在两种访问中都可以对集群进行扩容和缩容。
Redis Cluster Helm chart 还包括一项附加功能:灾难恢复和故障转移。如果主节点甚至所有节点宕机,集群会自动恢复并提升新的主节点,以维持集群的平衡并确保读写操作继续不间断。
四、Redis 主要的架构
Redis Helm chart 支持的架构如下:
Default: Master-Replicas
When installing the chart with architecture=replication, it will deploy a Redis™ master StatefulSet (only one master node allowed) and a Redis™ replicas StatefulSet. The replicas will be read-replicas of the master. Two services will be exposed:
- Redis™ Master service: Points to the master, where read-write operations can be performed
- Redis™ Replicas service: Points to the replicas, where only read operations are allowed.
In case the master crashes, the replicas will wait until the master node is respawned again by the Kubernetes Controller Manager.
Standalone
When installing the chart with architecture=standalone, it will deploy a standalone Redis™ StatefulSet (only one node allowed) and a Redis™ replicas StatefulSet. A single service will be exposed:
- Redis™ Master service: Points to the master, where read-write operations can be performed
Master-Replicas with Sentinel
When installing the chart with architecture=replication and sentinel.enabled=true, it will deploy a Redis™ master StatefulSet (only one master allowed) and a Redis™ replicas StatefulSet. In this case, the pods will contain an extra container with Redis™ Sentinel. This container will form a cluster of Redis™ Sentinel nodes, which will promote a new master in case the actual one fails. In addition to this, only one service is exposed:
- Redis™ service: Exposes port 6379 for Redis™ read-only operations and port 26379 for accessing Redis™ Sentinel.
For read-only operations, access the service using port 6379. For write operations, it’s necessary to access the Redis™ Sentinel cluster and query the current master using the command below (using redis-cli or similar):
SENTINEL get-master-addr-by-name <name of your MasterSet. e.g: mymaster>
This command will return the address of the current master, which can be accessed from inside the cluster.
In case the current master crashes, the Sentinel containers will elect a new master node.