华为CCE安装isito实现ASM服务网格

华为云在国内上线的有商业版的服务网格产品ASM(基于isito的商业产品),不过在海外regions没有上线,刚好遇的一个项目需要在华为云CCE上使用服务网格,跟同事简单沟通了下客户需求,就手动安装了一个isito实现服务网格。

一、安装istio

istio的安装配置需要使用到istioctl指令,这里使用官方方法,比较简单:

curl -L https://istio.io/downloadIstio | sh -
# 可以通过以下参数指定版本和平台
curl -L https://istio.io/downloadIstio | ISTIO_VERSION=1.13.3 TARGET_ARCH=x86_64 sh -

上面的步骤是把isito下载到本地目录并解压,需要将istioctl指令放到/usr/local/bin目录,或者配置直接通过环境变量配置实现生效:

export PATH=PWD/bin:PATH

istio profile有几个选择,简单一点可以选择demo方式安装,具体的差别见下表:
isito-profile

该表的官方链接为:https://istio.io/latest/docs/setup/additional-setup/config-profiles/ 。

出于测试目的,这里选择demo安装,并给default namespace打上相关lable:

$ istioctl install --set profile=demo -y
$ kubectl label namespace default istio-injection=enabled

二、安装bookinfo测试应用

bookinfo是istio进行功能测试的一个微服务程序,其在第一步执行的curl目录里已经附带了,如果没有,也可以通过 https://github.com/istio/istio.git 链接拿到。
应用安装:

[root@testcce-68506-l3jp4 istio]# kubectl apply -f samples/bookinfo/platform/kube/bookinfo.yaml
service/details created
serviceaccount/bookinfo-details created
deployment.apps/details-v1 created
service/ratings created
serviceaccount/bookinfo-ratings created
deployment.apps/ratings-v1 created
service/reviews created
serviceaccount/bookinfo-reviews created
deployment.apps/reviews-v1 created
deployment.apps/reviews-v2 created
deployment.apps/reviews-v3 created
service/productpage created
serviceaccount/bookinfo-productpage created
deployment.apps/productpage-v1 created
[root@testcce-68506-l3jp4 istio]# kubectl get services
NAME          TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)          AGE
details       ClusterIP   10.247.188.112   <none>        9080/TCP         32s
kubernetes    ClusterIP   10.247.0.1       <none>        443/TCP          7d18h
productpage   ClusterIP   10.247.129.177   <none>        9080/TCP         32s
ratings       ClusterIP   10.247.126.86    <none>        9080/TCP         32s
reviews       ClusterIP   10.247.43.118    <none>        9080/TCP         32s
[root@testcce-68506-l3jp4 istio]# kubectl get pods
NAME                              READY   STATUS    RESTARTS   AGE
details-v1-79f774bdb9-xmsw9       2/2     Running   0          3m1s
productpage-v1-699b85f86b-wdx6r   2/2     Running   0          3m1s
ratings-v1-b6994bb9-wtwmg         2/2     Running   0          3m
reviews-v1-6ff96557f5-s7xgw       2/2     Running   0          3m1s
reviews-v2-7f87f9d489-phvv6       2/2     Running   0          3m1s
reviews-v3-7f59677cc9-gff95       2/2     Running   0          3m1s

安装完成后,可以通过容器内部地址访问,命令如下:

[root@testcce-68506-l3jp4 istio]# kubectl exec "$(kubectl get pod -l app=ratings -o jsonpath='{.items[0].metadata.name}')" -c ratings -- curl -sS productpage:9080/productpage | grep -o "<title>.*</title>"
<title>Simple Bookstore App</title>

三、外部访问配置

外部访问需要使用到 Istio Ingress Gateway,这个可以通过如下指令进行安装:

[root@testcce-68506-l3jp4 istio]# kubectl apply -f samples/bookinfo/networking/bookinfo-gateway.yaml
gateway.networking.istio.io/bookinfo-gateway created
virtualservice.networking.istio.io/bookinfo created
# 以下指令确认没有报错
[root@testcce-68506-l3jp4 istio]# istioctl analyze
✔ No validation issues found when analyzing namespace: default.

这个时候在华为CCE上查看会发现还会有问题,因为对应的ELB地址会一直获取不到,如下:

[root@testcce-68506-l3jp4 istio]# kubectl get svc istio-ingressgateway -n istio-system
NAME                   TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)                                                                      AGE
istio-ingressgateway   LoadBalancer   10.247.224.23   <pending>     15021:30371/TCP,80:31414/TCP,443:32343/TCP,31400:31843/TCP,15443:32645/TCP   71m

会发现其状态一直是pending状态,其实这个问题,官方已经在相关文档中做了说明:

If the EXTERNAL-IP value is set, your environment has an external load balancer that you can use for the ingress gateway. If the EXTERNAL-IP value is (or perpetually ), your environment does not provide an external load balancer for the ingress gateway. In this case, you can access the gateway using the service’s node port.

上面是告诉我们需要配置一个外部LB地址,如果LoadBalancer不行,可以换为node port 。而华为云CCE是支持ELB配置的,这在isito的文档中也进行了特别的说明:https://istio.io/latest/docs/setup/platform-setup/huaweicloud/
huaweicloud-istio-elb
这里我先在华为云上创建了一个ELB,点开最前面的名称(elb-361way)就可以拿到id等详细信息,这部分一会儿我们需要进行替换配置:

kubectl apply -f - <<EOF
apiVersion: v1
kind: Service
metadata:
  annotations:
    kubernetes.io/elb.class: union
    kubernetes.io/elb.id: 7081b003-0c99-4ece-8da8-4bd07fea9fd4 # ELB ID
    kubernetes.io/elb.lb-algorithm: ROUND_ROBIN
  labels:
    app: istio-ingressgateway
    install.operator.istio.io/owning-resource: unknown
    install.operator.istio.io/owning-resource-namespace: istio-system
    istio: ingressgateway
    istio.io/rev: default
    operator.istio.io/component: IngressGateways
    operator.istio.io/managed: Reconcile
    operator.istio.io/version: 1.9.0
    release: istio
  name: istio-ingressgateway
  namespace: istio-system
spec:
  clusterIP: 10.247.224.23        ## clusterIP
  externalTrafficPolicy: Cluster
  loadBalancerIP: 110.238.85.11     ## ELB EIP
  ports:
  - name: status-port
    nodePort: 32484
    port: 15021
    protocol: TCP
    targetPort: 15021
  - name: http2
    nodePort: 30294
    port: 80
    protocol: TCP
    targetPort: 8080
  - name: https
    nodePort: 31301
    port: 443
    protocol: TCP
    targetPort: 8443
  - name: tcp
    nodePort: 30229
    port: 31400
    protocol: TCP
    targetPort: 31400
  - name: tls
    nodePort: 32028
    port: 15443
    protocol: TCP
    targetPort: 15443
  selector:
    app: istio-ingressgateway
    istio: ingressgateway
  sessionAffinity: None
  type: LoadBalancer
EOF

注意这里需要修改的地方有三部分,除了官方文档标注的ELB的两部分外,还需要修改clusterIP的地址,应用生效后,我们可以再查看下SVC信息:

[root@testcce-68506-l3jp4 istio]# kubectl get svc istio-ingressgateway -n istio-system
NAME                   TYPE           CLUSTER-IP      EXTERNAL-IP     PORT(S)                                                                      AGE
istio-ingressgateway   LoadBalancer   10.247.224.23   110.238.85.11   15021:32484/TCP,80:30294/TCP,443:31301/TCP,31400:30229/TCP,15443:32028/TCP   158m

四、验证确认

这时候我们可以通过内部变量获取访问信息,可以直接在浏览器上访问,也可以在命令行下验证:

[root@testcce-68506-l3jp4 istio]# export INGRESS_HOST=(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
 -n istio-system get service istio-ingressgateway -o jsonpath='{.spec.ports[?(@.name=="http2")].port}')
export SECURE_INGRESS_PORT=(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.spec.ports[?(@.name=="https")].port}')
[root@testcce-68506-l3jp4 istio]# export INGRESS_PORT=(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.spec.ports[?(@.name=="http2")].port}')
[root@testcce-68506-l3jp4 istio]# export SECURE_INGRESS_PORT=(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.spec.ports[?(@.name=="https")].port}')
[root@testcce-68506-l3jp4 istio]# export GATEWAY_URL=INGRESS_HOST:INGRESS_PORT
[root@testcce-68506-l3jp4 istio]# echo "GATEWAY_URL"
110.238.85.11:80
[root@testcce-68506-l3jp4 istio]# echo "http://GATEWAY_URL/productpage"
http://110.238.85.11:80/productpage
[root@testcce-68506-l3jp4 istio]# curl "http://$GATEWAY_URL/productpage"

还可以通过Kiali的界面来查看具体的调用链拓扑和访问信息。这里需要先安装下相关配套插件:

[root@testcce-68506-l3jp4 istio]# kubectl apply -f samples/addons
serviceaccount/grafana created
configmap/grafana created
service/grafana created
deployment.apps/grafana created
configmap/istio-grafana-dashboards created
configmap/istio-services-grafana-dashboards created
deployment.apps/jaeger created
service/tracing created
service/zipkin created
service/jaeger-collector created
serviceaccount/kiali created
configmap/kiali created
clusterrole.rbac.authorization.k8s.io/kiali-viewer created
clusterrole.rbac.authorization.k8s.io/kiali created
clusterrolebinding.rbac.authorization.k8s.io/kiali created
role.rbac.authorization.k8s.io/kiali-controlplane created
rolebinding.rbac.authorization.k8s.io/kiali-controlplane created
service/kiali created
deployment.apps/kiali created
serviceaccount/prometheus created
configmap/prometheus created
Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply
clusterrole.rbac.authorization.k8s.io/prometheus configured
Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply
clusterrolebinding.rbac.authorization.k8s.io/prometheus configured
service/prometheus created
deployment.apps/prometheus created
[root@testcce-68506-l3jp4 istio]# kubectl rollout status deployment/kiali -n istio-system
Waiting for deployment "kiali" rollout to finish: 0 of 1 updated replicas are available...
deployment "kiali" successfully rolled out

安装完成后,将 kiali 服务的管理界面暴漏出去:

[root@testcce-68506-l3jp4 ~]# istioctl dashboard kiali --address 0.0.0.0
http://0.0.0.0:20001/kiali
Failed to open browser; open http://0.0.0.0:20001/kiali in your browser.

因为需求公网访问,我这里把监听地址改成了0.0.0.0,这里需要注意下,需要在云SG安全策略里开下20001端口的访问,这里需要注意下,这里监听的地址不是ELB的地址,是node节点的公网或私网地址。
通过如下命令进行压测:

for i in (seq 1 10000); do curl -s -o /dev/null "http://GATEWAY_URL/productpage"; done

这里通过graph查看下几个图,这里看下workload图,如下:
kiali-graph-workload

如果需要查看进一步测试istio的功能,可以查看 https://istio.io/latest/docs/setup/getting-started/#next-steps 链接里的demo示例,可以对应的测试功能如下:
– Request routing
– Fault injection
– Traffic shifting
– Querying metrics
– Visualizing metrics
– Accessing external services
– Visualizing your mesh

五、删除

清理战场的命令如下:

kubectl delete -f samples/addons
istioctl manifest generate --set profile=demo | kubectl delete --ignore-not-found=true -f -
istioctl tag remove default
kubectl delete namespace istio-system
kubectl label namespace default istio-injection-

发表评论

您的电子邮箱地址不会被公开。