1. 程式人生 > >Kebernetes學習總結(5) Ingress

Kebernetes學習總結(5) Ingress

大致 uwsgi http exec 就是 rule nts 生產環境 read

雖然kubernetes集群內部署的pod、server都有自己的IP,但是卻無法提供外網訪問,以前可以通過監聽NodePort的方式暴露服務,但是這種方式並不靈活,生產環境也不建議使用。Ingresss是kubernetes集群中的一個API資源對象,扮演邊緣路由器(edge router)的角色,也可以理解為集群防火墻、集群網關,可以自定義路由規則來轉發、管理、暴露服務(一組pod),非常靈活,生產環境建議使用這種方式。另外LoadBlancer也可以暴露服務,不過這種方式需要向雲平臺申請負債均衡器;雖然目前很多雲平臺都支持,但是這種方式深度耦合了雲平臺。
Ingress相關組件:
1) Ingress controlle
r
kubernetes中的controller有很多,比如CronJob、DeamonSet、Deployment、ReplicationSet、StatefulSet等,它們的作用就是監控集群的變化,使集群始終保持期望的最終狀態(yml文件)。同理Ingress controller的作用就是實時感知Ingress路由規則集合的變化,再與Api Server交互,獲取Service、Pod在集群中的IP等信息,然後發送給反向代理web服務器(ingress pods),刷新其路由配置信息,這就是它的服務發現機制。
2) 反向代理web服務器(ingress pods) 負責轉發外部請求到後端service,比如Nginx、Apache、traefik等等
3) Ingress 定義路由規則集合
4) NodePort類型的service 負責將外部流量引入kubernetes cluster,然後由反向代理web服務器處理。

技術分享圖片
本篇總結中的示例大致如下:首先部署ingress controller及 ingress pods;然後利用NodePort類型的service將外部流最引入kubernetes cluster中;然後再定義後端deployment及後端service;然後定義ingress 轉發規則集合;最後訪問測試。
1) 部署ingress pods
[root@docker79 ingress]# mkdir example
[root@docker79 ingress]# cd example/

[root@docker79 example]# for file in namespace.yaml configmap.yaml rbac.yaml tcp-services-configmap.yaml with-rbac.yaml default-backend.yaml udp-services-configmap.yaml ; do wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/$file ; done
[root@docker79 example]# kubectl apply -f .
configmap/nginx-configuration created
deployment.extensions/default-http-backend created
service/default-http-backend created
Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply
namespace/ingress-nginx configured
serviceaccount/nginx-ingress-serviceaccount created
clusterrole.rbac.authorization.k8s.io/nginx-ingress-clusterrole created
role.rbac.authorization.k8s.io/nginx-ingress-role created
rolebinding.rbac.authorization.k8s.io/nginx-ingress-role-nisa-binding created
clusterrolebinding.rbac.authorization.k8s.io/nginx-ingress-clusterrole-nisa-binding created
configmap/tcp-services created
configmap/udp-services created
deployment.extensions/nginx-ingress-controller created
[root@docker79 example]#
[root@docker79 example]# kubectl get pods -n ingress-nginx
NAME READY STATUS RESTARTS AGE
default-http-backend-6586bc58b6-8rxtp 1/1 Running 0 59m
nginx-ingress-controller-6bd7c597cb-xdrcm 1/1 Running 0 59m
[root@docker79 example]#

2) 在ingress-nginx 名稱空間定義nodePort類型的service 引入外部流量,過程如下所示:

[root@docker79 ingress]# cat nginx-frontend-ingress-svc.yaml
apiVersion: v1
kind: Service
metadata:
  name: nginx-frontend-ingress-svc
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
spec:
  type: NodePort
  ports:
  - name: http
    port: 80
    targetPort: 80
    protocol: TCP
    nodePort: 30080
  - name: https
    port: 443
    targetPort: 443
    protocol: TCP
    nodePort: 30443
  selector:
    app.kubernetes.io/name: ingress-nginx
[root@docker79 ingress]#
[root@docker79 ingress]# kubectl apply -f nginx-frontend-ingress-svc.yaml
service/nginx-frontend-ingress-svc created
[root@docker79 ingress]# kubectl get svc -n ingress-nginx
NAME                   TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)                      AGE
default-http-backend   ClusterIP   10.102.71.36   <none>        80/TCP                       1h
nginx-frontend-ingress-svc     NodePort    10.103.52.44   <none>        80:30080/TCP,443:30443/TCP   11s
[root@docker79 ingress]#

3) 下例中部署兩個後端nginx-backend-deploy、tomcat-backend-deploy
首先部署nginx-backend-deploy及nginx-backend-svc,過程如下:

[root@docker79 ingress]# vim backend-nginx.yaml
[root@docker79 ingress]# cat backend-nginx.yaml
apiVersion: v1
kind: Service
metadata:
  name: nginx-backend-svc
  namespace: default
spec:
  selector:
    app: nginx-backend
    release: canary
  ports:
  - name: http
    targetPort: 80
    port: 80
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-backend-deploy
  namespace: default
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx-backend
      release: canary
  template:
    metadata:
      labels:
        app: nginx-backend
        release: canary
    spec:
      containers:
      - name: nginx-container
        image: nginx:1.15-alpine
        ports:
        - name: http
          containerPort: 80
[root@docker79 ingress]#
[root@docker79 ingress]# kubectl apply -f backend-nginx.yaml
service/nginx-backend-svc created
deployment.apps/nginx-backend-deploy created
[root@docker79 ingress]# kubectl get pods
NAME                                   READY     STATUS    RESTARTS   AGE
nginx-backend-deploy-9c9b85bf4-6587j   1/1       Running   0          8s
nginx-backend-deploy-9c9b85bf4-k64tv   1/1       Running   0          8s
[root@docker79 ingress]#
[root@docker79 ingress]# kubectl get svc
NAME                TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)   AGE
kubernetes          ClusterIP   10.96.0.1        <none>        443/TCP   3d
nginx-backend-svc   ClusterIP   10.110.150.187   <none>        80/TCP    1m
[root@docker79 ingress]#

然後部署tomcat-backend-deploy及tomcat-backend-svc,過程如下:

[root@docker79 ingress]# vim backend-tomcat.yaml
[root@docker79 ingress]# cat backend-tomcat.yaml
apiVersion: v1
kind: Service
metadata:
  name: tomcat-backend-svc
  namespace: default
spec:
  selector:
    app: tomcat-backend
    release: canary
  ports:
  - name: http
    port: 8080
    targetPort: 8080
  - name: ajp
    port: 8009
    targetPort: 8009
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: tomcat-deploy
  namespace: default
spec:
  replicas: 3
  selector:
    matchLabels:
      app: tomcat-backend
      release: canary
  template:
    metadata:
      labels:
        app: tomcat-backend
        release: canary
    spec:
      containers:
      - name: tomcat
        image: tomcat:8.5.32-jre8-alpine
        ports:
        - name: http
          containerPort: 8080
        - name: ajp
          containerPort: 8009
[root@docker79 ingress]# kubectl apply -f backend-tomcat.yaml
service/tomcat-backend-svc created
deployment.apps/tomcat-deploy created
[root@docker79 ingress]# kubectl get pods
NAME                                   READY     STATUS    RESTARTS   AGE
nginx-backend-deploy-9c9b85bf4-6587j   1/1       Running   0          24m
nginx-backend-deploy-9c9b85bf4-k64tv   1/1       Running   0          24m
tomcat-deploy-7b4c97ff9f-fv8rx         1/1       Running   0          2m
tomcat-deploy-7b4c97ff9f-rj94r         1/1       Running   0          2m
[root@docker79 ingress]# kubectl get svc
NAME                 TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)             AGE
kubernetes           ClusterIP   10.96.0.1        <none>        443/TCP             3d
nginx-backend-svc    ClusterIP   10.110.150.187   <none>        80/TCP              24m
tomcat-backend-svc   ClusterIP   10.105.189.191   <none>        8080/TCP,8009/TCP   2m
[root@docker79 ingress]#

4) 部署 ingress 轉發規則集合
首先部署nginx-backend-svc的ingress 轉發規則集合,過程如下所示:

[root@docker79 ingress]# vim ingress-rules-ngx.yaml
[root@docker79 ingress]# cat ingress-rules-ngx.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ingress-rule-ngx
  namespace: default
  annotations:
    kubernetes.io/ingress.class: "nginx"
spec:
  rules:
  - host: nginx.inspiry.com
    http:
      paths:
      - path:
        backend:
          serviceName: nginx-backend-svc
          servicePort: 80
[root@docker79 ingress]#
[root@docker79 ingress]# kubectl apply -f ingress-rules.yaml
ingress.extensions/ingress-rule-ngx created
[root@docker79 ingress]# kubectl get ingress
NAME               HOSTS               ADDRESS   PORTS     AGE
ingress-rule-ngx   nginx.inspiry.com             80        9s
[root@docker79 ingress]#
[root@docker79 ingress]# kubectl describe ingress  ingress-rule-ngx
Name:             ingress-rule-ngx
Namespace:        default
Address:
Default backend:  default-http-backend:80 (<none>)
Rules:
  Host               Path  Backends
  ----               ----  --------
  nginx.inspiry.com
                        nginx-backend-svc:80 (<none>)
Annotations:
  kubectl.kubernetes.io/last-applied-configuration:  {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernetes.io/ingress.class":"nginx"},"name":"ingress-rule-ngx","namespace":"default"},"spec":{"rules":[{"host":"nginx.inspiry.com","http":{"paths":[{"backend":{"serviceName":"nginx-backend-svc","servicePort":80},"path":null}]}}]}}

  kubernetes.io/ingress.class:  nginx
Events:                         <none>
[root@docker79 ingress]#

然後再部署tomcat-backend-svc的ingress 轉發規則集合,過程如下所示:

[root@docker79 ingress]# vim ingress-rules-tomcat.yaml
[root@docker79 ingress]# cat ingress-rules-tomcat.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ingress-rule-tomcat
  namespace: default
  annotations:
    kubernetes.io/ingress.class: "nginx"
spec:
  rules:
  - host: tomcat.inspiry.cn
    http:
      paths:
      - path:
        backend:
          serviceName: tomcat-backend-svc
          servicePort: 8080
[root@docker79 ingress]# kubectl apply -f ingress-rules-tomcat.yaml
ingress.extensions/ingress-rule-tomcat created
[root@docker79 ingress]# kubectl get ingress
NAME                  HOSTS               ADDRESS   PORTS     AGE
ingress-rule-ngx      nginx.inspiry.com             80        11m
ingress-rule-tomcat   tomcat.inspiry.cn             80        6s
[root@docker79 ingress]#
[root@docker79 ingress]# kubectl describe ingress ingress-rule-tomcat
Name:             ingress-rule-tomcat
Namespace:        default
Address:
Default backend:  default-http-backend:80 (<none>)
Rules:
  Host               Path  Backends
  ----               ----  --------
  tomcat.inspiry.cn
                        tomcat-backend-svc:8080 (<none>)
Annotations:
  kubectl.kubernetes.io/last-applied-configuration:  {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernetes.io/ingress.class":"nginx"},"name":"ingress-rule-tomcat","namespace":"default"},"spec":{"rules":[{"host":"tomcat.inspiry.cn","http":{"paths":[{"backend":{"serviceName":"tomcat-backend-svc","servicePort":8080},"path":null}]}}]}}

  kubernetes.io/ingress.class:  nginx
Events:                         <none>
[root@docker79 ingress]#

5) 測試
在kubernetes cluster 外的任何一臺主機上使用 curl http://nginx.inspiry.com:30080/ 或 curl http://tomcat.inspiry.cn:30080/ 即可訪問相應nginx-backend 、tomcat-backend 的pods。
生產環境中還可以在k8s cluster外再創建一層 LB,將接收到的 nginx.inspiry.com、tomcat.inspiry.cn 域名分別轉發到192.168.20.79、192.168.20.78、192.168.20.77為後端的upstream上。

觀察ingress controller的內容如下:

[root@docker79 ~]# kubectl get pods -n ingress-nginx
NAME                                        READY     STATUS    RESTARTS   AGE
default-http-backend-6586bc58b6-8rxtp       1/1       Running   0          1h
nginx-ingress-controller-6bd7c597cb-xdrcm   1/1       Running   0          1h
[root@docker79 ~]# kubectl exec -n ingress-nginx -it nginx-ingress-controller-6bd7c597cb-xdrcm -- /bin/sh
$ ls
fastcgi.conf        geoip    mime.types      nginx.conf         scgi_params      uwsgi_params.default
fastcgi.conf.default    koi-utf  mime.types.default  nginx.conf.default     scgi_params.default  win-utf
fastcgi_params      koi-win  modsecurity         opentracing.json       template
fastcgi_params.default  lua  modules         owasp-modsecurity-crs  uwsgi_params
$ cat nginx.conf
...... (可仔細觀察該文件的內容,發現相應的轉發規則是隨著 ingress rules的變化而變化的)

Kebernetes學習總結(5) Ingress