k8s實踐4:kube-proxy問題iptables轉發規則不顯示
阿新 • • 發佈：2019-03-12
int gre routing bili 配置 loop rpo ati rop 1.
今天想看看kube-proxy的iptables轉發規則,執行命令iptables-save,見下:
今天想看看kube-proxy的iptables轉發規則,執行命令iptables-save,見下:
[root@k8s-master1 test]# iptables-save |grep "^-A KUBE" -A KUBE-FIREWALL -m comment --comment "kubernetes firewall for dropping marked packets" -m mark --mark 0x8000/0x8000 -j DROP -A KUBE-FORWARD -m comment --comment "kubernetes forwarding rules" -m mark --mark 0x4000/0x4000 -j ACCEPT -A KUBE-FORWARD -s 172.30.0.0/16 -m comment --comment "kubernetes forwarding conntrack pod source rule" -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT -A KUBE-FORWARD -d 172.30.0.0/16 -m comment --comment "kubernetes forwarding conntrack pod destination rule" -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT -A KUBE-FIREWALL -j KUBE-MARK-DROP -A KUBE-LOAD-BALANCER -j KUBE-MARK-MASQ -A KUBE-MARK-DROP -j MARK --set-xmark 0x8000/0x8000 -A KUBE-MARK-MASQ -j MARK --set-xmark 0x4000/0x4000 -A KUBE-NODE-PORT -p tcp -m comment --comment "Kubernetes nodeport TCP port for masquerade purpose" -m set --match-set KUBE-NODE-PORT-TCP dst -j KUBE-MARK-MASQ -A KUBE-POSTROUTING -m comment --comment "kubernetes service traffic requiring SNAT" -m mark --mark 0x4000/0x4000 -j MASQUERADE -A KUBE-POSTROUTING -m comment --comment "Kubernetes endpoints dst ip:port, source ip for solving hairpin purpose" -m set --match-set KUBE-LOOP-BACK dst,dst,src -j MASQUERADE -A KUBE-SERVICES ! -s 172.30.0.0/16 -m comment --comment "Kubernetes service cluster ip + port for masquerade purpose" -m set --match-set KUBE-CLUSTER-IP dst,dst -j KUBE-MARK-MASQ -A KUBE-SERVICES -m addrtype --dst-type LOCAL -j KUBE-NODE-PORT -A KUBE-SERVICES -m set --match-set KUBE-CLUSTER-IP dst,dst -j ACCEPT
發現,根本看不到詳細的svc的和pod的iptables轉發規則.
為什麽看不到呢？以前kubeadm部署是可以看到的.
2.
查資料做對比.
發現kubeadm部署使用kube-proxy的ipatbles模式
現在手動部署的集群使用的是kube-proxy的ipvs模式
可以通過檢查配置文件,檢索模式?
[root@k8s-master2 ~]# cat /etc/kubernetes/kube-proxy.config.yaml apiVersion: kubeproxy.config.k8s.io/v1alpha1 bindAddress: 192.168.32.129 clientConnection: ? kubeconfig: /etc/kubernetes/kube-proxy.kubeconfig clusterCIDR: 172.30.0.0/16 healthzBindAddress: 192.168.32.129:10256 hostnameOverride: k8s-master2 kind: KubeProxyConfiguration metricsBindAddress: 192.168.32.129:10249 mode: "ipvs"
把ipvs模式改成iptables,重啟kube-proxy.
執行命令iptables-save命令,見下:
[root@k8s-master2 ~]# iptables-save |grep "^-A KUBE-SVC" -A KUBE-SVC-ERIFXISQEP7F7OF4 -j KUBE-SEP-GU5WDSRFVPTYJ5QU -A KUBE-SVC-NPX46M4PTMTKRN6Y -m statistic --mode random --probability 0.33332999982 -j KUBE-SEP-TZ3VXNY2EEVCTOTN -A KUBE-SVC-NPX46M4PTMTKRN6Y -m statistic --mode random --probability 0.50000000000 -j KUBE-SEP-BU2YJ53RTO4VMWUK -A KUBE-SVC-NPX46M4PTMTKRN6Y -j KUBE-SEP-NOTEWJDBBN5H3PPR -A KUBE-SVC-R2VK7O5AFVLRAXSH -m statistic --mode random --probability 0.50000000000 -j KUBE-SEP-55H4YX2333AS44RT -A KUBE-SVC-R2VK7O5AFVLRAXSH -j KUBE-SEP-YZEC6Y7BVIHLFR3L -A KUBE-SVC-RL3JAE4GN7VOGDGP -m statistic --mode random --probability 0.20000000019 -j KUBE-SEP-VB7GMOVJXYUHR5XB -A KUBE-SVC-RL3JAE4GN7VOGDGP -m statistic --mode random --probability 0.25000000000 -j KUBE-SEP-SOERVJR7HCE5UQCC -A KUBE-SVC-RL3JAE4GN7VOGDGP -m statistic --mode random --probability 0.33332999982 -j KUBE-SEP-2EMOF3UUDCCOTQCO -A KUBE-SVC-RL3JAE4GN7VOGDGP -m statistic --mode random --probability 0.50000000000 -j KUBE-SEP-ZMLJTAH443KZVOBZ -A KUBE-SVC-RL3JAE4GN7VOGDGP -j KUBE-SEP-OTUMA5HXXG654BGO -A KUBE-SVC-TCOU7JCQXEZGVUNU -j KUBE-SEP-4H6QT2QBUKHBI7U2 -A KUBE-SVC-VPEW22VBKQ5JUV7N -m statistic --mode random --probability 0.25000000000 -j KUBE-SEP-6ZCL2K4RTLLEWKG3 -A KUBE-SVC-VPEW22VBKQ5JUV7N -m statistic --mode random --probability 0.33332999982 -j KUBE-SEP-HBKM2LVSQ4YLR7GU -A KUBE-SVC-VPEW22VBKQ5JUV7N -m statistic --mode random --probability 0.50000000000 -j KUBE-SEP-NGAS2WWEJXLUMON5 -A KUBE-SVC-VPEW22VBKQ5JUV7N -j KUBE-SEP-ZEKL4SKLF2DTYX5K [root@k8s-master2 ~]#
[root@k8s-master2 ~]# iptables-save |grep "^-A KUBE-SEP"
-A KUBE-SEP-2EMOF3UUDCCOTQCO -s 172.30.78.2/32 -j KUBE-MARK-MASQ
-A KUBE-SEP-2EMOF3UUDCCOTQCO -p tcp -m tcp -j DNAT --to-destination 172.30.78.2:80
-A KUBE-SEP-4H6QT2QBUKHBI7U2 -s 172.30.60.4/32 -j KUBE-MARK-MASQ
-A KUBE-SEP-4H6QT2QBUKHBI7U2 -p udp -m udp -j DNAT --to-destination 172.30.60.4:53
-A KUBE-SEP-55H4YX2333AS44RT -s 172.30.60.5/32 -j KUBE-MARK-MASQ
-A KUBE-SEP-55H4YX2333AS44RT -p tcp -m tcp -j DNAT --to-destination 172.30.60.5:80
-A KUBE-SEP-6ZCL2K4RTLLEWKG3 -s 172.30.60.2/32 -j KUBE-MARK-MASQ
-A KUBE-SEP-6ZCL2K4RTLLEWKG3 -p tcp -m tcp -j DNAT --to-destination 172.30.60.2:80
-A KUBE-SEP-BU2YJ53RTO4VMWUK -s 192.168.32.129/32 -j KUBE-MARK-MASQ
-A KUBE-SEP-BU2YJ53RTO4VMWUK -p tcp -m tcp -j DNAT --to-destination 192.168.32.129:6443
-A KUBE-SEP-GU5WDSRFVPTYJ5QU -s 172.30.60.4/32 -j KUBE-MARK-MASQ
-A KUBE-SEP-GU5WDSRFVPTYJ5QU -p tcp -m tcp -j DNAT --to-destination 172.30.60.4:53
-A KUBE-SEP-HBKM2LVSQ4YLR7GU -s 172.30.7.2/32 -j KUBE-MARK-MASQ
-A KUBE-SEP-HBKM2LVSQ4YLR7GU -p tcp -m tcp -j DNAT --to-destination 172.30.7.2:80
-A KUBE-SEP-NGAS2WWEJXLUMON5 -s 172.30.78.3/32 -j KUBE-MARK-MASQ
-A KUBE-SEP-NGAS2WWEJXLUMON5 -p tcp -m tcp -j DNAT --to-destination 172.30.78.3:80
-A KUBE-SEP-NOTEWJDBBN5H3PPR -s 192.168.32.130/32 -j KUBE-MARK-MASQ
-A KUBE-SEP-NOTEWJDBBN5H3PPR -p tcp -m tcp -j DNAT --to-destination 192.168.32.130:6443
-A KUBE-SEP-OTUMA5HXXG654BGO -s 172.30.80.3/32 -j KUBE-MARK-MASQ
-A KUBE-SEP-OTUMA5HXXG654BGO -p tcp -m tcp -j DNAT --to-destination 172.30.80.3:80
-A KUBE-SEP-SOERVJR7HCE5UQCC -s 172.30.7.3/32 -j KUBE-MARK-MASQ
-A KUBE-SEP-SOERVJR7HCE5UQCC -p tcp -m tcp -j DNAT --to-destination 172.30.7.3:80
-A KUBE-SEP-TZ3VXNY2EEVCTOTN -s 192.168.32.128/32 -j KUBE-MARK-MASQ
-A KUBE-SEP-TZ3VXNY2EEVCTOTN -p tcp -m tcp -j DNAT --to-destination 192.168.32.128:6443
-A KUBE-SEP-VB7GMOVJXYUHR5XB -s 172.30.60.3/32 -j KUBE-MARK-MASQ
-A KUBE-SEP-VB7GMOVJXYUHR5XB -p tcp -m tcp -j DNAT --to-destination 172.30.60.3:80
-A KUBE-SEP-YZEC6Y7BVIHLFR3L -s 172.30.78.4/32 -j KUBE-MARK-MASQ
-A KUBE-SEP-YZEC6Y7BVIHLFR3L -p tcp -m tcp -j DNAT --to-destination 172.30.78.4:80
-A KUBE-SEP-ZEKL4SKLF2DTYX5K -s 172.30.80.4/32 -j KUBE-MARK-MASQ
-A KUBE-SEP-ZEKL4SKLF2DTYX5K -p tcp -m tcp -j DNAT --to-destination 172.30.80.4:80
-A KUBE-SEP-ZMLJTAH443KZVOBZ -s 172.30.80.2/32 -j KUBE-MARK-MASQ
-A KUBE-SEP-ZMLJTAH443KZVOBZ -p tcp -m tcp -j DNAT --to-destination 172.30.80.2:80
[root@k8s-master2 ~]# 可以看到改回iptables模式之後,所有的轉發規則全部顯示出來了.
3.
ipvs模式比iptables模式強大,測試完後,記得改回ipvs模式.
k8s實踐4:kube-proxy問題iptables轉發規則不顯示