1. 程式人生 > >Docker跨主機通信網絡

Docker跨主機通信網絡

Docker 跨主機通信

Docker跨主機通信網絡

跨主機網絡方案包括:

docker原生的 overlaymacvlan
第三方方案:常用的包括flannel、weave 和 calico

(1)準備 Overlay 網絡實驗環境

[root@linux-node1 ~]# docker run -d -p 8500:8500 -h consul --name consul progrium/consul -server -bootstrap
[root@linux-node1 ~]# netstat -tulnp |grep 8500
tcp6       0      0 :::8500                 :::*                    LISTEN      61092/docker-proxy- 
容器啟動後,可以通過 http://192.168.56.110:8500 訪問 Consul。

技術分享圖片

接下來修改 node1 和 node2 的 docker daemon 的配置文件/var/lib/systemd/system/docker.service。
--cluster-store 指定 consul 的地址。
--cluster-advertise 告知 consul 自己的連接地址。
[root@linux-node1 ~]# cat /usr/lib/systemd/system/docker.service 
......
ExecStart=/usr/bin/dockerd-current           --add-runtime docker-runc=/usr/libexec/docker/docker-runc-current           --default-runtime=docker-runc           --exec-opt native.cgroupdriver=systemd           --userland-proxy-path=/usr/libexec/docker/docker-proxy-current           --seccomp-profile=/etc/docker/seccomp.json           --cluster-store=consul://192.168.56.110:8500           --cluster-advertise=eth0:2376           $OPTIONS ......
[root@linux-node1 ~]# systemctl daemon-reload
[root@linux-node1 ~]# systemctl restart docker
node1 和 node2 將自動註冊到 Consul 數據庫中。

技術分享圖片

(2)創建 overlay 網絡

在 node1 中創建 overlay 網絡 ov_net1:
[root@linux-node1 ~]# docker network create -d overlay ov_net1  #-d overlay 指定 driver 為 overaly。
[root@linux-node1 ~]# docker network ls  #查看當前網絡
NETWORK ID          NAME                DRIVER              SCOPE
8eb7fd71a52c        bridge              bridge              local
6ba20168e34f        host                host                local
4e896f9ac4bc        none                null                local
d9652d84d9de        ov_net1             overlay             global
[root@linux-node2 ~]# docker network ls  #查看當前網絡
NETWORK ID          NAME                DRIVER              SCOPE
94a3bc259414        bridge              bridge              local
f8443f6cb8d2        host                host                local
2535ab8f3493        none                null                local
d9652d84d9de        ov_net1             overlay             global

node2 上也能看到 ov_net1。這是因為創建 ov_net1 時 node1 將 overlay 網絡信息存入了 consul,node2 從 consul 讀取到了新網絡的數據。之後 ov_net 的任何變化都會同步到 node1 和 node2。

[root@linux-node1 ~]# docker network inspect ov_net1  #查看 ov_net1 的詳細信息
[
    {
        "Name": "ov_net1",
        "Id": "d9652d84d9de6d1145c77d0254c90164b968f72f2eda4aee43d56ab03f8530ed",
        "Created": "2018-04-19T21:50:29.128801226+08:00",
        "Scope": "global",
        "Driver": "overlay",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": {},
            "Config": [
                {
                    "Subnet": "10.0.0.0/24",
                    "Gateway": "10.0.0.1"
                }
            ]
        },
        "Internal": false,
        "Attachable": false,
        "Containers": {},
        "Options": {},
        "Labels": {}
    }
]

(3) overlay 中運行容器

[root@linux-node1 ~]# docker run -itd --name bbox1 --network ov_net1 busybox
340f748b06786c0f81c3e26dd9dbd820dafcdf73baa9232f02aece8d4c89a73b
[root@linux-node1 ~]# docker exec bbox1 ip r  #查看容器的網絡配置
default via 172.18.0.1 dev eth1 
10.0.0.0/24 dev eth0 scope link  src 10.0.0.2 
172.18.0.0/16 dev eth1 scope link  src 172.18.0.2 

bbox1有兩個網絡接口eth0eth1eth0 IP10.0.0.2,連接的是 overlay 網絡ov_net1eth1 IP 172.18.0.2,容器的默認路由是走 eth1eth1是哪兒來的呢?

其實,docker會創建一個bridge網絡 “docker_gwbridge”,為所有連接到 overlay 網絡的容器提供訪問外網的能力。

[root@linux-node1 ~]# docker network ls
NETWORK ID          NAME                DRIVER              SCOPE
8eb7fd71a52c        bridge              bridge              local
751bd423a345        docker_gwbridge     bridge              local
6ba20168e34f        host                host                local
4e896f9ac4bc        none                null                local
d9652d84d9de        ov_net1             overlay             global
[root@linux-node1 ~]# docker network inspect docker_gwbridge 
[
    {
        "Name": "docker_gwbridge",
        "Id": "751bd423a345a7beaa6b4cbf2a69a7687e3d8b7e656952090c4b94aec54ec1b5",
        "Created": "2018-04-21T16:11:57.684140362+08:00",
        "Scope": "local",
        "Driver": "bridge",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": null,
            "Config": [
                {
                    "Subnet": "172.18.0.0/16",
                    "Gateway": "172.18.0.1"
                }
            ]
        },
        "Internal": false,
        "Attachable": false,
        "Containers": {
            "340f748b06786c0f81c3e26dd9dbd820dafcdf73baa9232f02aece8d4c89a73b": {
                "Name": "gateway_340f748b0678",
                "EndpointID": "64cd599aaa2408ca0a1e595264e727b09d26482ba4d2aa18d97862ed29e23b51",
                "MacAddress": "02:42:ac:12:00:02",
                "IPv4Address": "172.18.0.2/16",
                "IPv6Address": ""
            }
        },
        "Options": {
            "com.docker.network.bridge.enable_icc": "false",
            "com.docker.network.bridge.enable_ip_masquerade": "true",
            "com.docker.network.bridge.name": "docker_gwbridge"
        },
        "Labels": {}
    }
]

docker network inspect docker_gwbridge輸出可確認 docker_gwbridge的 IP 地址範圍是 172.18.0.0/16,當前連接的容器就是bbox1(172.18.0.2)
而且此網絡的網關就是網橋 docker_gwbridgeIP 172.17.0.1

[root@linux-node1 ~]# ifconfig docker_gwbridge 
docker_gwbridge: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 172.18.0.1  netmask 255.255.0.0  broadcast 0.0.0.0
        inet6 fe80::42:e4ff:feb8:22cb  prefixlen 64  scopeid 0x20<link>
        ether 02:42:e4:b8:22:cb  txqueuelen 0  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

這樣容器 bbox1 就可以通過 docker_gwbridge 訪問外網。

[root@linux-node1 ~]# docker exec bbox1 ping -c 2 www.baidu.com
PING www.baidu.com (58.217.200.112): 56 data bytes
64 bytes from 58.217.200.112: seq=0 ttl=127 time=32.465 ms
64 bytes from 58.217.200.112: seq=1 ttl=127 time=32.754 ms

--- www.baidu.com ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 32.465/32.609/32.754 ms

(4)overlay 如何實現跨主機通信?

[root@linux-node2 ~]# docker run -itd --name bbox2 --network ov_net1 busybox
[root@linux-node2 ~]# docker ps
CONTAINER ID      IMAGE     COMMAND        CREATED       STATUS      PORTS       NAMES
68c81b90fb86     busybox    "sh"        2 days ago      Up 2 days                bbox2
[root@linux-node2 ~]# docker exec bbox2 ip r
default via 172.18.0.1 dev eth1 
10.0.0.0/24 dev eth0 scope link  src 10.0.0.3   
172.18.0.0/16 dev eth1 scope link  src 172.18.0.2 

##bbox2 IP 為 10.0.0.3,可以直接 ping bbox1
[root@linux-node2 ~]# docker exec bbox2 ping -c 3 bbox1
PING bbox1 (10.0.0.2): 56 data bytes
64 bytes from 10.0.0.2: seq=0 ttl=64 time=154.064 ms
64 bytes from 10.0.0.2: seq=1 ttl=64 time=0.789 ms
64 bytes from 10.0.0.2: seq=2 ttl=64 time=0.539 ms

--- bbox1 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 0.539/51.797/154.064 ms

docker會為每個overlay網絡創建一個獨立的network namespace,其中會有一個linux bridge br0endpoint 還是由veth pair 實現,一端連接到容器中(即 eth0),另一端連接到 namespacebr0上。

br0除了連接所有的 endpoint,還會連接一個 vxlan 設備,用於與其他 host建立 vxlan tunnel。容器之間的數據就是通過這個tunnel通信的。邏輯網絡拓撲結構如圖所示:

技術分享圖片

(5)overlay 是如何隔離的?

不同的overlay網絡是相互隔離的。我們創建第二個 overlay網絡ov_net2 並運行容器bbox3

[root@linux-node1 ~]# docker run -itd --name bbox3 --network ov_net2 busybox
946def609a7b183f68b8398b35fd3f72dc28bff47cc2ba63467f266fde297d5a

[root@linux-node1 ~]# docker exec -it bbox3 ip r
default via 172.18.0.1 dev eth1 
10.0.1.0/24 dev eth0 scope link  src 10.0.1.2   ##bbox3的ip為10.0.1.2
172.18.0.0/16 dev eth1 scope link  src 172.18.0.4 

[root@linux-node1 ~]# docker exec -it bbox3 ping -c 2 10.0.0.3  #bbox3無法ping通bbox2
PING 10.0.0.3 (10.0.0.3): 56 data bytes
^C
--- 10.0.0.3 ping statistics ---
2 packets transmitted, 0 packets received, 100% packet loss

如果要實現 bbox3 與 bbox2 通信,可以將 bbox3 也連接到 ov_net1。
[root@linux-node1 ~]# docker network connect ov_net1 bbox3
[root@linux-node1 ~]# docker exec -it bbox3 ping -c 2 10.0.0.3
PING 10.0.0.3 (10.0.0.3): 56 data bytes
64 bytes from 10.0.0.3: seq=0 ttl=64 time=34.110 ms
64 bytes from 10.0.0.3: seq=1 ttl=64 time=0.745 ms

--- 10.0.0.3 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 0.745/17.427/34.110 ms

docker默認為 overlay網絡分配 24 位掩碼的子網(10.0.X.0/24),所有主機共享這個 subnet,容器啟動時會順序從此空間分配 IP。當然我們也可以通過--subnet 指定 IP 空間。

[root@linux-node1 ~]# docker network create -d overlay --subnet 10.22.1.0/24 ov_net3a111191fa67e500015a2f3ab8166793d23f0adef4d66bfcee81166127915ff9f
[root@linux-node1 ~]# docker network ls
NETWORK ID          NAME                DRIVER              SCOPE
8eb7fd71a52c        bridge              bridge              local
751bd423a345        docker_gwbridge     bridge              local
6ba20168e34f        host                host                local
4e896f9ac4bc        none                null                local
d9652d84d9de        ov_net1             overlay             global
667cc7ef7427        ov_net2             overlay             global
a111191fa67e        ov_net3             overlay             global

Docker跨主機通信網絡