前言
虽然docker发展得很快,能实现容器互联的方式已经有很多,本文则介绍原生实现:Docker1.9版本的docker network功能。
各种非原生Docker网络方案的优劣
1.Weave
由Zett.io开发的一个基于overlay的工具,可以创建出虚拟网络,用于连接部署在多台主机上的容器。在保证外部设备能访问Weave网络上的应用程序所提供的服务的同时也能令已有的内部系统暴露在应用程序容器上。
优点:支持通讯加密;能穿透防火墙等
缺点:部署之后需要额外的维护
2.Pipework
由一位Docker工程师实现的一个很简单但功能强大的shell脚本,使用了cgroups和namespace。
优点:灵活,可以采用各种方式实现容器的互联(笔者之前就是利用OVS和Pipework实现了一个简单的跨主机互联方案,当时的Docker版本为1.4.1)
缺点:实现容器互联后还需要借助其他程序才能实现自动化管理,增加额外的维护成本。
3.Kubernetes
Google推出的针对容器管理和编排的开源项目,让用户能在跨主机集群的情况下轻松地管理、监测容器化应用部署,其某些概念与SDN非常相似。
优点:完整方案,集成得比较完善的功能
缺点:它并非一个简单的工具,而是一个系统,对于纯粹想要一个能实现互联的工具的人来说是:有太多不需要的东西
4.Flannel
CoreOS团队实现的针对Kubernentes的一个overlay网络工具,让每个使用kubernetes得CoreOS主机拥有一个完整的子网。
优点:可与CoreOS紧密结合,对于那些打算使用CoreOS的人来说是个不错的选择
缺点:这个方案基本上都要与Kubernetes搭配使用。另外在某些情况下,这个方案会导致IP地址的浪费
Docker1.9的安装事项
因为Docker正在快速迭代,一些旧版本的功能不被支持,虽然可以卸载旧的版本再重新安装1.9。这里笔者在新版本系统上重新安装Docker1.9。笔者使用的系统是ubuntu server 14.04.3,内核版本为3.19(内核版本低于3.16无法支持libnetwork)。
查看内核版本:
1 2 3 |
root@machine1:~# uname -r 3.19.0-25-generic |
安装Docker1.9:
添加GPG key:
1 2 |
sudo apt-key adv --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys 58118E89F3A912897C070ADBF76221572C52609D |
增加Docker的源:
1 2 |
echo "deb https://apt.dockerproject.org/repo ubuntu-trusty main" >> /etc/apt/sources.list.d/docker.list |
更新一下:
1 2 |
apt-get update |
安装Docker:
1 2 |
apt-get install docker-engine |
检查Docker的版本:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
root@machine1:~# docker version Client: Version: 1.9.1 API version: 1.21 Go version: go1.4.2 Git commit: a34a1d5 Built: Fri Nov 20 13:12:04 UTC 2015 OS/Arch: linux/amd64 Server: Version: 1.9.1 API version: 1.21 Go version: go1.4.2 Git commit: a34a1d5 Built: Fri Nov 20 13:12:04 UTC 2015 OS/Arch: linux/amd64 |
Docker1.9默认的三种网络
通过docker network可以看到docker创建了三个网络:
1 2 3 4 5 6 |
root@machine1:~# docker network ls NETWORK ID NAME DRIVER 77c49087f27a bridge bridge 4d58099b6fbd none null a51102ab7699 host host |
第一个是bridge,和过去的版本一样,默认情况下,通过docker创建的容器默认都会连接到该网桥,网桥会自动为容器分配ip地址:
1 2 3 4 5 6 7 |
4: docker0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 02:42:32:e0:df:cd brd ff:ff:ff:ff:ff:ff inet 172.17.0.1/16 scope global docker0 valid_lft forever preferred_lft forever inet6 fe80::42:32ff:fee0:dfcd/64 scope link valid_lft forever preferred_lft forever |
创建一个新的容器:
1 2 3 |
root@machine1:~# docker run -itd --name=c1 busybox 1c8f6d73b50e161572e1cb84834b55200d2ae76fea9eb32a8a9c862911511148 |
可以看到bridge网桥上增加了我们的c1:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 |
root@machine1:~# docker network inspect bridge [ { "Name": "bridge", "Id": "77c49087f27aa80d5dbf2c58618929de5cd72d5d7ca358eae1301d8ef4c1e6c8", "Scope": "local", "Driver": "bridge", "IPAM": { "Driver": "default", "Config": [ { "Subnet": "172.17.0.0/16" } ] }, "Containers": { "1c8f6d73b50e161572e1cb84834b55200d2ae76fea9eb32a8a9c862911511148": { "EndpointID": "6259d2e9b606f1dfef726757311b4c57aa1948a7bb533258e5711be37eaebf67", "MacAddress": "02:42:ac:11:00:02", "IPv4Address": "172.17.0.2/16", "IPv6Address": "" } }, "Options": { "com.docker.network.bridge.default_bridge": "true", "com.docker.network.bridge.enable_icc": "true", "com.docker.network.bridge.enable_ip_masquerade": "true", "com.docker.network.bridge.host_binding_ipv4": "0.0.0.0", "com.docker.network.bridge.name": "docker0", "com.docker.network.driver.mtu": "1500" } } ] |
在c1中可以看到相关的网络信息:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
root@machine1:~# docker attach c1 / # ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 5: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue link/ether 02:42:ac:11:00:02 brd ff:ff:ff:ff:ff:ff inet 172.17.0.2/16 scope global eth0 valid_lft forever preferred_lft forever inet6 fe80::42:acff:fe11:2/64 scope link valid_lft forever preferred_lft forever / # |
我们再创建一个容器,指定其网络为none:
1 2 3 |
root@machine1:~# docker run -itd --net=none --name=c2 busybox 8d03b9d602a396514565d56fe550c16c123de8640f88db13a2cd688449a2e57f |
可以看到none中增加了该容器的信息:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 |
root@machine1:~# docker network inspect none [ { "Name": "none", "Id": "4d58099b6fbd4fd4764929d657808554041c00a5df151000f1aa60484c77b080", "Scope": "local", "Driver": "null", "IPAM": { "Driver": "default", "Config": [] }, "Containers": { "8d03b9d602a396514565d56fe550c16c123de8640f88db13a2cd688449a2e57f": { "EndpointID": "2027b0c57223699e77f0a719f45db95b9f4963de8b761747d956e4db2d7d4d94", "MacAddress": "", "IPv4Address": "", "IPv6Address": "" } }, "Options": {} } ] |
进入容器:
1 2 3 4 5 6 7 8 9 10 11 |
root@machine1:~# docker attach c2 / # ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever / # |
可以看到容器只有本地lo
至于最后的host网络,我们同样创建一个容器:
1 2 3 |
root@machine1:~# docker run -itd --net=host --name=c3 busybox 01e334c41f23e58d92279346cbfe65a76205c5a0a8dc75f186aa5e59f4cb554e |
检查数据:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 |
root@machine1:~# docker network inspect host [ { "Name": "host", "Id": "a51102ab76994610ad6005930c8749b2dd1cfbb5d2d7e326e46376ee0b354cfa", "Scope": "local", "Driver": "host", "IPAM": { "Driver": "default", "Config": [] }, "Containers": { "01e334c41f23e58d92279346cbfe65a76205c5a0a8dc75f186aa5e59f4cb554e": { "EndpointID": "39a59c507fcea723af48c8d3dba31a88d2efeb1d518056fbe4c7e53845a9ffd5", "MacAddress": "", "IPv4Address": "", "IPv6Address": "" } }, "Options": {} } ] |
进入容器,检查网络信息:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 |
root@machine1:~# docker attach c3 / # ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000 link/ether 00:0c:29:68:d3:ec brd ff:ff:ff:ff:ff:ff inet 172.16.77.181/24 brd 172.16.77.255 scope global eth0 valid_lft forever preferred_lft forever inet6 fe80::20c:29ff:fe68:d3ec/64 scope link valid_lft forever preferred_lft forever 3: docker_gwbridge: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue link/ether 02:42:ff:1b:f4:9c brd ff:ff:ff:ff:ff:ff inet 172.18.0.1/16 scope global docker_gwbridge valid_lft forever preferred_lft forever 4: docker0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue link/ether 02:42:32:e0:df:cd brd ff:ff:ff:ff:ff:ff inet 172.17.0.1/16 scope global docker0 valid_lft forever preferred_lft forever inet6 fe80::42:32ff:fee0:dfcd/64 scope link valid_lft forever preferred_lft forever 6: veth28dc068: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker0 link/ether 9e:95:80:ce:b3:13 brd ff:ff:ff:ff:ff:ff inet6 fe80::9c95:80ff:fece:b313/64 scope link valid_lft forever preferred_lft forever |
实际上和我的宿主机machine1的网络信息是一样的:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 |
root@machine1:~# ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 00:0c:29:68:d3:ec brd ff:ff:ff:ff:ff:ff inet 172.16.77.181/24 brd 172.16.77.255 scope global eth0 valid_lft forever preferred_lft forever inet6 fe80::20c:29ff:fe68:d3ec/64 scope link valid_lft forever preferred_lft forever 3: docker_gwbridge: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default link/ether 02:42:ff:1b:f4:9c brd ff:ff:ff:ff:ff:ff inet 172.18.0.1/16 scope global docker_gwbridge valid_lft forever preferred_lft forever 4: docker0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 02:42:32:e0:df:cd brd ff:ff:ff:ff:ff:ff inet 172.17.0.1/16 scope global docker0 valid_lft forever preferred_lft forever inet6 fe80::42:32ff:fee0:dfcd/64 scope link valid_lft forever preferred_lft forever 6: veth28dc068: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker0 state UP group default link/ether 9e:95:80:ce:b3:13 brd ff:ff:ff:ff:ff:ff inet6 fe80::9c95:80ff:fece:b313/64 scope link valid_lft forever preferred_lft forever |
我们再创建一个连接到bridge的容器c4:
1 2 3 |
root@machine1:~# docker run -itd --name=c4 busybox 725d878fcadf7624415a6d7c5a1ca353a048b7c0fca7fb5a8ee2f5cde79c8f0f |
进入c4,尝试ping容器c1:
1 2 3 4 5 6 7 8 9 10 11 12 13 |
/ # ping -w 4 c1 ping: bad address 'c1' / # ping -w 4 172.17.0.2 PING 172.17.0.2 (172.17.0.2): 56 data bytes 64 bytes from 172.17.0.2: seq=0 ttl=64 time=0.160 ms 64 bytes from 172.17.0.2: seq=1 ttl=64 time=0.129 ms 64 bytes from 172.17.0.2: seq=2 ttl=64 time=0.122 ms 64 bytes from 172.17.0.2: seq=3 ttl=64 time=0.125 ms --- 172.17.0.2 ping statistics --- 4 packets transmitted, 4 packets received, 0% packet loss round-trip min/avg/max = 0.122/0.134/0.160 ms |
可以看到,由于没有服务发现,无法直接通过容器名ping通c2,但可以直接通过ip地址ping通。如果启动c2时指定了--link为c1,就可以直接通过容器名ping通。
在docker1.9中,用户可以自定义一个网络驱动,通过参数-d或--driver指定,比如:
1 2 3 4 5 6 7 8 9 |
root@machine1:~# docker network create -d bridge b1 32adb8c91216604fda461c7ccdb340a20a65de5f621e19d22888858e2efa5a28 root@machine1:~# docker network ls NETWORK ID NAME DRIVER 4d58099b6fbd none null a51102ab7699 host host 32adb8c91216 b1 bridge 77c49087f27a bridge bridge |
我们创建了一个名为b1的bridge驱动的网络,-d还可以指定overlay或其他plugin比如weave。
使用Docker network命令
目前docker network的功能还比较简单,只有下面几个子命令:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
root@machine1:~# docker network help Usage: docker network [OPTIONS] COMMAND [OPTIONS] Commands: ls List all networks rm Remove a network create Create a network connect Connect container to a network disconnect Disconnect container from a network inspect Display detailed network information Run 'docker network COMMAND --help' for more information on a command. --help=false Print usage |
当我们开始使用Docker network时,可以先为Docker创建网络,在不加-d指定网络驱动时,默认创建基于网桥的网络:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 |
root@machine1:~# docker network create mynetwork 5f2db170b3d84ce73d32090db677ec161f6353cfbf0cdd95da21850077cd0475 root@machine1:~# docker network inspect mynetwork [ { "Name": "mynetwork", "Id": "5f2db170b3d84ce73d32090db677ec161f6353cfbf0cdd95da21850077cd0475", "Scope": "local", "Driver": "bridge", "IPAM": { "Driver": "default", "Config": [ {} ] }, "Containers": {}, "Options": {} } ] |
可以通过inspect查看一个具体网络的信息,Driver那一项指明了网络类型。
但是,对于以网桥为基础的网络,还能有更多特性,这些特性可以通过创建网络时传入参数启用,参考下面的例子:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
root@machine1:~# docker network create --opt "com.docker.network.enable_ipv6"=True mynetwork1 root@machine1:~# docker network inspect mynetwork1 [ { "Name": "mynetwork1", "Id": "37a3faecedc4e7645573eb5bba6c7dc58a52c4529b8830069d2c22ebe55e3ef3", "Scope": "local", "Driver": "bridge", "IPAM": { "Driver": "default", "Config": [ {} ] }, "Containers": {}, "Options": { "com.docker.network.enable_ipv6": "True" } } ] |
创建以网桥为驱动的网络时,通过-o或--opt传入配置项和对应的参数,在上面的例子中,让mynetwork1启用了ipv6的支持,使用的配置项格式是"com.docker.network.bridge.xxx"。在Docker network中有以下的几种可用配置项:
1 2 3 4 5 6 7 8 9 10 11 12 |
1.com.docker.network.bridge.name:用于指定一个名字,创建网桥时使用 2.com.docker.network.bridge.enable_ip_masquerade:启用ip伪装 3.com.docker.network.bridge.enable_icc:是否允许容器间互相连通 4.com.docker.network.bridge.host_binding_ipv4:指定绑定port时的默认ip地址 5.com.docker.network.mtu:设定MTU 6.com.docker.network.enable_ipv6:是否启用ipv6的支持 |
下面就以enable_icc这个参数测试一下效果:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 |
root@machine1:~# docker network create -o "com.docker.network.bridge.enable_icc"="False" -o "com.docker.network.bridge.name"="netBridge2" mynetwork2 adb97f9fe3463a1f83766bd2a7ffbc2adf634ce59fe15a1c1412e233650383fa root@machine1:~# docker network inspect mynetwork2 [ { "Name": "mynetwork2", "Id": "adb97f9fe3463a1f83766bd2a7ffbc2adf634ce59fe15a1c1412e233650383fa", "Scope": "local", "Driver": "bridge", "IPAM": { "Driver": "default", "Config": [ {} ] }, "Containers": {}, "Options": { "com.docker.network.bridge.enable_icc": "False", "com.docker.network.bridge.name": "netBridge2" } } ] |
通过brctl可以看到刚才创建的网络的网桥:
1 2 |
netBridge2 8000.024270292357 no |
启动容器:
1 2 3 4 5 |
root@machine1:~# docker run -itd --name c1 ubuntu 580c0399fa9cec09764e71e84002117331afeed227e43451da4d8fd3af1b9649 root@machine1:~# docker run -itd --name c2 ubuntu b747df584654e17164345e15985e28484852b7338a137d3fd0577017a26b8555 |
有问题?接下来会使用docker network的connect功能把容器挂载到指定网络上:
1 2 3 |
root@machine1:~# docker network connect mynetwork2 c1 root@machine1:~# docker network connect mynetwork2 c2 |
在docker network中提供了connect功能,可以把容器挂载到指定的网络上。但请注意,如果启动容器时指明容器网络为none的话,即传入了--net=none的参数的话,容器就不可以再挂载到别的网络上,这一点还请读者注意。(无法挂载到别的网络时的错误提示如下):
1 2 |
Error response from daemon: Container cannot be connected to multiple networks with one of the networks in --none mode |
通过查询,可以知道c1的两个地址分别为:
1 2 3 4 5 6 7 8 9 10 11 12 13 |
9: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 02:42:ac:11:00:02 brd ff:ff:ff:ff:ff:ff inet 172.17.0.2/16 scope global eth0 valid_lft forever preferred_lft forever inet6 fe80::42:acff:fe11:2/64 scope link valid_lft forever preferred_lft forever 13: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 02:42:ac:15:00:02 brd ff:ff:ff:ff:ff:ff inet 172.21.0.2/16 scope global eth1 valid_lft forever preferred_lft forever inet6 fe80::42:acff:fe15:2/64 scope link valid_lft forever preferred_lft forever |
c2的地址分别为172.17.0.3和172.21.0.3
eth0是在docker初始时的bridge网桥上的接口,eth1是mynetwork2上的接口,我们在c1中尝试ping c2的不同网络接口试试:
eth0:
1 2 3 4 |
--- 172.17.0.3 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2004ms rtt min/avg/max/mdev = 0.108/0.161/0.215/0.046 ms |
eth1:
1 2 3 |
--- 172.21.0.3 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 2001ms |
可以看到icc被设为false后的确生效了。
如果我们想把容器与网络解除关系,直接使用disconnect
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 |
root@machine1:~# docker network disconnect mynetwork2 c1 root@machine1:~# docker network disconnect mynetwork2 c2 root@machine1:~# docker network inspect mynetwork2 [ { "Name": "mynetwork2", "Id": "adb97f9fe3463a1f83766bd2a7ffbc2adf634ce59fe15a1c1412e233650383fa", "Scope": "local", "Driver": "bridge", "IPAM": { "Driver": "default", "Config": [ {} ] }, "Containers": {}, "Options": { "com.docker.network.bridge.enable_icc": "False", "com.docker.network.bridge.name": "netBridge2" } } ] |
可以看到该网络中已经没有相关容器的信息了。
从上面的例子中可以看到docker network中也提供了inspect用于查看网络的相关信息。
在创建网络时,可以添加-d参数指定网络驱动(可以是weave、bridge等),在上面的icc那个例子中,我们必须进入容器或inspect容器才能知道容器在mynetwork2中的ip地址,我们可以在创建网络时就指定子网地址范围:
1 2 3 |
root@machine1:~# docker network create -d bridge --subnet 172.27.0.0/16 mynetwork3 863023cf826a0f08569ba32d256ca216e5be742affe3ad83bc71752addd5ed73 |
这样一来,凡是加入该网络的容器都会自动从这个范围得到ip地址。
以上就是对Docker network的命令的使用指引。
Docker1.9的跨主机容器网络体验
在使用overlay类型的网络时,要做好几个准备:
1.一个可以供集群中所有主机访问的键值储存系统,docker engine支持etcd,consul和zookeeper,libnetwork通过kv store同步信息,在一个节点上创建的overlay网络,你会发现在其他节点也能看到。
2.在每个节点上配置好docker daemon,需要指定--cluster-store,--cluster-store-opt, --cluster-advertise。 但如果你有使用Swarm就会方便得多。
在创建overlay类型的网络时还可以指定好几个参数(引用官方的例子):
1 2 3 4 5 6 7 8 |
$ docker network create -d overlay --subnet=192.168.0.0/16 --subnet=192.170.0.0/16 --gateway=192.168.0.100 --gateway=192.170.0.100 --ip-range=192.168.1.0/24 --aux-address a=192.168.1.5 --aux-address b=192.168.1.6 --aux-address a=192.170.1.5 --aux-address b=192.170.1.6 my-multihost-network |
笔者没有像官方文档一样使用docker swarm和docker engine,而是手动处理下面的实验。
开始体验
下载,安装consul:
1 2 3 4 |
wget https://releases.hashicorp.com/consul/0.6.3/consul_0.6.3_linux_amd64.zip unzip consul_0.6.3_linux_amd64.zip mv consul /usr/local/bin |
笔者做实验时使用Vmware创建了两个虚拟机,分别是machine1(172.16.77.181)和machine2(172.16.77.182),这里我使用machine1作为consul的server,machine2作为agent。
启动consul:
在m1上:
1 2 |
consul agent -server -bootstrap -data-dir /tmp/consul -bind=172.16.77.181 |
在m2上:
1 2 3 |
consul agent -data-dir /tmp/consul -bind 172.16.77.181 & consul join 172.16.77.181 |
设置docker daemon:
在m1上:
1 2 |
docker daemon -D --cluster-store=consul://localhost:8500 --label=com.docker.network.driver.overlay.bind_interface=eth0 |
在m2上:
1 2 |
docker daemon -D --cluster-store=consul://localhost:8500 --label=com.docker.network.driver.overlay.bind_interface=eth0 --label=com.docker.network.driver.overlay.neighbor_ip=172.16.77.181 |
在任意一个节点创建一个Overlay网络:
1 2 |
docker network create -d overlay o1 |
可以看到这个网络已经被创建(实际上这个网络的数据通过consul同步到了另外一个vm上,即在任意节点创建一个overlay网络,在所有加入consul的agent节点都能看到):
1 2 3 4 5 6 7 |
root@machine1:~# docker network ls NETWORK ID NAME DRIVER 7a7be616b3ef o1 overlay daf10496ff7a host host bb6b43205ffc bridge bridge b2eed4029d7b none null |
启动容器并连接o1:
在m1上:
1 2 |
docker run -itd --net=o1 --name=c1 busybox |
在m2上:
1 2 |
docker run -itd --net=o1 --name=c2 busybox |
在连接到网络之后,docker创建了一个docker_gwbridge的网桥用于NAT:
1 2 3 4 5 6 7 8 |
root@machine1:~# docker network ls NETWORK ID NAME DRIVER 7a7be616b3ef o1 overlay daf10496ff7a host host bb6b43205ffc bridge bridge af63d717993f docker_gwbridge bridge b2eed4029d7b none null |
接下来收集几个必要的数据:
c1,c2的IP地址(这里只给出我收集c1数据的例子):
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
root@machine1:~# docker inspect c1 | grep IP "LinkLocalIPv6Address": "", "LinkLocalIPv6PrefixLen": 0, "SecondaryIPAddresses": null, "SecondaryIPv6Addresses": null, "GlobalIPv6Address": "", "GlobalIPv6PrefixLen": 0, "IPAddress": "", "IPPrefixLen": 0, "IPv6Gateway": "", "IPAddress": "10.0.0.2", "IPPrefixLen": 24, "IPv6Gateway": "", "GlobalIPv6Address": "", "GlobalIPv6PrefixLen": 0, |
MAC地址:
1 2 3 4 |
root@machine1:~# docker inspect c1 | grep Mac "MacAddress": "", "MacAddress": "02:42:0a:00:00:02" |
用于overlay网络的namespace ID:
1 2 3 |
root@machine1:~# ls /var/run/docker/netns 2-7a7be616b3 d96446acbf28 |
c1的namespace ID:
1 2 3 4 |
root@machine1:~# docker inspect c1 | grep Sand "SandboxID": "d96446acbf287732a59ea8b5b25feb856084c7b28e488bfb03d58074062d0660", "SandboxKey": "/var/run/docker/netns/d96446acbf28", |
c1有关的全部数据:
IP:10.0.0.2
MAC:02:42:0a:00:00:02
overlay namespace ID:2-7a7be616b3(全都有一个'-'字符)
c1 namespace ID:d96446acbf28
对于c2也同样收集这些数据(overlay namespace ID在两个节点间是一样的)
c2的数据:
IP:10.0.0.3
MAC:02:42:0a:00:00:03
namespace:6c277da83c39
用ln让namespace可以访问:
以m1为例:
1 2 3 |
ln -s /var/run/docker/netns/2-7a7be616b3 /var/run/netns/2-7a7be616b3 ln -s /var/run/docker/netns/d96446acbf28 /var/run/netns/d96446acbf28 |
检查下:
1 2 3 4 |
root@machine1:~# ip net list d96446acbf28 2-7a7be616b3 |
查看namespace的信息(overlay网络实际上是基于vxlan的实现):
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
root@machine1:~# ip netns exec 2-7a7be616b3 ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default link/ether 3a:5c:e4:68:c4:fd brd ff:ff:ff:ff:ff:ff inet 10.0.0.1/24 scope global br0 valid_lft forever preferred_lft forever inet6 fe80::40c8:ccff:fe0a:6f8e/64 scope link valid_lft forever preferred_lft forever 24: vxlan1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br0 state UNKNOWN group default link/ether da:93:3c:8b:ef:61 brd ff:ff:ff:ff:ff:ff inet6 fe80::d893:3cff:fe8b:ef61/64 scope link valid_lft forever preferred_lft forever 26: veth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue master br0 state UP group default link/ether 3a:5c:e4:68:c4:fd brd ff:ff:ff:ff:ff:ff inet6 fe80::385c:e4ff:fe68:c4fd/64 scope link valid_lft forever preferred_lft forever |
在m1上输入:
1 2 3 |
ip netns exec 2-7a7be616b3 bridge fdb add 02:42:0a:00:00:03 dev vxlan1 dst 172.16.77.182 ip netns exec d96446acbf28 arp -s 10.0.0.3 02:42:0a:00:00:03 |
m2上:
1 2 3 |
ip netns exec 2-7a7be616b3 bridge fdb add 02:42:0a:00:00:02 dev vxlan1 dst 172.16.77.181 ip netns exec 6c277da83c39 arp -s 10.0.0.2 02:42:0a:00:00:02 |
这样一来就可以实现c1和c2的互通了。
结语
Docker的迭代速度真的好快(笔者写此文时1.10已经出到rc3了),很多东西都变了,但如果能跟上官方的动态的话,大概是没啥问题。Docker通过libnetwork实现了原生的网络方案,在很大程度上弥补了之前网络功能缺失的问题,希望这个方案能尽快成熟吧。
参考资料:
https://docs.docker.com/engine/userguide/networking/work-with-networks/
https://docs.docker.com/engine/userguide/networking/get-started-overlay/
https://docs.docker.com/engine/userguide/networking/dockernetworks/
http://hustcat.github.io/docker-overlay-network-practice/
作者简介:何智刚,2015至今,现为广东的一名在校高三学生,在学习之余,主要研究Docker,OpenStack,SDN,对各种领域都有所涉猎,目标是迈向full stack。