1. Deploy Topology
图 1 Deploy Topology
图 2 ovs拓扑
br-int负责东西流量与tunnel建立,br-ex负责南北流量,通过patch接口连接起来。
注意:当前netvirt只支持tunnel不支持vlan。
控制节点ovs配置:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
vagrant@devstack-control:~/devstack$ sudo ovs-vsctl show 96c4b076-edbd-4a5a-b82b-da3def978a9c Manager "tcp:192.168.50.20:6640" is_connected: true Bridge br-ex Controller "tcp:192.168.50.20:6653" is_connected: true fail_mode: secure Port br-ex Interface br-ex type: internal Port "eth2" Interface "eth2" Bridge br-int Controller "tcp:192.168.50.20:6653" is_connected: true fail_mode: secure Port br-int Interface br-int type: internal ovs_version: "2.3.1" |
控制节点openflow流表:br-ex
1 2 3 4 |
$sudo ovs-ofctl -O OpenFlow13 dump-flows br-ex OFPST_FLOW reply (OF1.3)(xid=0x2): cookie=0x0, duration=15.660s, table=0, n_packets=2, n_bytes=218, priority=0 actions=NORMAL cookie=0x0, duration=5.198s, table=0, n_packets=1, n_bytes=109, dl_type=0x88cc actions=CONTROLLER:65535 |
计算节点流表与控制节点一样,不再赘述。
2. Tenants Dopology
这里我们创建2个tenant,租户内的VM通过Distributed Router访问外网,如下所示,每个租户有一个分布式路由,通过eth2连接到RotuerVM。
图 3 Tenants Topology
RouterVM为外部路由器,网关为192.168.111.254。
VM一般通过SNAT与DNAT方式访问外网,当前netvirt仅支持DNAT方式,根据neutron的实现VM访问外网需要进行如下配置:
1 创建router
2 创建externalnetwork,网络类型为flat
3 将外部网络关联到router
4 创建floatingip
tenantinfo:
Device type | tenant | host | ip | mac | floatingip | vxlan |
T1-VM1 | Tenant1 | control | 10.1.0.3 | fa:16:3e:0e:36:3e | 192.168.111.22 | 0x442 |
T1-VM2 | Tenant1 | compute | 10.1.0.4 | fa:16:3e:fa:03:a0 | 192.168.111.23 | 0x442 |
T1-DHCP | Tenant1 | control | 10.1.0.2 | fa:16:3e:e7:fc:a5 | ||
T1_Router | Tenant1 | 10.1.0.1 | fa:16:3e:36:43:ca | 192.168.111.21 |
External ip allocation:
External ip | mac | Device type | |
Router Interface | 192.168.111.21 | fa:16:3e:24:8f:05 | t1 router interface |
floatingip | 192.168.111.22 | fa:16:3e:e1:b4:76 | t1_vm1 |
flaotingip | 192.168.111.23 | fa:16:3e:2a:20:73 | T1_vm2 |
每个floatingip都有一个mac与port,用于处理floatingip arp。
3. pipeline
控制节点:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 |
$ sudo ovs-ofctl -O OpenFlow13 dump-flows br-int | cut -d',' -f3- OFPST_FLOW reply (OF1.3) (xid=0x2): ### table=0, n_packets=1860, n_bytes=215428, in_port=4,dl_src=fa:16:3e:0e:36:3f actions=set_field:0x442->tun_id,load:0x1->NXM_NX_REG0[],goto_table:20 table=0, n_packets=2405, n_bytes=221997, in_port=1,dl_src=fa:16:3e:e7:fc:a5 actions=set_field:0x442->tun_id,load:0x1->NXM_NX_REG0[],goto_table:20 table=0, n_packets=15, n_bytes=1824, tun_id=0x442,in_port=3 actions=load:0x2->NXM_NX_REG0[],goto_table:20 table=0, n_packets=8692, n_bytes=964812, dl_type=0x88cc actions=CONTROLLER:65535 table=0, n_packets=0, n_bytes=0, priority=8192,in_port=1 actions=drop table=0, n_packets=0, n_bytes=0, priority=8192,in_port=4 actions=drop table=0, n_packets=8021, n_bytes=575011, priority=0 actions=goto_table:20 ### table=20, n_packets=12257, n_bytes=1012412, priority=0 actions=goto_table:30 table=20, n_packets=4, n_bytes=168, priority=1024,arp,tun_id=0x442,arp_tpa=10.1.0.4 actions=move:NXM_OF_ETH_SRC[]->NXM_OF_ETH_DST[],set_field:fa:16:3e:fa:03:a0->eth_src,load:0x2->NXM_OF_ARP_OP[],move:NXM_NX_ARP_SHA[]->NXM_NX_ARP_THA[],move:NXM_OF_ARP_SPA[]->NXM_OF_ARP_TPA[],load:0xfa163efa03a0->NXM_NX_ARP_SHA[],load:0xa010004->NXM_OF_ARP_SPA[],IN_PORT table=20, n_packets=2, n_bytes=84, priority=1024,arp,tun_id=0x442,arp_tpa=10.1.0.2 actions=move:NXM_OF_ETH_SRC[]->NXM_OF_ETH_DST[],set_field:fa:16:3e:e7:fc:a5->eth_src,load:0x2->NXM_OF_ARP_OP[],move:NXM_NX_ARP_SHA[]->NXM_NX_ARP_THA[],move:NXM_OF_ARP_SPA[]->NXM_OF_ARP_TPA[],load:0xfa163ee7fca5->NXM_NX_ARP_SHA[],load:0xa010002->NXM_OF_ARP_SPA[],IN_PORT table=20, n_packets=1, n_bytes=42, priority=1024,arp,tun_id=0x442,arp_tpa=10.1.0.3 actions=move:NXM_OF_ETH_SRC[]->NXM_OF_ETH_DST[],set_field:fa:16:3e:0e:36:3f->eth_src,load:0x2->NXM_OF_ARP_OP[],move:NXM_NX_ARP_SHA[]->NXM_NX_ARP_THA[],move:NXM_OF_ARP_SPA[]->NXM_OF_ARP_TPA[],load:0xfa163e0e363f->NXM_NX_ARP_SHA[],load:0xa010003->NXM_OF_ARP_SPA[],IN_PORT table=20, n_packets=16, n_bytes=672, priority=1024,arp,tun_id=0x442,arp_tpa=10.1.0.1 actions=move:NXM_OF_ETH_SRC[]->NXM_OF_ETH_DST[],set_field:fa:16:3e:36:43:ca->eth_src,load:0x2->NXM_OF_ARP_OP[],move:NXM_NX_ARP_SHA[]->NXM_NX_ARP_THA[],move:NXM_OF_ARP_SPA[]->NXM_OF_ARP_TPA[],load:0xfa163e3643ca->NXM_NX_ARP_SHA[],load:0xa010001->NXM_OF_ARP_SPA[],IN_PORT #floatingip arp responser table=20, n_packets=21, n_bytes=882, priority=1024,arp,in_port=2,arp_tpa=192.168.111.22 actions=move:NXM_OF_ETH_SRC[]->NXM_OF_ETH_DST[],set_field:fa:16:3e:e1:b4:76->eth_src,load:0x2->NXM_OF_ARP_OP[],move:NXM_NX_ARP_SHA[]->NXM_NX_ARP_THA[],move:NXM_OF_ARP_SPA[]->NXM_OF_ARP_TPA[],load:0xfa163ee1b476->NXM_NX_ARP_SHA[],load:0xc0a86f16->NXM_OF_ARP_SPA[],IN_PORT #### table=30, n_packets=11762, n_bytes=777765, priority=0 actions=goto_table:40 #inbound nat table=30, n_packets=495, n_bytes=234647, priority=1024,ip,in_port=2,nw_dst=192.168.111.22 actions=set_field:10.1.0.3->ip_dst,load:0x442->NXM_NX_REG3[],goto_table:40 #### table=40, n_packets=1831, n_bytes=213408, priority=36001,ip,in_port=4,dl_src=fa:16:3e:0e:36:3f,nw_src=10.1.0.3 actions=goto_table:50 table=40, n_packets=10426, n_bytes=799004, priority=0 actions=goto_table:50 #drop dhcp replay from vm, the port of dhcp server is 67 table=40, n_packets=0, n_bytes=0, priority=61011,udp,in_port=4,tp_src=67,tp_dst=68 actions=drop #### table=50, n_packets=12257, n_bytes=1012412, priority=0 actions=goto_table:60 table=60, n_packets=11762, n_bytes=777765, priority=0 actions=goto_table:70 #routing table=60, n_packets=495, n_bytes=234647, priority=2048,ip,reg3=0x442,nw_dst=10.1.0.0/24 actions=set_field:fa:16:3e:36:43:ca->eth_src,dec_ttl,set_field:0x442->tun_id,goto_table:70 #### table=70, n_packets=7957, n_bytes=379282, priority=0 actions=goto_table:80 table=70, n_packets=1392, n_bytes=175518, priority=1024,ip,tun_id=0x442,nw_dst=10.1.0.2 actions=set_field:fa:16:3e:e7:fc:a5->eth_dst,goto_table:80 table=70, n_packets=2894, n_bytes=455721, priority=1024,ip,tun_id=0x442,nw_dst=10.1.0.3 actions=set_field:fa:16:3e:0e:36:3f->eth_dst,goto_table:80 table=70, n_packets=9, n_bytes=1401, priority=1024,ip,tun_id=0x442,nw_dst=10.1.0.4 actions=set_field:fa:16:3e:fa:03:a0->eth_dst,goto_table:80 table=80, n_packets=12252, n_bytes=1011922, priority=0 actions=goto_table:90 table=90, n_packets=12248, n_bytes=1010492, priority=0 actions=goto_table:100 table=90, n_packets=4, n_bytes=1430, priority=61006,udp,dl_src=fa:16:3e:e7:fc:a5,tp_src=67,tp_dst=68 actions=goto_table:100 #### #outbound nat table=100, n_packets=423, n_bytes=36361, priority=512,ip,tun_id=0x442,dl_dst=fa:16:3e:36:43:ca,nw_src=10.1.0.3 actions=set_field:fa:16:3e:e1:b4:76->eth_src,dec_ttl,set_field:00:00:5e:00:01:01->eth_dst,set_field:192.168.111.22->ip_src,output:2 table=100, n_packets=7528, n_bytes=342333, priority=0 actions=goto_table:110 table=100, n_packets=4301, n_bytes=633228, priority=1024,ip,tun_id=0x442,nw_dst=10.1.0.0/24 actions=goto_table:110 #### table=110, n_packets=9, n_bytes=843, priority=8192,tun_id=0x442 actions=drop table=110, n_packets=7506, n_bytes=339572, priority=0 actions=drop table=110, n_packets=8, n_bytes=1138, priority=16384,reg0=0x2,tun_id=0x442,dl_dst=01:00:00:00:00:00/01:00:00:00:00:00 actions=output:1,output:4 table=110, n_packets=11, n_bytes=1368, priority=16383,reg0=0x1,tun_id=0x442,dl_dst=01:00:00:00:00:00/01:00:00:00:00:00 actions=output:1,output:3,output:4 table=110, n_packets=2894, n_bytes=455721, tun_id=0x442,dl_dst=fa:16:3e:0e:36:3f actions=output:4 table=110, n_packets=9, n_bytes=1401, tun_id=0x442,dl_dst=fa:16:3e:fa:03:a0 actions=output:3 table=110, n_packets=1392, n_bytes=175518, tun_id=0x442,dl_dst=fa:16:3e:e7:fc:a5 actions=output:1 |
计算节点与其类似,不再赘述。
4. 小结
floatingip通过Table30,60与100实现的,通过配置静态流表,修改报文的destinationip与source ip来实现,是一种无状态的nat实现,对于snat与port-forwarding还不支持。
作者简介:胡西宁,搞过neutron,对SDN/NFV有浓厚兴趣,一直从事相关工作,现在某通信公司工作