1. Tenants Topology
图 1 TenantsTopology
此图为netvirtdemo生成的拓扑，包括两个租户Tenant1与Tenant2，其配置如下：
| vm | tenant | host | ip | mac | ofport | vxlan | |
| T1-VM1 | Tenant1 | control | 1.0.0.2 | fa:16:3e:32:d8:e8 | 4 | ||
| T1-VM2 | Tenant1 | compute | 1.0.0.3 | fa:16:3e:4c:39:0e | 2 | ||
| T1-DHCP | Tenant1 | control | 1.0.0.1 | fa:16:3e:49:68:47 | 1 | ||
| T1_Router | Tenant1 | 1.0.0.254 | fa:16:3e:69:5a:42 | ||||
| vxlan tunnel | host | control | 72:8a:8e:af:52:57 | 2 | 1001(0x3e9) | ||
| T2-VM1 | Tenant2 | control | 2.0.0.2 | fa:16:3e:41:56:ec | 5 | ||
| T2-VM2 | Tenant2 | compute | 2.0.0.3 | fa:16:3e:94:75:95 | 3 | ||
| T2-DHCP | Tenant2 | control | 2.0.0.1 | fa:16:3e:a8:c2:66 | 3 | ||
| T2_Router | Tenant2 | 2.0.0.254 | fa:16:3e:cb:14:47 | ||||
| vxlan tunnel | host | compute | 0e:1e:92:c3:c7:5a | 1 | 1002(0x3ea) |
启动VM后，控制节点的ovs配置如下：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 |
vagrant@devstack-control:~/devstack$ sudo ovs-vsctl show 65d2cad6-e0ed-4a57-869a-85686c485930 Manager "tcp:192.168.50.20:6640" is_connected: true Bridge br-int Controller "tcp:192.168.50.20:6633" is_connected: true fail_mode: secure Port "tap83f1bf16-3c" //t1_vm1 Interface "tap83f1bf16-3c" Port br-int Interface br-int type: internal Port "tap9f750bfb-4a" //t1_dhcp Interface "tap9f750bfb-4a" type: internal Port "tapa1d116c9-93" //t2_dhcp Interface "tapa1d116c9-93" type: internal Port "tapad13ec08-42" //t2_vm1 Interface "tapad13ec08-42" Port "vxlan-192.168.50.21" Interface "vxlan-192.168.50.21" type: vxlan options: {key=flow, local_ip="192.168.50.20", remote_ip="192.168.50.21"} ovs_version: "2.3.2" |
计算节点配置如下：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
vagrant@devstack-compute-1:~$ sudo ovs-vsctl show a31569c6-314f-41dd-972d-a75806b4ee3f Manager "tcp:192.168.50.20:6640" is_connected: true Bridge br-int Controller "tcp:192.168.50.20:6633" is_connected: true fail_mode: secure Port "vxlan-192.168.50.20" Interface "vxlan-192.168.50.20" type: vxlan options: {key=flow, local_ip="192.168.50.21", remote_ip="192.168.50.20"} Port "tapb58febde-6f" //t1_vm2 Interface "tapb58febde-6f" Port br-int Interface br-int Port "tap2a008646-41" //t2_vm2 Interface "tap2a008646-41" ovs_version: "2.3.2" |
2. Pipeline
控制节点：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 |
controller＃sudo ovs-ofctl -O OpenFlow13 dump-flows br-int OFPST_FLOW reply (OF1.3) (xid=0x2): #add tunnel and set flow direction from vms and dhcps cookie=0x0, duration=9640.568s, table=0, n_packets=309, n_bytes=34846, in_port=4,dl_src=fa:16:3e:32:d8:e8 actions=set_field:0x3e9->tun_id,load:0x1->NXM_NX_REG0[],goto_table:20 cookie=0x0, duration=9672.344s, table=0, n_packets=35, n_bytes=4086, in_port=3,dl_src=fa:16:3e:a8:c2:66 actions=set_field:0x3ea->tun_id,load:0x1->NXM_NX_REG0[],goto_table:20 cookie=0x0, duration=9612.070s, table=0, n_packets=28, n_bytes=2848, in_port=5,dl_src=fa:16:3e:41:56:ec actions=set_field:0x3ea->tun_id,load:0x1->NXM_NX_REG0[],goto_table:20 cookie=0x0, duration=9687.978s, table=0, n_packets=279, n_bytes=28627, in_port=1,dl_src=fa:16:3e:49:68:47 actions=set_field:0x3e9->tun_id,load:0x1->NXM_NX_REG0[],goto_table:20 #drop unexception mac pkts from vms and dhcps cookie=0x0, duration=9671.841s, table=0, n_packets=0, n_bytes=0, priority=8192,in_port=3 actions=drop cookie=0x0, duration=9611.564s, table=0, n_packets=0, n_bytes=0, priority=8192,in_port=5 actions=drop cookie=0x0, duration=9687.493s, table=0, n_packets=0, n_bytes=0, priority=8192,in_port=1 actions=drop cookie=0x0, duration=9640.052s, table=0, n_packets=0, n_bytes=0, priority=8192,in_port=4 actions=drop #set reg0 to 2 for from other nodes cookie=0x0, duration=9684.398s, table=0, n_packets=94, n_bytes=11730, tun_id=0x3e9,in_port=2 actions=load:0x2->NXM_NX_REG0[],goto_table:20 cookie=0x0, duration=9668.768s, table=0, n_packets=17, n_bytes=2050, tun_id=0x3ea,in_port=2 actions=load:0x2->NXM_NX_REG0[],goto_table:20 #push lldp to controller cookie=0x0, duration=28587.063s, table=0, n_packets=1939, n_bytes=219107, dl_type=0x88cc actions=CONTROLLER:65535 #default got to 20, default drop? cookie=0x0, duration=28586.366s, table=0, n_packets=22, n_bytes=2250, priority=0 actions=goto_table:20 #arp responser from other nodes #t1_vm1 arp replay cookie=0x0, duration=9645.244s, table=20, n_packets=2, n_bytes=84, priority=1024,arp,tun_id=0x3e9,arp_tpa=1.0.0.3 actions=move:NXM_OF_ETH_SRC[]->NXM_OF_ETH_DST[],set_field:fa:16:3e:4c:39:0e->eth_src,load:0x2->NXM_OF_ARP_OP[],move:NXM_NX_ARP_SHA[]->NXM_NX_ARP_THA[],move:NXM_OF_ARP_SPA[]->NXM_OF_ARP_TPA[],load:0xfa163e4c390e->NXM_NX_ARP_SHA[],load:0x1000003->NXM_OF_ARP_SPA[],IN_PORT #t1_dhcp cookie=0x0, duration=9673.863s, table=20, n_packets=3, n_bytes=126, priority=1024,arp,tun_id=0x3e9,arp_tpa=1.0.0.1 actions=move:NXM_OF_ETH_SRC[]->NXM_OF_ETH_DST[],set_field:fa:16:3e:49:68:47->eth_src,load:0x2->NXM_OF_ARP_OP[],move:NXM_NX_ARP_SHA[]->NXM_NX_ARP_THA[],move:NXM_OF_ARP_SPA[]->NXM_OF_ARP_TPA[],load:0xfa163e496847->NXM_NX_ARP_SHA[],load:0x1000001->NXM_OF_ARP_SPA[],IN_PORT #t1 vm gatewayport cookie=0x0, duration=9677.419s, table=20, n_packets=3, n_bytes=126, priority=1024,arp,tun_id=0x3e9,arp_tpa=1.0.0.254 actions=move:NXM_OF_ETH_SRC[]->NXM_OF_ETH_DST[],set_field:fa:16:3e:69:5a:42->eth_src,load:0x2->NXM_OF_ARP_OP[],move:NXM_NX_ARP_SHA[]->NXM_NX_ARP_THA[],move:NXM_OF_ARP_SPA[]->NXM_OF_ARP_TPA[],load:0xfa163e695a42->NXM_NX_ARP_SHA[],load:0x10000fe->NXM_OF_ARP_SPA[],IN_PORT #t1_vm1, local arp responser cookie=0x0, duration=9656.405s, table=20, n_packets=1, n_bytes=42, priority=1024,arp,tun_id=0x3e9,arp_tpa=1.0.0.2 actions=move:NXM_OF_ETH_SRC[]->NXM_OF_ETH_DST[],set_field:fa:16:3e:32:d8:e8->eth_src,load:0x2->NXM_OF_ARP_OP[],move:NXM_NX_ARP_SHA[]->NXM_NX_ARP_THA[],move:NXM_OF_ARP_SPA[]->NXM_OF_ARP_TPA[],load:0xfa163e32d8e8->NXM_NX_ARP_SHA[],load:0x1000002->NXM_OF_ARP_SPA[],IN_PORT #t2_dhcp cookie=0x0, duration=9654.361s, table=20, n_packets=2, n_bytes=84, priority=1024,arp,tun_id=0x3ea,arp_tpa=2.0.0.1 actions=move:NXM_OF_ETH_SRC[]->NXM_OF_ETH_DST[],set_field:fa:16:3e:a8:c2:66->eth_src,load:0x2->NXM_OF_ARP_OP[],move:NXM_NX_ARP_SHA[]->NXM_NX_ARP_THA[],move:NXM_OF_ARP_SPA[]->NXM_OF_ARP_TPA[],load:0xfa163ea8c266->NXM_NX_ARP_SHA[],load:0x2000001->NXM_OF_ARP_SPA[],IN_PORT #t2 vm gatewayport cookie=0x0, duration=9660.025s, table=20, n_packets=3, n_bytes=126, priority=1024,arp,tun_id=0x3ea,arp_tpa=2.0.0.254 actions=move:NXM_OF_ETH_SRC[]->NXM_OF_ETH_DST[],set_field:fa:16:3e:cb:14:47->eth_src,load:0x2->NXM_OF_ARP_OP[],move:NXM_NX_ARP_SHA[]->NXM_NX_ARP_THA[],move:NXM_OF_ARP_SPA[]->NXM_OF_ARP_TPA[],load:0xfa163ecb1447->NXM_NX_ARP_SHA[],load:0x20000fe->NXM_OF_ARP_SPA[],IN_PORT #t2_vm1, local arp responser cookie=0x0, duration=9642.164s, table=20, n_packets=1, n_bytes=42, priority=1024,arp,tun_id=0x3ea,arp_tpa=2.0.0.2 actions=move:NXM_OF_ETH_SRC[]->NXM_OF_ETH_DST[],set_field:fa:16:3e:41:56:ec->eth_src,load:0x2->NXM_OF_ARP_OP[],move:NXM_NX_ARP_SHA[]->NXM_NX_ARP_THA[],move:NXM_OF_ARP_SPA[]->NXM_OF_ARP_TPA[],load:0xfa163e4156ec->NXM_NX_ARP_SHA[],load:0x2000002->NXM_OF_ARP_SPA[],IN_PORT #t2_vm2 cookie=0x0, duration=9629.870s, table=20, n_packets=3, n_bytes=126, priority=1024,arp,tun_id=0x3ea,arp_tpa=2.0.0.3 actions=move:NXM_OF_ETH_SRC[]->NXM_OF_ETH_DST[],set_field:fa:16:3e:94:75:95->eth_src,load:0x2->NXM_OF_ARP_OP[],move:NXM_NX_ARP_SHA[]->NXM_NX_ARP_THA[],move:NXM_OF_ARP_SPA[]->NXM_OF_ARP_TPA[],load:0xfa163e947595->NXM_NX_ARP_SHA[],load:0x2000003->NXM_OF_ARP_SPA[],IN_PORT #default forwarding all flow cookie=0x0, duration=28585.859s, table=20, n_packets=766, n_bytes=85681, priority=0 actions=goto_table:30 #forwarding normal for local subnet cookie=0x0, duration=9659.015s, table=30, n_packets=62, n_bytes=6828, priority=1024,ip,tun_id=0x3ea,nw_dst=2.0.0.0/24 actions=goto_table:40 cookie=0x0, duration=9676.412s, table=30, n_packets=540, n_bytes=61729, priority=1024,ip,tun_id=0x3e9,nw_dst=1.0.0.0/24 actions=goto_table:40 #default forwarding all flow cookie=0x0, duration=28585.349s, table=30, n_packets=164, n_bytes=17124, priority=0 actions=goto_table:40 #pipeline process cookie=0x0, duration=28584.843s, table=40, n_packets=766, n_bytes=85681, priority=0 actions=goto_table:50 cookie=0x0, duration=28584.338s, table=50, n_packets=766, n_bytes=85681, priority=0 actions=goto_table:60 #routing cookie=0x0, duration=9662.085s, table=60, n_packets=122, n_bytes=11222, priority=2048,ip,tun_id=0x3e9,nw_dst=2.0.0.0/24 actions=set_field:fa:16:3e:cb:14:47->eth_src,dec_ttl,set_field:0x3ea->tun_id,goto_table:70 cookie=0x0, duration=9661.045s, table=60, n_packets=4, n_bytes=392, priority=2048,ip,tun_id=0x3ea,nw_dst=1.0.0.0/24 actions=set_field:fa:16:3e:69:5a:42->eth_src,dec_ttl,set_field:0x3e9->tun_id,goto_table:70 #default forwarding all flow cookie=0x0, duration=28583.831s, table=60, n_packets=640, n_bytes=74067, priority=0 actions=goto_table:70 #L3 fowarding table cookie=0x0, duration=9645.756s, table=70, n_packets=15, n_bytes=1924, priority=1024,ip,tun_id=0x3e9,nw_dst=1.0.0.3 actions=set_field:fa:16:3e:4c:39:0e->eth_dst,goto_table:80 cookie=0x0, duration=9642.698s, table=70, n_packets=17, n_bytes=2120, priority=1024,ip,tun_id=0x3ea,nw_dst=2.0.0.2 actions=set_field:fa:16:3e:41:56:ec->eth_dst,goto_table:80 cookie=0x0, duration=9656.933s, table=70, n_packets=337, n_bytes=36287, priority=1024,ip,tun_id=0x3e9,nw_dst=1.0.0.2 actions=set_field:fa:16:3e:32:d8:e8->eth_dst,goto_table:80 cookie=0x0, duration=9630.376s, table=70, n_packets=135, n_bytes=12950, priority=1024,ip,tun_id=0x3ea,nw_dst=2.0.0.3 actions=set_field:fa:16:3e:94:75:95->eth_dst,goto_table:80 cookie=0x0, duration=9654.869s, table=70, n_packets=26, n_bytes=2392, priority=1024,ip,tun_id=0x3ea,nw_dst=2.0.0.1 actions=set_field:fa:16:3e:a8:c2:66->eth_dst,goto_table:80 cookie=0x0, duration=9674.371s, table=70, n_packets=186, n_bytes=23322, priority=1024,ip,tun_id=0x3e9,nw_dst=1.0.0.1 actions=set_field:fa:16:3e:49:68:47->eth_dst,goto_table:80 #default forwarding all flow cookie=0x0, duration=28583.313s, table=70, n_packets=50, n_bytes=6686, priority=0 actions=goto_table:80 #pipeline process, default forwarding cookie=0x0, duration=28582.807s, table=80, n_packets=766, n_bytes=85681, priority=0 actions=goto_table:90 cookie=0x0, duration=28582.275s, table=90, n_packets=766, n_bytes=85681, priority=0 actions=goto_table:100 cookie=0x0, duration=28581.334s, table=100, n_packets=38, n_bytes=5510, priority=0 actions=goto_table:110 #local net cookie=0x0, duration=9657.954s, table=100, n_packets=184, n_bytes=18050, priority=1024,ip,tun_id=0x3ea,nw_dst=2.0.0.0/24 actions=goto_table:110 cookie=0x0, duration=9675.405s, table=100, n_packets=544, n_bytes=62121, priority=1024,ip,tun_id=0x3e9,nw_dst=1.0.0.0/24 actions=goto_table:110 #drop pkts for unexcepted subnet in this node cookie=0x0, duration=9684.903s, table=110, n_packets=6, n_bytes=588, priority=8192,tun_id=0x3e9 actions=drop cookie=0x0, duration=9669.305s, table=110, n_packets=6, n_bytes=588, priority=8192,tun_id=0x3ea actions=drop #flood all ports for boardcast cookie=0x0, duration=9685.913s, table=110, n_packets=4, n_bytes=798, priority=16384,reg0=0x1,tun_id=0x3e9,dl_dst=01:00:00:00:00:00/01:00:00:00:00:00 actions=output:1,output:2,output:4 cookie=0x0, duration=9670.825s, table=110, n_packets=2, n_bytes=658, priority=16384,reg0=0x2,tun_id=0x3ea,dl_dst=01:00:00:00:00:00/01:00:00:00:00:00 actions=output:3,output:5 cookie=0x0, duration=9686.418s, table=110, n_packets=2, n_bytes=658, priority=16384,reg0=0x2,tun_id=0x3e9,dl_dst=01:00:00:00:00:00/01:00:00:00:00:00 actions=output:1,output:4 cookie=0x0, duration=9670.320s, table=110, n_packets=2, n_bytes=658, priority=16384,reg0=0x1,tun_id=0x3ea,dl_dst=01:00:00:00:00:00/01:00:00:00:00:00 actions=output:3,output:2,output:5 #vxlan table, why is it in this table? cookie=0x0, duration=9611.059s, table=110, n_packets=17, n_bytes=2120, tun_id=0x3ea,dl_dst=fa:16:3e:41:56:ec actions=output:5 cookie=0x0, duration=9686.987s, table=110, n_packets=186, n_bytes=23322, tun_id=0x3e9,dl_dst=fa:16:3e:49:68:47 actions=output:1 cookie=0x0, duration=9671.330s, table=110, n_packets=26, n_bytes=2392, tun_id=0x3ea,dl_dst=fa:16:3e:a8:c2:66 actions=output:3 cookie=0x0, duration=9639.542s, table=110, n_packets=337, n_bytes=36287, tun_id=0x3e9,dl_dst=fa:16:3e:32:d8:e8 actions=output:4 #forward to tunnel for remote vm cookie=0x0, duration=9620.686s, table=110, n_packets=15, n_bytes=1924, tun_id=0x3e9,dl_dst=fa:16:3e:4c:39:0e actions=output:2 cookie=0x0, duration=9597.367s, table=110, n_packets=135, n_bytes=12950, tun_id=0x3ea,dl_dst=fa:16:3e:94:75:95 actions=output:2 #default drop cookie=0x0, duration=28580.779s, table=110, n_packets=28, n_bytes=2738, priority=0 actions=drop |
计算节点转发与其类似这里不再赘述。
3. 关注表项
Table 20 ，60，70 andTable 110
作者简介：胡西宁，搞过neutron，对SDN/NFV有浓厚兴趣，一直从事相关工作，现在某通信公司工作
