5 链跟踪，定位位置

首先，查看FORWARD链：

[root@w-openstack32 ~]# iptables -nxvL FORWARD
Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
    pkts      bytes target     prot opt in     out     source               destination
46935472750 20715430397978 neutron-filter-top  all  --  *      *       0.0.0.0/0            0.0.0.0/0
46935472750 20715430397978 neutron-openvswi-FORWARD  all  --  *      *       0.0.0.0/0            0.0.0.0/0

往下走，跟踪neutron-openvswi-FORWARD链：

[root@w-openstack32 ~]# iptables -nxvL neutron-openvswi-FORWARD
Chain neutron-openvswi-FORWARD (1 references)
    pkts      bytes target     prot opt in     out     source               destination
46934035761 20714724813160 neutron-openvswi-scope  all  --  *      *       0.0.0.0/0            0.0.0.0/0
459609600 121602465610 neutron-openvswi-sg-chain  all  --  *      *       0.0.0.0/0            0.0.0.0/0            PHYSDEV match --physdev-out tapbddxxxxc-bd --physdev-is-bridged /* Direct traffic from the VM interface to the security group chain. */
418512415 87324679449 neutron-openvswi-sg-chain  all  --  *      *       0.0.0.0/0            0.0.0.0/0            PHYSDEV match --physdev-in tapbddxxxxc-bd --physdev-is-bridged /* Direct traffic from the VM interface to the security group chain. */

继续往下走，跟踪neutron-openvswi-sg-chain链：

[root@w-openstack32 ~]# iptables -nxvL neutron-openvswi-sg-chain
Chain neutron-openvswi-sg-chain (10 references)
    pkts      bytes target     prot opt in     out     source               destination
459619285 121606141606 neutron-openvswi-ibddxxxxc-b  all  --  *      *       0.0.0.0/0            0.0.0.0/0            PHYSDEV match --physdev-out tapbddxxxxc-bd --physdev-is-bridged /* Jump to the VM specific chain. */
418522179 87328347603 neutron-openvswi-obddxxxxc-b  all  --  *      *       0.0.0.0/0            0.0.0.0/0            PHYSDEV match --physdev-in tapbddxxxxc-bd --physdev-is-bridged /* Jump to the VM specific chain. */
46927558621 20715211464005 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0

往下走，跟踪进入neutron-openvswi-ibddxxxxc-b链：

[root@w-openstack32 ~]# iptables -nxvL neutron-openvswi-ibddxxxxc-b
Chain neutron-openvswi-ibddxxxxc-b (1 references)
    pkts      bytes target     prot opt in     out     source               destination
428583997 119531816501 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0            state RELATED,ESTABLISHED /* Direct packets associated with a known session to the RETURN chain. */
       0        0 DROP       all  --  *      *       0.0.0.0/0            0.0.0.0/0            state INVALID /* Drop packets that appear related to an existing connection (e.g. TCP ACK/FIN) but do not have an entry in conntrack. */
 1937647 124006974 neutron-openvswi-sg-fallback  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* Send unmatched traffic to the fallback chain. */

最后看下，如果以上规则都不匹配，做的动作：

[root@w-openstack32 ~]# iptables -nxvL neutron-openvswi-sg-fallback
Chain neutron-openvswi-sg-fallback (10 references)
    pkts      bytes target     prot opt in     out     source               destination
 9841017 629783588 DROP       all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* Default drop rule for unmatched traffic. */

6 Neutron安全组白名单机制

Openstack Neutron安全组使用白名单机制，此时网络的访问能力 = 用户自定义的规则允许的能力。如果这些规则都不匹配，没看错，是drop掉该数据包。

可以通过上边的iptables命令看到，安全组没有放开IPIP协议的规则，所以数据包没进到虚机中。

控制节点上再确认下安全组策略，的确没有IPIP协议规则。

7 确认IPIP协议号

查看kernel源码中头文件include/uapi/linux/in.h，可以看到IPIP协议号是4；

/* Standard well-defined IP protocols.  */
enum {
  IPPROTO_IP = 0,   /* Dummy protocol for TCP   */
  IPPROTO_ICMP = 1,   /* Internet Control Message Protocol  */
  IPPROTO_IGMP = 2,   /* Internet Group Management Protocol */
  IPPROTO_IPIP = 4,   /* IPIP tunnels (older KA9Q tunnels use 94) */
  IPPROTO_TCP = 6,    /* Transmission Control Protocol  */
  IPPROTO_EGP = 8,    /* Exterior Gateway Protocol    */
  IPPROTO_PUP = 12,   /* PUP protocol       */
  IPPROTO_UDP = 17,   /* User Datagram Protocol   */
  IPPROTO_IDP = 22,   /* XNS IDP protocol     */
  IPPROTO_DCCP = 33,    /* Datagram Congestion Control Protocol */
  IPPROTO_RSVP = 46,    /* RSVP protocol      */
  IPPROTO_GRE = 47,   /* Cisco GRE tunnels (rfc 1701,1702)  */
...

8 加入规则，纠正网络

在Openstack Neutron安全组dashboard上加入规则：

在控制节点上看到规则已经生成：

9 单元测试

再去虚机抓包，客户端发包通信接收到应答包，并且在虚机上抓到IPIP数据包。同时用命令行查看iptables，已有数据包匹配并执行（prot为4的pkts不为0）。

[root@w-openstack32 ~]# iptables -nxvL neutron-openvswi-ibddxxxxc-b
Chain neutron-openvswi-ibddxxxxc-b (1 references)
    pkts      bytes target     prot opt in     out     source               destination
428583997 119531816501 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0            state RELATED,ESTABLISHED /* Direct packets associated with a known session to the RETURN chain. */
   12266  7040888 RETURN     4    --  *      *       0.0.0.0/0            0.0.0.0/0
       0        0 DROP       all  --  *      *       0.0.0.0/0            0.0.0.0/0            state INVALID /* Drop packets that appear related to an existing connection (e.g. TCP ACK/FIN) but do not have an entry in conntrack. */
 1937647 124006974 neutron-openvswi-sg-fallback  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* Send unmatched traffic to the fallback chain. */

在虚机上抓包，没有看到其他类型数据包进入：

tcpdump -i any \(not icmp\) and \(not udp\) and \(not tcp\) and \(not arp\)

10 推入生产，回归测试

经验证，和测试环境效果一致，done。

总结

通过问题的探索，我们了解Openstack Neutron的安全组策略相关知识了，也对Neutron网络数据面的网络拓扑架构有了基本的认识。之后，我会结合源码展开谈谈iptables模式匹配与执行，以及iptables的高级用法和原理，以及部分netfilter的原理。

本文转载自公众号360云计算（ID：hulktalk）。

原文链接：

https://mp.weixin.qq.com/s/jMBRgscKJVgjQZRZ3ZES_A

创作场景

带着问题了解 Openstack Neutron 安全组（下）