Previously, we introduced the architecture and packet flow of Open vSwitch with Provider Networks. Today, we'll cover Open vSwitch with self-service networks.

Table of Contents

Open vSwitch: Self-service Networks

Architecture

The above diagram shows the overall architecture of Open vSwitch – Self-service networks, clearly illustrating which components run on the controller node, which on the compute node, and which on the network node.

This diagram illustrates how the various components are interconnected under a single untagged (flat) provider network and a single self-service network. In this example, the instance and DHCP agent are on the same host, but in real-world scenarios, the DHCP agent might be located on a different compute node.

Traffic Flow

This section will explain how packets actually flow through the architecture in various scenarios.

Architecture Setup

Below is the envisioned architecture, featuring two self-service networks separated by VXLAN, each with one instance.

Provider Network 1 (VLAN)
- VLAN ID 101 (Tagged)
Self-service network 1 (VXLAN)
- VXLAN ID (VNI) 101
Self-service network 2 (VXLAN)
- iptables (3) on Linux bridge will handle firewall and connection tracking
OVS on Linux bridge via veth pair forwards packets to OVS integration bridge security group interface (4)
- OVS tunnel bridge (8) encapsulates packets with VNI 101
- OVS integration bridge - "int-br-provider" patch interface (6) forwards packets to OVS tunnel bridge r patch interface (7)
- OVS integration bridge adds internal VLAN tag to packets
Instance 1
Instance 2

North-south traffic flow 1: Instance with fixed IP

For instances with fixed IPv4 addresses, the network node performs SNAT to route traffic from self-service networks to external networks like the Internet. For instances with fixed IPv6 addresses, the network node forwards packets via standard routing to external networks.

Instance on compute node 1 and using self-service network 1
Instance forwarding packets to external network

Instance network interface (1) through veth pair (2) sends packets to the instance port on the Linux bridge
在 linux bridge 上 iptables (3) 會處理防火牆跟 connection tracking
Linux bridge 上的 ovs 介 (4) 面透過 veth pair 將封包送往 OVS integration bridge security group 介面 (5)
OVS integration bridge 將封包加上 internal VLAN tag
OVS integration bridge int-br-provider patch 介面 (6) 將封包送往 OVS tunnel bridge r patch 介面 (7)
OVS tunnel bridge (8) 將封包包入 VNI 101
Physical network interface (9) sends packets through overlay network (10) to network node

Packet flow after reaching the network node:

Overlay network interface (11) sends packets to OVS tunnel bridge (12)
OVS tunnel bridge decapsulates VXLAN packets and adds internal tunnel ID
OVS tunnel bridge converts internal tunnel ID into internal VLAN tag
OVS tunnel bridge patch port (13) sends packets to OVS integration bridge patch port (14)
Self-service network OVS integration bridge port (15) strips internal VLAN tag and forwards packet to self-service network interface (16) in router namespace
- In IPv4 mode, the router performs SNAT, changing the source IP to the router's IP, then forwards the packet through the gateway on the provider network (17) to the gateway
- In IPv6 mode, the router forwards packets to the next-hop IP address, typically the provider network gateway, which also routes through the gateway on the provider network (17)
The router forwards packets to the corresponding OVS integration bridge interface (18) on the provider network
OVS integration adds internal VLAN tag to the packet
The int-br-provider patch port (19) on the OVS integration bridge will forward the packet to the OVS provider bridge phy-br-provider patch port (20).
The OVS Provider bridge will map the internal VLAN tag to the VLAN tag 101 of the actual provider network.
The provider physical network interface (14) will add VLAN tag 101 to the packets and forward them to the provider network device (15)
The provider network interface (21) on the OVS Provider bridge will forward the packet to the physical network interface (22).
The physical network interface will forward the packet to the switch (23) in the physical network device.

The following portion works as described in standard networking—packets are forwarded through the actual network to the router and then sent out. The return path is simply reversed.

North-south traffic flow 2: Instance with Floating IP

For instances with fixed IPv4 addresses, the network node performs SNAT to route traffic from self-service networks to external networks like the Internet, and also performs DNAT to route incoming packets from external networks back to the instance. Floating IPs are not applicable to IPv6.

The packet flow from instance to external network is similar to Example 1. No further explanation is needed, except that the SNAT IP is replaced with a floating IP instead of the router’s IP. This will be discussed in detail for incoming traffic from external networks.

Instance on compute node 1 and using self-service network 1
The external network will forward packets to the instance

Packet flow from external network after reaching the network node:

The provider network device (1) will forward packets to the provider physical network interface (2)
The provider physical network interface (2) strips VLAN tag 101 and forwards the packet to the provider network port (3) on the OVS provider bridge.
OVS provider bridge will map VLAN tag 101 to internal VLAN tag
The phy-br-provider port (4) on the OVS provider bridge will forward packets to the int-br-provider port (5) on the OVS integration bridge.
The provider network port (6) on the OVS integration bridge will strip the internal VLAN tag and forward packets to the provider network interface (7) in the router namespace.
- In IPv4 mode, the router will use DNAT to change the destination IP to the instance's IP, then forward packets through the self-service interface (8) to the self-service network.
- In IPv6 mode, the router will forward packets to the next-hop IP address, typically the gateway of the self-service network, and also route them through the self-service interface (8).
路由器將封包送往 OVS integration bridge 對應 self-service network 的介面 (9)
OVS integration adds internal VLAN tag to the packet
OVS tunnel bridge converts internal tunnel ID into internal VLAN tag
OVS integration bridge 上的 patch-tun patch port (10)) 會將封包送往 OVS tunnel bridge patch-int patch port (11)
OVS tunnel bridge (12) 將封包包進 VNI 101
實體網路介面 (13) 將 VXLAN 封包透過 overlay 網路 (14) 送往 compute node

Packet flow after reaching the compute node:

Overlay 網路對應的實體網路介面 (15) 將封包送往 OVS tunnel bridge (16)
OVS tunnel bridge decapsulates VXLAN packets and adds internal tunnel ID
OVS tunnel bridge converts internal tunnel ID into internal VLAN tag
OVS tunnel bridge patch-int port (17) forwards packets to OVS integration bridge patch-tun patch port (18).
OVS integration bridge strips internal VLAN tag
OVS integration bridge security group interface (19) forwards packets to OVS interface on Linux bridge (20)
On Linux bridge, iptables (21) handles firewall and connection tracking
Linux bridge instance port (22) uses veth pair to forward packets to instance network interface (23)

The above is the packet flow when using Floating IP.

East-West Traffic 1: Instances on the same network

Instance 1 on compute node 1 also uses self-service network 1
Instance 2 on compute node 2 also uses self-service network 1
Instance 1 sends the packet to Instance 2

On compute node 1:

Instance network interface (1) through veth pair (2) sends packets to the instance port on the Linux bridge
在 linux bridge 上 iptables (3) 會處理防火牆跟 connection tracking
Linux bridge 上的 ovs 介 (4) 面透過 veth pair 將封包送往 OVS integration bridge security group 介面 (5)
OVS integration bridge 將封包加上 internal VLAN tag
OVS integration bridge int-br-provider patch 介面 (6) 將封包送往 OVS tunnel bridge r patch 介面 (7)
OVS tunnel bridge (8) 將封包包入 VNI 101
實體網路介面 (9) 透過 overlay 網路 (10) 將封包送往 compute node 2

After reaching compute node 2:

Overlay network interface (11) sends packets to OVS tunnel bridge (12)
OVS tunnel bridge decapsulates VXLAN packets and adds internal tunnel ID
OVS tunnel bridge converts internal tunnel ID into internal VLAN tag
OVS tunnel bridge patch-int port (13) 將封包送往 OVS integration bridge patch-tun patch port (14).
OVS integration bridge strips internal VLAN tag
OVS integration bridge security group 介面 (15) 封包送往 Linux bridge 上的 ovs 介面 (16)
在 linux bridge 上 iptables (17) 會處理防火牆跟 connection tracking
linux bridge 的 instance port (18) 透過 veth pair 將封包送往 Instance 的網路介面 (19)

East-West Traffic 2: Instances on different networks

Instance 1 on compute node 1 and uses self-service network 1
Instance 2 on compute node 1 and uses self-service network 2
Instance 1 sends the packet to Instance 2

On compute node:

Instance network interface (1) through veth pair (2) sends packets to the instance port on the Linux bridge
在 linux bridge 上 iptables (3) 會處理防火牆跟 connection tracking
Linux bridge 上的 ovs 介 (4) 面透過 veth pair 將封包送往 OVS integration bridge security group 介面 (5)
OVS integration bridge 將封包加上 internal VLAN tag
OVS integration bridge int-br-provider patch 介面 (6) 將封包送往 OVS tunnel bridge r patch 介面 (7)
OVS tunnel bridge (8) 將封包包入 VNI 101
Physical network interface (9) sends packets through overlay network (10) to network node

Packet flow after reaching the network node:

Overlay network interface (11) sends packets to OVS tunnel bridge (12)
OVS tunnel bridge decapsulates VXLAN packets and adds internal tunnel ID
OVS tunnel bridge converts internal tunnel ID into internal VLAN tag
OVS tunnel bridge patch port (13) forwards packets to OVS integration bridge patch-tun patch port (14)
OVS integration bridge port (15) for self-service network 1 strips internal VLAN tag and forwards packets to self-service network 1 interface (16) in router namespace
Router forwards packets to next-hop IP address, typically via self-service network 2 interface (17) to self-service network 2 gateway
Router forwards packets to corresponding OVS integration bridge interface (18) for self-service network 2
OVS integration adds internal VLAN tag to the packet
OVS integration bridge converts internal VLAN tag to internal tunnel ID
The patch-tun patch port (19) on the OVS integration bridge will forward the packet to the patch-int patch port (20) on the OVS tunnel bridge.
The OVS tunnel bridge (21) will encapsulate the packet with VNI 102
The physical network interface (22) will forward the VXLAN packet through the overlay network (23) to the compute node

After the packet returns to the compute node:

The overlay network interface (24) will forward the packet to the OVS tunnel bridge (25)
OVS tunnel bridge decapsulates VXLAN packets and adds internal tunnel ID
OVS tunnel bridge converts internal tunnel ID into internal VLAN tag
The patch-int patch port (26) on the OVS tunnel bridge will forward the packet to the patch-tun patch port (27) on the OVS integration bridge.
OVS integration bridge strips internal VLAN tag
OVS integration bridge security group overview (28) forwarding packets to OVS interface (29) on Linux bridge
iptables (30) on Linux bridge will handle firewall and connection tracking
Linux bridge instance port (31) using veth pair to forward packets to instance network interface (32)

The above describes the packet flows under different conditions for Open vSwitch self-service networks.

Summary

This article explained the architecture and packet flow of Open vSwitch self-service networks. After understanding how packets travel through the network, you’ll have a much clearer concept when debugging network connectivity issues. The next article will still focus on Neutron, but this time we’ll introduce a relatively new plugin—the OVN plugin—and its architecture.

From the Edge to the Core: OpenStack Neutron Introduction - OVS Self-service Networks