Networking
A standard Xen installation offers two distinct methods of connecting virtualised guests to the physical network - bridging and routing. Both methods create virtual interfaces for the guest domain on the host machine. The only difference between the two is how those virtual interfaces are connected to the host network stack.
When using app-emulation/xen-tools package from the Hacking Networked Solutions overlay for Gentoo Linux a third method of connecting the virtualised guests to the physical network is introduced - link routing. This is similar to standard routing however instead of a point-to-point link being created between the host and the guest a complete subnet is used for each virtual machine.
As you would expect with three similar but subtly different networking configurations each has some advantages and disadvantages over the others. To enable you to make a more informed decision as to which networking method to use each is described in more detail below.
More information and a detailed description of each option used in the example configurations below may be found in the XL Network Configuration documentation◳ and the official Xen Project website◳.
Bridged using vif-bridge
The most commonly deployed method of connecting virtualised guest domains to the host network is to use a bridge. As this is a fairly straightforward approach it is often favoured by tutorials and how-to documents, such as Configuring Gentoo with Xen◳.
One of the key reasons that this method is preferred in such documents is that it requires little, if any, knowledge of IP network routing. The main problem with this method is that it requires the network interface of the host machine to be placed into promiscuous mode so that all packets transmitted on the local subnet will be received by the host machine. If the domain operator is un-trusted it also requires that iptables (or similar) be used to restrict the network addresses which may be used by the guest to ensure that it does not use other addresses without permission.
# General name = "example"; memory = 512; vcpus = 1; # Booting kernel = "/usr/xen/kernels/linux-3.10.5-gentoo-r1"; root = "/dev/xvda1 rw"; extra = "console=hvc0 vdso32=0 raid=noautodetect rootflags=barrier=0 net.ifnames=0"; # Virtual harddisk disk = [ "phy:/dev/volumes/example-vm,xvda1,w" ];
# Virtual Network vif = [ 'script=vif-bridge, ip=10.0.0.49, bridge=br1, vifname=vif.example' ];
config_eth0="10.0.0.49/24"
routes_eth0="default via 10.0.0.1
Routed using vif-route
As mentioned previously a standard Xen installation offers a second method of connecting virtualised guests to the host network. This method uses routing to connect a guest (on a different subnet to the host) to the host network.
This method has the notable advantage that the network interface in the host machine does not need to be placed into promiscuous mode which results in much more efficient operation. Unfortunately, as you can see from the example below, this method requires the guest to have knowledge of the host's network configuration as it needs to use the host's IP address as its default gateway. This problem alone renders this method useless if we wish to be able to utilise the live migration features offered by Xen.
# General name = "example"; memory = 512; vcpus = 1; # Booting kernel = "/usr/xen/kernels/linux-3.10.5-gentoo-r1"; root = "/dev/xvda1 rw"; extra = "console=hvc0 vdso32=0 raid=noautodetect rootflags=barrier=0 net.ifnames=0"; # Virtual harddisk disk = [ "phy:/dev/volumes/example-vm,xvda1,w" ];
# Virtual Network vif = [ 'script=vif-route, ip=10.0.1.49, vifname=vif.example' ];
config_eth0="10.0.1.49/32"
routes_eth0="10.0.0.20/32 default via 10.0.0.20"
Another serious problem with this method, as shown in the example below, is that multiple virtual network interfaces with the same IP address will be created on the host. This seriously confuses some software (such as SMTPD) which mistakenly believes that there are multiple instances of a single network address on the same network!
vif.examp Link encap:Ethernet HWaddr fe:ff:ff:ff:ff:ff
inet addr:10.0.0.20 Bcast:10.255.255.255 Mask:255.255.255.255
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:166892 errors:0 dropped:0 overruns:0 frame:0
TX packets:159748 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:32
RX bytes:27873012 (26.5 MiB) TX bytes:52071546 (49.6 MiB) Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 0.0.0.0 10.0.0.1 0.0.0.0 UG 3 0 0 eth0 10.0.0.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0 10.0.1.49 0.0.0.0 255.255.255.255 UH 0 0 0 vif.example
Routed using vif-link-route
The final method of connecting a virtualised guest to the host network which we shall describe here is to use the vif-link-route script. As we mentioned above this script is not provided in a standard Xen installation and, unless you are using the app-emulation/xen-tools package from the Hacking Networked Solutions overlay for Gentoo Linux, you will need to install the app-emulation/xen-vif-hacking package (also from the overlay).
This networking method is very similar to the vif-route method above. Instead of creating a host route to the network address of the guest via a virtual interface with the same address as the host however a normal route is used and the virtual interface is assigned the gateway address of a guest specific subnet.
As you can see from the examples below this method has the advantages that the guest does not need any knowledge of the host's network configuration (because it uses an address on its own subnet as a gateway address) and that the host has virtual interfaces with unique addresses. It also works correctly with the live migration features offered by Xen.
# General name = "example"; memory = 512; vcpus = 1; # Booting kernel = "/usr/xen/kernels/linux-3.10.5-gentoo-r1"; root = "/dev/xvda1 rw"; extra = "console=hvc0 vdso32=0 raid=noautodetect rootflags=barrier=0 net.ifnames=0"; # Virtual harddisk disk = [ "phy:/dev/volumes/example-vm,xvda1,w" ];
# Virtual Network vif = [ 'script=vif-link-route, ip=10.0.1.50:255.255.255.252, vifname=vif.example' ];
config_eth0="10.0.1.49/30"
routes_eth0="default via 10.0.1.50
vif.examp Link encap:Ethernet HWaddr fe:ff:ff:ff:ff:ff
inet addr:10.0.1.50 Bcast:10.0.1.51 Mask:255.255.255.252
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:166892 errors:0 dropped:0 overruns:0 frame:0
TX packets:159748 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:32
RX bytes:27873012 (26.5 MiB) TX bytes:52071546 (49.6 MiB) Unfortunately, as with everything in life, a tradeoff must be made for this to work. As you can see from the example below the tradeoff in this case is that four network addresses (as indicated by the /30 suffix or subnet mask of 255.255.255.252) are now required by every guest instead of a single address (as indicated by the /32 suffix or subnet mask of 255.255.255.255) with the previous two methods.
Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 0.0.0.0 10.0.0.1 0.0.0.0 UG 3 0 0 eth0 10.0.0.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0 10.0.1.48 0.0.0.0 255.255.255.252 U 0 0 0 vif.example