Avoid IP conflicts in Xen DomU

Recently I’m playing with Xen virtualization, and I came across one protential problem. As I need to share my guest machines to clients,  I must give them root privileges… that’s whats VPS-es all about… having root access to OS without having to purchase expensive physical ones.  So having that in mind they are by default untrusted and unpredictable. Probably only god knows what they will do on their VMs!

So what’s the exact problem here?

By default xen, and all those tools, like Cpanel / WHM and  Webmin, don’t really have a way of sorting out ip conflicts. So basically you have some scripts that will setup clients ip address during his machine startup, you have vif ip statment in the config file. But what’s really holds clients from entering:

# ifconfig eth0 x.x.x.x

Where x.x.x.x is the IP of some super important server in same netmask. Luckily for me I came across this problem while still in testing. Xen supports ip declaration in vif statment of dom config file:

vif = ['ip=x.x.x.x, more parametars here....']

Also you can declare multiple ip’s by simply putting space between them, like this:

vif = ['ip=xx.xx.xx.x1 xx.xx.xx.x2, more parametars here....']

For the purpose of ip conflict prevention make sure you declare unique MAC address in vif section too.

Next step is to install ebtables on your Dom0 box. After that, all we need to do is to apply the following patch for the script vif-bridge located in /etc/xen/scripts/:

[root@dom0 scripts]# diff -u vif-bridge-org vif-bridge
--- vif-bridge-org      2009-11-12 16:31:56.000000000 +0800
+++ vif-bridge  2009-11-12 16:40:38.000000000 +0800
@@ -57,15 +57,41 @@
     online)
        setup_bridge_port "$vif"
        add_to_bridge "$bridge" "$vif"
+
+       ebtables -N $vif
+       ebtables -P $vif DROP
+       ebtables -A INPUT -i $vif -j $vif
+       ebtables -A FORWARD -i $vif -j $vif
+       ebtables -A $vif -p ARP --arp-opcode 1 -j ACCEPT
+
+       if [ ! -z "$ip" ]
+       then
+       for oneip in $ip
+       do
+               ebtables -A $vif -p IPv4 --ip-src $oneip -j ACCEPT
+               ebtables -A $vif -p IPv4 --ip-dst $oneip -j ACCEPT
+               ebtables -A $vif -p ARP --arp-opcode 2 --arp-ip-src $oneip -j ACCEPT
+       done
+
+       ebtables -A $vif --log-prefix="arp-drop" --log-arp -j DROP
+
+       fi
+
         ;;

     offline)
         do_without_error brctl delif "$bridge" "$vif"
         do_without_error ifconfig "$vif" down
+
+       do_without_error ebtables -D INPUT -i $vif -j $vif
+       do_without_error ebtables -D FORWARD -i $vif -j $vif
+       do_without_error ebtables -F $vif
+       do_without_error ebtables -X $vif
+
         ;;
 esac

-handle_iptable
+#handle_iptable

 log debug "Successful vif-bridge $command for $vif, bridge $bridge."
 if [ "$command" == "online" ]

It’s tested OK in my latest CentOS-5.4 box.

Presuming you use bridging scripts this effectively binds ip address-es from “vif = [‘ip=x.x.x.x’]” list to mac addresses from vif list. Binding is done while enabling vps machine and undone when powering it off.

So this way untrusted user is limited only to the ip addresses defined in xen guest conf file, trying to change existing ip address into another one on same network will only cause that machine unresponsive.

Warning: This post is original created by Branko at his blog site.  I copied his content, and updated some settings so that it can work in CentOS-5.4.

Share Button

3 thoughts on “Avoid IP conflicts in Xen DomU

  1. Thank you very much for this, I have been struggling to find a solution to this exact problem.

  2. I am running Debain dom0 using kernel 2.6.26-2-xen-amd64.

    When I started one domU attached to xenbr0, it would be able to ping the gateway a dom0, along with other LAN pc’s. However, when I started subsequent domU’s on the host, they would not be able to arp or ping any LAN IP, including dom0.

    I had a heck of a time setting up dummy interfaces, and then using a “multi.network-bridge” config as follows. I finally added IP’s to the vif interfaces and xenbrX interfaces.:

    #!/bin/sh
    dir=$(dirname “$0”)
    “$dir/network-bridge” “$@” vifnum=0 netdev=eth0 bridge=xenbr0
    “$dir/network-bridge” “$@” vifnum=1 netdev=dummy1 bridge=xenbr1
    “$dir/network-bridge” “$@” vifnum=2 netdev=dummy2 bridge=xenbr2

    ifconfig xenbr2 up
    ifconfig xenbr1 up
    ifconfig xenbr0 up
    ifconfig xenbr0 192.168.15.253 netmask 255.255.255.0
    ifconfig xenbr1 10.10.10.1 netmask 255.255.255.0
    ifconfig xenbr2 10.20.20.1 netmask 255.255.255.0
    ifconfig vif3.0 192.168.15.30 netmask 255.255.255.0
    ifconfig vif3.1 10.20.20.30 netmask 255.255.255.0
    ifconfig vif2.2 10.20.20.20 netmask 255.255.255.0
    ifconfig vif2.1 10.10.10.20 netmask 255.255.255.0
    ifconfig vif2.0 192.168.15.20 netmask 255.255.255.0
    ifconfig vif1.0 192.168.15.25 netmask 255.255.255.0
    ifconfig vif1.1 10.10.10.25 netmask 255.255.255.0
    ip route flush table main
    ip route add 192.168.15.0/24 dev xenbr0
    ip route add 10.10.10.0/24 dev xenbr1
    ip route add 10.20.20.0/24 dev xenbr2
    ip route add default via 192.168.15.1

    Even after modifying the bridge script, restarting xend and networking; and finally rebooting several times, I still had the same problem.

    I installed ebtables and modified the vif-bridge script as indicated with this post, and now all the IPs are pingable. Thnx

Leave a comment

Your email address will not be published. Required fields are marked *