New projects and toys

Learning some fun new stuff this week.
* Linux KVM - onboard virtualization, snazzy and fast once you work through the kinks. For instance, bridging to the VLAN is not the default networking setup, so you have to craft your own network interface (br0) and then hack the XML file defining the VM to attach to it.
* Keepalived - a replacement for LVS and Heartbeat to provide redundant load-balancers.

Plus a nice Christmas gifts: 20" LCD monitor and the ever gratifying Costco-sized Peppered beef jerky .

pfSense and CARP on vmware-server

Before I forget, I wanted to document the necessary details for getting CARP to work on pfSense running under vmware-server 1.0.7. IT IS BROKEN by default, because the vmnet driver does not recognize the (emulated) MAC address used by CARP (and VRRP): 00-00-5E-00-01-XX

The symptom is that the carp0 interface appears but cannot be communicate, and failover does not happen. Reason being, the vmnet driver is silently dropping the packets on the floor!

To get it working requires a patch (on the host) to vmnet-only/driver.c and recompile.

Here is the patch for vmware-server-1.0.7 for Linux. Save it into /tmp/driver.c.patch

--- vmnet-only/driver.c.orig 2008-10-08 15:37:23.000000000 -0500
+++ vmnet-only/driver.c 2008-10-08 15:44:50.000000000 -0500
@@ -1284,6 +1284,9 @@
 
    return ((flags & IFF_PROMISC) || MAC_EQ(destAddr, ifAddr) || 
     ((flags & IFF_BROADCAST) && MAC_EQ(destAddr, broadcast)) || 
+  ((destAddr[0] == 0) && (destAddr[1] == 0) &&
+   (destAddr[2] == 0x5e) && (destAddr[3] == 0) &&
+   (destAddr[4] == 1)) ||
     ((destAddr[0] & 0x1) && (flags & IFF_ALLMULTI || 
       (flags & IFF_MULTICAST && 
        VNetMulticastFilter(destAddr, ladrf)))));

The file is a little hard to find, being hidden in a vmnet.tar below /usr/lib/vmware/modules/source/ ...

cd /usr/lib/vmware/modules/source
tar xvf vmnet.tar
patch < /tmp/driver.c.patch
tar cvf vmnet.tar vmnet-only
vmware-config.pl

Then, choose the option to recompile the kernel drivers, specifically vmnet.

The one other addl. need (I'm not 100% sure on this) is that it may be necessary to allow promiscous on the ethernet device, in the .vmx file...

ethernet0.nopromisc = "false"

References...

• VRRP & CARP with VMware Server
  
• CARP problem with VMWare Server
  

VMDK conversions

Moving virtual disks back-and-forth between Vmware ESX and Vmware Server is getting to be an unintentional hobby of mine. It is actually pretty easy, though tedious, once you get the hang of it.

The secret sauce lies both in knowing what values are understood by each product inside the VMDK file and making sure you pass the right options to vcbExport.

Going from ESX to Server, you want to export your disk like so:

 export VMNAME=YOURVMNAME (for bash)
 setenv VMNAME YOURVMNAME (for csh, tcsh)
 /usr/sbin/vcbExport -M 0 -F 0 \
 -d /path/to/exported/$VMNAME/$VMNAME.vmdk \
 -s /vmfs/volumes/datastore1/$VMNAME/$VMNAME.vmdk

Going back to ESX from Server can be accomplished with:

 /usr/sbin/vcbExport -M 1 -F 1 \
 -d /vmfs/volumes/datastore1/$VMNAME/$VMNAME.vmdk
 -s /path/to/exported/$VMNAME/$VMNAME.vmdk

Afterwards, you need to "tweek" the resulting VMDK file...
Change
createType="monolithicFlat"
to
createType="vmfs"

and change
# Extent description
RW 80668672 FLAT "YOURVMNAME-flat.vmdk" 0
to
RW 80668672 VMFS "YOURVMNAME-flat.vmdk" 0

FreeBSD re-supported in VMware ESX server

Just discovered that the latest 2.5.x release of Vmware's ESX server supports FreeBSD after dropping support in some of the earlier releases.

This is great!

However it is also a bittersweet circumstance, in that the version they choose to support is on a very short leash. Support for 4.11-RELEASE by the FreeBSD team is due to expire in just a few months (Jan '07). Hopefully VMware will recognize the situation for what it is and get the 6.x series into a supported state. 6.x is the horse they should be riding at this point.

References:
http://www.vmware.com/support/esx25/doc/releasenotes_esx254.html
http://lists.freebsd.org/pipermail/freebsd-security/2006-October/004045.html

Right Tool For the Job - Nessus

Good article about vulnerability scanning with nessus. Interesting that they chose to implement the scanner from a virtual machine, my recent installation of nessus 3.x on a VM (running under VMware's ESX server) produced a warning that running under a VM was not a great idea for performance reasons.

VMware benchmarks pulled

Due to a recent discovery that VMware censors their customers' freedom of speech, I have pulled my benchmarks showing Xen vs. ESX comparitive analysis. I do not like to be sued.

If you must know, Xen beat ESX hands down in performance. I have the data to prove it.

See Xen's official performance analysis for a legally-safe alternative. They were able to publish only because they used an older version of VMware (Workstation 3.2). The quote that tipped me off was "the latest version of any VMware product which allows the publication of comparative benchmarks".

The VMware ESX license agreement can be found here. The license states "you may not disclose the results of any benchmark test of the Software to any third party without VMware's prior written approval".

Sounds like they don't want the public to know their product might be inferior - at least in terms of performance.