Posted: Sun, 13 May 2012
When I wrote [lvmsync](http://theshed.hezmatt.org/lvmsync) [late last
didn't realise I was being typecast. Before too long, I realised that the
logic that I'd implemented for lvmsync would also help me with a separate
migration project I'd been dreading -- getting [the day
job](http://www.anchor.com.au/) off VMware.
Back in the early days of virtualisation, management made the decision to
run VMware, for all the usual reasons ("commercially supported!", "industry
standard!", and so on). Unsurprisingly (to me, anyway) it didn't take too
long for management to realise that it wasn't the best choice for us. When
you've got umpty-billion dollars to spend on hardware, software, and
support, VMware might be the right option (although Amazon doesn't seem to
think so). Anchor's company culture, on the other hand, is build around
"smart staff, simple systems" over "dumb staff, smart vendors", because [no
vendor is ever going to care about our customers as much as we
So VMware was never going to work for us.
Unfortunately, as happens all too often, once VMware was in place, there was
very little motivation to get rid of it and move those customers onto the
chosen replacement (that we were deploying all new customers on). I happen
to think this is a terrible attitude in general -- one that makes life so
much harder in the long term. I believe strongly in retrofitting old
systems to keep them up-to-date with the current state of the art, and
keeping technical debt under control. But, I wasn't running the show back
when we stopped putting new customers on VMware, so the few VMware servers
we had stayed around far longer than they should have.
Recently, though, bad things started to happen. The VMware servers were
starting to fall apart. The Windows machine we had to keep around to use
the VMware management console started crapping out, and when the choice was
between doing unspeakable things to Windows, and just ditching VMware...
well, it wasn't much of a choice. The only remaining question was how to do
the migration off VMware with the least amount of downtime to our customers.
I was really quite surprised that nobody out in Internet land appeared to
have come up with a simple, robust tool to do this. Sure, some vendors had
all-singing, all-dancing toolkits that cost ridiculous amounts of money,
required you to install their agent on the machine involved, and promised
the earth, but it all smelt of snakeoil and bullshit.
In true hacker style, then, I decided to write something myself. The model
I came up with mirrored lvmsync's quite closely -- because that one worked,
and it turned out to be surprisingly easy to implement once I managed to
reverse-engineer the file format (VMware has a PDF spec of a bunch of it's
file formats, but whoever wrote it was enough of an evil genius to make it
utterly incomprehensible to anyone who doesn't already know the file format,
whilst making perfect sense to anyone who already does).
The result: [vmdksync](http://theshed.hezmatt.org/vmdksync/). It is nothing
but 80-odd lines of ruby whose sole purpose is to take a `delta.vmdk` file
and write the changes that are stored in that file to a file or block device
that is a copy of the `flat.vmdk` file that you can copy while the VM is
still running (after you've made a snapshot, of course). It helped me
provide a painless migration path away from VMware, and I'd be really
pleased if it helped some other people do the same. Share and enjoy!