vSphere
Shrinking VMDK virtual hard disks in vSphere
27/01/11 15:12
There is a bug in VMware vSphere 4.1.
You should be able to reclaim the space in a virtual hard disk (VMDK) by zeroing out all the free virtual disk space with “sdelete” or “dd”, and then migrate it (Storage vMotion) to a different datastore as a Thin provisioned disk. VMware will look for all the blocks full of zeroes and remove them from the VMDK image, often drastically reducing the amount of physical disk space it occupies.
The bug is that if the 2 datastores (source and destination of the migration) have the same block size, VMware does not remove the blocks of zeroes.
To make it work, just migrate it to a datastore with a different block size, and migrate it back where you want to keep it.
To find the block size of a datastore, in vSphere Client go to Home —> Inventory —> Datastores. Select the datastore, and switch to the “Configuration” tab on the right-hand side of the window.
I appreciate that most people format all their datastores with 8MB block size, as that allows the greatest maximum VMDK file size (2Tbytes). This is a very good reason to keep at least 1 datastore with a different block size!
So the full procedure for Windows is
For Linux, it’s like this
You should be able to reclaim the space in a virtual hard disk (VMDK) by zeroing out all the free virtual disk space with “sdelete” or “dd”, and then migrate it (Storage vMotion) to a different datastore as a Thin provisioned disk. VMware will look for all the blocks full of zeroes and remove them from the VMDK image, often drastically reducing the amount of physical disk space it occupies.
The bug is that if the 2 datastores (source and destination of the migration) have the same block size, VMware does not remove the blocks of zeroes.
To make it work, just migrate it to a datastore with a different block size, and migrate it back where you want to keep it.
To find the block size of a datastore, in vSphere Client go to Home —> Inventory —> Datastores. Select the datastore, and switch to the “Configuration” tab on the right-hand side of the window.
I appreciate that most people format all their datastores with 8MB block size, as that allows the greatest maximum VMDK file size (2Tbytes). This is a very good reason to keep at least 1 datastore with a different block size!
So the full procedure for Windows is
- Download “sdelete” from Microsoft.
- Run the command “sdelete -c C:” from a Command Prompt, while logged in as an Administrator.
- Migrate the VM from its current datastore to one with a different block size. Be sure to select “Thin provisioned format”.
- Migrate it back again.
- Bingo! You have all your unused disk space back.
For Linux, it’s like this
- Run the command “dd if=/dev/zero of=BIGFILE bs=1024000 ; rm -f BIGFILE”. If you have several large filesystems in the VM, then do this while your current directory is inside each one in turn. You only need to be root if you have user quotas enforced (most people don’t). But it does no harm to run the command as root if you want to.
- Migrate the VM from its current datastore to one with a different block size. Be sure to select “Thin provisioned format”.
- Migrate it back again.
- Bingo! You have all your unused disk space back.
Comments
VMware Data Recovery log analyser
08/01/11 14:42
Update: Improved error reporting, so script generates a “Failed” email if the log copy fails.
vSphere 4.1 and ESXi 4.1 are both very bad at letting you monitor what has happened with the VMware Data Recovery backup system. You cannot currently even email the log.
So I have written a log analyser which takes the raw VDR log file (similar to the log file viewable in the vSphere plugin) and works out what has happened since it was last run.
The log you see in the plugin is actually a heavily processed version of the raw log file, which is not remotely readable.
You can download my script for free. It’s a short Perl script that uses “iconv” to do the UTF-16 to UTF-8 conversion needed, and the “sendmail” binary (which is provided by all major email systems) to send the email report.
There is a comment at the top that you should read, and there is a little block of definitions which you should adapt for your setup. But otherwise it should work on any Linux distribution without further modification.
You may use my script for free, but please do not charge anyone any fee, monetary or otherwise, for the ability to use it in any form or derivative.
As an example of the output, here’s a little chunk of the email it sends:
21/12/10 08:22 VM Win2008R2x64 Server Source Ok
21/12/10 08:23 FAIL kanga-az1g10 Execution incomplete Trouble reading files, error -3948 ( vcb api exception)
21/12/10 08:24 VM kanga-hl1g09 Ok
21/12/10 08:31 VM kanga-cl14e10 Ok
21/12/10 08:31 VM kanga-cm7e09 Ok
21/12/10 08:34 VM kanga-bc2e10 Ok
21/12/10 08:56 Operation Verify (Execution unit 1) Ok
21/12/10 08:59 VM kanga-amg Ok
vSphere 4.1 and ESXi 4.1 are both very bad at letting you monitor what has happened with the VMware Data Recovery backup system. You cannot currently even email the log.
So I have written a log analyser which takes the raw VDR log file (similar to the log file viewable in the vSphere plugin) and works out what has happened since it was last run.
The log you see in the plugin is actually a heavily processed version of the raw log file, which is not remotely readable.
You can download my script for free. It’s a short Perl script that uses “iconv” to do the UTF-16 to UTF-8 conversion needed, and the “sendmail” binary (which is provided by all major email systems) to send the email report.
There is a comment at the top that you should read, and there is a little block of definitions which you should adapt for your setup. But otherwise it should work on any Linux distribution without further modification.
You may use my script for free, but please do not charge anyone any fee, monetary or otherwise, for the ability to use it in any form or derivative.
As an example of the output, here’s a little chunk of the email it sends:
21/12/10 08:22 VM Win2008R2x64 Server Source Ok
21/12/10 08:23 FAIL kanga-az1g10 Execution incomplete Trouble reading files, error -3948 ( vcb api exception)
21/12/10 08:24 VM kanga-hl1g09 Ok
21/12/10 08:31 VM kanga-cl14e10 Ok
21/12/10 08:31 VM kanga-cm7e09 Ok
21/12/10 08:34 VM kanga-bc2e10 Ok
21/12/10 08:56 Operation Verify (Execution unit 1) Ok
21/12/10 08:59 VM kanga-amg Ok
Shrinking VMDK Virtual Hard Disks
26/12/10 19:39
In the process of building a template or using a VM, it is likely you will at some point use a lot more space than you finish with, and you will have also scattered files across the disk. All of this results in the Thin-Provisioned disk growing as free space is written to (and then possibly deleted again).
So when you have cleared off all the junk, you need to reclaim all that free disk space, and take advantage of Thin Provisioning to not have to store loads of copies of the free space as you deploy VMs from the template.
Reclaiming unused free space from thin disks used to be easy.
You just used "sdelete -c" or "wipe" (both attached to this article) to zero all the free space on the disk, then migrated it to a different datastore in thin format, then migrated it back again.
Then VMware released 4.1 and broke it. It no longer skips blocks of zeroes when copying vmdk virtual disks to a thin vmdk. They have acknowledged it is a bug and will fix it in a future patch or release.
For the time being, there is 1 way around it, but it's not simple. This only works with Windows. To shrink a Linux VM, you need to use the converter with the VM running and the source type set to "Powered on machine". See my guide to converting a Linux system to a VM, that works perfectly well with a virtual Linux system instead of a physical Linux system. But do remember, like in the instructions below, that you must change the size of all of the filesystems. You may also need to remove+replace the network adapter, and then re-install the VMware Tools.
Then shutdown the VM.
Remote Desktop to the vSphere administration VM.
Run the VMware Standalone Converter, and login to localhost as administrator. (Or run the Converter locally and connect to the vSphere administration VM)
Convert a machine.
Source is a VMware Infrastructure virtual machine, and find the source VM.
Destination is a VMware Infrastructure virtual machine, also on localhost logged in as administrator. Give it a name you will be able to find later!
Put it on a different host and a different datastore. You can migrate it back where you want it at the very end.
Edit the disk layout, choose "Select volumes to copy", advanced view.
Change the size of the main partition slightly. It doesn't matter if you expand or shrink it, you just have to change it.
Note: If you cannot change the numbers, then cancel the wizard, run vSphere Client, Edit settings to slightly enlarge the size of the disk by a couple of GB. Then switch on the VM, login to it as an administrator, use Start / Computer / Manage / Disk Management to expand the volume to fill the empty space. Then shutdown the VM again and restart the "Convert a machine" wizard. Now you should be able to resize the disk volume.
Set the NIC connection correctly, and just check the other settings, they should all be correct.
Finish the wizard and it will start the conversion.
Migrate the output VM back to the host and datastore you want, and it should now be much smaller than it was.
Next, Edit Settings on the VM and remove and re-add the network adapter as a "VMXNET3" network adapter instead of the "Flexible" adapter you will currently have, with the correct VLAN assigned to it.
Power on the new VM, and do an interactive VMware Tools installation and "Repair" the installation.
Then configure the network settings on the new network adapter, and assign it an IP so you get an internet connection.
Now shutdown the VM again.
So when you have cleared off all the junk, you need to reclaim all that free disk space, and take advantage of Thin Provisioning to not have to store loads of copies of the free space as you deploy VMs from the template.
Reclaiming unused free space from thin disks used to be easy.
You just used "sdelete -c" or "wipe" (both attached to this article) to zero all the free space on the disk, then migrated it to a different datastore in thin format, then migrated it back again.
Then VMware released 4.1 and broke it. It no longer skips blocks of zeroes when copying vmdk virtual disks to a thin vmdk. They have acknowledged it is a bug and will fix it in a future patch or release.
For the time being, there is 1 way around it, but it's not simple. This only works with Windows. To shrink a Linux VM, you need to use the converter with the VM running and the source type set to "Powered on machine". See my guide to converting a Linux system to a VM, that works perfectly well with a virtual Linux system instead of a physical Linux system. But do remember, like in the instructions below, that you must change the size of all of the filesystems. You may also need to remove+replace the network adapter, and then re-install the VMware Tools.
Then shutdown the VM.
Remote Desktop to the vSphere administration VM.
Run the VMware Standalone Converter, and login to localhost as administrator. (Or run the Converter locally and connect to the vSphere administration VM)
Convert a machine.
Source is a VMware Infrastructure virtual machine, and find the source VM.
Destination is a VMware Infrastructure virtual machine, also on localhost logged in as administrator. Give it a name you will be able to find later!
Put it on a different host and a different datastore. You can migrate it back where you want it at the very end.
Edit the disk layout, choose "Select volumes to copy", advanced view.
Change the size of the main partition slightly. It doesn't matter if you expand or shrink it, you just have to change it.
Note: If you cannot change the numbers, then cancel the wizard, run vSphere Client, Edit settings to slightly enlarge the size of the disk by a couple of GB. Then switch on the VM, login to it as an administrator, use Start / Computer / Manage / Disk Management to expand the volume to fill the empty space. Then shutdown the VM again and restart the "Convert a machine" wizard. Now you should be able to resize the disk volume.
Set the NIC connection correctly, and just check the other settings, they should all be correct.
Finish the wizard and it will start the conversion.
Migrate the output VM back to the host and datastore you want, and it should now be much smaller than it was.
Next, Edit Settings on the VM and remove and re-add the network adapter as a "VMXNET3" network adapter instead of the "Flexible" adapter you will currently have, with the correct VLAN assigned to it.
Power on the new VM, and do an interactive VMware Tools installation and "Repair" the installation.
Then configure the network settings on the new network adapter, and assign it an IP so you get an internet connection.
Now shutdown the VM again.
VMware Delete Consolidate Helper Snapshot
26/12/10 19:38
Sometimes a VM, particularly a VDR VMware Data Recovery VM can gain a huge snapshot with a name like "consolidate helper". You cannot simply delete this snapshot, VMware will not let you.
Shutdown the guest operating system on the VM, to shut it down.
Once it has completely stopped, add a new snapshot.
Then open the Snapshot Manager (from the right-button menu in the VM index).
Then "Delete All" snapshots.
Don't worry if that times out and produces an error, this can happen but it is just the GUI timing out, it will still delete all the snapshots.
Power on the VM.
Shutdown the guest operating system on the VM, to shut it down.
Once it has completely stopped, add a new snapshot.
Then open the Snapshot Manager (from the right-button menu in the VM index).
Then "Delete All" snapshots.
Don't worry if that times out and produces an error, this can happen but it is just the GUI timing out, it will still delete all the snapshots.
Power on the VM.
VMware Data Recovery (VDR) Setup
26/12/10 19:35
This is a description of the setup and use of VMware Data Recovery (known as VDR).
You can only have 1 VDR VM per CPU host.
Each VDR VM can backup 100 VMs at most.
Each VDR VM can have 2 Destinations at most.
In strict terms, a "destination" is a "deduplication store".
Each destination can be 1 Tbyte at most.
Each destination should be a virtual hard disk stored inside the VM.
Do not use "thin" provisioning for destination vmdk disks.
Do not use independent disks for destination vmdk disks.
Set the SCSI id of each destionation vmdk to be (1,0) or (2,0) or (3,0) and so on. The second digit must always be 0.
Before you start you must install the "VMwareDataRecoveryPlugin.msi" program. Download the msi file from datastore "ugstore1-vol1"/ISOs onto your PC, quit the vSphere Client and double-click on the file to install it. Restart the vSphere Client and on the bottom row of "Home" you will now have the VMware Data Recovery solution.
You can then connect to any one of the VDR VMs.
There are currently 2 VDR VMs:
The actual backup and restore operations are pretty intuitive, the on-line help will answer most of the obvious questions.
The backup time window is shifted from the default to not backup during 9am-7pm Mon-Fri, and 11am-9pm Sat-Sun.
The Reports and Configuration tabs will show you what is going on, any warnings and errors, and how much space is available in the destinations deduplication stores.
The VDR system checks the integrity of the destinations daily automatically.
When looking at the list of destinations in the Configuration tab, the line for each destination will list a low figure for free space, but a very different figure for the "Deduplicated Size" at the bottom of the pane.
The VDR system automatically consolidates identical and unused blocks daily automatically.
Creating a New VDR VM
You need a Virtual Appliance (ovf file) from the download of "VMwareDataRecovery-....-i386.iso" which is on ecsvm-admin in Administrator's Downloads directory. Mount this iso as a new drive letter using MagicISO (installed on ecsvm-admin) or Daemon-Tools.
Using vSphere Client, File / Deploy OVF Template... and point it to the file \VMwareDataRecovery-ovf-i386\VMwareDataRecovery_OVF10.ovf.
Set the VM's name to something useful like "VMware Data Recovery UG1", and put the location somewhere sensible.
Set the host to a CPU server that doesn't already have a VMware Data Recovery VM on it, there must only be 1 VM per server.
Set the datastore to the place you want all the backups to go.
Use Thick Provisioned format.
Set the Destination Network appropriately for the IP you have.
Set the Timezone to Europe/London.
Finish.
Edit the settings to add a new Hard Disk, which will become the destination/deduplication store.
Make it thick provisioned, SCaSI id (1:0) or (2:0) and so on (2nd digit must be 0).
In Options / VMware Tools, tick "Synchronize guest time with host".
You can install the VDR VM from the OVF file on the vSphere iso image which you can download from your account on www.vmware.com.
Power on the VDR VM. Remember to place it on the correct host (only 1 VDR VM per host).
Open a console window on it.
Once it has booted, set the network configuration to use a static IP.
The username is "root" and the default password is vmw@re.
As soon as you have configured the networking, login using the console window and change the root password to something secret.
Now use the vSphere Client Home / Solutions and Applications / VMware Data Recovery to set it up.
It will walk you through a wizard.
Format the Physical Disk which is the VMDK you created earlier. It will mount it automatically.
Create a new backup job.
If you are trying to select to backup another VDR VM, open the tree on that VM and only backup the 5GB VMDK and not the whole VM!
Set the times so it is *not* backed up 9am-8pm Mon-Fri and 11am-10pm Sat-Sun.
Set the number of backups to keep to something sensible, which may be less than the supplied defaults. We use 6,4,3,0,0.
Manually run a first backup by right-clicking on the job name and using the popup menu.
You can only have 1 VDR VM per CPU host.
Each VDR VM can backup 100 VMs at most.
Each VDR VM can have 2 Destinations at most.
In strict terms, a "destination" is a "deduplication store".
Each destination can be 1 Tbyte at most.
Each destination should be a virtual hard disk stored inside the VM.
Do not use "thin" provisioning for destination vmdk disks.
Do not use independent disks for destination vmdk disks.
Set the SCSI id of each destionation vmdk to be (1,0) or (2,0) or (3,0) and so on. The second digit must always be 0.
Before you start you must install the "VMwareDataRecoveryPlugin.msi" program. Download the msi file from datastore "ugstore1-vol1"/ISOs onto your PC, quit the vSphere Client and double-click on the file to install it. Restart the vSphere Client and on the bottom row of "Home" you will now have the VMware Data Recovery solution.
You can then connect to any one of the VDR VMs.
There are currently 2 VDR VMs:
- Teaching / Backups / VMware Data Recovery UG1 - DNS name ecsvm-vdr-ug1 - backs up teaching VMs
- Systems / VMware Data Recovery Infra1 - DNS name ecsvm-vdr-inf1 - backs up infrastructure VMs
The actual backup and restore operations are pretty intuitive, the on-line help will answer most of the obvious questions.
The backup time window is shifted from the default to not backup during 9am-7pm Mon-Fri, and 11am-9pm Sat-Sun.
The Reports and Configuration tabs will show you what is going on, any warnings and errors, and how much space is available in the destinations deduplication stores.
The VDR system checks the integrity of the destinations daily automatically.
When looking at the list of destinations in the Configuration tab, the line for each destination will list a low figure for free space, but a very different figure for the "Deduplicated Size" at the bottom of the pane.
The VDR system automatically consolidates identical and unused blocks daily automatically.
Creating a New VDR VM
You need a Virtual Appliance (ovf file) from the download of "VMwareDataRecovery-....-i386.iso" which is on ecsvm-admin in Administrator's Downloads directory. Mount this iso as a new drive letter using MagicISO (installed on ecsvm-admin) or Daemon-Tools.
Using vSphere Client, File / Deploy OVF Template... and point it to the file \VMwareDataRecovery-ovf-i386\VMwareDataRecovery_OVF10.ovf.
Set the VM's name to something useful like "VMware Data Recovery UG1", and put the location somewhere sensible.
Set the host to a CPU server that doesn't already have a VMware Data Recovery VM on it, there must only be 1 VM per server.
Set the datastore to the place you want all the backups to go.
Use Thick Provisioned format.
Set the Destination Network appropriately for the IP you have.
Set the Timezone to Europe/London.
Finish.
Edit the settings to add a new Hard Disk, which will become the destination/deduplication store.
Make it thick provisioned, SCaSI id (1:0) or (2:0) and so on (2nd digit must be 0).
In Options / VMware Tools, tick "Synchronize guest time with host".
You can install the VDR VM from the OVF file on the vSphere iso image which you can download from your account on www.vmware.com.
Power on the VDR VM. Remember to place it on the correct host (only 1 VDR VM per host).
Open a console window on it.
Once it has booted, set the network configuration to use a static IP.
The username is "root" and the default password is vmw@re.
As soon as you have configured the networking, login using the console window and change the root password to something secret.
Now use the vSphere Client Home / Solutions and Applications / VMware Data Recovery to set it up.
It will walk you through a wizard.
Format the Physical Disk which is the VMDK you created earlier. It will mount it automatically.
Create a new backup job.
If you are trying to select to backup another VDR VM, open the tree on that VM and only backup the 5GB VMDK and not the whole VM!
Set the times so it is *not* backed up 9am-8pm Mon-Fri and 11am-10pm Sat-Sun.
Set the number of backups to keep to something sensible, which may be less than the supplied defaults. We use 6,4,3,0,0.
Manually run a first backup by right-clicking on the job name and using the popup menu.
Unable to query the live linux source machine
26/12/10 19:34
You may get this error message from an installation of the VMware Converter Standalone Client or Server.
All you need to do is put a copy of the PuTTY plink.exe and pscp.exe programs into
C:\Program Files (x86)\VMware\VMware vCenter Standalone Converter
All you need to do is put a copy of the PuTTY plink.exe and pscp.exe programs into
C:\Program Files (x86)\VMware\VMware vCenter Standalone Converter
Fast VMware VCB Backups
29/11/09 11:42
Update 11th Mar 2010 : Less noisy output and retries backup request if it failed
Update 29th Nov 2009 : Major improvement to VM selection rules
Update 18th Nov 2009 : Bug fix release.
Update 15th Nov 2009 : Now handles zombie processes correctly so will terminate correctly when called from .cmd batch scripts.
Update 10th Nov 2009 : Now deletes old backup snapshots left behind by failed previous backups.
This is a new program which uses the VMware "Virtual Consolidated Backup" (or VCB) system to work, and will run on any Windows system. It is highly parallel, very fast and will easily saturate a 1Gbps network link. I have had performance of over 100Mbytes/second across our network, with a sustained rate of over 70Mbytes/second for several hours.
It is far faster than my previous JKFBackup.sh script mentioned in a previous article. It also does not require any ssh or other "unsupported" access to the VMware host server.
It takes a command-line parameter which is the full pathname of the configuration file that it uses. This can be surrounded in double-quotes (") if there are spaces in the path or filename.
The configuration file is a simple text file:
The INCLUDE and EXCLUDE lines work like rules in a firewall. A rule can match several VMs by using “*” wildcard characters in them. As a simple example, the line “INCLUDE kanga-*” would tell the program to backup all VMs whose names start with “kanga-”. The first rule that matches the name of the VM says whether it will be backed up or not. I strongly advise you put either “INCLUDE *” or “EXCLUDE *” as the last rule in the file, to specify the default action for any VMs that do not match any other rules, such as VMs that have been added since you updated the configuration file.
Currently the script uses a different directory for each day of the week, so you have the last week's worth of backups on the disk. If you want to change this, learn a little Perl and you will find the code is very simple and straightforward.
As an example, the backups for Thursday for the VM named "Dummy" will be stored in DIRECTORY\Thu\Dummy.
If you want to backup several VMware host servers, then have a separate configuration file for each server. This script is currently aimed at relatively small vSpheres where you only have a few VMware host servers. If you have hundreds of VMware host servers, then please feel free to rip the script apart and use any bits of it you want. If you are prepared to let me include your improvements in a future release, then please contact me!
To backup VMs as fast as possible, the script splits the list of VMs into 3 and does 3 backups simultaneously in parallel.
Using VCB, data recovery is very simple. Just look in the directory named after the VM you want to restore, and use the VMware Standalone Converter to turn the files in there back into a VM on your VMware server. Easy as that!
You can download version 1.10 of VCBBackup.pl here.
I suggest you right-click on that link and choose "Save as..." to download the file.
It is written in Perl so you will need to install the free Perl distribution from www.activestate.com. You will also need to download and install the "VMware Consolidated Backup" distribution from VMware, which you should be able to access if you have purchased VMware products.
I do my backups onto a little Dell 860 1U rackmount server running Windows Server 2008R2 x64. It backs up to cheap external disk connected via a Firewire-800 interface. This is quite sufficient and can maintain over 700 Mbits/second backup speed.
Every couple of nights I swap out the hard disk and take it off-site. That provides us with backups of the last fortnight with some off-site storage.
I always welcome constructive feedback, and suggestions and requests for new features. If you need any help, just contact me!
Update 29th Nov 2009 : Major improvement to VM selection rules
Update 18th Nov 2009 : Bug fix release.
Update 15th Nov 2009 : Now handles zombie processes correctly so will terminate correctly when called from .cmd batch scripts.
Update 10th Nov 2009 : Now deletes old backup snapshots left behind by failed previous backups.
This is a new program which uses the VMware "Virtual Consolidated Backup" (or VCB) system to work, and will run on any Windows system. It is highly parallel, very fast and will easily saturate a 1Gbps network link. I have had performance of over 100Mbytes/second across our network, with a sustained rate of over 70Mbytes/second for several hours.
It is far faster than my previous JKFBackup.sh script mentioned in a previous article. It also does not require any ssh or other "unsupported" access to the VMware host server.
It takes a command-line parameter which is the full pathname of the configuration file that it uses. This can be surrounded in double-quotes (") if there are spaces in the path or filename.
Configuration
The configuration file is a simple text file:
- All leading and trailing spaces are ignored
- All blank lines are ignored
- Comments start with a hash character (pound sign in the USA) like this "#" and continue to the end of the line
- Each line must start with one of the following keywords, followed by its single parameter:
- DIRECTORY : the full path, including drive letter, under which all the backups will be stored
- HOST : the name of the VMware host server running the VMs to be backed up
- USER : the username to access the VMware host server (usually root)
- PASSWORD : the password for the stated USER
- INCLUDE : backup this named VM. Use the “*” character as a wildcard to mean “0 or more characters” to specify several VMs
- EXCLUDE : do not backup this named VM. Use the “*” character as a wildcard to mean “0 or more characters” to specify several VMs
The INCLUDE and EXCLUDE lines work like rules in a firewall. A rule can match several VMs by using “*” wildcard characters in them. As a simple example, the line “INCLUDE kanga-*” would tell the program to backup all VMs whose names start with “kanga-”. The first rule that matches the name of the VM says whether it will be backed up or not. I strongly advise you put either “INCLUDE *” or “EXCLUDE *” as the last rule in the file, to specify the default action for any VMs that do not match any other rules, such as VMs that have been added since you updated the configuration file.
Currently the script uses a different directory for each day of the week, so you have the last week's worth of backups on the disk. If you want to change this, learn a little Perl and you will find the code is very simple and straightforward.
As an example, the backups for Thursday for the VM named "Dummy" will be stored in DIRECTORY\Thu\Dummy.
If you want to backup several VMware host servers, then have a separate configuration file for each server. This script is currently aimed at relatively small vSpheres where you only have a few VMware host servers. If you have hundreds of VMware host servers, then please feel free to rip the script apart and use any bits of it you want. If you are prepared to let me include your improvements in a future release, then please contact me!
To backup VMs as fast as possible, the script splits the list of VMs into 3 and does 3 backups simultaneously in parallel.
Data Recovery
Using VCB, data recovery is very simple. Just look in the directory named after the VM you want to restore, and use the VMware Standalone Converter to turn the files in there back into a VM on your VMware server. Easy as that!
Download
You can download version 1.10 of VCBBackup.pl here.
I suggest you right-click on that link and choose "Save as..." to download the file.
It is written in Perl so you will need to install the free Perl distribution from www.activestate.com. You will also need to download and install the "VMware Consolidated Backup" distribution from VMware, which you should be able to access if you have purchased VMware products.
My Setup
I do my backups onto a little Dell 860 1U rackmount server running Windows Server 2008R2 x64. It backs up to cheap external disk connected via a Firewire-800 interface. This is quite sufficient and can maintain over 700 Mbits/second backup speed.
Every couple of nights I swap out the hard disk and take it off-site. That provides us with backups of the last fortnight with some off-site storage.
Feedback
I always welcome constructive feedback, and suggestions and requests for new features. If you need any help, just contact me!
VMware vSphere Client in Parallels Desktop
15/11/09 20:20
This is a quick solution to a problem I suffered, and thought it was a bug in Parallels Desktop for Mac. You run the vSphere Client application, open a Console to a virtual machine, and after that all your Windows keystrokes and mouse clicks go into the Console and not into any other Windows applications at all, not even the taskbar or Start menu.
The key is that the vSphere Client uses the same "Ctrl Alt" key combination to get out of the console, as does Parallels to get out of Windows programs.
When running in Coherence or Crystal mode, you never really need this key combination as it works automatically anyway when you click in any other Mac application.
The solution is to change the Parallels "Release Input" key combination. Get out of Crystal mode, then go to the Parallels / Preferences window. Select the "Keyboard & Mouse" pane of the dialog box, and work through each Profile in turn. In each profile, set the key combination for "Release Input" to "Cmd-Ctrl-Space" (that's what I used).
Then quit and re-run Parallels and you will find the problem has gone away.
The key is that the vSphere Client uses the same "Ctrl Alt" key combination to get out of the console, as does Parallels to get out of Windows programs.
When running in Coherence or Crystal mode, you never really need this key combination as it works automatically anyway when you click in any other Mac application.
The solution is to change the Parallels "Release Input" key combination. Get out of Crystal mode, then go to the Parallels / Preferences window. Select the "Keyboard & Mouse" pane of the dialog box, and work through each Profile in turn. In each profile, set the key combination for "Release Input" to "Cmd-Ctrl-Space" (that's what I used).
Then quit and re-run Parallels and you will find the problem has gone away.
VMware Converter - Converting an Existing Windows System
18/10/09 20:23
The Converter is a client/server program, where the converter server runs on ecsvm-admin.ecs and the client can run anywhere, such as win-admin.
Firewall Rules
If the source physical Windows system is in the DMZ, you will need to add a couple of rules to the firewall "Short-Term Rules" section allowing all traffic from ECS-internal to the source host and from the source host to ECS-internal. You should remove these rules again once the conversion has completed.
Starting the Conversion
Start the VMWare Converter Standalone Client program.
IP Address/Name = ecsvm-admin:7443
Username = administrator (or your own sys* username if you are registered on the vSphere with an account).
Password = Windows-infrastructure password
Click the "Convert Machine" button in the toolbar.
Source type = Powered-on machine
Give remote machine full hostname.
Give username and password of a user in the "Administrators" group on the source Windows system, such as the "ECS2000\administrator" account and domain password..
OS Family = Windows.
Next.
It should connect to the source physical system.
If it fails, then give it the IP address of the source physical system instead of the name of it.
If it still fails, then Remote Desktop to the source physical system, download the converter (from browsing the datastores starting at http://ecsvm-admin.ecs.soton.ac.uk/) and install the converter agent (not the client/server setup) onto the source system, then run it again from there.
When it asks for the name of the source system to convert, give it the IP address of the source system (which is the same system you are now running the Converter on).
Tell it to automatically uninstall the files when the import succeeds.
It will then deploy the agent, which takes a few seconds.
VMware Infrastructure server details:
Server = ecsvm-admin
Username = administrator (or your sys* username if you have an account on vSphere).
Password = Windows-infrastructure password (or yours).
Next.
Select host to run the VM on = ecsvm-admin1.ecs.soton.ac.uk.
Virtual machine name = short hostname (eg. major-backup).
Datastore = infrastore1-Vol2 (an infrastore volume with plenty of free space).
Virtual machine version = Version 7.
Next.
Options
Destination Attributes: Name = short hostname, Folder = ecs
Data to copy: Click "Advanced...", then "Target Layout...".
Set all large, non-swap filesystems to "Thin provisioning" (this is "Dynamic disks" in Microsoft speak, where only the disk space in use is actually allocated on disk, used disk space expands as necessary to hold the data, up to the maximum set by the size of the filesystem).
Set the Size of the normal large filesystems (eg. "C:") to a reasonable number, no point in making them huge, most will fit in 60Gbytes.
Switch to the "Source Volumes" tab to see how much space is actually in use at the moment.
Normally set C: to thin provisioning, 60GBytes or more.
It is important not to waste disk space on Flat-provisioned disks that are not going to use all their space. Disk is relatively expensive.
Devices: Numer of processors = 1.
Disk controller = SCSI LSI Logic or SCSI Buslogic.
Memory = usually the default will be fine.
Networks: Network adapters to connect = 1
Set the network for the network adapter to be the same Virtual Machine Network VLAN as the physical machine you are converting.
The new virtual machine will take over the IP address of the physical source machine.
Services: Source Services: switch off services that should not be left running when the VM is created (such as SQL Server or WWW Publishing Service, and Hyper-V services if moving from Hyper-V to VMware). Destination Services: set all Hyper-V services to "Disabled".
Advanced Options: Power on target machine = yes
Power off source machine = yes
Install VMware Tools on the imported virtual machine = yes
Configure guest preferences for the virtual machine = no
Remove System Restore checkpoints on destination = yes
Reconfigure destination virtual machine = yes
Next.
It should now just show you the final option settings and then start the process of converting the host. Wait for the whole process to finish before touching either the source (physical) or destination (virtual) machines.
How long it takes depends on the quantity of data that has to be moved. You can expect about 20 to 25 MBytes/second conversion speed. A machine with about 7GB of used disk takes about 25 minutes to convert.
As the process is run by the Converter Server (running on ecsvm-admin), it doesn't matter if you quit the Windows app used to watch the process, you can track the progress of the conversion from the Converter Windows app running on any PC.
After the Conversion has Finished
Open a console on the new VM.
In the "VM/Guest" menu, install the VM tools. This will insert a CD into the virtual CD drive, what happens then is dependent on the Windows Autoplay preferences in the virtual machine.
This will force a restart of the virtual machine.
Shutdown the virtual machine.
Once the VM has stopped, edit the settings of the VM and choose the middle "Options" tab. In the "VMware Tools" settings, right at the bottom right of the dialog there is an option to "Synchronize guest time with host". Tick this box.
Okay that, then power on the VM.
It should successfully boot.
The VM should now be fully running happily and serving its services to the users.
Now just remove the two temporary firewall rules you added at the start of this process.
Firewall Rules
If the source physical Windows system is in the DMZ, you will need to add a couple of rules to the firewall "Short-Term Rules" section allowing all traffic from ECS-internal to the source host and from the source host to ECS-internal. You should remove these rules again once the conversion has completed.
Starting the Conversion
Start the VMWare Converter Standalone Client program.
IP Address/Name = ecsvm-admin:7443
Username = administrator (or your own sys* username if you are registered on the vSphere with an account).
Password = Windows-infrastructure password
Click the "Convert Machine" button in the toolbar.
Source type = Powered-on machine
Give remote machine full hostname.
Give username and password of a user in the "Administrators" group on the source Windows system, such as the "ECS2000\administrator" account and domain password..
OS Family = Windows.
Next.
It should connect to the source physical system.
If it fails, then give it the IP address of the source physical system instead of the name of it.
If it still fails, then Remote Desktop to the source physical system, download the converter (from browsing the datastores starting at http://ecsvm-admin.ecs.soton.ac.uk/) and install the converter agent (not the client/server setup) onto the source system, then run it again from there.
When it asks for the name of the source system to convert, give it the IP address of the source system (which is the same system you are now running the Converter on).
Tell it to automatically uninstall the files when the import succeeds.
It will then deploy the agent, which takes a few seconds.
VMware Infrastructure server details:
Server = ecsvm-admin
Username = administrator (or your sys* username if you have an account on vSphere).
Password = Windows-infrastructure password (or yours).
Next.
Select host to run the VM on = ecsvm-admin1.ecs.soton.ac.uk.
Virtual machine name = short hostname (eg. major-backup).
Datastore = infrastore1-Vol2 (an infrastore volume with plenty of free space).
Virtual machine version = Version 7.
Next.
Options
Destination Attributes: Name = short hostname, Folder = ecs
Data to copy: Click "Advanced...", then "Target Layout...".
Set all large, non-swap filesystems to "Thin provisioning" (this is "Dynamic disks" in Microsoft speak, where only the disk space in use is actually allocated on disk, used disk space expands as necessary to hold the data, up to the maximum set by the size of the filesystem).
Set the Size of the normal large filesystems (eg. "C:") to a reasonable number, no point in making them huge, most will fit in 60Gbytes.
Switch to the "Source Volumes" tab to see how much space is actually in use at the moment.
Normally set C: to thin provisioning, 60GBytes or more.
It is important not to waste disk space on Flat-provisioned disks that are not going to use all their space. Disk is relatively expensive.
Devices: Numer of processors = 1.
Disk controller = SCSI LSI Logic or SCSI Buslogic.
Memory = usually the default will be fine.
Networks: Network adapters to connect = 1
Set the network for the network adapter to be the same Virtual Machine Network VLAN as the physical machine you are converting.
The new virtual machine will take over the IP address of the physical source machine.
Services: Source Services: switch off services that should not be left running when the VM is created (such as SQL Server or WWW Publishing Service, and Hyper-V services if moving from Hyper-V to VMware). Destination Services: set all Hyper-V services to "Disabled".
Advanced Options: Power on target machine = yes
Power off source machine = yes
Install VMware Tools on the imported virtual machine = yes
Configure guest preferences for the virtual machine = no
Remove System Restore checkpoints on destination = yes
Reconfigure destination virtual machine = yes
Next.
It should now just show you the final option settings and then start the process of converting the host. Wait for the whole process to finish before touching either the source (physical) or destination (virtual) machines.
How long it takes depends on the quantity of data that has to be moved. You can expect about 20 to 25 MBytes/second conversion speed. A machine with about 7GB of used disk takes about 25 minutes to convert.
As the process is run by the Converter Server (running on ecsvm-admin), it doesn't matter if you quit the Windows app used to watch the process, you can track the progress of the conversion from the Converter Windows app running on any PC.
After the Conversion has Finished
Open a console on the new VM.
In the "VM/Guest" menu, install the VM tools. This will insert a CD into the virtual CD drive, what happens then is dependent on the Windows Autoplay preferences in the virtual machine.
This will force a restart of the virtual machine.
Shutdown the virtual machine.
Once the VM has stopped, edit the settings of the VM and choose the middle "Options" tab. In the "VMware Tools" settings, right at the bottom right of the dialog there is an option to "Synchronize guest time with host". Tick this box.
Okay that, then power on the VM.
It should successfully boot.
The VM should now be fully running happily and serving its services to the users.
Now just remove the two temporary firewall rules you added at the start of this process.
VMware Converter - Converting an Existing Linux System
18/10/09 20:22
The Converter is a client/server program, where the converter server runs on ecsvm-admin.ecs and the client can run anywhere, such as win-admin.
Firewall Rules
If the source physical Linux system is in the DMZ, you will need to add a couple of rules to the firewall "Short-Term Rules" section allowing all traffic from ECS-internal to the source host and from the source host to ECS-internal. You should remove these rules again once the conversion has completed.
Before You Start
On the final virtual machine, you will need to install the VMware tools and set the VM settings to synchronise the time on the VM with that of the ecsvm-admin server. So you won't want ntpd to be running. Also, you are going to need to reboot the VM at least once after you have converted it, so stop and disable the primary user services (e.g. httpd, mysqld) that are running on the server. You can start up the user services again as the last step after getting the VM Linux system running. So I would start with
service ntpd stop
service httpd stop
service mysqld stop
chkconfig ntpd off
chkconfig httpd off
chkconfig mysqld off
Also, if the physical source machine is actually a Windows 2008 Hyper-V VM, then you will want to do the same to the "inputvsc" service, and copy the seth0 device settings to eth0 (remembering to change the device name in /etc/sysconfig/network-scripts/ifcfg-eth0 after copying the ifcfg-seth0 file onto it) as VMware will use the eth0 device and not the seth0 device.
Once running in VMware, the kernel will need to be able to "probe" the disk controllers in order to be able to find the controller types. So in /boot/grub/grub.conf, edit the kernel command line arguments and remove any settings that set "hda=noprobe" or similar. You don't need to reboot after this change, this will be applied once the VM conversion has finished and VMware starts the new VM.
Starting the Conversion
Start the VMWare Converter Standalone Client program.
IP Address/Name = ecsvm-admin:7443
Username = administrator (or your own sys* username if you are registered on the vSphere with an account).
Password = Windows-infrastructure password
Click the "Convert Machine" button in the toolbar.
Source type = Powered-on machine
Give remote machine full hostname.
Give root username and password.
Next.
VMware Infrastructure server details:
Server = ecsvm-admin
Username = administrator (or your sys* username if you have an account on vSphere).
Password = Windows-infrastructure password (or yours).
Next.
Select host to run the VM on = ecsvm-admin1.ecs.soton.ac.uk.
Virtual machine name = short hostname (eg. gander).
Datastore = infrastore1-Vol2 (an infrastore volume with plenty of free space).
Virtual machine version = Version 7.
Next.
Options
Destination Attributes: Name = short hostname, Folder = ecs
Data to copy: Click "Advanced...", then "Target Layout...".
Set all large, non-swap filesystems to "Thin provisioning" (this is "Dynamic disks" in Microsoft speak, where only the disk space in use is actually allocated on disk, used disk space expands as necessary to hold the data, up to the maximum set by the size of the filesystem).
Set the Size of the normal large filesystems (eg. "/") to a reasonable number, no point in making them huge, most will fit in 40Gbytes.
Switch to the "Source Volumes" tab to see how much space is actually in use at the moment.
Normally set / to thin provisioning, 40GBytes, leave /boot and swap alone.
Devices: Numer of processors = 1.
Disk controller = SCSI LSI Logic or SCSI Buslogic.
Memory = usually the default will be fine.
Networks: Network adapters to connect = 1
Set the network for the network adapter to be the same Virtual Machine Network VLAN as the physical machine you are converting.
The new virtual machine will take over the IP address of the physical source machine.
Advanced Options: Power on target machine = yes
Power off source machine = yes
Reconfigure destination virtual machine = yes
Helper VM Network: Look up the IP address of the host "ecsvm--helper" which is a spare IP address unused by anything else in the same "" as the physical host you are converting. There are currently hosts defined such as "ecsvm-systems-helper", "ecsvm-dmz-helper" and "ecsvm-servers-helper". The IP configuration parameters and DNS setup for the Helper VM Network must all be consistent with its IP address. This "helper" machine is a temporary system setup by the VMware Converter just for use during the process of converting a Linux box, it disappears again at the end.
Next.
It should now just show you the final option settings and then start the process of converting the host. Wait for the whole process to finish before touching either the source (physical) or destination (virtual) machines.
How long it takes depends on the quantity of data that has to be moved. You can expect about 20 to 25 MBytes/second conversion speed. A machine with about 7GB of used disk takes about 25 minutes to convert.
As the process is run by the Converter Server (running on ecsvm-admin), it doesn't matter if you quit the Windows app used to watch the process, you can track the progress of the conversion from the Converter Windows app running on any PC.
After the Conversion has Finished
Open a console on the new VM.
In the "VM/Guest" menu, install the VM tools.
mount /dev/cdrom /mnt
cd /tmp
tar xzf /mnt/VM*
umount /mnt
cd vmware*
./vmware-tools-install.pl
Accept all the defaults. It will find suitable modules for your kernel, or else will compile them itself, and build a new initrd, grub.conf and modprobe.conf for your kernel.
Assuming that succeeded, then
shutdown -h now
Once the VM has stopped, edit the settings of the VM and choose the middle "Options" tab. In the "VMware Tools" settings, right at the bottom right of the dialog there is an option to "Synchronize guest time with host". Tick this box.
Okay that, then power on the VM.
It should successfully boot.
The only remaining tasks are to enable and start up the user services, but not ntpd.
chkconfig mysqld on
chkconfig httpd on
service mysqld start
service httpd start
The VM should now be fully running happily and serving its services to the users. If you prefer, you can always reboot the VM again instead of running those last two "service ... start" commands to prove it is all okay and booting correctly.
Now just remove the two temporary firewall rules you added at the start of this process.
Firewall Rules
If the source physical Linux system is in the DMZ, you will need to add a couple of rules to the firewall "Short-Term Rules" section allowing all traffic from ECS-internal to the source host and from the source host to ECS-internal. You should remove these rules again once the conversion has completed.
Before You Start
On the final virtual machine, you will need to install the VMware tools and set the VM settings to synchronise the time on the VM with that of the ecsvm-admin server. So you won't want ntpd to be running. Also, you are going to need to reboot the VM at least once after you have converted it, so stop and disable the primary user services (e.g. httpd, mysqld) that are running on the server. You can start up the user services again as the last step after getting the VM Linux system running. So I would start with
service ntpd stop
service httpd stop
service mysqld stop
chkconfig ntpd off
chkconfig httpd off
chkconfig mysqld off
Also, if the physical source machine is actually a Windows 2008 Hyper-V VM, then you will want to do the same to the "inputvsc" service, and copy the seth0 device settings to eth0 (remembering to change the device name in /etc/sysconfig/network-scripts/ifcfg-eth0 after copying the ifcfg-seth0 file onto it) as VMware will use the eth0 device and not the seth0 device.
Once running in VMware, the kernel will need to be able to "probe" the disk controllers in order to be able to find the controller types. So in /boot/grub/grub.conf, edit the kernel command line arguments and remove any settings that set "hda=noprobe" or similar. You don't need to reboot after this change, this will be applied once the VM conversion has finished and VMware starts the new VM.
Starting the Conversion
Start the VMWare Converter Standalone Client program.
IP Address/Name = ecsvm-admin:7443
Username = administrator (or your own sys* username if you are registered on the vSphere with an account).
Password = Windows-infrastructure password
Click the "Convert Machine" button in the toolbar.
Source type = Powered-on machine
Give remote machine full hostname.
Give root username and password.
Next.
VMware Infrastructure server details:
Server = ecsvm-admin
Username = administrator (or your sys* username if you have an account on vSphere).
Password = Windows-infrastructure password (or yours).
Next.
Select host to run the VM on = ecsvm-admin1.ecs.soton.ac.uk.
Virtual machine name = short hostname (eg. gander).
Datastore = infrastore1-Vol2 (an infrastore volume with plenty of free space).
Virtual machine version = Version 7.
Next.
Options
Destination Attributes: Name = short hostname, Folder = ecs
Data to copy: Click "Advanced...", then "Target Layout...".
Set all large, non-swap filesystems to "Thin provisioning" (this is "Dynamic disks" in Microsoft speak, where only the disk space in use is actually allocated on disk, used disk space expands as necessary to hold the data, up to the maximum set by the size of the filesystem).
Set the Size of the normal large filesystems (eg. "/") to a reasonable number, no point in making them huge, most will fit in 40Gbytes.
Switch to the "Source Volumes" tab to see how much space is actually in use at the moment.
Normally set / to thin provisioning, 40GBytes, leave /boot and swap alone.
Devices: Numer of processors = 1.
Disk controller = SCSI LSI Logic or SCSI Buslogic.
Memory = usually the default will be fine.
Networks: Network adapters to connect = 1
Set the network for the network adapter to be the same Virtual Machine Network VLAN as the physical machine you are converting.
The new virtual machine will take over the IP address of the physical source machine.
Advanced Options: Power on target machine = yes
Power off source machine = yes
Reconfigure destination virtual machine = yes
Helper VM Network: Look up the IP address of the host "ecsvm-
Next.
It should now just show you the final option settings and then start the process of converting the host. Wait for the whole process to finish before touching either the source (physical) or destination (virtual) machines.
How long it takes depends on the quantity of data that has to be moved. You can expect about 20 to 25 MBytes/second conversion speed. A machine with about 7GB of used disk takes about 25 minutes to convert.
As the process is run by the Converter Server (running on ecsvm-admin), it doesn't matter if you quit the Windows app used to watch the process, you can track the progress of the conversion from the Converter Windows app running on any PC.
Open a console on the new VM.
In the "VM/Guest" menu, install the VM tools.
mount /dev/cdrom /mnt
cd /tmp
tar xzf /mnt/VM*
umount /mnt
cd vmware*
./vmware-tools-install.pl
Accept all the defaults. It will find suitable modules for your kernel, or else will compile them itself, and build a new initrd, grub.conf and modprobe.conf for your kernel.
Assuming that succeeded, then
shutdown -h now
Once the VM has stopped, edit the settings of the VM and choose the middle "Options" tab. In the "VMware Tools" settings, right at the bottom right of the dialog there is an option to "Synchronize guest time with host". Tick this box.
Okay that, then power on the VM.
It should successfully boot.
The only remaining tasks are to enable and start up the user services, but not ntpd.
chkconfig mysqld on
chkconfig httpd on
service mysqld start
service httpd start
The VM should now be fully running happily and serving its services to the users. If you prefer, you can always reboot the VM again instead of running those last two "service ... start" commands to prove it is all okay and booting correctly.
Now just remove the two temporary firewall rules you added at the start of this process.
Install VMware Tools on a Linux Client
23/09/09 20:13
First, log in to the VM as root and "yum update".
Then "yum install gcc make binutils kernel-devel kernel-headers".
Then reboot the VM.
Then start up the vSphere Client, right click on the VM and choose Guest - Install VMware Tools.
Back into the VM's root session.
mount /dev/cdrom /mnt
cd /tmp
tar xzf /mnt/VM*
umount /mnt
cd vmware-tools-distrib
./vmware-install.pl
Accept all the defaults, and let it do everything it wants. If you have a fully updated and correct system, it should install flawlessly.
Check the "ifconfig -a" and ensure that all the network devices that exist have startup scripts in /etc/sysconfig/network-scripts/ifcfg-eth*.
Go into the vSphere Client again and right-click on the VM. Look in the "Guest" sub-menu and tell it to stop installing the VMware Tools if it offers you that.
Reboot the VM.
If you have problems...
The most likely problem is that you are running an el5xen kernel or some other xen kernel, which you don't want to be doing.
Once you've done a yum update, take the xen kernel you are running (uname -a will tell you) and do something like this:
yum install kernel-2.6.18-164.el5
Then edit /boot/grub/grub.conf and ensure that the "default=" setting at the top is set to boot the kernel you just installed and not the xen kernel (they start numbering from the top of the file from 0).
Then reboot so you are running the non-xen kernel.
Then
yum install kernel-devel-2.6.18-164.el5
Then re-run /tmp/vmware-tools-distrib/vmware-install.pl and if it asks you for the kernel headers location, give it
/usr/src/kernels/2.6.18-164.el5/include
But you will most likely find that it just happily works on its own!
Then just reboot to pick up all the VMware tools in a fresh boot.
Then "yum install gcc make binutils kernel-devel kernel-headers".
Then reboot the VM.
Then start up the vSphere Client, right click on the VM and choose Guest - Install VMware Tools.
Back into the VM's root session.
mount /dev/cdrom /mnt
cd /tmp
tar xzf /mnt/VM*
umount /mnt
cd vmware-tools-distrib
./vmware-install.pl
Accept all the defaults, and let it do everything it wants. If you have a fully updated and correct system, it should install flawlessly.
Check the "ifconfig -a" and ensure that all the network devices that exist have startup scripts in /etc/sysconfig/network-scripts/ifcfg-eth*.
Go into the vSphere Client again and right-click on the VM. Look in the "Guest" sub-menu and tell it to stop installing the VMware Tools if it offers you that.
Reboot the VM.
If you have problems...
The most likely problem is that you are running an el5xen kernel or some other xen kernel, which you don't want to be doing.
Once you've done a yum update, take the xen kernel you are running (uname -a will tell you) and do something like this:
yum install kernel-2.6.18-164.el5
Then edit /boot/grub/grub.conf and ensure that the "default=" setting at the top is set to boot the kernel you just installed and not the xen kernel (they start numbering from the top of the file from 0).
Then reboot so you are running the non-xen kernel.
Then
yum install kernel-devel-2.6.18-164.el5
Then re-run /tmp/vmware-tools-distrib/vmware-install.pl and if it asks you for the kernel headers location, give it
/usr/src/kernels/2.6.18-164.el5/include
But you will most likely find that it just happily works on its own!
Then just reboot to pick up all the VMware tools in a fresh boot.
VMware Converter Fails for Linux Client
23/09/09 20:11
You need to change the installed SCSI controller to be a LSI Logic controller. The original machine may have been configured for a Transtec 3Ware SCSI controller.
The main relevant article is here:
http://tipstricksandmore.blogspot.com/2009/01/after-converting-physical-rhel4-system.html
Once the VM has been converted, you can change the hardware of the VM in the "Edit Settings..." menu from right-clicking on the VM. You can only change it when the VM is powered off.
Edit the VM settings and connect the DVD drive to a "Datastore ISO File": infra1-localDisk/vSphere Management Assistant/rhel-5-server-i386-dvd.iso.
Set the Device Status to "Connected" and "Connect at power on".
In the VM options page, set it to go into the BIOS setup at next boot.
Open a console on the VM and power it on. It will go into the BIOS setup.
In the "Boot" BIOS menu, select the DVD drive and press + to move it to the top.
Save and exit the BIOS setup.
It will boot from DVD.
Enter "linux rescue".
You don't need any network interfaces.
Let it look for the installed system to mount under /mnt/sysimage.
chroot /mnt/sysimage
Replace hda with sda in /etc/fstab, /boot/grub/device.map and /boot/grub/grub.conf
grub-install /dev/sda
Make sure /etc/modules.conf is empty or non-existent.
Edit /etc/modprobe.conf and set
alias eth0 pcnet32
alias eth1 pcnet32
alias scsi_hostadapter mptbase
alias scsi_hostadapter1 mptscsih
Work out the full path to the initrd image you are going to rebuild. They are in /boot and are called initrd*.
The /boot/grub/grub.conf will point to the right one.
So in my example it is "/boot/initrd-2.6.18-128.7.1.el5xen.img".
There should be a directory under /lib/modules called the same version number.
Then you use a command like this to rebuild it
mkinitrd -v -f /boot/initrd-2.6.18-128.7.1.el5xen.img 2.6.18-128.7.1.el5xen
exit
reboot
Press Esc to get the boot menu and force it to boot from the hard disk.
Hopefully it will boot this time!
Shut it down again, edit the VM settings and set the DVD drive back to "Client Device".
Boot your VM normally.
The main relevant article is here:
http://tipstricksandmore.blogspot.com/2009/01/after-converting-physical-rhel4-system.html
Once the VM has been converted, you can change the hardware of the VM in the "Edit Settings..." menu from right-clicking on the VM. You can only change it when the VM is powered off.
Edit the VM settings and connect the DVD drive to a "Datastore ISO File": infra1-localDisk/vSphere Management Assistant/rhel-5-server-i386-dvd.iso.
Set the Device Status to "Connected" and "Connect at power on".
In the VM options page, set it to go into the BIOS setup at next boot.
Open a console on the VM and power it on. It will go into the BIOS setup.
In the "Boot" BIOS menu, select the DVD drive and press + to move it to the top.
Save and exit the BIOS setup.
It will boot from DVD.
Enter "linux rescue".
You don't need any network interfaces.
Let it look for the installed system to mount under /mnt/sysimage.
chroot /mnt/sysimage
Replace hda with sda in /etc/fstab, /boot/grub/device.map and /boot/grub/grub.conf
grub-install /dev/sda
Make sure /etc/modules.conf is empty or non-existent.
Edit /etc/modprobe.conf and set
alias eth0 pcnet32
alias eth1 pcnet32
alias scsi_hostadapter mptbase
alias scsi_hostadapter1 mptscsih
Work out the full path to the initrd image you are going to rebuild. They are in /boot and are called initrd*.
The /boot/grub/grub.conf will point to the right one.
So in my example it is "/boot/initrd-2.6.18-128.7.1.el5xen.img".
There should be a directory under /lib/modules called the same version number.
Then you use a command like this to rebuild it
mkinitrd -v -f /boot/initrd-2.6.18-128.7.1.el5xen.img 2.6.18-128.7.1.el5xen
exit
reboot
Press Esc to get the boot menu and force it to boot from the hard disk.
Hopefully it will boot this time!
Shut it down again, edit the VM settings and set the DVD drive back to "Client Device".
Boot your VM normally.
VMware Hot Clone of an Existing System
23/09/09 20:10
This article describes how to create a VMware virtual machine from an existing live (Windows) system.
The same process should work with Linux but I've not tested it yet.
Starting the Client
1. Open a Remote Desktop session to ecsvm-admin.ecs.soton.ac.uk and login as ecs2000\administrator
2. Start the VmWare Converter Standalone Client on ecsvm-admin. Start --> All Programs --> VMWare --> Converter Standalone Client
3. Select "Connect to local Server" and click ok.
4. The client software should start
Creating the VM
1. Click the "Convert Machine" button (towards the top left)
2. Select the "A Remote machine" radio button
3. Enter the IP address of the machine to be cloned
4. Enter the user ID and password of an Administrator on the machine being cloned
5. Click Next
6. You should see a prompt asking if you want to "Automatically uninstall" the agent software when the clone has finished. Select the option and click Yes.
7. It'll now wait while the client agent is installed.
8. Next you're prompted for the VMware Virtual Infrastructure details:
Server=ecsvm-admin.ecs.soton.ac.uk
User Name=ecs2000\administrator
9. You should now see a view of our VI Infrastructure:
a. Click server "ecsvm-infra1.ecs.soton.ac.uk"
b. Select one of the infrastore1 disk volumes for the datastore
c. Type the virtual machine name you want to use (normally just the host name and not the FQDN)
10. Click next
11. You now get a chance to configure the VM "hardware" and select which disks to clone etc
12 Click Next
13. Click finish and the clone should start.
14 After the clone has finished you need to make sure the VM is connected to the correct virtual network:
Using the Vsphere client right click on the VM and click edit settings.
Click the network adapter, choose the correct network in the list box and click OK.
15. Install/Upgrade VMware tools:
Using vSphere client right click on the VM and click Guest --> Install/Upgrade VMware tools
The same process should work with Linux but I've not tested it yet.
Starting the Client
1. Open a Remote Desktop session to ecsvm-admin.ecs.soton.ac.uk and login as ecs2000\administrator
2. Start the VmWare Converter Standalone Client on ecsvm-admin. Start --> All Programs --> VMWare --> Converter Standalone Client
3. Select "Connect to local Server" and click ok.
4. The client software should start
Creating the VM
1. Click the "Convert Machine" button (towards the top left)
2. Select the "A Remote machine" radio button
3. Enter the IP address of the machine to be cloned
4. Enter the user ID and password of an Administrator on the machine being cloned
5. Click Next
6. You should see a prompt asking if you want to "Automatically uninstall" the agent software when the clone has finished. Select the option and click Yes.
7. It'll now wait while the client agent is installed.
8. Next you're prompted for the VMware Virtual Infrastructure details:
Server=ecsvm-admin.ecs.soton.ac.uk
User Name=ecs2000\administrator
9. You should now see a view of our VI Infrastructure:
a. Click server "ecsvm-infra1.ecs.soton.ac.uk"
b. Select one of the infrastore1 disk volumes for the datastore
c. Type the virtual machine name you want to use (normally just the host name and not the FQDN)
10. Click next
11. You now get a chance to configure the VM "hardware" and select which disks to clone etc
12 Click Next
13. Click finish and the clone should start.
14 After the clone has finished you need to make sure the VM is connected to the correct virtual network:
Using the Vsphere client right click on the VM and click edit settings.
Click the network adapter, choose the correct network in the list box and click OK.
15. Install/Upgrade VMware tools:
Using vSphere client right click on the VM and click Guest --> Install/Upgrade VMware tools
vSphere Client will not Run on Windows 7
23/09/09 20:08
When you start the vSphere Client on Windows 7, you get an error about it being unable to read the "clients.xml" file, followed by an another error "The type initializer for "VirtualInfrastructure.Utils.HttpWebRequestProxy" threw an exception".
The workaround is this:
You can now use the VpxClient.cmd (or the shortcut) to launch the vSphere client in Windows 7.
This workaround bypasses the normal .NET Framework loading mechanism. Assembly versions found in the DEVPATH folder are not checked.
The workaround is this:
- Create a folder (e.g. Lib) in the Windows 7 machine where the vSphere client is installed (%ProgramFiles%\VMware\Infrastructure\Virtual Infrastructure Client\Launcher\).
- Obtain a copy of %SystemRoot%\Microsoft.NET\Framework\v2.0.50727\System.dll from a non-Windows 7 machine that has .NET 3.5 SP1 installed. Copy this file into the folder created in Step 1.
- In the vSphere client launcher directory, open the VpxClient.exe.config file in a text editor and add a <runtime> element and a <developmentMode> element within the <configuration> element. Save the file.
- <?xml version="1.0" encoding="utf-8"?>
<configuration>
...
<runtime>
<developmentMode developerInstallation="true"/>
</runtime>
</configuration>
- Create a batch file (e.g. VpxClient.cmd) in a suitable location (e.g. Desktop). In this file add a command to set the DEVPATH environment variable to the folder where you copied the System.dll assembly in step 2 and a second command to launch the vSphere client. Save the file.
- SET DEVPATH=%ProgramFiles%\VMware\Infrastructure\Virtual Infrastructure Client\Launcher\Lib
"%ProgramFiles%\VMware\Infrastructure\Virtual Infrastructure Client\Launcher\VpxClient.exe"
You can now use the VpxClient.cmd (or the shortcut) to launch the vSphere client in Windows 7.
This workaround bypasses the normal .NET Framework loading mechanism. Assembly versions found in the DEVPATH folder are not checked.