Archive

Archive for the ‘Virtualisation’ Category

Free VMware training!!!

April 21, 2011 Leave a comment

Well . .some free training . . and no . . not VCDX or anything too exciting, but VMware have at least provided some online training / eBooks to help all those people who will soon need to transition to ESXi, from ESX (ESX of course will be end of lifed soon)

On VMware.com blogs:

Great news for all VMware customers: the VMware Education Services team has just made available a new, FREE elearning course dedicated to ESXi , “Transition to ESXi Essentials”. The course is a self-paced three-hour online training that provides the knowledge necessary to make fundamental design decisions to successfully add VMware ESXi to a vSphere environment and to take advantage of all of the new features included in ESXi 4.1. The training is ideal for system administrators, consultants and engineers responsible for managing and supporting a vSphere environment.


More here . . .

‘Upgrading’ from ESX to ESXi – a multipart series – Tools

March 2, 2011 Leave a comment

Setting up the test environment

In order to get this whole build tested, we need a repeatable and accessible Lab environment.

If you are unfortunate enough to not have an ESX lab environment that you can play on, you could build a workstation and emulate your production environment right on your desktop using VMware workstation.

Tools that we’ll require (so need to download if you do not have them already) are as follows:

PowerCli

· Installation guide http://www.vmware.com/support/developer/windowstoolkit/wintk40/doc/viwin_install.pdf

· Download : http://www.vmware.com/download/download.do?downloadGroup=sdkwin41u1 (you’ll need a free logon)

You favourite script editor

· I use PowerGui : http://www.powergui.org/index.jspa – but you can use anything you like

A VMware ESXi Server, or VMware workstation to run tests on, along with a copy of the ESXi Installable media

· https://www.vmware.com/tryvmware – you can get trials of both from here

Copies of the various tools we’ll be testing

· UDA (Ultimate Deployment Appliance) – http://www.ultimatedeployment.org/download.html

· EDA (ESX Deployment appliance) – http://virtuall.eu/downloads/

· VMware’s own ‘Auto Deploy’ – http://labs.vmware.com/flings/vmware-auto-deploy

· SD / USB duplication – WinImage – http://www.winimage.com/download.htm

· V-PXEServer – http://www.epic.ca/lab/V-PXEServer.zip

· Manual installation – Just the Install ISO above

Assuming that not everyone has spare hardware at their disposal, I guess it would be useful to create VMs to act as ESXi hardware on which to test our installations.

If you’re using VMWare workstation as your lab – Full guide at: http://www.vladan.fr/how-to-install-esxi-4-1-inside-of-vmware-workstation-7-1/

Quick Video:

If you are using an ESX(i) host to run your test ESXi VMs on, follow the guide at : http://www.vcritical.com/2009/05/vmware-esx-4-can-even-virtualize-itself/

For each appliance based deployment method, we’ll create a new VM – these will be detailed as we test each appliance

Categories: Virtualisation, VMWare

‘Upgrading’ from ESX to ESXi – a multipart series – Intro

March 2, 2011 Leave a comment

‘Upgrading’ from VMWare ESX 4.0 to ESXi 4.1(u1) – with as few button clicks as possible.

Anyone reading the official docs from VMware at : http://www.vmware.com/products/vsphere/esxi-and-esx/upgrade.html#c177045 is likely to be annoyed and frustrated at the lack of actual information as to how to manage the ‘upgrade’ (migration) from ESX to ESXi.

There is no direct way to upgrade directly as the 2 products are effectively 2 different operating systems that behave in exactly the same way. (well not really, but they are installed totally differently and are managed slightly differently)

Anyway, we have about 80 ESX hosts to migrate to ESXi, so I decided to find the easiest method to do so.

A scour of the web found several great resources. – and some valuable information. It seems that nobody has yet delivered a full ‘upgrade’ solution, but several people have provided automated deployments of the core installation and several other people have created configuration scripts for the newly installed ESX. So I have decided to add an additional ‘information gathering’ step, then ‘borrow’ some of the work done elsewhere and put together an ‘upgrade’ solution.

Scratching my head, I have been working on a simpler way to manage the deployment / upgrade.

The way I see it, the ‘upgrade’ needs 3 parts:

1) Capture configuration from the existing ESX host that we will be replacing (we may need to interpret and reformat it for our new ESXi Host

2) Deploy new ESXi instance on the hardware

3) Deploy captured config to our new ESXi host.

Whilst I realise that this probably will not be enough to make it a full ‘upgrade’ the idea is to get as much done as quickly as possible.

Coming up will be several posts, documenting comparisons of 5 methods of deployment, as well as some PowerCli code and a look into Host Profiles for easing this process.

On the off chance that one of the vendors that do ‘migration’ tools feel like offering me a free license to trial their tool and write up a process doc, I may even be inclined to review that for people with $$$ – but it is important that it is noted that I promise no allegiance to any tools, as would like this review to be fair and open to all. The end product of the series should provide a decision as to my preferred FREE method for doing the migration.

At the end of the series, I’ll create a comparison table comparing the various products, as a springboard for anyone who’ll soon be in the same situation as me.

For environments where there are only 2 or 3 ESX hosts, I’d consider building a solution like this going overboard, so I will make the assumption that readers of the series are people who have large numbers of hosts to upgrade and therefore will be wanting to work on ESX / ESXi rather than VMware workstation. As such, I will endeavour to get all tools working on ESXi and will highlight tweaks required (several of the tools I have tested so far were created on VMware workstation and therefore do not import directly to ESX hosts)

Products that I will be including in my review for the deployment include:

1. UDA (Ultimate Deployment Appliance) –

2. EDA (ESX Deployment appliance) – (0.90 in VMware appliance Marketplace, but 0.94 available at: )

3. VMware’s own ‘Auto Deploy’ –

4. SD / USB duplication

5. Manual installation (I will include this as a benchmark – remember ESXi requires very few clicks as is to get running, so manual may still be the way to go)

I will go through the options for the deployment of ESXi, before doing the capture and deploy steps above (I need a deployment workframe to work on, so it makes sense in this case)

All tests for now will be run in my isolated lab, with the convenient luxury of the deployment servers being located on the same subnets etc as the Hosts I’ll be deploying to, but once I have selected a solution and start the migration of 80 ESX hosts to ESXi, I will of course post details of any further tweaks required for deployment.

 

Categories: Virtualisation, VMWare Tags:

Monitoring VM Logs – Nearly real time monitoring – Script of the day

February 9, 2011 Leave a comment

Today’s Script of the day is not one of my own, but one I have used a few times in the past few weeks – and I figured I should post here as a reminder for when I next need it.

Written by the scripting wizard LucD, it is a tool that basicall ymonitors and displays VM logs . . almost as they happen.

http://www.lucd.info/2010/10/08/nearly-real-time-monitoring/

A great and very effective script for troubleshooting realtime VM issues.

ESX backup and monitoring for your home labs – FREE

February 8, 2011 Leave a comment

http://www.veeam.com/nfr/free-nfr-license

As a special holiday gift to VMware vExperts, VMware Certified Professionals and VMware Certified Instructors, Veeam is offering FREE two-socket licenses of their products for non-production use.

Just register here to receive the Veeam products of your choice for evaluation, demonstration and training purposes. You can choose one or both:

Veeam Backup & Replication v5 with vPower
Veeam ONE Solution

— Veeam Monitor Plus (with Veeam Reporter)
— Veeam Business View

Categories: Virtualisation Tags: , , ,

Enabling SSH on ESXi

February 8, 2011 Leave a comment

Info below copied from : http://www.vm-help.com

By default SSH is not enabled on ESXi – though every time you log a call with VMWare, the first thing that they do (of course) is ask you to enable it?

To enable SSH, do the following:

1) At the console of the ESXi host, press ALT-F1 to access the console window.
2) Enter unsupported in the console and then press Enter. You will not see the text you type in.
3) If you typed in unsupported correctly, you will see the Tech Support Mode warning and a password prompt. Enter the password for the root login.
4) You should then see the prompt of ~ #. Edit the file inetd.conf (enter the command vi /etc/inetd.conf).
5) Find the lines that begins with #ssh and remove the #. Then save the file. If you’re new to using vi, then move the cursor down to #ssh line and then press the Insert key. Move the cursor over one space and then hit backspace to delete the #. Then press ESC and type in :wq to save the file and exit vi. If you make a mistake, you can press the ESC key and then type it :q! to quit vi without saving the file. Note: there are two lines for SSH with ESXi 4.0 now – one for regular IP and the other for IPv6. You should
6) Once you’ve closed the vi editor, you can either restart the host or restart the inetd process. To restart inetd run ps | grep inetd to determine the process ID for the inetd process. The output of the command will be something like 1299 1299 busybox inetd, and the process ID is 1299. Then run kill -HUP <process_id> (kill -HUP 1299 in this example) and you’ll then be able to access the host via SSH.

Tip – with some applications like WinSCP, the default encryption cipher used is AES. If you change that to Blowfish you will likely see significantly faster transfers.

Changing the port for SSH

To change the port for SSH, edit the file /etc/services and change the SSH port listed in the file. Save the file and repeat step 6 above.

Enable Telnet

The steps are the same as with SSH, but you’ll remove the # from the 2 telnet entries in /etc/inetd.conf. Enabling telnet is not recommended if security is a concern.

Enable SSH access for a non-root account

Use the following process to enable SSH access for a non-root account
1) Access SSH or the console with a root account.
2) Create a new account with the command useradd <account_name> -M -d/ . This will set the home directory to / instead of requiring a /home directory.
3) Use the command passwd <account_name> to set the password for your new login.
4) Edit the passwd file with vi /etc/passwd. For the entry for your new account, change the /bin/sh part to /bin/ash. Save the file and exit. See the example for the test1 user below.

root:x:0:0:Administrator:/:/bin/ash
nobody:x:99:99:Nobody:/:/sbin/nologin
nfsnobody:x:65534:65534:Anonymous NFS User:/:/sbin/nologin
dcui:x:100:100:DCUI User:/:/sbin/nologin
daemon:x:2:2:daemon:/:/sbin/nologin
vimuser:x:12:20:vimuser:/sbin:/sbin/nologin
test1:x:500:500:Linux User,,,:/:/bin/ash

You should now be able to connect with SSH using this new account.

Disable SSH access for the root account

If you have created non-root accounts for SSH access you can also disable root access via SSH. Edit the /etc/inetd.conf file using the initial process on this page and add the option -w after the -i option. The line in inetd.conf will appear similar to the below.

ssh stream tcp nowait root /sbin/dropbearmulti dropbear ++min=0,swap,group=shell -i -w -K60

One you have made the change, save the file and run the kill -HUP command to restart the inetd process. You will now be able to login with a non-root account, but will get access denied if you use a root account. Once you have established a SSH session with your non-root account you can issue the command su – to switch to the root account.

Copying VMware Network config / Port Groups

February 7, 2011 Leave a comment

How often do you need to duplicate the port group config from one ESX host to another – easy if you can use Host Profiles . . but maybe you are not licensed for it?

I found a great (FREE . . the best type) tool for this:

http://www.vladan.fr/itq-vlan-and-portgroup-manager-free-utility/

The author is Flores Eken from ITQ Consultancy in the Netherlands. He is a VMware SDK programmer. He wrote this application in C# based on the new ESX3.x /VC2.x SDK, but it works in ESX 4

you can download it at :

http://www.run-virtual.com/files/ITQ-VI-Setup.msi

Script of the Day – import all VMs from a Datastore to VMWare ESX / ESXi

February 3, 2011 Leave a comment

So your DC fell over . . but you have a copy of all your vmdks etc and would like to import them to a new ESX host . .

The following script will run you through a series of prompt and then import all VMs from a DS to the specified ESX cluster


# Code by Alan van Wyk
# This script prompts for a Cluster and a Datastore and imports all VMs from the DS to the Cluster (to be used in DR emergencies etc)

# Simple function to tidy display at prompt screen
Function Selections () # Refresh screen at top of page, shows user selections
{
cls
cls
Write-Host ("Virtual Center: ") -ForegroundColor Blue -NoNewLine; Write-Host (" " + $vc)
Write-Host ("Destination Data Center: ") -ForegroundColor Blue -NoNewLine; Write-Host (" "  + $DC2)
Write-Host ("Destination Cluster: ") -ForegroundColor Blue -NoNewLine; Write-Host (" "  + $Cluster2)
Write-Host ("Destination Datastore: ") -ForegroundColor Blue -NoNewline; Write-Host (" "  + $Datastore2)
Write-Host
}
Function quit #quits script
{
exit
}
Function Pause ($Message="Press any key to continue...") #pause and wait for user to hit key to continue
{
Write-Host -NoNewLine $Message
$null = $Host.UI.RawUI.ReadKey("NoEcho,IncludeKeyDown")
Write-Host ""
}

$vc = Read-Host "Please specify your Virtual Center"
Write-Host "Connecting to Virtual Center... "
 Connect-VIServer $vc

selections
#Select DC
 $x = @(0) * 20 # initialize the array
 $i=0 # intialize the indexer
 Get-Datacenter `
 | select -first 20 `
 | % { $x[$i]=$_.Name; "{0} {1}" -f $i++, $_.Name } # store the DCs into an array and display the array
 Write-Host ("`n Please select Destination DataCenter from index above:`n") -ForegroundColor Yellow -BackgroundColor Red
 $index = read-host index # ask for an index
 $dc2 = $x[$index] # select index

 Selections
 # Select cluster
 $x = @(0) * 20 # initialize the array
 $i=0 # intialize the indexer
 Get-Datacenter $dc2    | Get-Cluster `
 | select -first 20 `
 | % { $x[$i]=$_.Name; "{0} {1}" -f $i++, $_.Name } # store the Clusters into an array and display the array
 Write-Host ("`n Please select Destination Cluster from index above:`n")    -ForegroundColor Yellow -BackgroundColor Red

 $index = read-host index # ask for an index
 $Cluster2 = $x[$index] # select index
 Selections

 $x = @(0) * 20 # initialize the array
 $i=0 # intialize the indexer
 get-cluster $cluster2 | get-vmhost | Get-Datastore `
 | select -first 20 `
 | % { $x[$i]=$_.Name; "{0} {1}" -f $i++, $_.Name } # store the Datastores into an array and display the array
 Write-Host ("`n Please select Destination Datastore from index above:`n")    -ForegroundColor Yellow -BackgroundColor Red
 $index = read-host index # ask for an index
 $Datastore2 = $x[$index] # select index
 Selections

 #Prompt for confirmation

 $confirmation = Read-Host "Type CONTINUE to confirm that you would like to import all VMs from $Datastore2 to $cluster2"
 If ($confirmation -cne "CONTINUE")
 {
 Write-Host ("Settings not confirmed - disconnecting from Virtual Center") -BackgroundColor Red -ForegroundColor Yellow
 disconnect-viserver -confirm:$false
 Pause
 Quit
 }

 $dsname = $Datastore2
 $datacenter = $dc2
 $cluster = $cluster2

#####################################################################################################
# Code below by LucD

$ESXname = Get-Cluster $cluster | Get-VMHost | select -First 1
$dsBrowser = Get-View (Get-View (Get-VMHost -Name $ESXname).ID).DatastoreBrowser
$folder = Get-View (Get-Datacenter -Name $datacenter | Get-Folder -Name "vm").ID
$pool = Get-View (Get-Cluster -Name $cluster | Get-ResourcePool -Name "Resources").ID
cls

foreach($dsImpl in $dsBrowser.Datastore){
 $ds = Get-View $dsImpl
 if($ds.Summary.Name -ne $dsname){continue}

 $datastorepath = "[" + $ds.Summary.Name + "]"

 $searchspec = New-Object VMware.Vim.HostDatastoreBrowserSearchSpec
 $searchSpec.matchpattern = "*.vmx"

 Write-Host "Searching in path" $datastorepath

 $task = Get-View ($dsBrowser.SearchDatastoreSubFolders_Task($datastorePath, $searchSpec))
 while ($task.Info.State -eq "running" -or $task.Info.State -eq "queued"){
 $task.UpdateViewData()
 sleep 5
 }
 if($task.info.result -ne $null){
 foreach ($file in $task.info.Result){

 if($file.FolderPath -match ".snapshot"){continue}
 $found = $FALSE
 foreach($vmx in $vms){
 if(($file.FolderPath + $file.File[0].Path) -eq $vmx){
 Write-Host "`tVM is registered"
 $found = $TRUE
 }
 }
 if (-not $found -and $task.Info.Result[0].File -ne $null){
 $vmx = $file.FolderPath + $file.File[0].Path
 $params = @($vmx,$null,$FALSE,$pool.MoRef,$null)
 Write-Host "Registering" $vmx
 $folder.GetType().GetMethod("RegisterVM_Task").Invoke($folder, $params)
 }

 }
 }
}

Accessing VMWare logs

February 3, 2011 Leave a comment

You can see ESX logs:

  • From the Service Console
  • From the vSphere Client connected directly to the ESX host (click Home > Administration > System Logs)
  • From the VMware Infrastructure Client connected directly to the ESX host (click Administration > System Logs)

The vmkernel logs (which log everything related to the kernel/core of the ESX) are located at /var/log/vmkernel.

The vmkwarning logs (which log warnings from the vmkernel) are located at /var/log/vmkwarning.

The vmksummary logs (which provide a summary of system activities such as uptime, downtime, reasons for downtime) are located at /var/log/vmksummary.

The hostd log (which is the log of the ESX management service of the ESX) are located at /var/log/vmware/hostd.log.

The messages log (which log activity on the Service Console operating system) is located at /var/log/messages.

The VirtualCenter Agent log is located at /var/log/vmware/vmware/vpx/vpxa.log.

The Automatic Availability Manager (AAM) logs are located at /var/log/vmware/aam/vmware_<hostname>-xxx.log.

The SW iSCSI logs are located at /var/log/vmkiscsid.log.

The System boot log is located at /var/log/boot-logs/sysboot.log.

 

More complete information available at: http://searchvmware.techtarget.com/tip/Manipulating-VMware-log-file-space-via-log-file-rotation-settings

ESXi host losing settings at reboot – checking system partitions of ESX host

February 2, 2011 Leave a comment

On 3 separate occasions I have seen ESX hosts that appeared to lose their config at reboot.
I stumbled across a useful doc once and copied some of it, but can not remember where I found it..

Anyway, in all instances, the problem was caused by the bootbank being corrupted and I followed the following process to resolve the issue.

You see, the way VMWare operates is that 3 Hypervisor partitions are created and used for normal operation. the are mounted as /bootbank, /altbootbank and /store.

/store is simply used to ‘store’ data (e.g. VMTools isos and VI client etc) as well as information for the vCenter Server agent and the HA agent

Once you configure the Scratchconfig.ConfiguredScratchLocation paramater for an ESX host (swap file) it will mount a 4th partition for this purpose.

Anyway, the first two partitions mentioned above are used for the ‘running’ config and ‘saved’ config . . very loosely similarly to the way in which a Cisco router stores 2 different configs.
What happens with VMware though is that the ‘running’ config
/bootbank and /altbootbank are effectively the running copy of the ESX firmware / config and the last saved version.

VMWare backs up its running config every hours (at the1 minute after the hour)

to see if you have been getting updated backups (so there is a failure in the process) you can check for a an up to date stage.tgz file.

At the ESXi hgost, hit  ALT+F1 / then type in ‘unsupported’, press enter and login with the root password)

Run
 ls -l /bootbank/

and check the timestamp of state.tgz (make sure this is 1 minute past an hour ago)

If this shows an old version, you could try force a backup:

/sbin/backup.sh 0 /bootbank/

Check again if  the timestamp on local.tgz gets updated

If not, we should try to fix corruption (if this is indeed the cause)

In order to identify the different paritions, you can use one of 2 methods:

~ # esxcfg-vmhbadevs -f
~ # ls -l | grep vmfs

what you need to do is simply identify which partitions are the /bootbank and /altbootbank partition, so that you can run a check.

to run a check on a partition, simply run dosfsck -v /dev/disks/<hba id>

e.g.
dosfsck -v /dev/disks/vmhba1:0:0:4

Of course, using this tool, you can perform various other checks, so I have included the contents of the man page below:

NAME
dosfsck - check and repair MS-DOS file systems
SYNOPSIS
dosfsck [-aAflnrtvVwy] [-d path -d ...] [-u path
-u ...] device
DESCRIPTION
dosfsck verifies the consistency of MS-DOS file systems and optionally
tries to repair them. The following file system problems can be cor-
rected (in this order):
- FAT contains invalid cluster numbers. Cluster is changed to EOF.
- File's cluster chain contains a loop. The loop is broken.
- Bad clusters (read errors). The clusters are marked bad and they
are removed from files owning them. This check is optional.
- Directories with a large number of bad entries (probably cor-
rupt). The directory can be dropped.
- Files . and .. are non-directories. They can be dropped or
renamed.
- Directories . and .. in root directory. They are dropped.
- Bad file names. They can be renamed.
- Duplicate directory entries. They can be dropped or renamed.
- Directories with non-zero size field. Size is set to zero.
- Directory . does not point to parent directory. The start
pointer is adjusted.
- Directory .. does not point to parent of parent directory. The
start pointer is adjusted.
- Start cluster number of a file is invalid. The file is trun-
cated.
- File contains bad or free clusters. The file is truncated.
- File's cluster chain is longer than indicated by the size
fields. The file is truncated.
- Two or more files share the same cluster(s). All but one of the
files are truncated. If the file being truncated is a directory
file that has already been read, the file system check is
restarted after truncation.
- File's cluster chain is shorter than indicated by the size
fields. The file is truncated.
- Clusters are marked as used but are not owned by a file. They
are marked as free.
Additionally, the following problems are detected, but not repaired:
- Invalid parameters in boot sector.
- Absence of . and .. entries in non-root directories
When dosfsck checks a file system, it accumulates all changes in memory
and performs them only after all checks are complete. This can be dis-
abled with the -w option.
OPTIONS
-a Automatically repair the file system. No user intervention is
necessary. Whenever there is more than one method to solve a
problem, the least destructive approach is used.
-A Use Atari variation of the MS-DOS filesystem. This is default if
dosfsck is run on an Atari, then this option turns off Atari
format. There are some minor differences in Atari format: Some
boot sector fields are interpreted slightly different, and the
special FAT entries for end-of-file and bad cluster can be dif-
ferent. Under MS-DOS 0xfff8 is used for EOF and Atari employs
0xffff by default, but both systems recognize all values from
0xfff8...0xffff as end-of-file. MS-DOS uses only 0xfff7 for bad
clusters, where on Atari values 0xfff0...0xfff7 are for this
purpose (but the standard value is still 0xfff7).
-d Drop the specified file. If more that one file with that name
exists, the first one is dropped.
-f Salvage unused cluster chains to files. By default, unused clus-
ters are added to the free disk space except in auto mode (-a).
-l List path names of files being processed.
-n No-operation mode: non-interactively check for errors, but don't
write anything to the filesystem.
-r Interactively repair the file system. The user is asked for
advice whenever there is more than one approach to fix an incon-
sistency.
-t Mark unreadable clusters as bad.
-u Try to undelete the specified file. dosfsck tries to allocate a
chain of contiguous unallocated clusters beginning with the
start cluster of the undeleted file.
-v Verbose mode. Generates slightly more output.
-V Perform a verification pass. The file system check is repeated
after the first run. The second pass should never report any
fixable errors. It may take considerably longer than the first
pass, because the first pass may have generated long list of
modifications that have to be scanned for each disk read.
-w Write changes to disk immediately.
-y Same as -a (automatically repair filesystem) for compatibility
with other fsck tools.
If -a and -r are absent, the file system is only checked, but not
repaired.
EXIT STATUS
0 No recoverable errors have been detected.
1 Recoverable errors have been detected or dosfsck has discovered
an internal inconsistency.
2 Usage error. dosfsck did not access the file system.
BUGS
Does not create . and .. files where necessary. Does not remove
entirely empty directories. Should give more diagnostic messages.
Undeleting files should use a more sophisticated algorithm.
AUTHORS
Werner Almesberger <werner.almesberger@lrc.di.epfl.ch> Extensions
(FAT32, VFAT) by and current maintainer: Roman Hodek <roman@hodek.net>