Archive

Archive for the ‘Uncategorized’ Category

How to migrate and convert physical RDMs

March 21, 2011 1 comment

Ken Gottleib, the Principle Virtualization Consultant at Winchester Systems, Inc. USA recently posted the following very useful bit of info over on the VMWare communities site.

I quickly dropped him a note and asked for permission to duplicate his work, as I have had this same issue before and never documented the process properly.

"This is what you need to do, fully explained the way each and every post should be. You must power the VM down to do this, there is no way to do it live. Period

1) log into ESX console as root
2) navigate to your virtual machine folder: /vmfs/volumes/vmfs-volume-freindlyname/vm-folder/
3) locate the pointer file for your disk, this is not the -rdmp file, this is the servername_1.vmdk, it is only a few KBs, inside it does nothing but point to the actual disk file, which in this case will be the servername_1-rdmp.vmdk. It is this servername_1-rdmp.vmdk file that maps IO to the actual lun which is always represented by ESX as something like vml.xxxxxxxxxxxxxxx and can be viewed in /vmfs/devices/disks/

The source you are looking to migrate (with this method it is actually a copy which is good, because the source is always left intact) can be a virtual disk, virtual rdm, or physical rrdm, it doesnt’ matter, you will always use the servername_x.vmdk file as the source in the command string you are about to see… but before I continune, if you want to migrate from physical compatibility mode lun to a new physical compatibility mode lun, be sure to have provisioned the lun so ESX can see it. If you are to convert the physical mode RDM to virtual RDM, the same applies. If you are to convert it to virtual disk, just be sure you have a vmfs volume with enough space to accomodate the new disk.

4) here it is:
To migrate and convert physical RDM to virtual disk:
vmkfstools -i servername_1.vmdk /vmfs/volumes/vm folder/servername-newdisk_1.vmdk
To migrate and convert physica RDM to virtual RDM, there are two methods
– in vi client edit VM settings of powered down VM, remove rdmp mapped disk, then re-add a new hard disk selecting the same disk only be sure to select virtual as the mode, you will log in and see all of your data and mount points are unchanged.
– from the ESX prompt in the virtual machine folder on the VMFS volume:
vmkfstools -i servername_1.vmdk -d rdm:/vmfs/devices/disks/vml.xxxxxxxxxxxxxxxxxxxxxxxx /vmfs/volumes/vm folder/servername_1-rdm.vmdk
To migrate physica RDM to a new physical RDM:
vmkfstools -i servername_1.vmdk -d rdmp:/vmfs/devices/disks/vml.xxxxxxxxxxxxxxxxxxxx /vmfs/volumes/vm folder/servername_1-rdmp.vmdk

Command strings EXPLAINED:
vmkfstools -i is the import \ clone command
the servername_x.vmdk is the source, the x will stand for the order of the device, in my example I use 1 which means this is the second disk on the VM. the first disk file is servername.vmdk, this is typically always the system disk.
the -d is the disk type for the destination you are converting to
the rdm(p) is part of the disk type telling the destination disk what the raw disk mode is, virtual or physical (the p means physical obviously)
the /vmfs/volumes/vm folder/servername_1-rdm.vmdk tells us the location of where to store the raw disks mapping file, I have chosen to put it in the virtual machine folder as you can see, you can put it on any vmfs file system, I like to keep things neat and clean.

Figuring out which vml.xxxxxxxxxxxxxxx is tricky.. I the ls-lh command on the /vmfs/devices/disks/ directory to list the unfriendly volume names out and figure out which one is mine by the LUN size, there are other ways to do this, there is a VMware KB article that describes other ways to dump out this information. and in some cases, depending, there may be a partition involved and it would look like vml.xxxxxxxxxxxxxxxx:1 or :2 or whatever.. vmware’s KB references it as vml.xxxxxxxxxxxx:p
you can also use vmhbax:x:x: in the string as such: /vmfs/devices/disks/vmhbax:x:x:p to represent the raw LUN
check out this article, it has what you need, but not everything.. the example they use for the disk is naa.xxxxxxxxxxxxxxxxx and I believe this is because there is different storage in use then I have… SO, be sure to navigate to /vmfs/devices/disks/ to see how vmware lists out the disks as the storage presents them

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=3443266

Hope you find this useful, don’t let anyone tell you that you can’t migrate a physical RDM. Its bogus information. This is how its done."

Categories: Uncategorized

Using the Vyatta VM appliance to simulate private networks / identify firewall port requirements and emulate routers.

February 21, 2011 Leave a comment

VMware is an amazing tool for emulating physical Firewalls, Routers, DHCP servers. It is especially useful for helping identify port requirements of various applications and tools.
Quite often, as a consultant, people ask you to implement some new product and expect you to provide all port requirements for the product, so that relevant firewall rules etc can be created – but they won’t allow you to drop anything like Wireshark on their live network.

My normal solution is to create my own ‘Private’ network on an ESX host (which could be your mobile lab)
This allows me to isolate traffic behind an ‘appliance’ firewall / router and if I like, drop a VM on that private network, to do port capture etc.

Of course, this is also a great tool for simulating routing / firewalls in your home lab, providing DHCP and so on.

In the following example, I’ll use Vyatta to build a Private network and then do some port monitoring.

First of all, we need to import the Vyatta appliance

Importing appliance
Download thje applicance form VMware Virtual Appliance store, or Vyatta’s website (you need the ESX / ESXi version)
Import Vyatta OVF (In VC, go to File – Deploy OVF template) – Follow the wizard.

Boot Vyatta (2vNics required – ideally on different vSwitches)
You’ll want one Nic attached to the public / office network and one to your new private network.
in this case, eth0 is public, eth1 is private.

We’ll make the private network be 172.16.0.0/24 with s default gateway of 172.16.1.2(eth1 on the Vyatta)
The public side of the Vyatta will be connected to Subnet 10.0.224.0/20 and will have a gateway of 10.0.239.251.
eth0 on that network will be 10.0.226.72 – this address will be an endpoint for that network, unless traffic is coming from within the isolate Private network we create – we’ll create some Nat rules to allow the Private Network to get to the public network

OK, once you have imported the Vyatta, you need to connect the 2 vNics to the relevant port groups.
The first will need to be connected to a port group that has access to the 10.0.224.0/20 network
The second can be connected to a portgroup that has been created on a vSwitch that is created on a pNic that does not even have network connectivity.

Defaul Login to vyatta is u: vyatta p:vyatta

So Let’s Configure the Vyatta

remember . . IP interfaces on Vyatta as follows

eth0 : 10.0.226.72/20, gw:10.0.239.251
eth1 : 172.16.1.2, gw: none

To enter edit mode

# >configure

assign IP / SM to eth0

# >set interfaces ethernet eth0 address 10.0.226.72/20

assign IP / SM to eth1

# >set interfaces ethernet eth1 address 172.16.1.2/16

assign default gateway for Live Network

# >set system gateway-address 10.0.239.251

commit changes

# >commit

save changes

# >save

At this point, we need to configure NAT rules as the ‘Live Network’ does not know anything about our Private network, so traffic can not route back.
These will basically tell the router to ‘disguise’ traffic so tha tthe source address is the public side of the router – so a known address on the network.

# >set service nat rule 1 source address 172.16.0.0/16
# >set service nat rule 1 outbound-interface eth0
# >set service nat rule 1 type masquerade
# >commit
# >save
# >exit

You can now route from the 172.16.0.0/16 network to any other network that is attached. – if not, commit changes, save and reboot.
Of course, it may well be that you do not want this to happen, so before dropping any VMs on the new network, you’d want to enable the Firewall(I’ll do that in a bit)

If you would like the Vyatta to provide IPs on the inside of the Private network, you can do so as follows:
Configuring DHCP from the Vyatta to give IP address to the 172.16.0.0/24 network

Enable DHCP Service (must be in edit mode)
go to edit Mode

# >configure

Enable DHCP

# >set service dhcp-server shared-network-name ETH1_POOL subnet 172.16.0.0/16 start 172.16.1.10 stop 172.16.1.250
# >set service dhcp-server shared-network-name ETH1_POOL subnet 172.16.0.0/16 default-router 172.16.1.2
# >set service dhcp-server shared-network-name ETH1_POOL subnet 172.16.0.0/16 dns-server 10.1.100.1
# >commit
# >save

At this point, a machine on the Private network should be able to boot and get an IP from the DHCP pool you have created, then access beyond the Vyatta box.

We now have an isolated network that has DHCP enabled and can send data requests to our Public network.
At this point, all traffic should route and no firewall rules etc should get in the way.

We said at the start that though, that we would like to use this tool to idetify port requirements etc.
Well, as the network is private, we could drop a VM on the Private network, running something like WireShark and it will catch all requests running on the network segment. Most importantly of all, it won’t get any of the usual broadcast traffic that you normally get when running Wireshark, as the only hosts on the segment are your VM and the Vyatta (which will not forward broadcasts)

If you have a second VM running the application that needs to get through the firewall, you simply need to track the requests coming form that host using Wireshark.

http://portforward.com/networking/wireshark.htm

As a bonus though, Vyatta also has a built in port monitor:

If you would like to monitor traffic on any ethernet interface, this can be done by issuing the following command

# show interfaces ethernet eth0 capture

To quit from the capture, type ‘q’

So, what we have done is created an isolated network, with limited traffic.

The next thing to do is enable the firewall on the Vyatta and only allow ports that your application requires.
Usually, I am a great fan of the Command line (automation is always the way forward as far as I am concerned) – but doing so for large numbers of rule can be quite arduous. Luckily, Adam over at http://www.arkf.net/blog/?p=217 has created an Excel spreasheet that generates everything we’ll need to create the relevant firewall rules (nice old school automation in itself)

to enable the firewall and block all traffic, you need ot create a rule as follows:

# >set firewall name eth1 rule 9999
# >set firewall name eth1 rule 9999 action drop
# >set firewall name eth1 rule 9999 description "Drop and log all others"
# >set firewall name eth1 rule 9999 log enable

This will again require to commit and save all changes.
We have purposely chosen the rule number as 9999, so if we create any further rules at lower numbers, they will override this one and allow the relevant bits of traffic.

If we enable the aobve rules now and then drop a Wireshark client on that network, we’ll be able to track all IP requests and calculate which ports / ips we’ll need to open

From here, the process is pretty simple, run your application on the private network, capture packets with Wireshark, create port rule (as per above) repeat process . . until your app starts to work – happy days.

For a quick scripted install, with a very similar config try:
http://www.sonoracomm.com/support/19-inet-support/233-vyatta-cable

Categories: Uncategorized

Script of the Day – remove duplicate lines in a CSV file

February 11, 2011 Leave a comment

Friday today, so we’ll keep it short and sweet.

Someone dropped me a CSV file recently and asked if there was a quick way to remove duplicates.

Easy . .

Sample source file:

Script:

$dest = @()
foreach ($row in (import-csv .\duplicates.csv)){if (!($dest -match $row)){$dest += $row}};
$dest

CDB_Cab                                   Computername                              FindIt_Cab
-------                                   ------------                              ----------
LAB SL07                               LAB9252                               Decommissioned
LAB SL07                               LAB9301                               SL 07
LAB SL07                               LAB9309                               SL 07
LAB SL07                               LAB9304                               SL 07
LAB SL07                               LAB9310                               SL 07
LAB SL07                               LAB9311                               SL 07
LAB SL07                               LAB9312                               SL 07

Pretty basic stuff, but he was getting caught up in the portion of the script that matches a row to a table.
thing to remember is that

if (!($dest -match $row))

is not the same as

if ($dest -notmatch $row)

We fixed that, but afterwards I showed him that there was perhaps a quicker, easier way:

</pre>
gc .\duplicates.csv | Get-Unique
"CDB_Cab","Computername","FindIt_Cab"
"LONDON SL07","loncmss9252","Decommissioned"
"LONDON SL07","loncmss9301","SL 07"
"LONDON SL07","loncmss9309","SL 07"
"LONDON SL07","loncmss9304","SL 07"
"LONDON SL07","loncmss9310","SL 07"
"LONDON SL07","loncmss9311","SL 07"
"LONDON SL07","loncmss9312","SL 07"
if (!($dest -match $row))

Just depends on the format of your file.

Interestingly, you’d assume that if ‘get-unique’ worked on the import using gc, it would work on an import using import-csv, but alas:

</pre>
PS:9 >import-csv .\duplicates.csv | get-Unique

CDB_Cab                                   Computername                              FindIt_Cab
-------                                   ------------                              ----------
LONDON SL07                               loncmss9252                               Decommissioned

Powershell can be a very strange beast sometimes

[tags Script of the day, Scripting, Powershell]
[category Scripting]

Categories: Uncategorized

Extending a Windows C:\ Dell EXTPart

January 26, 2011 Leave a comment

I get many requests to extend the C:\ (or any other drive)
Most times these are for VMs – so it is pretty easy right? Extend the amount of disk allocated to the virtual machine, then extend in Computer Management – or using diskpart?

Well this does not always work – see, Windows does not really like you messing with system drive.

Well, Dell have come up with a brilliant tool called EXTPart
http://support.dell.com/support/downloads/download.aspx?c=us&cs=19&l=en&s=dhs&releaseid=R64398&formatcnt=2&fileid=83929

All you need to do now is provision the extra space to the VM, then run the tool at the command line and follow the wizard:

C:\>extpart.exe
ExtPart - Utility to extend basic disks (Build 1.0.4)
(c) Dell Computer Corporation 2003

Volume to extend (drive letter or mount point): c:
Current volume size : 66285 MB (69504860160 bytes)
Current partition size : 76285 MB (79990815744 bytes)
Size to expand the volume (MB): 76285

that’s it – job done . . zero downtime (watch out of course . . this works differently if you have a clustered disk to extend – see:  https://geekseat.wordpress.com/2011/01/25/replacing-clustered-storage-for-a-sql-cluster-emc-ce-ms-clustering/ )

Building a custom Windows PE Image

January 25, 2011 Leave a comment

The first step in creating a customized Windows® PE 3.0 image is to modify the base Windows PE image (winpe.wim) by using the Deployment Image Servicing and Management (DISM) tool. DISM extracts the files to a local directory and enables you to add and remove packages (optional components and language packs). In addition, you can add out-of-box drivers. DISM provides the same mounting and unmounting operations as ImageX.

The general process for creating a custom Windows PE image includes:
1. Mount the base image by using the DISM tool to a local directory share. For example,

Dism /Mount-Wim /WimFile:C:\winpe_x86\winpe.wim /index:1 /MountDir:C:\winpe_x86\mount

2. Using the Dism command with the /Get-Package option to see which packages are installed. For example,

Dism /image:C:\winpe_x86\mount /Get-Packages

3. Add packages, and language packs as appropriate by using the Dism command with the /Add-Package option. For example, to add the HTA package you must add both the language neutral package along with the language specific package. For example:

Dism /image:C:\winpe_x86\mount /Add-Package /PackagePath:"C:\Program Files\<version>\Tools\PETools\x86\WinPE_FPs\WinPE-HTA.cab"
Dism /image:C:\winpe_x86\mount /Add-Package /PackagePath:"C:\Program Files\<version>\Tools\PETools\x86\WinPE_FPs\en-us\WinPE-HTA_en-us.cab"

Where <version> can be the OEM Preinstallation Kit (OPK) or the Automated Installation Kit (AIK).
4. Add drivers as appropriate by using the Dism command with the /Add-Driver option. For example:

Dism /image:C:\winpe_x86\mount /Add-Driver /driver:C:\test\drivers\mydriver.inf

5. Add any additional custom files or tools that you intend to include in the image within the \mount directory. For example, you can include ImageX within your image,

copy "C:\Program Files\<version>\Tools\x86\imagex.exe" C:\winpe_x86\mount\Windows\System32\

Where <version> can be Windows OPK or Windows AIK.
6. Commit the changes using the Dism command with the /Unmount-Wim /Commit option. For example,

Dism /Unmount-Wim /MountDir:C:\winpe_x86\mount /Commit

7. Copy your custom image into \ISO\sources folder and rename to boot.wim. For example,

copy c:\winpe_x86\winpe.wim c:\winpe_x86\ISO\sources\boot.wim
Categories: SCCM / SMS, Uncategorized Tags: , ,

SCCM Log file locations

January 25, 2011 Leave a comment

Operating System Deployment Log Files

The following table provides information about the log files and their default locations that are created when using Configuration Manager 2007 operating system deployment.

Log File Name Description
CCMSetup.log Provides information about client-based operating system actions.
Log file location:
%Windir%\System32\Ccmsetup
CreateTSMedia.log Provides information about task sequence media when it is created. This log is generated on the computer running the Configuration Manager 2007 administrator console.
Log file location:
<ConfigMgrInstallationPath>\AdminUI\AdminUILog
Dism.log Provides information about drivers installed during operating system deployment.
Configuration Manager 2007 SP2 installs drivers by using the Deployment Image Servicing and Management (DISM) tool in Windows Automated Installation (AIK) 2.0.
Log file location:
%Temp%\SMSTSLOG\Dism.log
DriverCatalog.log Provides information about device drivers that have been imported into the driver catalog.
Log file location:
<ConfigMgrInstallationPath>\Logs
MP_ClientIDManager.log Provides information about the Configuration Manager 2007 management point when it responds to Configuration Manager 2007 client ID requests from boot media or PXE. This log is generated on the Configuration Manager 2007 management point.
Log file location:
<ConfigMgrInstallationPath>\SMS_CCM\Logs
MP_DriverManager.log Provides information about the Configuration Manager 2007 management point when it responds to a request from the Auto Apply Driver task sequence action. This log is generated on the Configuration Manager 2007 management point.
Log file location:
<ConfigMgrInstallationPath>\SMS_CCM\Logs
MP_Location.log Provides information about the Configuration Manager 2007 management point when it responds to request state store or release state store requests from the state migration point. This log is generated on the Configuration Manager 2007 management point.
Log file location:
<ConfigMgrInstallationPath>\SMS_CCM\Logs
PkgMgr.log Provides information about drivers installed during operating system deployment.
Configuration Manager 2007 SP1 installs drivers by using the Package Manager tool.
Log file location:
%Temp%\SMSTSLOG\Pkgmgr.log
Pxecontrol.log Provides information about the PXE Control Manager.
Log file location:
<ConfigMgrInstallationPath>\Logs
PXEMsi.log Provides information about the PXE service point and is generated when the PXE service point site server has been created.
Log file location:
<ConfigMgrInstallationPath>\Logs
PXESetup.log Provides information about the PXE service point and is generated when the PXE service point site server has been created.
Log file location:
<ConfigMgrInstallationPath>\Logs
Setupact.log, Setupapi.log, Setuperr.log Provides information about Windows Sysprep and Setup logs.
Log file location:
%Windir%
SmpIsapi.log Provides information about the state migration point Configuration Manager 2007 client request responses.
Log file location:
<InstallationPath>\SMS_CCM\Logs
Smpmgr.log Provides information about the results of state migration point health checks and configuration changes.
Log file location:
<ConfigMgrInstallationPath>\Logs
SmpMSI.log Provides information about the state migration point and is generated when the state migration point site server has been created.
Log file location:
<ConfigMgrInstallationPath>\Logs
Smsprov.log Provides information about the SMS Provider.
Log file location:
<ConfigMgrInstallationPath>\Logs
Smspxe.log Provides information about the Configuration Manager 2007 PXE service point.
Log file location:
<ConfigMgrInstallationPath>\Sms_ccm\Logs
SMSSMPSetup.log Provides information about the state migration point and is generated when the state migration point site server has been created.
Log file location:
<ConfigMgrInstallationPath>\Logs
Smsts.log General location for all operating system deployment and task sequence log events.
Log file location:

  • If task sequence completes when running in the full operating system with an Configuration Manager 2007 client installed on the computer: <ConfigMgrInstallationPath>\Logs
  • If a task sequence is completed when running in the full operating system with no Configuration Manager 2007 client installed on the computer: %temp%\SMSTSLOG
  • If a task sequence is completed when running in Windows PE: <largest fixed partition>\SMSTSLOG
Note
<ConfigMgrInstallationPath> is %Windir%\System32\Ccm\Logs for most Configuration Manager 2007 clients and <ConfigMgrInstallationPath>\SMS_CCM for the Configuration Manager 2007 site server.
TaskSequenceProvider.log Provides information about task sequences when they are imported, exported, or edited.
Log file location:
<ConfigMgrInstallationPath>\Logs
USMT Log loadstate.log Provides information about the User State Migration Tool (USMT) regarding the restoring of user state data.
Log file location:
%Windir%\System32\Ccm\Logs
USMT Log scanstate.log Provides information about the User State Migration Tool (USMT) regarding the capture of user state data.
Log file location:
%Windir%\System32\Ccm\Logs
Categories: SCCM / SMS, Uncategorized Tags: , ,

News from Google – a restructure?

January 21, 2011 Leave a comment

News from over at Google: http://googleblog.blogspot.com/2011/01/update-from-chairman.html

the below text was posted by Eric Schmidt

"An update from the Chairman
1/20/2011 01:01:00 PM
When I joined Google in 2001 I never imagined—even in my wildest dreams—that we would get as far, as fast as we have today. Search has quite literally changed people’s lives—increasing the collective sum of the world’s knowledge and revolutionizing advertising in the process. And our emerging businesses—display, Android, YouTube and Chrome—are on fire. Of course, like any successful organization we’ve had our fair share of good luck, but the entire team—now over 24,000 Googlers globally—deserves most of the credit.

And as our results today show, the outlook is bright. But as Google has grown, managing the business has become more complicated. So Larry, Sergey and I have been talking for a long time about how best to simplify our management structure and speed up decision making—and over the holidays we decided now was the right moment to make some changes to the way we are structured.

For the last 10 years, we have all been equally involved in making decisions. This triumvirate approach has real benefits in terms of shared wisdom, and we will continue to discuss the big decisions among the three of us. But we have also agreed to clarify our individual roles so there’s clear responsibility and accountability at the top of the company.

Larry will now lead product development and technology strategy, his greatest strengths, and starting from April 4 he will take charge of our day-to-day operations as Google’s Chief Executive Officer. In this new role I know he will merge Google’s technology and business vision brilliantly. I am enormously proud of my last decade as CEO, and I am certain that the next 10 years under Larry will be even better! Larry, in my clear opinion, is ready to lead.

Sergey has decided to devote his time and energy to strategic projects, in particular working on new products. His title will be Co-Founder. He’s an innovator and entrepreneur to the core, and this role suits him perfectly.

As Executive Chairman, I will focus wherever I can add the greatest value: externally, on the deals, partnerships, customers and broader business relationships, government outreach and technology thought leadership that are increasingly important given Google’s global reach; and internally as an advisor to Larry and Sergey.

We are confident that this focus will serve Google and our users well in the future. Larry, Sergey and I have worked exceptionally closely together for over a decade—and we anticipate working together for a long time to come. As friends, co-workers and computer scientists we have a lot in common, most important of all a profound belief in the potential for technology to make the world a better place. We love Google—our people, our products and most of all the opportunity we have to improve the lives of millions of people around the world"

Categories: Uncategorized Tags: