TechOnTip Weblog

Run book for Technocrats

Archive for the ‘ServerVirtualisation’ Category

P2V Migration Gotcha 3: Unable to create a VSS snapshot of the source volumes. Error code 2147754776 (0x80042318)

Posted by Brajesh Panda on April 9, 2012

  • I was running P2V conversion of a Citrix server & below error came up;

    “Failed: Unable to create a VSS snapshot of the source volumes. Error code 2147754776 (0x80042318)”

  • Afterwards I have come across this issue multiple times & almost every time it resulted due to corrupted vss registry keys.

  • Logon to the source server and check vss providers & writers
    • Vssadmin list providers
    • It should show you out put like below

  • Vssadmin list providers
  • It should show you out put like below screenshot

  • If you are not getting about output, you have to delete the registry key & restart respective services; it will rebuilt the VSS configuration.
    • Click Start, click Run, type Regedit, and then click OK.
      • Locate and then click the following registry subkey:
      • HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\EventSystem\{26c409cc-ae86-11d1-b616-00805fc79216}\Subscriptions
      • On the Edit menu, click Delete, and then click Yes to confirm that you want to delete the subkey.
      • Exit Registry Editor.
      • Click Start, click Run, type services.msc, and then click OK.
    • Right-click the following services one at a time. For each service, click Restart:
      • COM+ Event System
      • COM+ System Application
      • Microsoft Software Shadow Copy Provider
      • Volume Shadow Copy

Here is a MS article how to fix it.

Posted in ServerVirtualisation | 7 Comments »

P2V Migration Gotcha 2: Event ID 6013 OLEDB Error, SQL Server doesn’t exist or access denied

Posted by Brajesh Panda on April 9, 2012

  • VMWare Converter uses Microsoft Software Shadow Copy Provider services to take VSS base snapshot of the source machine & transport the same to vSphere environment.
  • As you may know VSS comes with different application related VSS writers (like Exchange, SQL etc), if any of the VSS writer didn’t did their job properly converter will not able to do it’s conversion job.
  • Migration may not reported as failed in the converter console but it will be in hung state at “Creating a snapshot for source system” for quite some hours before you kill it with frustation.
  • In this case verify Application event log for any kind of VSS errors. Before you re-run the converter tool again make sure this VSS error is fixed.


  • Mostly you can disable the problamatic application & run the converter tool again. Here is a screenshot of SQL (MSDE) VSS error.
  • In this case I disabled MSDE service to make the converter run & after conversion we restarted the MSDE service again.

Posted in ServerVirtualisation | Leave a Comment »

P2V Migration Gotcha 1: x00007B Blue Screen Error while migrating Windows 2003 x64bit Server

Posted by Brajesh Panda on April 9, 2012

Lately I was migrating some Windows 2003 x64bit servers to virtual machines using vmware converter. After multiple attempts realised converter is failing at a certain stage; not completing the task. Even if we start the converted virtual machine, it shows blue screen error with x00007B error. This error refers to disk driver issues. Here are my findings about this issue!

  • Mostly all Windows 2003 x64bit servers do not come with LSI logic SCSI drivers in their driver folder. So if you migrate a Win2003x64 server, your migration may fail at VM reconfiguration phase of migration. And if you start the same VM, it will end up with x00007B Blue Screen of Death error!
  • Here are 3 possible reason for this kind of failure (mostly the 1st one)
    • Missing driver file
    • Missing registry keys for these drivers/devices (Registry Entry)
    • Missing start up services entry to load the driver & registy at the boot. (Registry Entry)
  • Missing Driver File
    • Copy symmpi.sys file to C:\Windows\system32\drivers folder of x64bit server.

       

  1. Missing registry keys for these drivers/devices (Registry Entry)

To edit/change registry keys you have to connect system drive VMDK to another live x64bit server & mount the registry hive to perform necessary action.

  • Shutdown broken VM & take copy backup of system drive (C Drive)
  • Connect the system drive to another x64bit working VM & make sure it is detected with a drive letter
  • In the work VM, Open registry, Click on HKLM, Select File & Load Hive, browse to the above drive letter\Windows\system32\Config, Double Click Syste, it will ask you a name, type BrokenSystem
  • Now you can browse HKLM hive of broken system
  • Expand BrokenSystem -> ControlSet001 -> Control -> CriticalDeviceDatatabase
  • Look for a key called pci#ven_1000&dev_0030,
  • If you have a key called pci#ven_1000&dev_0030, Select the same;
  • There should be two strings in pci#ven_1000&dev_0030
  • ClassGUID which has a value of {4D36E97B-E325-11CE-BFC1-08002BE10318}
  • Service which has a value of symmpi
  • If these two strings are either not present in pci#ven_1000&dev_0030 or contain different information, change them.
  • If you do not have a key called pci#ven_1000&dev_0030
  • RIGHT-CLICK on the LEFT PANE and click new -> key
  • key name is pci#ven_1000&dev_0030
  • RIGHT-CLICK in the RIGHT PANE and click new -> string value the name of this value is ClassGUID the value is {4D36E97B-E325-11CE-BFC1-08002BE10318}
  • RIGHT-CLICK in the RIGHT-PANE and click new -> string value the name of this value is Service the value is symmpi

     

  1. Configure “Missing start up services entry to load the driver & registy at the boot” following below process
    1. Shutdown broken VM & take copy backup of system drive (C Drive)
    2. Connect the system drive to another x64bit working VM & make sure it is detected with a drive letter
    3. In the work VM, Open registry, Click on HKLM, Select File & Load Hive, browse to the above drive letter\Windows\system32\Config, Double Click Syste, it will ask you a name, type BrokenSystem
    4. Now you can browse HKLM hive of broken system
    5. Expand BorkenSystem -> ControlSet001 -> Services; if you have a service called symmpi left-click on it Or if you do not have a service called symmpi RIGHT-CLICK on the LEFT-PANE and click new -> key
      1. Key name is symmpi
      2. f the symmpi key is setup properly you should have 7 keys in the right pane.
      3. (Default) (value not set) STRING
      4. ErrorControl (DECIMAL) 1 DWORD
      5. Group SCSI miniport STRING
      6. ImagePath system32\DRIVERS\symmpi.sys STRING
      7. Start 0 (DECIMAL) DWORD
      8. Tag 33 (DECIMAL) DWORD
      9. Type 1 (DECIMAL) DWORD
      10. If you have all of the keys in the right pane, most likely you will notice that the ImagePath key is missing and Start is set to 4 instead of 0.
      11. RIGHT-CLICK on the RIGHT PANE and select new -> String Value name = ImagePath value = system32\DRIVERS\symmpi.sys
      12. Double-Click on Start in the RIGHT-PANE, make sure DECIMAL is selected and change it to 0 if it is not 0 already.
      13. if you do not have all of the keys listed above create them using the method listed below.
      14. RIGHT-CLICK on the RIGHT PANE and select new -> type is either String Value or DWORD (see table above for types, names a values)
      15. Note: when entering DWORD values it helps to ensure that you have DECMIAL selected.

         

Make sure after you have added, adjusted or just verified the configuration of the symmpi service configuration take back the registry hive until you reach to SYSTEM2 and then unload it.

Posted in ServerVirtualisation | 9 Comments »

Hyper-V Virtual Machine Recovery

Posted by Brajesh Panda on August 15, 2010

  1. Scenario: 1

Best way for backup/restore/movement of Hype-V virtual machine is export & import process.

Export Process

  • Switch off the Virtual Machine
  • Right Click the Virtual Machine
  • Click On Export
  • Provide the path where you like to export the virtual machine

Import Process

  • In case you are importing to same Hyper-V server make sure you rename the old VM for as “-Old”. This is for only identification purpose
  • Open Hyper-V MMC Console
  • Right Click Hyper-V Server
  • Click Import New Virtual Machine
  • Provide Path to the folder where exported virtual machine is located
  • Select Move/Copy as per your scenario. If you are importing to same Hyper-V server, better to select generate a new SID for the virtual machine. It will avoid any conflict with old server

     

  1. Scenario: 2

Sometime we wrongly delete virtual machines from hyper-v console or if host OS get’s corrupted we found orphan virtual machines. These Orphan VMs can’t be directly imported to Hyper-V console. But no need to be panic… there are some manual steps by which you can register those VMs to the hyper-v console. But I think this is a unsupported process by Microsoft.

Note:

  • In case you would like to re-build/re-install the Hyper-V OS make sure none of the config file or snapshot files are in OS system drive. Else you will lose data at the time of formatting.
  • You may need same version Hyper-V instance to recover

How much you are lucky??

  • Do you have complete VM i.e. Virtual Machine Config File, All VHDs in a single location or you know the location where are they? – If not, it is worst!
  • Do you have multi snapshot virtual machines?? – If yes, it is worst. Hold your horses!

 

 

Disconnected VM Re-registration Process

If Virtual Machine VHD, AVHD & corresponding XML files are intact in the original location, No worries. Else, if you are coping/collecting/restoring all files to a single folder, you may have to edit XML files to mention actual VHD & AVHD paths.

  1. Note down system path to the virtual machine config file. Let suppose S:\BRAJESHTESTVM\
  2. Note down VM GUID name from the XML file. You can select the xml file & press F2, then Ctrl + C to copy the name. Let suppose “51CB2AF3-64D3-468A-863D-7E0C349F2261
  3. Open XML file & Verify VHD & AVHD are present as per mentioned locations
  4. In Windows Explorer open
  • C:\ProgramData\Microsoft\Windows\Hyper-V\Virtual Machines
  1. Right Click on Virtual Machines folder with Shift Key. Click on Start Command Prompt from here
  2. It will open up command prompt from the directory
  3. Create a symbolic link to the VM Config XML file using below command
  • C:\ProgramData\Microsoft\Windows\Hyper-V\Virtual Machines>mklink 51CB2AF3-64D3-468A-863D-7E0C349F2261.xmlS:\BRAJESHTESTVM\Virtual Machines\51CB2AF3-64D3-468A-863D-7E0C349F2261.xml
  • If syntax of the command is correct you will get a result like below

    Symbolic link created for 51CB2AF3-64D3-468A-863D-7E0C349F2261.xml <<===>> S:\BRAJESHTESTVM\Virtual Machines\51CB2AF3-64D3-468A-863D-7E0C349F2261.xml

  • Now you will able to see a symbolic link XML file inside the C:\ProgramData\Microsoft\Windows\Hyper-V\Virtual Machines folder
  • And also you will able to see the VM has been registered in the Hyper-V Console.
  • However if you try to start/switch this VM on you will get below error; as file access permissions are not configured correctly.

     

  1. In the same command prompt use below command line parameters to set right permissions. Over here we are configuring Full Control permissions, however you may configure respective granular permission. I would like to recommend configure Full Control & later on after recovery fine tune the permissions accordingly if you want.
  • C:\ProgramData\Microsoft\Windows\Hyper-V\Virtual Machines>icacls “C:\ProgramData \Microsoft\Windows\Hyper-V\Virtual Machines\51CB2AF3-64D3-468A-863D-7E0C349F2261.xml” /grant “NT VIRTUAL MACHINE\51CB2AF3-64D3-468A-863D-7E0C349F2261“:(F) /L
  • Observe the F & L at the end of the command, L means we are working on a symbolic link. F means full control permission. “NT VIRTUAL MACHINE\51CB2AF3-64D3-468A-863D-7E0C349F2261″ is the service SID which need permission to start the VM. It is a combination between “NT VIRTUAL MACHINE” & VM GUID.
  • If syntax of your command is correct it will result below output on your cmd prompt console processed file: C:\ProgramData\Microsoft\Windows\Hyper-V\Virtual Machines\51CB2AF3-64D3-468A-863D-7E0C349F2261.xml

    Successfully processed 1 files; Failed processing 0 files

  1. Before you start/switch on the Virtual Machine I would like to suggest to verify if this VM has any previous snapshots. If yes, you have to complete same procedure for each snapshot files before you switch on your VM.
    For each snapshot follow step 10 to 16. Else move to step 17!

Snapshot Registration Process

  1. In Original Virtual Machine data folder browse snapshot folder. Note down system path & GUID of the virtual machine snapshot config file. (If you have multiple snapshots you have to perform same task for all of them one by one). Let suppose we have one snapshot & it’s GUID is
    21B074A8-DA46-4FF1-A163-2E4B1551DCFF
  2. Open XML file & Verify VHD & AVHD are present as per mentioned locations
  3. In Windows Explorer open
  • C:\ProgramData\Microsoft\Windows\Hyper-V\Virtual Machines\Snapshots
  1. Right Click on Virtual Machines folder with Shift Key. Click on Start Command Prompt from here
  2. It will open up command prompt from the directory
  3. Create a symbolic link to the VM Config XML file using below command
  • C:\ProgramData\Microsoft\Windows\Hyper-V\Snapshots>mklink 21B074A8-DA46-4FF1-A163-2E4B1551DCFF.xmlS:\BRAJESHTESTVM\Snapshots\21B074A8-DA46-4FF1-A163-2E4B1551DCFF.xml
  • If syntax of the command is correct you will get a result like below

    Symbolic link created for 21B074A8-DA46-4FF1-A163-2E4B1551DCFF.xml <<===>> S:\BRAJESHTESTVM\Snapshots\21B074A8-DA46-4FF1-A163-2E4B1551DCFF.xml

  • Use below command syntax to provide proper permission to the VM GUID C:\ProgramData\Microsoft\Windows\Hyper-V\Snapshots>icacls “C:\ProgramData\Microsoft\Windows\Hyper-V\Snapshots\21B074A8-DA46-4FF1-A163-2E4B1551DCFF.xml” /grant “ NT VIRTUAL MACHINE\51CB2AF3-64D3-468A-863D-7E0C349F2261“:(F) /L
  • Remember NT VIRTUAL MACHINE\51CB2AF3-64D3-468A-863D-7E0C349F2261 is the VM GUID.
  • If syntax of above command is correct it will give below results in the command prompt

    Processed file: C:\ProgramData\Microsoft\Windows\Hyper-V\Snapshots\21B074A8-DA46-4FF1-A163-2E4B1551DCFF.xml

    Successfully processed 1 files; Failed processing 0 files

  1. From hyper-v console verify respective vm is showing reference to all snapshots & VHD config is pointing to the right VHD & AVHD files. If yes move to step 17 for next action.

Final Permission configuration

  1. Before you switch on the VM, make sure VM Service ID (GUID) has right permission on each VM files including VHDs. Use below syntax to configure the same.

    S:\>icacls
    S:\BRAJESHTESTVM\ /T /grant “NT VIRTUAL MACHINE\51CB2AF3-64D3-468A-863D-7E0C349F2261“:(F)

  • If syntax of above command is correct you will get below result in the command prompt

processed file: S:\BRAJESHTESTVM\

processed file: S:\BRAJESHTESTVM\BRAJESHTESTVM_C_Drive.vhd

processed file: S:\BRAJESHTESTVM\BRAJESHTESTVM_C_Drive_6DA4425D-5C8E-4247-9AB4-43A0F8F362DE.avhd

processed file: S:\BRAJESHTESTVM\BRAJESHTESTVM_D_Drive.vhd

processed file: S:\BRAJESHTESTVM\BRAJESHTESTVM_D_Drive_E52E811F-7946-4FAE-BA4F-7545C363EF73.avhd

processed file: S:\BRAJESHTESTVM\Snapshots

processed file: S:\BRAJESHTESTVM\Virtual Machines

processed file: S:\BRAJESHTESTVM\Snapshots\21B074A8-DA46-4FF1-A163-2E4B1551DCFF

processed file: S:\BRAJESHTESTVM\Snapshots\21B074A8-DA46-4FF1-A163-2E4B1551DCFF.xml

processed file: S:\BRAJESHTESTVM\Virtual Machines\51CB2AF3-64D3-468A-863D-7E0C349F2261

processed file: S:\BRAJESHTESTVM\Virtual Machines\51CB2AF3-64D3-468A-863D-7E0C349F2261.xml

processed file: S:\BRAJESHTESTVM\Virtual Machines\51CB2AF3-64D3-468A-863D-7E0C349F2261\51CB2AF3-64D3-468A-863D-7E0C349F2261.bin

processed file: S:\BRAJESHTESTVM\Virtual Machines\51CB2AF3-64D3-468A-863D-7E0C349F2261\51CB2AF3-64D3-468A-863D-7E0C349F2261.vsv

Successfully processed 13 files; Failed processing 0 files

  1. Let’s switch on the virtual machine; BANG, vm is not starting. It will throw below error

  • It is related to Virtual Ethernet Network Card. If you go to setting of virtual machine you will see “configuration error” for network cards.

  • Either connect them to a valid virtual network switch or configure them as not connected.
  • Now you should able to switch on your virtual machine.
  • After you start your virtual machine, you may need to clean up unused & hidden network cards to use same old IP address.

 

 

 

 

 

 

Posted in ServerVirtualisation | Tagged: | 2 Comments »

 
%d bloggers like this: