iSCSI virtual disks left behind by vss provider?

Hello all,

I have a Windows Server 2012 Failover cluster running a number of virtual machines. Storage for the cluster is provided by a Windows Storage Server 2012 instance and mapped to the nodes via iSCSI. In addition, I use DPM 2012 SP1 to protect the clustered VMs. The snapshot provider configuration was set up in accordance with the instructions and guidance at http://blogs.technet.com/b/filecab/archive/2012/10/08/iscsi-target-storage-vds-vss-provider.aspx . I had the same setup with Server 2008 R2, but recently upgraded to take advantage of the hardware snapshot provider and performance improvements claimed by Microsoft.

Since configuring the protection group for the clustered virtual machines, the group shows that all of the members are in an ideal state and there are no errors in the cluster event log. However, there are some issues in the event logs for the cluster nodes around the time the backups are being taken. This isn't the reason for this post, however, though it may be related.

While having a look at the iSCSI configuration on the storage server yesterday, I noticed the following...

Get-iSCSIVirtualDisk

ClusterGroupName   :
ComputerName       : myserver
Description        : Cluster Shared Volume
DiskType           : Fixed
HostVolumeId       : {67F71B49-BD09-457E-8E33-34607970BE1C}
LocalMountDeviceId :
OriginalPath       :
ParentPath         :
Path               : e:\csv.vhd
SerialNumber       : C4A45A4D-E7E0-4AEE-B800-403DD32E8A61
Size               : 8796093022208
SnapshotIds        : {{19C35EF1-C02D-49E4-9C38-B3AE500E2524}, {82FA960F-529E-4A41-A0B7-4F4981846051},
                     {E59C4532-5280-4465-B2EB-9FD31175F665}, {ED2F4661-2882-4C11-9B27-C946D6695219}...}
Status             : Connected
VirtualDiskIndex   : 1777681073

ClusterGroupName   :
ComputerName       : myserver
Description        : Snapshot of virtual disk 1777681073 taken at 12/8/2013 12:04:38 AM
DiskType           : ReadOnlySnapshot
HostVolumeId       : {67F71B49-BD09-457E-8E33-34607970BE1C}
LocalMountDeviceId :
OriginalPath       : e:\csv.vhd
ParentPath         :
Path               : \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy{2F56E973-560E-11E3-93F6-00259067F05B}\csv.vhd
SerialNumber       : C4A45A4D-E7E0-4AEE-B800-403DD32E8A61
Size               : 8796093022208
SnapshotIds        :
Status             : NotConnected
VirtualDiskIndex   : 1977840284

MANY others listed here...

ClusterGroupName   :
ComputerName       : myserver
Description        : Snapshot of virtual disk 1777681073 taken at 12/18/2013 1:03:52 AM
DiskType           : ReadOnlySnapshot
HostVolumeId       : {67F71B49-BD09-457E-8E33-34607970BE1C}
LocalMountDeviceId :
OriginalPath       : e:\csv.vhd
ParentPath         :
Path               : \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy{F1BD4A63-60EC-11E3-93F6-00259067F05B}\csv.vhd
SerialNumber       : C4A45A4D-E7E0-4AEE-B800-403DD32E8A61
Size               : 8796093022208
SnapshotIds        :
Status             : NotConnected
VirtualDiskIndex   : 2007241302

My searches for an explanation for this has, so far, turned up nothing.

When I do a "vssadmin list shadows" a whole bunch of crap is spit out as well:

vssadmin 1.1 - Volume Shadow Copy Service administrative command-line tool
(C) Copyright 2001-2012 Microsoft Corp.

Contents of shadow copy set ID: {2020c103-c81f-4c9f-a3e9-43ab496ba5b4}
   Contained 1 shadow copies at creation time: 11/29/2013 5:14:45 PM
      Shadow Copy ID: {09b6cd4f-28d2-4498-a132-3f9d6c716580}
         Original Volume: (E:)\\?\Volume{67f71b49-bd09-457e-8e33-34607970be1c}\
         Shadow Copy Volume: \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy1
         Originating Machine: myserver
         Service Machine: myserver
         Provider: 'Microsoft Software Shadow Copy provider 1.0'
         Type: DataVolumeRollback
         Attributes: Persistent, No auto release, No writers, Differential, Auto recovered

....

Contents of shadow copy set ID: {b3650eb5-ae1c-4ff1-92f0-90a33d0f82b1}
   Contained 1 shadow copies at creation time: 12/18/2013 12:34:08 AM
      Shadow Copy ID: {757b8c99-265a-4206-8a16-31b28597cf2f}
         Original Volume: (E:)\\?\Volume{67f71b49-bd09-457e-8e33-34607970be1c}\
         Shadow Copy Volume: \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy40
         Originating Machine: myserver
         Service Machine: myserver
         Provider: 'Microsoft Software Shadow Copy provider 1.0'
         Type: DataVolumeRollback
         Attributes: Persistent, No auto release, No writers, Differential, Auto recovered

Contents of shadow copy set ID: {4f58d81d-ebe1-48fe-a88b-17ff0b20757f}
   Contained 1 shadow copies at creation time: 12/18/2013 1:03:52 AM
      Shadow Copy ID: {318a9fb3-1476-4a41-98eb-00a60718cb3f}
         Original Volume: (E:)\\?\Volume{67f71b49-bd09-457e-8e33-34607970be1c}\
         Shadow Copy Volume: \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy41
         Originating Machine: myserver
         Service Machine: myserver
         Provider: 'Microsoft Software Shadow Copy provider 1.0'
         Type: DataVolumeRollback
         Attributes: Persistent, No auto release, No writers, Differential, Auto recovered

I can't find anything related in the system event log, but the timing of the majority of the creation times coincides somewhat with my backup schedule. Last night, there were 40 copies; this morning, there were 43. There are 3 virtual machines in the protection group, not sure if this is coincidence or not.

I was wondering if anyone could possibly enlighten me as to what exactly is going on here? Why are these snapshots appearing as iSCSI virtual disks? This looks like a potential cause for a major disaster...

Thanks for reading.

Greg