Hi,
I've been attempting to assist someone rebuild a failed RAID5 VDISK in a 2 enclosure MSA 2321FC system. Sadly when a new drive is inserted the rebuild runs for some time and eventually fails. When I examine the logs I can see the reconstruction has failed as below
B9320 2019-05-30 12:45:55 44 W B Unwritable cache data exists for a volume. (volume: , SN: 00c0ffd811c500003261c44a01000000) It comprises 1% of cache space.
B9321 2019-05-30 12:45:55 44 W B Unwritable cache data exists for a volume. (volume: , SN: 00c0ffd811c500003261c44a02000000) It comprises 1% of cache space.
B9322 2019-05-30 12:45:55 44 W B Unwritable cache data exists for a volume. (volume: , SN: 00c0ffd811c500003261c44a03000000) It comprises 1% of cache space.
B9323 2019-05-30 12:45:55 44 W B Unwritable cache data exists for a volume. (volume: , SN: 00c0ffd811c500003261c44a04000000) It comprises 1% of cache space.
B9324 2019-05-30 12:45:55 44 W B Unwritable cache data exists for a volume. (volume: , SN: 00c0ffd811c500003261c44a05000000) It comprises 1% of cache space.
B9325 2019-05-30 12:45:55 18 W B Vdisk reconstruct failed. Command failed. (error code: 1). (vdisk: VDISKAA SN: 00c0ffd811c50000c358c34a00000000)
B9326 2019-05-30 12:45:55 172 W B A vdisk was quarantined. (vdisk: VDISKAA SN: 00c0ffd811c50000c358c34a00000000)
B9341 2019-05-31 11:50:37 44 W B Unwritable cache data exists for a volume. (volume: , SN: 00c0ffd811c500003261c44a01000000) It comprises 1% of cache space.
B9342 2019-05-31 11:50:37 44 W B Unwritable cache data exists for a volume. (volume: , SN: 00c0ffd811c500003261c44a02000000) It comprises 1% of cache space.
B9343 2019-05-31 11:50:37 44 W B Unwritable cache data exists for a volume. (volume: , SN: 00c0ffd811c500003261c44a03000000) It comprises 1% of cache space.
B9344 2019-05-31 11:50:37 44 W B Unwritable cache data exists for a volume. (volume: , SN: 00c0ffd811c500003261c44a04000000) It comprises 1% of cache space.
B9345 2019-05-31 11:50:37 44 W B Unwritable cache data exists for a volume. (volume: , SN: 00c0ffd811c500003261c44a05000000) It comprises 1% of cache space.
B9346 2019-05-31 11:50:38 172 W B A vdisk was quarantined. (vdisk: VDISKAA, SN: 00c0ffd811c50000c358c34a00000000)
B9347 2019-05-31 11:50:38 18 W B Vdisk reconstruct failed. Command failed. (error code: 1). (vdisk: VDISKAA, SN: 00c0ffd811c50000c358c34a00000000)
It seems I end up with this unwritable cache data error everytime this drive is replaced. It's always the same drive at the same slot.
I'm a bit stumped why this may be occurring. I've tried clearing the cache, clearing the metadata on the newly inserted drive and repeating the process of reconstruction again (with a couple of different replacement drives) but the outcome is always the same.
Details of the volume referred to above taken from the log dump.
<OBJECT basetype="volumes" name="volume" oid="157" format="rows">
<PROPERTY name="virtual-disk-name">VDISKAA</PROPERTY>
<PROPERTY name="volume-name">VDISKAA_v002</PROPERTY>
<PROPERTY name="size" units="GB">998.9GB</PROPERTY>
<PROPERTY name="size-numeric">1951171872</PROPERTY>
<PROPERTY name="preferred-owner">B</PROPERTY>
<PROPERTY name="preferred-owner-numeric">0</PROPERTY>
<PROPERTY name="owner">B</PROPERTY>
<PROPERTY name="owner-numeric">0</PROPERTY>
<PROPERTY name="serial-number" key="true">00c0ffd811c500003261c44a02000000</PROPERTY>
<PROPERTY name="write-policy">write-back</PROPERTY>
<PROPERTY name="write-policy-numeric">1</PROPERTY>
<PROPERTY name="cache-optimization">standard</PROPERTY>
<PROPERTY name="cache-optimization-numeric">0</PROPERTY>
<PROPERTY name="read-ahead-size">Default</PROPERTY>
<PROPERTY name="read-ahead-size-numeric">-1</PROPERTY>
<PROPERTY name="volume-type">standard</PROPERTY>
<PROPERTY name="volume-type-numeric">0</PROPERTY>
<PROPERTY name="volume-class">standard</PROPERTY>
<PROPERTY name="volume-class-numeric">0</PROPERTY>
<PROPERTY name="blocks" blocksize="512">1951171872</PROPERTY>
<PROPERTY name="volume-parent"></PROPERTY>
<PROPERTY name="snap-pool"></PROPERTY>
<PROPERTY name="virtual-disk-serial">00c0ffd811c50000c358c34a00000000</PROPERTY>
<PROPERTY name="volume-description"></PROPERTY>
<PROPERTY name="progress">0%</PROPERTY>
I am wondering if other drives which are part of this VDISK are also failing however I could not see anything from the log dump to indicate this during the reconstruction.
Firmware is M112R14 which I am aware is deprecated. I know there is an Oct 2013 firmware available (M114P01-01) which I may suggest they update to,
Has anyone ever seen this behaviour? Is there a workaround or fix? We are now considering deleting the entire VDISK, rebuilding from scratch and restoring to see if that will resolve it.
Any advice appreciated.
thanks
Ad