@JonPaul thanks for the response.

We had a session with support, and whilst the disks were able to be dequarantined by support, the disk groups could not be removed (something to do with them being part of a virtual pool)

The case was sent to secodn line support, and logs provided, however it has been determined that the deivce is out of support, so for now it is closed whilst I wait for quote for Time&Materials.

I understand the concerns around data loss, but surely with appropriate caveats that should be on the customer, not the vendor to gatekeep?

Hi,

I've a MSA2050 with a degraded disk not failed. I've 1 disk configured as a global spare. What is the right procedure to replace that disk? Is there an option to failed the disk?

@javier12312 
A degraded drive does not need immediate replacement but if the system is stating it is degraded then it has seen issues with the drive and suspects that it could soon fail.  When the issues become more serious the system will FAIL the drive and rebuild to a spare drive.

If you do want to pre-emptively replace the degraded drive.
First check to see if a disk-group scrub has completed on that disk-group recently.  This will let you know that an end to end read of the disk-group was successful and other drives are not experiencing issues.

  A. Make sure that you have a good backup -- this is just in case things go catastrophic during the rebuild
  B. Make sure you have an appropriate replacement drive
  C. Make sure the disk-group is not degraded for any other reason than this disk 
  D. Validate that your spare drive is in a slot that is acceptable to your preferrence -- MSA traditional RAID sets will rebuild to the SPARE slot and will not return to the original slot. If you want to keep your disk-group physically in the same slots with traditional RAID levels then unconfigure and remove the spare drive(s) and put the spare drive in the slot of the degraded drive (make sure to give >30 seconds from drive removal to new drive insertion) if you have dynamic spares enabled the drive should be pulled in to reconstruct in a couple minutes, if not then add the drive as a SPARE.
  E. Set the Drive identify LED on/enabled -- Gen5 and lower is a BLUE LED,  Gen6 is the AMBER LED on for 3 seconds off for 1 second
      If something doesn't look right with the placement of the LED on drive do a thorough review of where the drive is and what drive you think should be removed.
  F.  Remove the drive -- this is not recommended as it is intentionally degrading the fault tolerance of your disk-group
     If you didn't do a good job in step C you, and all your customers, will know right away as data access halts.
  G. Wait a few minutes and see if reconstruction starts 
Depending on a number of factors reconstruction can take a long time.  It would be best to reconstruct when there is low I/O to the system.  During the reconstruction your disk-group is degraded or Critical and your data is at risk.  Once you remove the drive there is no going back, a reconstruction must complete for your data to be fault tolerant again.  

If you have an MSA Gen6 system and are using the MSA-DP+ RAID level then reconstruction will happen internal to the disk-group.  A drive placed into the removed drive slot will automatically be pulled into the disk-group and rebalance will occur.
Information about this and other topics is available in the MSA 6th Gen Virtual Storage Technical Guide:  https://www.hpe.com/psnow/doc/a00103247enw

@mbrownpng 
I think that link was intending to go the MSA 2050 Best Practices Guide.  Try this link:  https://www.hpe.com/psnow/doc/a00015961enw

@gwijnants 
Is your Pool Overcommitted?
In the event that a Pool is overcommitted and the Pool High Threshold is exceeded the controller will go into WRITE-THROUGH mode to protect data in flight.  The high threshold is automatically set to 200GB or (400GB in IN210R004 firmware) below the pool capacity.
I can see from the below that the Pool Overcommit is enabled but only by adding up the volume capacity will you be able to determine if the Pool is overcommitted.
You can see some detail on this in the MSA 2060 Best Practices Guide:  https://www.hpe.com/psnow/doc/a00105260enw
Look for "Threshold"

Hi,

@JonPaulthank you, very clear your explanation. I'll monitor the disk and I'll replaced when the disk fail.

Do you recommend update MSA firmware with the degraded disk?

Regards

Hi JonPaul,

Thank you for the solution! Indeed the pool was overcommitted. After adding capacity the status went back to Active.

Best Regards,

Guy

Hello @gwijnants,

That's awesome! 

We are extremely glad to know the problem has been resolved. 

Does the MSA 2052 dual controllers - run by default in an ACTIVE-ACTIVE or ACTIVE-PASSIVE state ?

We are currently installing the MSA - the DL360 Gen10 has a single SN1100Q Dual port 16gb FC card, port 1 connected to controller A - port 2 connected to B,  we can only see 1 initiator in the MSA GUI HOSTS page ?  is this correct and does this mean the MSA is running ACTIVE-ACTIVE ?

disk 1.3 works -> insert by reseller when i buy the refurbished 6 month ago

disk 1.4 don't work -> send last month for upgrade pool

they have same part number 804170-001, same firmware version, same supplier part number and model number but the firt work's, the second no.

set cli-parameters json pager off
new APIData([
new APIStatus([
new APIProp({name:"response-type",value:"Success"}),
new APIProp({name:"response-type-numeric",value:"0"}),
new APIProp({name:"response",value:"Command completed successfully. - The settings were changed successfully. (2023-10-30 12:06:15)"}),
new APIProp({name:"return-code",value:"0"}),
new APIProp({name:"component-id",value:""}),
new APIProp({name:"time-stamp",value:"2023-10-30 12:06:15"}),
new APIProp({name:"time-stamp-numeric",value:"1698667575"})],
{basetype:"status",name:"",oid:"1"})
])
# show disks 1.3
new APIData([
new APIDrives([
new APIProp({name:"durable-id",value:"disk_01.03"}),
new APIProp({name:"enclosure-id",value:"1"}),
new APIProp({name:"drawer-id",value:"255"}),
new APIProp({name:"slot",value:"3"}),
new APIProp({name:"location",value:"1.3"}),
new APIProp({name:"port",value:"0"}),
new APIProp({name:"scsi-id",value:"2"}),
new APIProp({name:"blocks",value:"7501476528"}),
new APIProp({name:"serial-number",value:"A00EFDA1"}),
new APIProp({name:"vendor",value:"SanDisk"}),
new APIProp({name:"model",value:"DOPM3840S5xnNMRI"}),
new APIProp({name:"revision",value:"3P07"}),
new APIProp({name:"secondary-channel",value:"0"}),
new APIProp({name:"container-index",value:"0"}),
new APIProp({name:"member-index",value:"0"}),
new APIProp({name:"description",value:"SSD SAS"}),
new APIProp({name:"description-numeric",value:"8"}),
new APIProp({name:"architecture",value:"SSD"}),
new APIProp({name:"architecture-numeric",value:"0"}),
new APIProp({name:"interface",value:"SAS"}),
new APIProp({name:"interface-numeric",value:"0"}),
new APIProp({name:"single-ported",value:"Disabled"}),
new APIProp({name:"single-ported-numeric",value:"0"}),
new APIProp({name:"type",value:"sSAS"}),
new APIProp({name:"type-numeric",value:"8"}),
new APIProp({name:"usage",value:"AVAIL"}),
new APIProp({name:"usage-numeric",value:"0"}),
new APIProp({name:"job-running",value:""}),
new APIProp({name:"job-running-numeric",value:"0"}),
new APIProp({name:"state",value:"AVAIL"}),
new APIProp({name:"current-job-completion",value:""}),
new APIProp({name:"blink",value:"0"}),
new APIProp({name:"locator-led",value:"Off"}),
new APIProp({name:"locator-led-numeric",value:"0"}),
new APIProp({name:"speed",value:"0"}),
new APIProp({name:"smart",value:"Enabled"}),
new APIProp({name:"smart-numeric",value:"1"}),
new APIProp({name:"dual-port",value:"1"}),
new APIProp({name:"error",value:"0"}),
new APIProp({name:"fc-p1-channel",value:"0"}),
new APIProp({name:"fc-p1-device-id",value:"2"}),
new APIProp({name:"fc-p1-node-wwn",value:"5001173100C2E548"}),
new APIProp({name:"fc-p1-port-wwn",value:"0000000000000000"}),
new APIProp({name:"fc-p1-unit-number",value:"0"}),
new APIProp({name:"fc-p2-channel",value:"0"}),
new APIProp({name:"fc-p2-device-id",value:"0"}),
new APIProp({name:"fc-p2-node-wwn",value:""}),
new APIProp({name:"fc-p2-port-wwn",value:""}),
new APIProp({name:"fc-p2-unit-number",value:"0"}),
new APIProp({name:"drive-down-code",value:"0"}),
new APIProp({name:"owner",value:"A"}),
new APIProp({name:"owner-numeric",value:"1"}),
new APIProp({name:"index",value:"2"}),
new APIProp({name:"rpm",value:"0"}),
new APIProp({name:"size",value:"3840.7GB"}),
new APIProp({name:"size-numeric",value:"7501476528"}),
new APIProp({name:"sector-format",value:"Unknown"}),
new APIProp({name:"sector-format-numeric",value:"4"}),
new APIProp({name:"transfer-rate",value:"6.0"}),
new APIProp({name:"transfer-rate-numeric",value:"3"}),
new APIProp({name:"attributes",value:" "}),
new APIProp({name:"attributes-numeric",value:"2"}),
new APIProp({name:"enclosure-wwn",value:"500C0FF02672D03C"}),
new APIProp({name:"status",value:"Up"}),
new APIProp({name:"recon-state",value:"N/A"}),
new APIProp({name:"recon-state-numeric",value:"0"}),
new APIProp({name:"copyback-state",value:"N/A"}),
new APIProp({name:"copyback-state-numeric",value:"0"}),
new APIProp({name:"virtual-disk-serial",value:""}),
new APIProp({name:"disk-group",value:""}),
new APIProp({name:"storage-pool-name",value:""}),
new APIProp({name:"storage-tier",value:"N/A"}),
new APIProp({name:"storage-tier-numeric",value:"0"}),
new APIProp({name:"ssd-life-left",value:"94%"}),
new APIProp({name:"ssd-life-left-numeric",value:"94"}),
new APIProp({name:"led-status",value:" Online"}),
new APIProp({name:"led-status-numeric",value:"0"}),
new APIProp({name:"disk-dsd-count",value:"1"}),
new APIProp({name:"spun-down",value:"0"}),
new APIProp({name:"number-of-ios",value:"0"}),
new APIProp({name:"total-data-transferred",value:"0B"}),
new APIProp({name:"total-data-transferred-numeric",value:"0"}),
new APIProp({name:"avg-rsp-time",value:"0"}),
new APIProp({name:"fde-state",value:"Not FDE Capable"}),
new APIProp({name:"fde-state-numeric",value:"1"}),
new APIProp({name:"lock-key-id",value:"00000000"}),
new APIProp({name:"import-lock-key-id",value:"00000000"}),
new APIProp({name:"fde-config-time",value:"N/A"}),
new APIProp({name:"fde-config-time-numeric",value:"0"}),
new APIProp({name:"temperature",value:"22 C"}),
new APIProp({name:"temperature-numeric",value:"22"}),
new APIProp({name:"temperature-status",value:"OK"}),
new APIProp({name:"temperature-status-numeric",value:"1"}),
new APIProp({name:"pi-formatted",value:"No"}),
new APIProp({name:"pi-formatted-numeric",value:"0"}),
new APIProp({name:"power-on-hours",value:"45908"}),
new APIProp({name:"health",value:"OK"}),
new APIProp({name:"health-numeric",value:"0"}),
new APIProp({name:"health-reason",value:""}),
new APIProp({name:"health-recommendation",value:""})],
{basetype:"drives",name:"drive",oid:"1"}),
new APIStatus([
new APIProp({name:"response-type",value:"Info"}),
new APIProp({name:"response-type-numeric",value:"2"}),
new APIProp({name:"response",value:" * Rates may vary. This is normal behavior. (2023-10-30 12:06:22)"}),
new APIProp({name:"return-code",value:"0"}),
new APIProp({name:"component-id",value:""}),
new APIProp({name:"time-stamp",value:"2023-10-30 12:06:22"}),
new APIProp({name:"time-stamp-numeric",value:"1698667582"})],
{basetype:"status",name:"",oid:"2"}),
new APIStatus([
new APIProp({name:"response-type",value:"Success"}),
new APIProp({name:"response-type-numeric",value:"0"}),
new APIProp({name:"response",value:"Command completed successfully. (2023-10-30 12:06:22)"}),
new APIProp({name:"return-code",value:"0"}),
new APIProp({name:"component-id",value:""}),
new APIProp({name:"time-stamp",value:"2023-10-30 12:06:22"}),
new APIProp({name:"time-stamp-numeric",value:"1698667582"})],
{basetype:"status",name:"",oid:"3"})
])
# show disks 1.4
new APIData([
new APIDrives([
new APIProp({name:"durable-id",value:"disk_01.04"}),
new APIProp({name:"enclosure-id",value:"1"}),
new APIProp({name:"drawer-id",value:"255"}),
new APIProp({name:"slot",value:"4"}),
new APIProp({name:"location",value:"1.4"}),
new APIProp({name:"port",value:"0"}),
new APIProp({name:"scsi-id",value:"3"}),
new APIProp({name:"blocks",value:"7501476528"}),
new APIProp({name:"serial-number",value:"A003B9F4"}),
new APIProp({name:"vendor",value:"SanDisk"}),
new APIProp({name:"model",value:"DOPM3840S5xnNMRI"}),
new APIProp({name:"revision",value:"3P07"}),
new APIProp({name:"secondary-channel",value:"0"}),
new APIProp({name:"container-index",value:"0"}),
new APIProp({name:"member-index",value:"0"}),
new APIProp({name:"description",value:"SSD SAS"}),
new APIProp({name:"description-numeric",value:"8"}),
new APIProp({name:"architecture",value:"SSD"}),
new APIProp({name:"architecture-numeric",value:"0"}),
new APIProp({name:"interface",value:"SAS"}),
new APIProp({name:"interface-numeric",value:"0"}),
new APIProp({name:"single-ported",value:"Disabled"}),
new APIProp({name:"single-ported-numeric",value:"0"}),
new APIProp({name:"type",value:"sSAS"}),
new APIProp({name:"type-numeric",value:"8"}),
new APIProp({name:"usage",value:"UNUSABLE"}),
new APIProp({name:"usage-numeric",value:"8"}),
new APIProp({name:"job-running",value:""}),
new APIProp({name:"job-running-numeric",value:"0"}),
new APIProp({name:"state",value:"UNUSABLE"}),
new APIProp({name:"current-job-completion",value:""}),
new APIProp({name:"blink",value:"0"}),
new APIProp({name:"locator-led",value:"Off"}),
new APIProp({name:"locator-led-numeric",value:"0"}),
new APIProp({name:"speed",value:"0"}),
new APIProp({name:"smart",value:"Enabled"}),
new APIProp({name:"smart-numeric",value:"1"}),
new APIProp({name:"dual-port",value:"1"}),
new APIProp({name:"error",value:"0"}),
new APIProp({name:"fc-p1-channel",value:"0"}),
new APIProp({name:"fc-p1-device-id",value:"3"}),
new APIProp({name:"fc-p1-node-wwn",value:"50011731009E15D8"}),
new APIProp({name:"fc-p1-port-wwn",value:"0000000000000000"}),
new APIProp({name:"fc-p1-unit-number",value:"0"}),
new APIProp({name:"fc-p2-channel",value:"0"}),
new APIProp({name:"fc-p2-device-id",value:"0"}),
new APIProp({name:"fc-p2-node-wwn",value:""}),
new APIProp({name:"fc-p2-port-wwn",value:""}),
new APIProp({name:"fc-p2-unit-number",value:"0"}),
new APIProp({name:"drive-down-code",value:"0"}),
new APIProp({name:"owner",value:"A"}),
new APIProp({name:"owner-numeric",value:"1"}),
new APIProp({name:"index",value:"3"}),
new APIProp({name:"rpm",value:"0"}),
new APIProp({name:"size",value:"3840.7GB"}),
new APIProp({name:"size-numeric",value:"7501476528"}),
new APIProp({name:"sector-format",value:"Unknown"}),
new APIProp({name:"sector-format-numeric",value:"4"}),
new APIProp({name:"transfer-rate",value:"6.0"}),
new APIProp({name:"transfer-rate-numeric",value:"3"}),
new APIProp({name:"attributes",value:" "}),
new APIProp({name:"attributes-numeric",value:"2"}),
new APIProp({name:"enclosure-wwn",value:"500C0FF02672D03C"}),
new APIProp({name:"status",value:"Up"}),
new APIProp({name:"recon-state",value:"N/A"}),
new APIProp({name:"recon-state-numeric",value:"0"}),
new APIProp({name:"copyback-state",value:"N/A"}),
new APIProp({name:"copyback-state-numeric",value:"0"}),
new APIProp({name:"virtual-disk-serial",value:""}),
new APIProp({name:"disk-group",value:""}),
new APIProp({name:"storage-pool-name",value:""}),
new APIProp({name:"storage-tier",value:"N/A"}),
new APIProp({name:"storage-tier-numeric",value:"0"}),
new APIProp({name:"ssd-life-left",value:"97%"}),
new APIProp({name:"ssd-life-left-numeric",value:"97"}),
new APIProp({name:"led-status",value:" Online"}),
new APIProp({name:"led-status-numeric",value:"0"}),
new APIProp({name:"disk-dsd-count",value:"0"}),
new APIProp({name:"spun-down",value:"0"}),
new APIProp({name:"number-of-ios",value:"0"}),
new APIProp({name:"total-data-transferred",value:"0B"}),
new APIProp({name:"total-data-transferred-numeric",value:"0"}),
new APIProp({name:"avg-rsp-time",value:"0"}),
new APIProp({name:"fde-state",value:"Not FDE Capable"}),
new APIProp({name:"fde-state-numeric",value:"1"}),
new APIProp({name:"lock-key-id",value:"00000000"}),
new APIProp({name:"import-lock-key-id",value:"00000000"}),
new APIProp({name:"fde-config-time",value:"N/A"}),
new APIProp({name:"fde-config-time-numeric",value:"0"}),
new APIProp({name:"temperature",value:"0 C"}),
new APIProp({name:"temperature-numeric",value:"0"}),
new APIProp({name:"temperature-status",value:"Unavailable"}),
new APIProp({name:"temperature-status-numeric",value:"7"}),
new APIProp({name:"pi-formatted",value:"No"}),
new APIProp({name:"pi-formatted-numeric",value:"0"}),
new APIProp({name:"power-on-hours",value:"0"}),
new APIProp({name:"health",value:"Degraded"}),
new APIProp({name:"health-numeric",value:"1"}),
new APIProp({name:"health-reason",value:"Configuration error: non FDE disk is installed in an FDE secured system."}),
new APIProp({name:"health-recommendation",value:"Remove the disk.<br />- The system is FDE secured but the disk is not FDE-capable.<br />- The disk is unusable in this configuration.<br />- Remove or replace the disk with an FDE-capable disk."})],
{basetype:"drives",name:"drive",oid:"1"}),
new APIStatus([
new APIProp({name:"response-type",value:"Info"}),
new APIProp({name:"response-type-numeric",value:"2"}),
new APIProp({name:"response",value:" * Rates may vary. This is normal behavior. (2023-10-30 12:06:39)"}),
new APIProp({name:"return-code",value:"0"}),
new APIProp({name:"component-id",value:""}),
new APIProp({name:"time-stamp",value:"2023-10-30 12:06:39"}),
new APIProp({name:"time-stamp-numeric",value:"1698667599"})],
{basetype:"status",name:"",oid:"2"}),
new APIStatus([
new APIProp({name:"response-type",value:"Success"}),
new APIProp({name:"response-type-numeric",value:"0"}),
new APIProp({name:"response",value:"Command completed successfully. (2023-10-30 12:06:39)"}),
new APIProp({name:"return-code",value:"0"}),
new APIProp({name:"component-id",value:""}),
new APIProp({name:"time-stamp",value:"2023-10-30 12:06:39"}),
new APIProp({name:"time-stamp-numeric",value:"1698667599"})],
{basetype:"status",name:"",oid:"3"})
])
# set cli-parameters console pager on
Success: Command completed successfully. - The settings were changed successfully. (2023-10-30 12:06:54)

Hello @Staylor76 ,

In both write-back and write-through caching strategies, active-active failover of the controllers is enabled.

When ULP is in use, the controllers' operating/redundancy mode is shown as Active-Active. ULP appears to the host as an
active-active storage system where the host can choose any available path to access an LUN regardless of disk group
ownership.

Hence the default state is Active-Active.

I hope this is useful!

Thanks and Regards,

Sahana S.

Turns out that read cache provides no benefit for my workload (linux page cache is sufficient).  Write cache is what would be helpful.  No longer trying to configure SSD read cache...but not encouraged by need to use the cli to do this...in properly designed user interface it would be possible to perform all functions from the SMU.  Did not attempt the instructions that were provided as I don't need read cache.  Consider this case resolved.

Hi All,

how to calculate MSA 2060 DP+ Fault fault tolerance?
HPE said DP+ using 2HDD for sparing and 20.1% for overhead.

how many hard drives can fail simultaneously? And how many hard drives can fail in total without disrupting the RAID?

if total number of HDD are 12, 18 and 24 and how caculate it?  

Hello @James1234_12 ,

The fault tolerance is 2 drives at a single time, more drives can be lost when the rebuilds are complete.  In a 12-drive MSA-DP+ you can lose 4 drives over time without losing data (2 drives worth of SPARE capacity and the 2 drives of parity for RAID 6).  In larger disk groups you can lose more disks depending on the capacity of the disk group used.  

For more information, please refer to the below document.
MSA Virtual Storage Tech reference

I hope this helps!

Thanks and Regards,
Sahana S.

@Staylor76 
I don't think that your question was fully answered.  When connecting a dual port HBA to the MSA you should see 2 initiators.  I believe that there is some blockage that is not enabling your second path from the HBA to the MSA.
Things to check:
CLI>  show ports      {do you have link on a port on A and a port on B}
CLI>  show initiators    {You should be able to see the WWPN (World Wide Port Number) for each of your HBA ports}
if you are connected to an FC switch, you should be able to see both HBA ports and MSA Host Ports connected (not your case)
ACTIVE-ACTIVE vs ACTIVE-PASSIVE is an easy and complex answer.  The system is ACTIVE-ACTIVE but each Pool is owned by a controller and that controller's ports are OPTIMIZED for I/O.  The other controller's ports are UN-OPTIMIZED according to ALUA (Asynchronous Logical Unit Access) standards. If you only have 1 Pool AND your multipathing solution is setup correctly, you will only see activity on one controller until that controller is unavailable this will look like an ACTIVE-PASSIVE configuration.  If you have 2 Pools then I/O for the volumes on the A pool will go through the A controller ports and the volumes from the B pool will go through the B controller ports.  

@javier12312 
it is a best practice to keep your system and disk firmware up to date. There are checks within the system and in the Array Firmware Smart Component which will notify if an upgrade is not recommended at that time and what should be reviewed to successfully update.

For other best practice recommendation, please take advantage of the MSA HealthCheck:  https://www.hpe.com/storage/MSAHealthCheck

@JonPaul  - thank you.

using the show ports command showed us that the SFP on controler B had failed.

We replaced the SFP and now everything is working !! woooooohoooooo

thanks again.

Hi, sorry if this has been answered befor. We have a MSA2060 SAS and need to add a new shelf.

Can this be done "Hot", that is just pluggerd in while the stack it in service.

Also do I need to cosider the firmware versions on the the old and new shelves.

This is supporting storage for a vmware servers.

Thanks.

System recommended content:

1. Notice: HPE MSA Storage - New SKUs For HPE MSA 1060/2060/2062 and MSA 2060 Disk Enclosures

2. HPE MSA 2060 Storage Array - Overview

Please click on "Thumbs Up/Kudo" icon to give a "Kudo".

Thank you for being a HPE valuable community member.