Post date: 22-May-2009 14:38:11
In one of our Data center which was being used for some Customers, there was a problem on SAN level with some disks went missing
due to some Human Error done on Storage level.
Resulting this some disks went missing which were being used by few ASM diskgroups for Production and Quality Assurance systems.
As there was redundancy set up on Oracle level provided by ASM, the data was still available on another failuregroups and the DB
was alive on all the cluster nodes as both instances were surviving.However the disks were in HUNG state and mount status was MISSING
Finally the problem was solved by addition of disks to existing failure groups and the disks which went missing were dropped
automatically as soon as ASM identified the availability of new disks to failure group and rebalanced all the extents available
from the disks with MISSING status to the newly added and existing disks which were fine.
Following is the simulated excerpt from one of my test servers to show how we handled this situation.