Embedded Files
References for AIX MPIO and SDDPCM multipath options with DS3000/DS4000/DS5000
While the RDAC driver is being deprecated with AIX, it is still the default driver on AIX 5.2 and 5.3 for DS4000 devices. With AIX 6.1, the default driver is MPIO for DS4000 devices. All future DS4000/DS5000 models will only be supported with MPIO. For DS3000 and DS5000 subsystems RDAC is no option. These subsystems are always configured as MPIO devices. The MPIO support for these devices removes the RDAC limitation of connecting only one adapter to each subsystem controller port (1:1 FC connection) and allows more than two paths to be configured to the device. This simplifies zoning and improves performance.
Overview of various multipath driver options for DS3000/DS4000/DS5000 subsystems:
DS3000: native MPIO only
DS4000: RDAC, native MPIO, MPIO w/SDDPCM
DS5000: native MPIO, MPIO w/SDDPCM
Note: RDAC support for AIX has been removed starting with 07.60.xx.xx firmware for DS4000 series. With controller firmware 7.60.xx or higher the AIX RDAC fail over driver on DS4000 storage can no longer work. There has been an inquiry change that will cause problems. Customers can continue to use 7.50.xx or lower levels of firmware on DS4000 series with RDAC or with MPIO but if they move to 7.60.xx or higher only MPIO will work. See "ibm_sw_ds4kfc1_10.60.xx.17_aix_anycpu.txt" which comes with the Storage Manager package for more details:
New limitations with Storage Manager version 10.60.xx.17 release (controller firmware 07.60.xx.xx):
DS4000 storage subsystems with controller firmware 7.60.xx or greater are supported only with the MPIO multipath driver on AIX. The fcp_array (RDAC) multipath driver support has been removed. To switch from fcp_array to MPIO consult your AIX documentation on the manage_disk_driver command.
For more details about prerequisites and supported OS and driver levels take a look at the particular interoperability matrices:
DS3000 family products interoperability matrix
http://www.ibm.com/systems/storage/disk/ds3000/pdf/interop.pdf
DS4000 family products interoperability matrix
http://www.ibm.com/systems/resources/systems_storage_disk_ds4000_pdf_interop-matrix.pdf
DS5000 family products interoperability matrix
http://www.ibm.com/systems/resources/systems_storage_disk_ds5000_interop-matrix.pdf
Always refer to the IBM System Storage Interoperation Center for latest and up-to-date information about supported environments and configurations, including supported multipath driver option:
IBM System Storage Interoperation Center (SSIC)
http://www.ibm.com/systems/support/storage/config/ssic
A compact overview of DS4000/DS5000 multipath options for AIX with RDAC, AIX MPIO and SDDPCM can now be found in the newly released IBM Redbook:
IBM Midrange System Storage Implementation and Best Practices Guide (SG24-6363-04, Chapter 12: DS5000 with AIX, PowerVM, and PowerHA)
http://www.redbooks.ibm.com/abstracts/sg246363.html?Open
General information about DS4000/DS5000 host attachment can be found in the
IBM System Storage DS Storage Manager - Installation and Host Support Guide
http://www.ibm.com/systems/support/supportsite.wss/docdisplay?lndocid=MIGR-5075652&brandind=5000028
(1) DS4000 and AIX MPIO
AIX MPIO is supported for DS4000 on AIX starting from AIX 5.2 TL10, AIX 5.3 TL6 and AIX 6.1 SP2 though various PTFs and fixes may be required (see interoperability matrix and AIX MPIO documentation below) on these earlier OS levels. It is supported with DS4000 firmware 6.60.xx.xx and higher on DS4800, DS4700, DS4200, DS4500 and DS4300. MPIO is the strategic driver for IBM midrange storage subsystem in AIX 6.1 and up - except for DS4400 which is RDAC only (note that also DS4100 is only listed with RDAC on the SSIC as supported configuration). RDAC is the default driver on all levels of AIX 5.2 and 5.3 for DS4000 devices while in AIX 6.1 the DS4000 products are configured as Multipath I/O (MPIO) devices by default. Please always check the IBM System Storage Interoperation Center (SSIC) for latest information about supported multipath drivers and configurations!
The first published references on how to migrate a DS4K product using the FCPARRAY (RDAC) driver to the MPIO driver, how to manage MPIO on AIX with DS4000 subsystems, the required DS4000 NVSRAM settings and the newly introduced manage_disk_driver and mpio_get_config commands in AIX can be found in the documentation below:
Procedures for AIX 6.1 and DS4000 Interoperability
http://www.ibm.com/systems/resources/systems_storage_disk_ds4000_pdf_aix.pdf (for reference only; unfortunately this link is no longer available)
http://w3.ibm.com/connections/files/app/file/e99a1584-8f81-4fea-af8b-fb9c7a20bc2d (IBM internal link to a copy of the document)
AIX61 TL02 Release Notes (p.19ff)
Please note that the information in these documents was released for early AIX 6.1 and DS4000 firmware releases starting to support AIX MPIO. The information in these documents is outdated for current AIX levels (e.g. AIX 6.1 TL4) and DS4000 firmware releases (V7.xx), for example, regarding the change of the DS4000 AIX HostNVSRAMBYTE at offset 0x27 and the manage_disk_drivers command options.
(2) DS4000 AIX HostNVSRAMBYTE at offset 0x27 for MPIO
Both documents above and even the latest AIX 6.1 TL5 Release Notes (p21ff) describe that the HostNVSRAMBYTE for host type 0x6 (which is AIX) at offset 0x27 has to be set to 0x0 on the DS4k subsystem when using AIX MPIO instead of RDAC as multipath driver by issuing the following SMcli commands to the DS4000 subsystem:
set controller [a] HostNVSRAMBYTE [6,0x27]=0x0;
set controller [b] HostNVSRAMBYTE [6,0x27]=0x0;
This was true for early DS4000 firmware releases prior to version 06.60.08 only!
Starting with DS4000 firmware release 06.60.08 and higher the factory default value of the AIX HostNVSRAMBYTE at offset changed to 0x8 which is fine for AIX MPIO. So before applying any change to the DS4000 NVSRAM for AIX MPIO first check if you are already at a firmware level with a default setting of 0x8 for host type 0x6 (AIX) at offset 0x27 by issuing the following SMcli script command:
show controller [a] HostNVSRAMBYTE [0x06,0x27];
show controller [b] HostNVSRAMBYTE [0x06,0x27];
If the output shows
with a default value of 0x8 then _no_ specific change of the DS4000 HostNVSRAMBYTE is required for AIX MPIO!
Please note, that the documents above describing the NVSRAM changes for AIX MPIO are not specific about the host type option required to be changed for AIX MPIO in order to work properly. A change of this HostNVSRAMBYTE effects several host type options each represented by a single bit setting. The DS4000 host type options related to offset 0x27 of the HostNVSRAMBYTE are listed in the table below:
The real requirement for AIX MPIO to work correctly is to have the RETAIN LOGINS option disabled which is controlled by bit 0 of the HostNVSRAMBYTE at offset 0x27. Disabling the RETAIN LOGINS option means setting bit 0 = 0 for the specific host type (here AIX = 0x6) at offset 0x27. This bit effects the fibre channel PLOGI behaviour. Setting this bit to 0 is required for AIX MPIO. If not set correctly and running with reservation_policy=single_path (which is the default for DS4k hdisk attributes on AIX with MPIO) a failover to another path may not work properly.
With a default value of the HostNVSRAMBYTE at offset 0x27 being 0x8 (8 = 0 x 2^0 + 1 x 2^3 meaning bit 0 = 0 and bit 3 = 1) the RETAIN LOGINS bit is already set to 0 (bit 0 = 0) and no specific change of the NVSRAM is required anymore for using DS4000 with AIX MPIO.
When having applied the NVSRAM change already as described in the AIX 6.1 TL2 (...TL5) Release Notes by setting the whole AIX NVSRAM host byte at offset 0x27 to 0x0still the RETAIN LOGINS bit is set correctly to 0 as required by AIX MPIO. But additionally also bit 3 is set to 0 (bit 3 = 0) compared to the default value of 0x8 (8 = 1 x 2^3 meaning bit 3 = 1). Bit 3 at offset 0x27 of the AIX host byte alters the behavior regarding SCSI-3 persistent reservations, which normally are not used in the AIX MPIO driver by default. The default for AIX MPIO is to use SCSI-2 reservations with algorithm=fail_over and reservation_policy=single_path, so typically a setting of 0x0 should not be of concern in most standard configurations unless persistent reservation policies are applied. When using persistent reservation policies it is recommended to change the AIX host byte in the DS4000 NVSRAM at offset 0x27 back to its factory default value of 0x8:
set controller [a] HostNVSRAMBYTE [0x06,0x27] = 0x8;
set controller [b] HostNVSRAMBYTE [0x06,0x27] = 0x8;
Note that a DS4000 controller reboot is required for any NVSRAM changes to take effect. A controller reboot can be initiated via the GUI (Select the controller and go to Advanced > Recovery > Reset Controller) or by using the additional SMcli script commands:
reset Controller [a];
reset Controller [b];
Also note that all applied changes to the default DS4000 NVSRAM will be lost with a DS4000 firmware update as the NVSRAM update (which is part of the firmware update) will reset all settings stored in the NVSRAM to their defaults. So if any changes to the NVSRAM settings have been applied make sure to check and - if required - re-apply them after a firmware update.
(3) Managing multipath driver options with AIX manage_disk_drivers command
The manage_disk_drivers command introduced in previous AIX releases up to AIX 6.1 TL3 displayed a list of DS4K storage models and the driver that manages each model. All disks of a storage model must be managed by the same driver. If you specify the -c flag, the command changes the driver selection to an alternate supported driver for the storage model. The previous release of the command only offers the option to switch between AIX MPIO and RDAC for selected DS4000 models using themanage_disk_drivers -c [1..5] command option.
# manage_disk_drivers
1: DS4100: currently MPIO; supported: RDAC/fcparray, MPIO
2: DS4300: currently MPIO; supported: RDAC/fcparray, MPIO
3: DS4500: currently MPIO; supported: RDAC/fcparray, MPIO
4: DS4700/DS4200: currently MPIO; supported: RDAC/fcparray, MPIO
5: DS4800: currently MPIO; supported: RDAC/fcparray, MPIO
With release of AIX 6.1 TL4 and VIOS 2.1.2.10 the manage_disk_drivers command changed substantially. It now offers a new syntax and more options to configure multipath drivers for XIV, DS3950, DS4100, DS4200, DS4300, DS4500, DS4700, DS4800, DS5100 (including DS5300) and DS5020 storage devices. With SDDPCM 2.5.1 (or higher) filesets installed it even allows to select the following AIX multipath driver options for DS4000 devices:
SDDPCM (displayed as AIX_SDDAPPCM for active/passive controller subsystems like DS4k/DS5k; note that SDDPCM version 2.5.1 or higher is required),
AIX native PCM (displayed as AIX_APPCM for active/passive controller subsystems like DS4k/DS5k) and
RDAC/AIX_FCP array driver (displayed as AIX_fcparray).
Below you can find an example of the manage_disk_drivers command and the available multipath driver options on AIX6.1 TL4 with SDDPCM 2.5.1.0 filesets installed:
# lslpp -Lc | grep -E "sdd|fcp.disk.ibm"
devices.fcp.disk.ibm.mpio:devices.fcp.disk.ibm.mpio.rte:1.0.0.20: : :C: :IBM MPIO FCP Disk Device: : : : : : :0:0:/:
devices.sddpcm.61:devices.sddpcm.61.rte:2.5.1.0: : :C: :IBM SDD PCM for AIX V61: : : : : : :0:0:/:
# manage_disk_drivers
Usage :
manage_disk_drivers [-l]
manage_disk_drivers -d device -o driver_option
manage_disk_drivers -h
# manage_disk_drivers -l
Device Present Driver Driver Options
2810XIV AIX_AAPCM AIX_AAPCM,AIX_non_MPIO
DS4100 AIX_SDDAPPCM AIX_APPCM,AIX_fcparray,AIX_SDDAPPCM
DS4200 AIX_SDDAPPCM AIX_APPCM,AIX_fcparray,AIX_SDDAPPCM
DS4300 AIX_SDDAPPCM AIX_APPCM,AIX_fcparray,AIX_SDDAPPCM
DS4500 AIX_SDDAPPCM AIX_APPCM,AIX_fcparray,AIX_SDDAPPCM
DS4700 AIX_SDDAPPCM AIX_APPCM,AIX_fcparray,AIX_SDDAPPCM
DS4800 AIX_SDDAPPCM AIX_APPCM,AIX_fcparray,AIX_SDDAPPCM
DS3950 AIX_SDDAPPCM AIX_APPCM,AIX_SDDAPPCM
DS5020 AIX_SDDAPPCM AIX_APPCM,AIX_SDDAPPCM
DS5100 AIX_SDDAPPCM AIX_APPCM,AIX_SDDAPPCM
When using native AIX MPIO the mpio_get_config -Av command can be used to display subsystem specific information for the DS3k/DS4k/DS5k hdisks (as known from thefget_config -Av command for RDAC):
# mpio_get_config -Av
Frame id 0:
Storage Subsystem worldwide name: 60ab8002643200004afa58c
Controller count: 2
Partition count: 1
Partition 0:
Storage Subsystem Name = 'DS4700_PFE'
hdisk LUN # Ownership User Label
hdisk2 0 A (preferred) MyLogicalDrive#1
hdisk3 1 B (preferred) MyLogicalDrive#2
(4) SAN configuration with AIX MPIO
Examples for SAN attachment and zoning with AIX native MPIO and more than two paths can be found in the
IBM System Storage DS3000 - Storage Manager v2 Installation and Support Guide for IBM AIX and Linux on POWER
http://www.ibm.com/systems/support/supportsite.wss/docdisplay?lndocid=MIGR-5073850&brandind=5000028
in chapter 3, Preparing for a SAN attachment (p25ff):
The following examples are a small subset of the connectivity configurations that are supported using the DS3400. The best practice is to use 2 to 4 adapters providing 4 to 8 paths.
Example 1 Create one zone that contains two Fibre Channel HBA ports, one Fibre Channel port from controller A and one Fibre Channel port from controller B of the DS3400. This configuration provides four paths to hdisks. Two of the paths that are associated with the preferred storage controller will service I/Os when the DS3400 is optimal. The other two paths are used if the preferred controller is not accessible by the host. Note: The hdisk attribute “algorithm” should be set to round_robin. If the attribute is set to round_robin, the hdisk attribute “reserve_policy” must be set to no_reserve, pr_exclusive, or pr_shared.
Example 2 Create separate zones for the connection between the HBA port and one port of the DS3400 controller. Two zones are required for a dual-controller storage configuration. One zone contains an HBA port and controller port from controller A. The other zone contains a different HBA port and controller port from controller B. This configuration provides two paths to hdisks. Note: Attributes “algorithm” and “algorithm of hdisks” do not need to be altered from default values.
Example 3 Create one zone that contains two Fibre Channel HBA ports and all four Fibre Channel ports from the DS3400. This configuration provides eight paths to hdisks. Four of the paths that are associated with the preferred storage controller will service I/Os when the DS3400 is Optimal. The other four paths are used if the preferred controller is inoperable. Note: The hdisk attribute ″algorithm″ should be set to round_robin. When this attribute is set to round_robin the hdisk attribute ″reserve_policy″ must be set to no_reserve, pr_exclusive, or pr_shared.
The default multipath algorithm for DS3k/DS4K/DS5K devices with native MPIO on AIX is fail_over with a reserve_policy of single_path.
# lsattr -El hdisk3
PCM PCM/friend/otherapdisk Path Control Module False
PR_key_value none Persistant Reserve Key Value True
algorithm fail_over Algorithm True
autorecovery no Path/Ownership Autorecovery True
clr_q no Device CLEARS its Queue on error True
cntl_delay_time 0 Controller Delay Time True
cntl_hcheck_int 0 Controller Health Check Interval True
dist_err_pcnt 0 Distributed Error Percentage True
dist_tw_width 50 Distributed Error Sample Time True
hcheck_cmd inquiry Health Check Command True
hcheck_interval 60 Health Check Interval True
hcheck_mode nonactive Health Check Mode True
location Location Label True
lun_id 0x0 Logical Unit Number ID False
lun_reset_spt yes LUN Reset Supported True
max_retry_delay 60 Maximum Quiesce Time True
max_transfer 0x40000 Maximum TRANSFER Size True
node_name 0x200600a0b81163b8 FC Node Name False
pvid none Physical volume identifier False
q_err yes Use QERR bit True
q_type simple Queuing TYPE True
queue_depth 10 Queue DEPTH True
reassign_to 120 REASSIGN time out value True
reserve_policy single_path Reserve Policy True
rw_timeout 30 READ/WRITE time out value True
scsi_id 0xda SCSI ID False
start_timeout 60 START unit time out value True
unique_id 3E213600A0B8000118FD000009A1F49D1EDB20F1815 FAStT03IBMfcp Unique device id False
ww_name 0x202700a0b81143a0 FC World Wide Name False
In order to use multiple paths to the preferred controller with native AIX MPIO it is required to set the hdisk's algorithm attribute to round_robin and the reserve_policy attribute to no_reserve (no_reserve will be fine for most situations but is depending on your specific SCSI reservation policy) using:
# chdev -l <hdisk#> -a algorithm=round_robin -a reserve_policy=no_reserve -P
This change requires a reboot. Alternatively one can stop using the hdisks (varyoff any VG with the disks) and eliminate the -P flag to go into effect without a reboot.
The MPIO reserve_policy defines whether a reservation methodology is employed when the device is opened. The values are as follows:
# lsattr -Rl <hdisk#> -a reserve_policy
no_reserve
single_path
PR_exclusive
PR_shared
no_reserve Does not apply a reservation methodology for the device. The device might be accessed by other initiators, and these initiators might be on other host systems.
single_path Applies a SCSI2 reserve methodology for the device, which means the device can be accessed only by the initiator that issued the reserve. This policy prevents other initiators in the same host or on other hosts from accessing the device. This policy uses the SCSI2 reserve to lock the device to a single initiator (path), and commands routed through any other path result in a reservation conflict.
Path selection algorithms that alternate commands among multiple paths can result in thrashing when the single_path reserve value is selected. As an example, assume a device-specific PCM has a required attribute that is set to a value that distributes I/O across multiple paths. When single_path reserve is in effect, the disk driver must issue a bus device reset (BDR) and then issue a reserve using a new path for sending the next command to break the previous reservation. Each time a different path is selected, thrashing occurs and performance degrades because of the overhead of sending a BDR and issuing a reserve to the target device. (The AIX® PCM does not allow you to select an algorithm that could cause thrashing.)
PR_exclusive Applies a SCSI3 persistent-reserve, exclusive-host methodology when the device is opened. The PR_key attribute value must be unique for each host system. The PR_key attribute is used to prevent access to the device by initiators from other host systems.
PR_shared Applies a SCSI3 persistent-reserve, shared-host methodology when the device is opened. The PR_key value must be a unique value for each host system. Initiators from other host systems are required to register before they can access the device.
The PR_key_value is required only if the device supports any of the persistent reserve policies (PR_exclusive or PR_shared).
For more information on native AIX MPIO and the related device attributes please take a look at:
System p and AIX Information Center - Multiple Path I/O
(5) SDDPCM for DS4000/DS5000
Starting with the release of SDDPCM version 2.4.0.0 DS4000 and DS5000 subsystems are also supported with the additional SDD path control module (SDDPCM). SDDPCM is compliant with the MPIO architecture and as such, is an alternative to using the native AIX MPIO PCM. SDDPCM additionally provides I/O load balancing algorithms and the pcmsrv daemon for enhanced path health checks and First Time Data Capture enhancing overall data availability. As such SDDPCM is preferable to just using MPIO. Furthermore, the SDDPCM package also includes the additional pcmpath command providing ease of use for overall SDDPCM device and path management.
Be aware that the default path selection algorithm with SDDPCM is load_balance with the device reserve policy set to no_reserve which means that no reservation methodology for the device is applied. A SCSI-2 reservation which is typically in place when doing a varyonvg of an AIX volume group is _not_ applied actually with SDDPCM default settings for the device (here the hdisks in that VG) which are reserve_policy=no_reserve and algorithm=load_balance. Such an AIX volume group might be varied on and accessed by other initiators and even other host systems in this case!
# lsdev -Cc disk
hdisk5 Available 02-08-02 IBM MPIO DS4700 Array Disk
hdisk6 Available 02-08-02 IBM MPIO DS4700 Array Disk
# lsattr -El hdisk5
PCM PCM/friend/sddappcm PCM False
PR_key_value none Reserve Key True
algorithm load_balance Algorithm True
autorecovery no Auto Recovery True
clr_q no Device CLEARS its Queue on error True
cntl_delay_time 0 Auto Recovery True
cntl_hchdeck_in 0 Retry Timeout True
dist_err_pcnt 0 Distributed Error Percentage True
dist_tw_width 50 Distributed Error Sample Time True
hcheck_interval 60 Health Check Interval True
hcheck_mode nonactive Health Check Mode True
location Location Label True
lun_id 0x0 Logical Unit Number ID False
lun_reset_spt yes Support SCSI LUN reset True
max_transfer 0x40000 Maximum TRANSFER Size True
node_name 0x200400a0b83313f0 FC Node Name False
pvid none Physical volume identifier False
q_err yes Use QERR bit True
q_type simple Queuing TYPE True
qfull_dly 2 delay in seconds for SCSI TASK SET FULL True
queue_depth 16 Queue DEPTH True
reserve_policy no_reserve Reserve Policy True
rw_timeout 30 READ/WRITE time out value True
scbsy_dly 20 delay in seconds for SCSI BUSY True
scsi_id 0x1b0400 SCSI ID False
start_timeout 60 START unit time out value True
unique_id 3E213600A0B80002642EA0000C1CF4BE273F40F1814 FAStT03IBMfcp PCM False
ww_name 0x200400a0b83313f1 FC World Wide Name False
If the AIX volume group is shared between hosts (not using any cluster services like PowerHA with enhanced concurrent volume groups) and should _not_ be accessed by other hosts using a SCSI-2 reservation methodology the device reserve_policy can be set set to single_path ( SCSI-2 reserve) after setting the device path selection algorithm to fail_over. Any attempt to set the algorithm to round_robin, load_balance, or load_balance_port with single_path reserve policy will fail.
For a general documentation of SDDPCM support with DS4000/DS5000 take a look at the
Multipath Subsystem Device Driver User's Guide
http://www.ibm.com/support/docview.wss?rs=540&context=ST52G7&uid=ssg1S7000303
and also the SDDPCM readme
SDDPCM 2.5.1.0 for AIX Readme
ftp://ftp.software.ibm.com/storage/subsystem/aix/2.5.1.0/sddpcm.readme.2.5.1.0.txt
which always provides latest information about prerequisites, APARs and other restrictions that currently apply.
Note that SDDPCM is not supported on VIOS with DS4k/DS5k devices:
VIOS is not supported with SDDPCM on DS4000/DS5000/DS5020 subsystem devices.
However, the SDDPCM fileset might be required on a VIOS partition for attached DS8000 and/or SVC devices and you might want to attach also DS4000/DS5000 devices to the same VIOS. A supported configuration with SDDPCM on VIOS2.1 for DS8000 and/or SVC devices and native AIX MPIO for DS4k/DS5k devices might be achived by using SDDPCM 2.5.1.0 on VIOS 2.1.2.10-FP-22.1 (or higher) and manually deselecting the SDDPCM driver option for the DS4k/DS5k devics with the manage_disk_drivers command. Such a configuration would be based on:
VIOS 2.1.2.10-FP-22.1 (or higher) required by SDDPCM 2.5.1.0 and including the new manage_disk_drivers command
SDDPCM 2.5.1.0 (or higher) fileset with appropriate SDDPCM prerequisites (e.g. devices.fcp.disk.ibm.mpio.rte, V1.0.0.20)
selecting AIX_APPCM as present driver for attached DS5k/DS4k subsystems with manage_disk_drivers command (i.e. deselect AIX_SDDAPPCM)
AIX MPIO using SDDPCM for DS8000 / SVC devices (i.e. only the DS8000/SVC devices are managed by SDDPCM)
AIX MPIO using native AIX PCM for all DS4k/DS5k devices (i.e. all DS4k/DS5k devices are managed by the default AIX MPIO driver with the native AIX PCM)
The pcmpath command of the SDDPCM package can be used to easily display and manage SDDPCM devices and paths. Here is an example of the pcmpath query devicecommand with attached DS8000, SVC and DS4800 devices on an AIX 6.1 partition using SDDPCM 2.4.x:
# pcmpath query device
Total Dual Active and Active/Asymmetrc Devices : 2
DEV#: 0 DEVICE NAME: hdisk1 TYPE: 2107900 ALGORITHM: Load Balance
SERIAL: 75KA3628201
==========================================================================
Path# Adapter/Path Name State Mode Select Errors
0 fscsi0/path0 OPEN NORMAL 844158 0
1 fscsi0/path1 OPEN NORMAL 843779 0
2 fscsi1/path2 OPEN NORMAL 833541 0
3 fscsi1/path3 OPEN NORMAL 833459 0
DEV#: 1 DEVICE NAME: hdisk2 TYPE: 2145 ALGORITHM: Load Balance
SERIAL: 60050768018082E29000000000000008
==========================================================================
Path# Adapter/Path Name State Mode Select Errors
0* fscsi0/path0 OPEN NORMAL 6 0
1 fscsi0/path1 OPEN NORMAL 2124784 0
2 fscsi1/path2 OPEN NORMAL 2148331 0
3* fscsi1/path3 OPEN NORMAL 8 0
Total Active/Passive Devices : 1
DEV#: 3 DEVICE NAME: hdisk3 TYPE: 1815 ALGORITHM: Load Balance
SERIAL: 600A0B8000118FD00000878E49C33524
==========================================================================
Path# Adapter/Path Name State Mode Select Errors
0* fscsi0/path0 OPEN NORMAL 0 0
1* fscsi1/path2 OPEN NORMAL 0 0
2 fscsi0/path1 OPEN NORMAL 94440 0
3 fscsi1/path3 OPEN NORMAL 94168 0
The DS4000 and DS5000 products are dual array controller disk subsystems. Each LUN is assigned to one controller, which is considered the owner, or the active (primary) controller, of a particular LUN. The other controller is considered as an alternate, or passive (secondary), controller. The '*' next to the path ID in the pcmpath command output indicates that the related path is a non-preferred path. The indicator '*' for preferred and non-preferred paths are only shown after the device is opened once.
SDDPCM distinguishes the following paths to a given LUN on a DS4000 and DS5000 subsystem:
Paths to the primary (active) controller that currently owns the LUN and processes the I/Os to that LUN
Paths to the alternate (passive/secondary) controller
With this type of active/passive dual-controller subsystem device, I/O can be sent only to the controller that currently owns the LUN. When the SDDPCM selects paths for I/O, it selects paths that are connected only to the active controller for a given LUN. If there is no path on the active controller that can be used, SDDPCM changes the LUN controller ownership to an alternate controller, switches the paths that were passive to active, and then selects these active paths for I/O.
As shown in the example above for a DS4800 (model type 1825) the '*' indicates the paths to the secondary or alternate controller for the LUN. That's why there were no I/Os seen on these paths. Here the I/O load for the given LUN is balanced only across the paths to the primary DS4K/DS5K controller that currently owns the LUN. With two host FC adapters (HBAs) in the server and each host adapter seeing both DS4K/DS5k controllers the load-balancing algorithm with SDDPCM will balance I/Os across both host FC adapters for all I/Os to all DS4K/DS5K LUNs.
The SDDPCM fileset also provides the sddpcm_get_config command for DS4000/DS5000 devices which displays information about all MPIO-based DS4K/DS5K subsystems using SDDPCM and the hdisks associated with them. Specifically, it displays information about the frame (subsystem), including the frame's assigned name and worldwide name, and a list of hdisks (only those currently in the Available state) associated with that subsystem, including the hdisk name, LUN number, current ownership, preferred path, and the user-assigned label for that volume.
# sddpcm_get_config -Av
Frame id 0:
Storage Subsystem worldwide name: 60ab8001143a0000049b88968
Controller count: 2
Partition count: 1
Partition 0:
Storage Subsystem Name = 'VIOS_DS4800'
hdisk LUN # Ownership User Label
hdisk3 0 A (non-preferred) lpar3_vol01
The SDDPCM support matrix and the SDDPCM filesets (devices.sddpcm.61.rte) for DS4000/DS5000 are available here:
Support Matrix for Subsystem Device Driver, Subsystem Device Driver Path Control Module
http://www.ibm.com/support/docview.wss?rs=540&context=ST52G7&dc=DA400&uid=ssg1S7001350
Subsystem Device Driver Path Control Module (SDDPCM)
http://www.ibm.com/support/docview.wss?rs=540&context=ST52G7&dc=D430&uid=ssg1S4000201
SDDPCM also requires the AIX host attachment script fileset (devices.fcp.disk.ibm.mpio.rte) to be installed as prerequisite:
Host Attachment for SDDPCM on AIX
http://www.ibm.com/support/docview.wss?rs=540&context=ST52G7&dc=D400&q1=host+script&uid=ssg1S4000203
The AIX host attachment script provides the device definitions and ODM attributes for the supported SDDPCM devices. Without the AIX host attachment fileset installed the devices will be managed by the default AIX MPIO PCM and simply show up as:
# lsdev -Cc disk
hdisk0 Available Virtual SCSI Disk Drive
hdisk1 Available 30-T1-01 MPIO Other FC SCSI Disk Drive
hdisk2 Available 30-T1-01 MPIO Other FC SCSI Disk Drive
hdisk3 Available 30-T1-01 MPIO Other DS4K Array Disk
With the AIX host attachment script installed the supported SDDPCM devices will show up as:
# lsdev -Cc disk
hdisk0 Available Virtual SCSI Disk Drive
hdisk1 Available 30-T1-01 IBM MPIO FC 2107
hdisk2 Available 30-T1-01 MPIO FC 2145
hdisk3 Available 30-T1-01 IBM MPIO DS4800 Array Disk
If the DS4K/DS5K hdisks still show up as "MPIO Other DS4K Array Disk" after the installation of the AIX host attachment script fileset they will not be managed by SDDPCM and will not be displayed by the pcmpath query command. Here it would be necessary to manually remove and reconfigure the devices so that they show up as "IBM MPIO DS4800 Array Disk" (if the appropriate version of the AIX host attachment script for SDDPCM 2.4.x.x is installed).
The Multipath Subsystem Device Driver User's Guide recommends a SAN attachment with four paths for DS4000 and DS5000 subsystems with SDDPCM:
Because switching a device to another controller is a time-consuming recovery action and affects I/O performance, you can use this redundancy to avoid an unnecessary controller failover if a path fails. Therefore, configure a minimum of four paths for each LUN with two host adapter ports and two storage controller ports where each host adapter port has redundancy to each storage controller port and vice versa. (p.8)
Page updated
Google Sites
Report abuse