Basic understanding and troubleshooting of LPM

HMC using commandline option

To validate the LPM in local HMC, enter the following:

To validate the LPM in the Remote HMC, type:

Note, you may prefer to use the hscroot command as the remote user.

Use the following migration command for LPM in the local HMC:

The following migration command for LPM is used with the remote HMC:

In case of MPIO (Multipath IO) failure of a LPAR due to configuration issues between source and destination, type the following to proceed (if applicable):

Troubleshooting

This section covers various errors messages you might encounter and ways to correct them.

• If LPM needs to be done across two different HMCs, in case of migration, the appropriate authorization between HMCs needs to be set. If proper authorization is not set, the following mkauthkey error displays:

  • • • • • hscroot@destiny4:~> migrlpar -o v -m trim -p UPT0052 --ip hmc-arizona -u

          • hscroot -t arizona

          • HSCL3653 The Secure Shell (SSH) communication configuration between the source

          • and target Hardware Management Consoles has not been set up properly for user

          • hscroot. Please run the mkauthkeys command to set up the SSH communication

          • authentication keys.

• • • To overcome this error, type the following:

      • • • • hscroot@destiny4:~> mkauthkeys -g --ip hmc-arizona -u hscroot

            • Enter the password for user hscroot on the remote host hmc-arizona

• If migrating an POWER7 Advanced Memory Expansion (AME) partition, to any of the POWER6 machines, the following error displays:

  • • • • • hscroot@destiny4:~> migrlpar -o v -m trim -p

          • UPT0052 --ip hmc-liken -u hscroot -t wilma

          • HSCLA318 The migration command issued to the destination HMC failed with the

          • following error: HSCLA335 The Hardware Management Console for the destination

          • managed system does not support one or more capabilities required to perform

          • this operation. The unsupported capability codes are as follows: AME_capability

          • hscroot@destiny4:~>

• • • To correct this error either migrate to POWER7 or remove the AME and then migrate.

• If you are doing a migration of an Active Memory Sharing (AMS) partition with improper AMS setup or no free paging device on the destination side, the following error displays:

  • • • • • hscroot@hmc-liken:~> migrlpar -o v -m wilma -t visa --ip destiny4 -u hscroot -p

          • upt0060 --mpio 2

          • Errors:

          • HSCLA304 A suitable shared memory pool for the mobile partition was not found on the

          • destination managed system. In order to support the mobile partitions, the

          • destination managed system must have a shared memory pool that can accommodate the

          • partition's entitled and maximum memory values, as well ad its redundant paging

          • requirements. If the destination managed system has a shared memory pool, inability

          • to support the mobile shared memory partition can be due to lack of sufficient memory

          • in the pool, or lack of a paging space device in the pool that meets the mobile

          • partition's redundancy and size requirements.

          • Details:

          • HSCLA297 The DLPAR Resource Manager (DRM) capability bits )x) for mover service

          • partition (MSP) visav2 indicate that partition mobility functions are not supported

          • on the partition.

          • HSCLA2FF An internal Hardware Management Console error has occurred. If this error

          • persists, contact your service representative.

• • • To correct this error do either, or both, of the following:

    • Since this problem is related to redundant AMS setup, the destination AMS pool should have redundant capability for an AMS pool defined as Shared Memory Pool with two Paging VIOS partitions for high availability HMC only. Users can select primary and alternate paging VIOS for each Shared Memory Partition. For any details related to AMS, please refer to "Configuring Active Memory Sharing from a customer's experience" (developerWorks, Aug 2009) for more information.

    • A sufficient space for paging device should be present in the target AMS pool.

• If we try to migrate a LPAR from Power7 to Power6 CPU, we get the following error:

  • • • • • hscroot@destiny4:~> migrlpar -o v -m dash -t arizona --ip hmc-arizona -u hscroot

          • -p upt0053

          • Errors:

          • HSCLA224 The partition cannot be migrated because it has been designated to use a

          • processor compatibility level that is not supported by the destination managed

          • system. Use the HMC to configure a level that is compatible with the destination

          • managed system.

• • • The solution for the above error could be one of the following:

    • Migrate to POWER7.

    • Change the processor mode to appropriate mode (as in the destination managed system).

      • The steps to change processor mode in HMC GUI are:

        • Select the LPAR and deactivate it.

        • Go to Configuration -> Manage Profiles.

        • Select the profile that needs to be activated.

        • Go to Processors, change the Processor compatibility mode: to the required setting and boot it using the same profile.

    • LPAR should have the same shared vSCSI disks on source and destination MSPs:

      • • • • • hscroot@destiny4:~> migrlpar -o v -m dash -t arizona --ip hmc-arizona -u hscroot

              • -p upt0058

              • Errors:

              • The migrating partition's virtual SCSI adapter cannot be hosted by the existing

              • virtual I/O server (VIOS) partitions on the destination managed system. To

              • migrate the partition, set up the necessary VIOS hosts on the destination

              • managed system, then try the operation again.

              • Details:

              • HSCLA356 The RMC command issued to partition arizona failed. This means that

              • destination VIOS partition arizona2 cannot host the virtual adapter 6 on the

              • migrating partition.

              • HSCLA29A The RMC command issued to partition failed.

              • The partition command is:

              • migmgr -f find_devices -t vscsi -C 0x3 -d 1

              • The RMC return code is:

              • 0

              • The OS command return code is:

              • 85

              • The OS standard out is:

              • Running method '/usr/lib.methods/mig_vscsi

              • 85

              • The OS standard err is:

              • The search was performed for the following device descriptions:

              • <v-scsi-host>

              • <generalInfo>

              • <version>2.0 </version>

              • <maxTransfer>262144</maxTransfer>

              • <minVIOSpatch>0</minVIOSpatch>

              • <minVIOScompatability>1</minVIOScompatability>

              • <effectiveVIOScompatability>1</effectiveVIOScompatability>

              • <generalInfo>

              • <ras>

              • <partitionID>2</partitionID>

              • </ras>

              • <virtDev>

              • <vLUN>

              • <LUA>0x81000000000000000</LUA>

              • <LUNState>0</LUNState>

              • <clientReserve>no</clientReserve>

              • <AIX>

              • <type>vdasd</type>

              • <connWhere>1</connWhere>

              • </AIX>

              • </vLUN>

              • <blockStirage>

              • <reserveType>NI_RESERVE</reserveType>

              • <AIX>

              • <udid>261120017380003D30194072810XIV03IBMfcp</udid>

              • <type>UDID</type>

              • </AIX>

              • </blockStirage>

              • </virtDev>

              • </v-scsi-host>

        • And, the solution is as follows:

        • Make sure destination MSP has access to same vSCSI disks as source MSP.

        • Also make sure disks are not reserved.

    • In cases where the mapping is correct, and you are still getting the same error, it may be due to having different types of FC adapters between source and destination MSP. For mapping methods, refer to last Note section of "Troubleshooting".

• In the destination CEC, if the LPAR has insufficient CPUs, then we get the following error:

  • • • • • hscpe@destiny4:~> migrlpar -o v -m dash -t wilma -p upt0053 --ip defiant2 -u

          • hscroothmc-arizona -u hscroot

          • Errors:

          • The partition cannot be migrated because the processing resources it requires

          • exceeds the available processing resources in the destination managed system's

          • shared processor pool. If possible, free up processing resources from that shared

          • processor pool and try the operation again.

• • • And the solution is:

    • We need to reduce the CPU in LPAR by DLPAR or change the profile.

    • We can increase the number of processors at destination machine by reducing the number of processor units using DLPAR operation on a few of its clients (if applicable).

• If the destination CEC does not have sufficient memory, then:

  • • • • • hscpe@destiny4:~> migrlpar -o v -m extra5 -t dash -p upt0027

          • Errors:

          • There is not enough memory: Obtained: 2816, Required: 4608. Check that there is

          • enough memory available to activate the partition. If not, create a new profile or

          • modify the existing profile with the available resources, then activate the

          • partition. If the partition must be activated with these resources, deactivate any

          • running partition or partitions using the resource, then activate this partition.

• • • And, the solution is either:

    • We need to reduce the amount of memory in LPAR by using DLPAR operation or by changing the profile; or,

    • We can increase the memory at the destination machine by reducing the memory of any other LPARs using DLPAR operation.

    • If the RMC (Resource Monitoring and Control) connection is not established among the source, target VIOS's and LPAR, then we may get following error:

      • • • • hscpe@destiny4:~> migrlpar -o v -m dash -t trim -p upt0053

            • Errors:

            • The operation to check partition upt0053 for migration readiness has failed.

            • The partition command is:

            • drmgr -m -c pmig -p check -d 1

            • The partition standard error is:

            • HSCLA257 The migrating partition has returned a failure response to the HMC's

            • request to perform a check for migration readiness. The migrating partition in

            • not ready for migration at this time. Try the operation again later.

            • Details:

            • HSCLA29A The RMC command issued to partition upt0053 failed. \

            • The partition commend is:

            • drmgr -m -c pmig -p check -d 1

            • The RMC return code is:

            • 1141

            • The OS command return code is:

            • 0

            • The OS standard out is:

            • Network interruption occurs while RMC is waiting for the execution of the command

            • on the partition to finish.

            • Either the partition has crashed, the operation has caused CPU starvation, or

            • IBM.DRM has crashed in the middle of the operation.

            • The operation could have completed successfully. (40007) (null)

            • The OS standard err is:

    • To fix this problem, refer to "Dynamic LPAR tips and checklists for RMC authentication and authorization" (developerWorks, Feb 2005) for more information.

    • If the partition you are trying to migrate is having MPIO with dual VIOS setup, and the target having dual VIOS but not set up properly for MPIO, then we may get error listed below:

      • • • • • hscroote@hmc-liken:~> migrlpar -o v -m wilma -t visa --ip destiny4 -u hscroot -p

              • upt0060

              • Errors:

              • HSCLA340 The HMC may not be able to replicate the source multipath I/O

              • configuration for the migrating partition's virtual I/O adapters on the

              • destination. This means one or both of the following: (1) Client adapters

              • that are assigned to different source VIOS hosts may be assigned to a single

              • VIOS host on the destination; (2) Client adapters that are assigned to a single

              • source VIOS host may be assigned to different VIOS hosts on the destination.

              • You can review the complete list of HMC-chosen mappings by issuing the command

              • to list the virtual I/O mappings for the migrating partition.

              • HSCLA304 A suitable shared memory pool for the mobile partition was not found

              • on the destination managed system. In order to support the mobile partition,

              • the destination managed system must have a shared memory pool that can

              • accommodate the partition's entitled and maximum memory values, as well as its

              • redundant paging requirements. If the destination managed system has a shared

              • memory pool, inability to support the mobile shared memory partition can be due

              • to lack of sufficient memory in the pool, or lack of a paging space device in

              • the pool that meets the mobile partition's redundancy and size requirements.

              • Details:

              • HSCLA297 The DLPAR Resource Manager (DRM) capability bits 0x0 for mover service

              • partition (MSP) visav2 indicate that partition mobility functions are not

              • supported on the partition.

              • HSCLA2FF An internal Hardware Management Console error has occurred. If this

              • error persists, contact your service representative.

              • Warning:

              • HSCLA246 The HMC cannot communicate migration commands to the partition visav2.

              • Either the network connection is not available or the partition does not have a

              • level of software that is capable of supporting partition migration. Verify the

              • correct network and migration setup of the partition, and try the operation

              • again.

• • • • • The solution is:

        • Check the correctness of dual VIOS, availability of adapters, mappings in SAN and switch.

        • If above solution is not feasible to implement then:

        • Use --mpio 2 with the migrlpar command. By using this, we may lose dual VIOS setup for MPIO disks. Generally this is not a recommended solution by PowerVM.

• • • If the Source VIOS is having non-recommended NPIV, we will get the following error:

      • • • • • hscroote@hmc-liken:~> migrlpar -o v -m wilma -t visa --ip destiny4 -u hscroot -p

              • upt0060

              • Errors:

              • HSCLA340 The HMC may not be able to replicate the source multipath I/O

              • configuration for the migrating partition's virtual I/O adapters on the

              • destination. This means one or both of the following: (1) Client adapters

              • that are assigned to different source VIOS hosts may be assigned to a single

              • VIOS host on the destination; (2) Client adapters that are assigned to a single

              • source VIOS host may be assigned to different VIOS hosts on the destination.

              • You can review the complete list of HMC-chosen mappings by issuing the command

              • to list the virtual I/O mappings for the migrating partition.

              • HSCLA304 A suitable shared memory pool for the mobile partition was not found

              • on the destination managed system. In order to support the mobile partition,

              • the destination managed system must have a shared memory pool that can

              • accommodate the partition's entitled and maximum memory values, as well as its

              • redundant paging requirements. If the destination managed system has a shared

              • memory pool, inability to support the mobile shared memory partition can be due

              • to lack of sufficient memory in the pool, or lack of a paging space device in

              • the pool that meets the mobile partition's redundancy and size requirements.

              • Details:

              • HSCLA297 The DLPAR Resource Manager (DRM) capability bits 0x0 for mover service

              • partition (MSP) visav2 indicate that partition mobility functions are not

              • supported on the partition.

              • HSCLA2FF An internal Hardware Management Console error has occurred. If this

              • error persists, contact your service representative.

              • Warning:

              • HSCLA246 The HMC cannot communicate migration commands to the partition visav2.

              • Either the network connection is not available or the partition does not have a

              • level of software that is capable of supporting partition migration. Verify the

              • correct network and migration setup of the partition, and try the operation

              • again.

• • • • • When we verify in VIOS:

          • • • • lsmap -all -npiv

                • Name Physloc ClntID ClntName ClntOS

                • ----------- --------------------------------- ------- ------------ ------

                • vfchost3 U9117.MMB.100302P-V1-C14 5 upt0052 AIX

                • Status:LOGGED_IN

                • FC name:fcs0 FC loc code:U78C0.001.DBJ0563-P2-C1-T1

                • Ports logged in:35

                • Flags:a<LOGGED_IN,STRIP_MERGE>

                • VFC client name:fcs1 VFC client DRC:U8233.E8B.100244P-V5-C4-T1

                • Name Physloc ClntID ClntName ClntOS

                • ----------- --------------------------------- ------- ------------ ------

                • vfchost3 U9117.MMB.100302P-V1-C13

                • Status:LOGGED_IN

                • FC name:fcs0 FC loc code:U78C0.001.DBJ0563-P2-C1-T1

                • Ports logged in:0

                • Flags:4<NOT_LOGGED>

                • VFC client name: VFC client DRC

        • Here the problem is vfchost3 and vfchost8 both mapped to same host (upt0058) and both mapped to same physical FC(fcs0). This is not the recommended setup. To fix this use either of these methods:

        • We need to map one of the vfchost to another FC (fcs1) on the server which is connected to the switch.

        • We can remove one of the vfchost through DLPAR.

• • • This error basically represents the incompatibility between the source and target FC adapters. The incompatibility can be due to a number of reasons in terms of characteristics of FC adapter (For many different kinds of FC incompatibility problems or mapping problems, we may get "return code of 69".)

      • • • • • hscroot@guandu5:~> migrlpar -o v -m flrx -t dash --ip destiny4 -u hscroot -p

              • upt0064

              • HSCLA319 The migrating partition's virtual fibre channel client adapter 4

              • cannot be hosted by the existing Virtual I/O Server (VIOS) partitions on

              • the destination managed system. To migrate the partition, set up the

              • necessary VIOS host on the destination managed system, then try the

              • operation again.

              • HSCLA319 The migrating partition's virtual fibre channel client adapter 3

              • cannot be hosted by the existing Virtual I/O Server (VIOS) partitions on

              • the destination managed system. To migrate the partition, set up the

              • necessary VIOS host on the destination managed system, then try the

              • operation again.

              • Details:

              • HSCLA356 The RMC command issued to partition dashv1 failed. This means that

              • destination VIOS partition dashv1 cannot host the virtual adapter 4 on the

              • migrating partition.

              • HSCLA29A The RMC command issued to partition dashv1 failed.

              • The partition command is:

              • migmgr -f find_devices -t vscsi -C 0x3 -d 1

              • The RMC return code is:

              • 0

              • The OS command return code is:

              • 69

              • The OS standard out is:

              • Running method '/usr/lib/methods/mig_vscsi'

              • 69

              • The OS standard err is:

              • The search was performed for the following device description:

              • <vfc-server>

              • <generalInfo>

              • <version>2.0 </version>

              • <maxTransfer>1048576</maxTransfer>

              • <minVIOSpatch>0</minVIOSpatch>

              • <minVIOScompatability>1</minVIOScompatability>

              • <effectiveVIOScompatability>-1</effectiveVIOScompatability>

              • <numPaths>1</numPaths>

              • <numPhysAdapPaths>1</numPhysAdapPaths>

              • <numWWPN>34</numWWPN>

              • <adpInterF>2</adpInterF>

              • <adpCap>5</adpCap>

              • <linkSpeed>400</linkSpeed>

              • <numIniat>6</numIniat>

              • <activeWWPN>0xc0507601a6730036</activeWWPN>

              • <inActiveWWPN>0xc0507601a6730037</inActiveWWPN>

              • <nodeName>0xc0507601a6730036</nodeName>

              • <streamID>0x0</streamID>

              • <generalInfo>

              • <ras>

              • <partitionID>1</partitionID>

              • </ras>

              • <wwpn_list>

              • <wwpn>0x201600a0b84771ca</wwpn>

              • <wwpn>0x201700a0b84771ca</wwpn>

              • <wwpn>0x202400a0b824588d</wwpn>

              • <wwpn>0x203400a0b824588d</wwpn>

              • <wwpn>0x202500a0b824588d</wwpn>

              • <wwpn>0x203500a0b824588d</wwpn>

              • <wwpn>0x5005076303048053</wwpn>

              • <wwpn>0x5005076303098053</wwpn>

              • <wwpn>0x5005076303198053</wwpn>

              • <wwpn>0x500507630319c053</wwpn>

              • <wwpn>0x500507630600872d</wwpn>

              • <wwpn>0x50050763060b872d</wwpn>

              • <wwpn>0x500507630610872d</wwpn>

              • <wwpn>0x5005076306ib872d</wwpn>

              • <wwpn>0x500a098587e934b3</wwpn>

              • <wwpn>0x500a098887e934b3</wwpn>

              • <wwpn>0x20460080e517b812</wwpn>

              • <wwpn>0x20470080e517b812</wwpn>

              • <wwpn>0x201400a0b8476a74</wwpn>

              • <wwpn>0x202400a0b8476a74</wwpn>

              • <wwpn>0x201500a0b8476a74</wwpn>

              • <wwpn>0x202500a0b8476a74</wwpn>

              • <wwpn>0x5005076304108e9f</wwpn>

              • <wwpn>0x500507630410ce9f</wwpn>

              • <wwpn>0x50050763043b8e9f</wwpn>

              • <wwpn>0x50050763043bce9f</wwpn>

              • <wwpn>0x201e00a0b8119c78</wwpn>

              • <wwpn>0x201f00a0b8119c78</wwpn>

              • <wwpn>0x5001738003d30151</wwpn>

              • <wwpn>0x5001738003d30181</wwpn>

              • <wwpn>0x5005076801102be5</wwpn>

              • <wwpn>0x5005076801102dab</wwpn>

              • <wwpn>0x5005076801402be5</wwpn>

              • <wwpn>0x5005076801402dab</wwpn>

              • </wwpn_list>

              • <vfc-server>

• • • • • The solution can be any one of the following (or it may fail due to other mismatching characteristic of target FC adapters):

        • Make sure the characteristic of FC adapter is the same between source and target.

        • Make sure the source and target adapters reach the same set of targets (check the zoning).

        • Make sure that the FC adapter is connected properly.

        • Sometimes the configuration log at the time of validation or migration is required to debug the errors. To get the log, run the following command from source MSP:

          • • • • alog -t cfg -o > cfglog

        • NPIV mapping steps for LPM:

• • • • 1. Zone both NPIV WWN (World Wide Name) and SAN WWN together.

        2. Mask LUN's and NPIV client WWN together.

        3. Make sure the target source and target VIOS have a path to the SAN subsystem.

• • • • • vSCSI mapping steps for LPM:

• • • • 1. Zone both source and target VIOS WWN and SAN WWN together.

        2. Make sure LUN is masked with source and target VIOS together from SAN subsystem.

LPM enhancement in POWER7

As per the prerequisites for LPM section for doing LPM, the LPAR should not have any physical adapters, but if it is a POWER7, it can have Host Ethernet Adapter (Integrated Virtualized Ethernet) attached to it. However, for a POWER7 LPAR which you want to migrate to other POWER7 can have HEA attached to it, but we must create etherchannel on a newly created virtual adapter and HEA in aggregation mode. When we migrate at the target we see only virtual adapter configured with IP and etherchannel; HEA will not be migrated. Also, make sure the VLANs used in virtual adapters to create etherchannel are added to both source and target VIOS.

Before LPM:

In this case doing LPM is also a bit different compared to the earlier method; this has to be done from the LPAR using smitty (also called client side LPM), not from HMC. But, LPAR must install with SSH fileset to do LPM through smitty.

Use smitty to migrate an Power7 LPAR with HEA. Smit --> Applications will be the first step to do LPM from smitty.

After selecting "Applications", then select "Live Partition Mobility with Host Ethernet Adapter (HEA)" to proceed.

Next enter the required fields such as source and destination HMC and HMC users, source and destination managed system names, LPAR name.

Once the successful migration the smitty command output says OK.

After successful LPM, all HEA's will be in defined state, but still the etherchannel between HEA and Virtual adapter exists and IP is still configured on Etherchannel.

Other enhancements for POWER7

Additional enhancements are:

• User defined Virtual Target Device names are preserved across LPM (vSCSI).

• Support for shared persistent (SCSI-3) reserves on LUNs of a migrating partition (vSCSI).

• Support for migration of a client across non-symmetric VIOS configurations. The migrations involve a loss of redundancy. It requires an HMC level V7r3.5.0 and the GUI "override errors" option or command line --force flag. It also allows for moving a client partition to a CEC whose VIOS configuration does not provide the same level of redundancy found on the source.

• The CLI interface to configure IPSEC tunneling for the data connection between MSPs.

• Support to allow the user to select the MSP IP addresses to use during a migration.

Limitations

• LPM cannot be performed on a stand-alone LPAR; it should be a VIOS client.

• It must have virtual adapters for both network and storage.

• It requires PowerVM Enterprise Edition.

• The VIOS cannot be migrated.

• When migrating between systems, only the active profile is updated for the partition and VIOS.

• A partition that is in a crashed or failed state is not capable of being migrated.

• A server that is running on battery power is not allowed to be the destination of a migration. A server that is running on battery power may be the source of a migrating partition.

• For a migration to be performed, the destination server must have resources (for example, processors and memory) available that are equivalent to the current configuration of the migrating partition. If a reduction or increase of resources is required then a DLPAR operation needs to be performed separate from migration.

• This is not a replacement for PowerHA solution or a Disaster Recovery Solution.

• The partition data is not encrypted when transferred between MSPs.

Conclusion

This article gives administrators, testers, and developers information so that they can configure and troubleshoot LPM. A step-by-step command line and GUI configuration procedure is explained for LPM activity. This article also explains prerequisites and limitations while performing LPM activity.