2018-05-22 Use iSCSI

Summary

When I tried to add REDFERN2 to the REDFERN RAC, I got the following errors as described in “REDFERN2 is unable to share disks with REDFERN1”.

File /OVS/shareDisk/REDFERN/FRA_01 is loopback-mounted through /dev/loop3,
which is mounted in a guest domain,

and so cannot be mounted now.

I will now use the allocated LUNs on the NARRABRI NAS for the shared disks.

References

Oracle Manuals

Procedure

The iSCSI target is the NARRABRI NAS.

This procedure follows that of “20.7.2 Configuring an iSCSI Initiator”.

Install iSCSI Initiator Software

Ran the following command as root on REDFERN1 to install the software for the iSCSI client (initiator):

yum install iscsi-initiator-utils

The output is:

Loaded plugins: ulninfo

ol7_UEKR4 | 1.2 kB 00:00

ol7_latest | 1.4 kB 00:00

(1/5): ol7_UEKR4/x86_64/updateinfo | 191 kB 00:01

(2/5): ol7_latest/x86_64/group | 659 kB 00:04

(3/5): ol7_latest/x86_64/updateinfo | 1.8 MB 00:04

(4/5): ol7_latest/x86_64/primary | 17 MB 00:48

(5/5): ol7_UEKR4/x86_64/primary | 36 MB 01:20

ol7_UEKR4 669/669

ol7_latest 26617/26617

Resolving Dependencies

--> Running transaction check

---> Package iscsi-initiator-utils.x86_64 0:6.2.0.874-7.0.1.el7 will be installed

--> Processing Dependency: iscsi-initiator-utils-iscsiuio >= 6.2.0.874-7.0.1.el7 for package: iscsi-initiator-utils-6.2.0.874-7.0.1.el7.x86_64

--> Running transaction check

---> Package iscsi-initiator-utils-iscsiuio.x86_64 0:6.2.0.874-7.0.1.el7 will be installed

--> Finished Dependency Resolution


Dependencies Resolved


================================================================================

Package Arch Version Repository Size

================================================================================

Installing:

iscsi-initiator-utils x86_64 6.2.0.874-7.0.1.el7 ol7_latest 421 k

Installing for dependencies:

iscsi-initiator-utils-iscsiuio x86_64 6.2.0.874-7.0.1.el7 ol7_latest 90 k


Transaction Summary

================================================================================

Install 1 Package (+1 Dependent package)


Total download size: 511 k

Installed size: 2.6 M

Is this ok [y/d/N]: y

Downloading packages:

warning: /var/cache/yum/x86_64/7Server/ol7_latest/packages/iscsi-initiator-utils-iscsiuio-6.2.0.874-7.0.1.el7.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID ec551f03: NOKEY

Public key for iscsi-initiator-utils-iscsiuio-6.2.0.874-7.0.1.el7.x86_64.rpm is not installed

(1/2): iscsi-initiator-utils-iscsiuio-6.2.0.874-7.0.1.el7. | 90 kB 00:00

(2/2): iscsi-initiator-utils-6.2.0.874-7.0.1.el7.x86_64.rp | 421 kB 00:00

--------------------------------------------------------------------------------

Total 700 kB/s | 511 kB 00:00

Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-oracle

Importing GPG key 0xEC551F03:

Userid : "Oracle OSS group (Open Source Software group) <build@oss.oracle.com>"

Fingerprint: 4214 4123 fecf c55b 9086 313d 72f9 7b74 ec55 1f03

Package : 7:oraclelinux-release-7.4-1.0.4.el7.x86_64 (@anaconda/7.4)

From : /etc/pki/rpm-gpg/RPM-GPG-KEY-oracle

Is this ok [y/N]: y

Running transaction check

Running transaction test

Transaction test succeeded

Running transaction

Installing : iscsi-initiator-utils-iscsiuio-6.2.0.874-7.0.1.el7.x86_64 1/2

Installing : iscsi-initiator-utils-6.2.0.874-7.0.1.el7.x86_64 2/2

Verifying : iscsi-initiator-utils-6.2.0.874-7.0.1.el7.x86_64 1/2

Verifying : iscsi-initiator-utils-iscsiuio-6.2.0.874-7.0.1.el7.x86_64 2/2


Installed:

iscsi-initiator-utils.x86_64 0:6.2.0.874-7.0.1.el7


Dependency Installed:

iscsi-initiator-utils-iscsiuio.x86_64 0:6.2.0.874-7.0.1.el7


Complete!

Discover Targets on Narrabri

Ran the following command as root on REDFERN1 to discover what iSCSI targets are available on the NARRABRI NAS:

iscsiadm -m discovery -t sendtargets -p narrabri

The output is:

192.168.1.253:3260,1 iqn.1994-11.com.netgear:narrabri:4eca4c51:redfern

This is what I set up for the REDFERN cluster. See the following screen-shot from the NAS console:

Display Target Information on Narrabri

Ran the following command as root on REDFERN1 to populate the local iSCSI discovery database with information about the iSCSI targets presented by the NARRABRI NAS:

iscsiadm -m discoverydb -t sendtargets -p narrabri

The output is:

# BEGIN RECORD 6.2.0.874-7

discovery.startup = manual

discovery.type = sendtargets

discovery.sendtargets.address = narrabri

discovery.sendtargets.port = 3260

discovery.sendtargets.auth.authmethod = None

discovery.sendtargets.auth.username = <empty>

discovery.sendtargets.auth.password = <empty>

discovery.sendtargets.auth.username_in = <empty>

discovery.sendtargets.auth.password_in = <empty>

discovery.sendtargets.timeo.login_timeout = 15

discovery.sendtargets.use_discoveryd = No

discovery.sendtargets.discoveryd_poll_inval = 30

discovery.sendtargets.reopen_max = 5

discovery.sendtargets.timeo.auth_timeout = 45

discovery.sendtargets.timeo.active_timeout = 30

discovery.sendtargets.iscsi.MaxRecvDataSegmentLength = 32768

Establish Session with REDFERN Target on Narrabri

Ran the following command as root on REDFERN1 to start an iSCSI initiator session with the NARRABRI NAS:

iscsiadm -m node -T iqn.1994-11.com.netgear:narrabri:4eca4c51:redfern -p narrabri:3260 -l

The output is:

Login to [iface: default, target: iqn.1994-11.com.netgear:narrabri:4eca4c51:redfern, portal: 192.168.1.253,3260] successful.

Establish Session with REDFERN Target on Narrabri

Ran the following command as root on REDFERN1 to verify that an iSCSI initiator session has been established with the NARRABRI NAS:

iscsiadm -m session -P 3

The output is:

iSCSI Transport Class version 2.0-870

version 6.2.0.874-7

Target: iqn.1994-11.com.netgear:narrabri:4eca4c51:redfern (non-flash)

Current Portal: 192.168.1.253:3260,1

Persistent Portal: 192.168.1.253:3260,1

**********

Interface:

**********

Iface Name: default

Iface Transport: tcp

Iface Initiatorname: iqn.1988-12.com.oracle:d3436e6a9eca

Iface IPaddress: 192.168.1.141

Iface HWaddress: <empty>

Iface Netdev: <empty>

SID: 1

iSCSI Connection State: LOGGED IN

iSCSI Session State: LOGGED_IN

Internal iscsid Session State: NO CHANGE

*********

Timeouts:

*********

Recovery Timeout: 120

Target Reset Timeout: 30

LUN Reset Timeout: 30

Abort Timeout: 15

*****

CHAP:

*****

username: <empty>

password: ********

username_in: <empty>

password_in: ********

************************

Negotiated iSCSI params:

************************

HeaderDigest: None

DataDigest: None

MaxRecvDataSegmentLength: 262144

MaxXmitDataSegmentLength: 262144

FirstBurstLength: 65536

MaxBurstLength: 262144

ImmediateData: Yes

InitialR2T: Yes

MaxOutstandingR2T: 1

************************

Attached SCSI devices:

************************

Host Number: 2 State: running

scsi2 Channel 00 Id 0 Lun: 0

Attached scsi disk sda State: running

scsi2 Channel 00 Id 0 Lun: 1

Attached scsi disk sdc State: running

scsi2 Channel 00 Id 0 Lun: 2

Attached scsi disk sdd State: running

scsi2 Channel 00 Id 0 Lun: 3

Attached scsi disk sde State: running

scsi2 Channel 00 Id 0 Lun: 4

Attached scsi disk sdb State: running

A less verbose output is generated by:

iscsiadm -m session

The output is:

tcp: [1] 192.168.1.253:3260,1 iqn.1994-11.com.netgear:narrabri:4eca4c51:redfern (non-flash)

However, the first version gives a list of disks that have been mounted through iSCSI:

************************

Attached SCSI devices:

************************

Host Number: 2 State: running

scsi2 Channel 00 Id 0 Lun: 0

Attached scsi disk sda State: running

scsi2 Channel 00 Id 0 Lun: 1

Attached scsi disk sdc State: running

scsi2 Channel 00 Id 0 Lun: 2

Attached scsi disk sdd State: running

scsi2 Channel 00 Id 0 Lun: 3

Attached scsi disk sde State: running

scsi2 Channel 00 Id 0 Lun: 4

Attached scsi disk sdb State: running

See What Disks Have Been Mounted

Ran the following command on REDFERN1 to see what iSCSI LUNs have been mounted from the NARRABRI NAS:

ls -l /dev/disk/by-path/

The output is:

total 0

lrwxrwxrwx. 1 root root 9 Jun 2 21:27 ip-192.168.1.253:3260-iscsi-iqn.1994-11.com.netgear:narrabri:4eca4c51:redfern-lun-0 -> ../../sda

lrwxrwxrwx. 1 root root 9 Jun 2 21:27 ip-192.168.1.253:3260-iscsi-iqn.1994-11.com.netgear:narrabri:4eca4c51:redfern-lun-1 -> ../../sdc

lrwxrwxrwx. 1 root root 9 Jun 2 21:27 ip-192.168.1.253:3260-iscsi-iqn.1994-11.com.netgear:narrabri:4eca4c51:redfern-lun-2 -> ../../sdd

lrwxrwxrwx. 1 root root 9 Jun 2 21:27 ip-192.168.1.253:3260-iscsi-iqn.1994-11.com.netgear:narrabri:4eca4c51:redfern-lun-3 -> ../../sde

lrwxrwxrwx. 1 root root 9 Jun 2 21:27 ip-192.168.1.253:3260-iscsi-iqn.1994-11.com.netgear:narrabri:4eca4c51:redfern-lun-4 -> ../../sdb

lrwxrwxrwx. 1 root root 9 Jun 2 20:19 pci-0000:00:01.1-ata-2.0 -> ../../sr0