Richmond Cluster (05)

Saturday 03 May, 2008 - 09:45

Set up the ssh user equivalency between central, richmond1, and richmond2 by going to central to download the ssh public keys from the other servers:

$ su - oracle

$ cd .ssh

$ rm known_hosts (stops conflict over the same IP address)

$ sftp richmond1-mgt

cd .ssh

get id_dsa.pub richmond1_dsa.pub

get id_rsa.pub richmond1_rsa.pub

exit

Repeated for richmond2.

Create a single repository for all public keys for oracle on central, then copied to the other servers:

$ su - oracle

$ cd .ssh

$ cat *.pub >authorized_keys (overwrites previous version of authorized_keys)

$ scp authorized_keys richmond1-mgt:.ssh/

$ scp authorized_keys richmond2-mgt:.ssh/

On each of central, richmond1, and richmond2, establish the bona-fides of the servers to each other (including itself to itself) by executing the following series of commands (the date command is just a simple command to execute and return to the calling host):

ssh-agent $SHELL

ssh-add

ssh central date

ssh central.yaocm.id.au date

ssh richmond1 date

ssh richmond1.yaocm.id.au date

ssh richmond1-mgt date

ssh richmond1-mgt.yaocm.id.au date

ssh richmond2 date

ssh richmond2.yaocm.id.au date

ssh richmond2-mgt date

ssh richmond2-mgt.yaocm.id.au date

Now to load the clusterware installation software on both richmond1 and richmond2:

$ su - oracle

$ mkdir installation

$ cd installation

$ unzip /mnt/hgfs/OCM/10201_clusterware_linux32.zip

Installed cvuqdisk package on both richmond1 and richmond2:

$ su -

# cd /home/oracle/installation/clusterware/rpm

# ls

cvuqdisk-1.0.1-1.rpm

# rpmquery cvuqdisk

package cvuqdisk is not installed

# export CVUQDISK_GRP=oinstall

# rpm -iv cvuqdisk-1.0.1-1.rpm

Preparing packages for installation...

cvuqdisk-1.0.1-1

# rpmquery cvuqdisk

cvuqdisk-1.0.1-1

On richmond1,

$ su -

# parted /dev/sdb

GNU Parted 1.6.3

...

Using /dev/sdb

Error: Unable to open /dev/sdb - unrecognised disk label.

Information: The operating system thinks the geometry on /dev/sdb is 261/255/63.

(parted) mklabel gpt

(parted) mkpart primary ext3 0 200

(parted) mkpart primary ext3 200 250

(parted) print

Disk geometry for /dev/sdb: 0.000-2048.000 megabytes

Disk label type: gpt

Minor Start End Filesystem Name Flags

1 0.017 200.000

2 200.000 250.000

(parted) select /dev/sdc

Using /dev/sdc

Error: Unable to open /dev/sdc - unrecognised disk label.

Information: The operating system thinks the geometry on /dev/sdc is 261/255/63.

(parted) mklabel gpt

(parted) mkpart primary ext3 0 200

(parted) mkpart primary ext3 200 250

(parted) print

Disk geometry for /dev/sdc: 0.000-2048.000 megabytes

Disk label type: gpt

Minor Start End Filesystem Name Flags

1 0.017 200.000

2 200.000 250.000

(parted) select /dev/sdd

Using /dev/sdd

Error: Unable to open /dev/sdd - unrecognised disk label.

Information: The operating system thinks the geometry on /dev/sdd is 261/255/63.

(parted) mklabel gpt

(parted) mkpart primary ext3 0 200

(parted) mkpart primary ext3 200 250

(parted) print

Disk geometry for /dev/sdd: 0.000-2048.000 megabytes

Disk label type: gpt

Minor Start End Filesystem Name Flags

1 0.017 200.000

2 200.000 250.000

(parted) quit

Information: Don't forget to update /etc/fstab, if necessary.

However when I use parted on richmond2, I get the following:

$ su -

# parted /dev/sdb

...

Using /dev/sdb

Warning: /dev/sdb contains GPT signatures, indicating that it has a GPT table. However, it does not have a valid fake msdos

partition table, as it should. Perhaps it was corrupted - possibly by a program that doesn't understand GPT partition

tables. Or perhaps you deleted the GPT table, and are now using an msdos partition table. Is this a GPT partition table?

Yes/No? yes

Error: The primary GPT table is corrupt, but the backup appears ok, so that will be used.

OK/Cancel? OK

Information: The operating system thinks the geometry on /dev/sdb is 261/255/63.

(parted) print

Disk geometry for /dev/sdb: 0.000-2048.000 megabytes

Disk label type: gpt

Minor Start End Filesystem Name Flags

1 0.017 200.000

2 200.000 250.000

This was also true for /dev/sdc and /dev/sdd as well. But this occurred only the first time on richmond2.

At this stage, I should be ready to install the clusterware. So, I run a preinstallation check:

$ su - oracle

$ cd ~/installation/clusterware/cluvfy

$ ./runcluvfy stage -pre crsinst -n richmond1,richmond2 -r 10gR2 -c /dev/sdb1 -q /dev/sdb2 -osdba dba -oinv oinstall -verbose

Performing pre-checks for cluster services setup

Checking node reachability...

Check: Node reachability from node "richmond1"

Destination Node Reachable?

------------------------------------ ------------------------

richmond2 yes

richmond1 yes

Result: Node reachability check passed from node "richmond1".

Checking user equivalence...

Check: User equivalence for user "oracle"

Node Name Comment

------------------------------------ ------------------------

richmond2 passed

richmond1 passed

Result: User equivalence check passed for user "oracle".

Checking administrative privileges...

Check: Existence of user "oracle"

Node Name User Exists Comment

------------ ------------------------ ------------------------

richmond2 yes passed

richmond1 yes passed

Result: User existence check passed for "oracle".

Check: Existence of group "oinstall"

Node Name Status Group ID

------------ ------------------------ ------------------------

richmond2 exists 600

richmond1 exists 600

Result: Group existence check passed for "oinstall".

Check: Membership of user "oracle" in group "oinstall" [as Primary]

Node Name User Exists Group Exists User in Group Primary Comment

---------------- ------------ ------------ ------------ ------------ ------------

richmond2 yes yes yes yes passed

richmond1 yes yes yes yes passed

Result: Membership check for user "oracle" in group "oinstall" [as Primary] passed.

Administrative privileges check passed.

Checking node connectivity...

Interface information for node "richmond2"

Interface Name IP Address Subnet

------------------------------ ------------------------------ ----------------

eth0 192.168.100.66 192.168.100.0

eth1 192.168.101.2 192.168.101.0

Interface information for node "richmond1"

Interface Name IP Address Subnet

------------------------------ ------------------------------ ----------------

eth0 192.168.100.65 192.168.100.0

eth1 192.168.101.1 192.168.101.0

Check: Node connectivity of subnet "192.168.100.0"

Source Destination Connected?

------------------------------ ------------------------------ ----------------

richmond2:eth0 richmond1:eth0 yes

Result: Node connectivity check passed for subnet "192.168.100.0" with node(s) richmond2,richmond1.

Check: Node connectivity of subnet "192.168.101.0"

Source Destination Connected?

------------------------------ ------------------------------ ----------------

richmond2:eth1 richmond1:eth1 yes

Result: Node connectivity check passed for subnet "192.168.101.0" with node(s) richmond2,richmond1.

Suitable interfaces for the private interconnect on subnet "192.168.100.0":

richmond2 eth0:192.168.100.66

richmond1 eth0:192.168.100.65

Suitable interfaces for the private interconnect on subnet "192.168.101.0":

richmond2 eth1:192.168.101.2

richmond1 eth1:192.168.101.1

ERROR:

Could not find a suitable set of interfaces for VIPs.

Result: Node connectivity check failed.

Checking shared storage accessibility...

ERROR: /dev/sdb1

Unable to determine the sharedness of /dev/sdb on nodes:

richmond2,richmond1

Shared storage check failed on nodes "richmond2,richmond1".

Checking shared storage accessibility...

ERROR: /dev/sdb2

Unable to determine the sharedness of /dev/sdb on nodes:

richmond2,richmond1

Shared storage check failed on nodes "richmond2,richmond1".

Checking system requirements for 'crs'...

Check: Total memory