KVM DG Migration

In the project described here a KVM guest running Oracle ASM and RDBMS standalone Oracle Grid Infrastructure separation environment was cloned to an exact replica KVM guest with a new name using virt-clone tool. Then the cloned KVM guest was prepared for use as an Oracle Dataguard Standby 4K Native Migration host, which means that an Oracle Dataguard Standby will be used to migrate the source database on KVM guest oracle631.vmem.org to a standby database on oracle632.vmem.org using RMAN Active Duplicate procedure to create the standby database. This blog post describes the detailed steps to create the 4K native Oracle standby database for the 512-byte format Oracle primary database and then migrate the source Oracle database from 512-byte legacy storage to all-4K native format storage. This is the same procedure that would be used to do a similar operation to migrate a database on legacy 512-byte format storage to Violin Memory flash storage presented to the server in 4K native format (no 512-emulation).

Create the KVM Guest Clone

The initial clone was created as shown below.

gstanden@vmem1:~$ man virt-clone

gstanden@vmem1:~$ virt-clone --connect qemu:///system --original oracle631 --name oracle632 --file /var/lib/libvirt/images/oracle632-0.img --file /var/lib/libvirt/images/oracle632-1.u00.img

Allocating 'oracle632-0.img'                                                                                                                       |  30 GB     01:40

Allocating 'oracle632-1.u00.img'                                                                                                                   |  20 GB     01:30

Clone 'oracle632' created successfully.
gstanden@vmem1:~$

The xml for the newly-created KVM guest oracle632 is shown below.

gstanden@vmem1:~$ virsh dumpxml oracle632
<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
<name>oracle632</name>
<uuid>a55c4432-8f82-547d-ecc0-6d07a40d68d0</uuid>
<memory unit='KiB'>5242880</memory>
<currentMemory unit='KiB'>5242880</currentMemory>
<vcpu placement='static'>2</vcpu>
<resource>
    <partition>/machine</partition>
</resource>
<os>
    <type arch='x86_64' machine='pc-i440fx-trusty'>hvm</type>
    <boot dev='hd'/>
    <boot dev='cdrom'/>
    <bootmenu enable='yes'/>
</os>
<features>
    <acpi/>
    <apic/>
    <pae/>
</features>
<clock offset='utc'/>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>restart</on_crash>
<devices>
    <emulator>/usr/bin/kvm-spice</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='none' io='native'/>
      <source file='/var/lib/libvirt/images/oracle632-0.img'/>
      <target dev='vda' bus='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </disk>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='none' io='native'/>
      <source file='/var/lib/libvirt/images/oracle632-1.u00.img'/>
      <target dev='vdb' bus='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/home/gstanden/Downloads/V41362-01.iso'/>
      <target dev='hdc' bus='ide'/>
      <readonly/>
      <address type='drive' controller='0' bus='1' target='0' unit='0'/>
    </disk>
    <controller type='usb' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pci-root'/>
    <controller type='ide' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <interface type='bridge'>
      <mac address='52:54:00:a4:7a:0a'/>
      <source bridge='sw1'/>
      <virtualport type='openvswitch'>
        <parameters interfaceid='b046f2fe-5f84-4c5a-9f00-7bcc642ba4b1'/>
      </virtualport>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>
    <interface type='bridge'>
      <mac address='52:54:00:ec:cd:15'/>
      <source bridge='sw2'/>
      <virtualport type='openvswitch'>
        <parameters interfaceid='827841a4-2e7d-40b5-ac86-5161e19f546b'/>
      </virtualport>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
    </interface>
    <interface type='bridge'>
      <mac address='52:54:00:7f:65:e2'/>
      <source bridge='sw3'/>
      <virtualport type='openvswitch'>
        <parameters interfaceid='2ae36348-31d7-462b-b3f4-a67d302773ee'/>
      </virtualport>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>
    </interface>
    <serial type='pty'>
      <target port='0'/>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
    </console>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <graphics type='vnc' port='-1' autoport='yes' listen='127.0.0.1'>
      <listen type='address' address='127.0.0.1'/>
    </graphics>
    <sound model='ich6'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </sound>
    <video>
      <model type='cirrus' vram='9216' heads='1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>
    <memballoon model='virtio'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </memballoon>
</devices>
<seclabel type='dynamic' model='apparmor' relabel='yes'/>
<qemu:commandline>
    <qemu:arg value='-set'/>
    <qemu:arg value='device.virtio-disk0.scsi=off'/>
    <qemu:arg value='-set'/>
    <qemu:arg value='device.virtio-disk0.config-wce=off'/>
    <qemu:arg value='-set'/>
    <qemu:arg value='device.virtio-disk0.x-data-plane=on'/>
    <qemu:arg value='-set'/>
    <qemu:arg value='device.virtio-disk1.scsi=off'/>
    <qemu:arg value='-set'/>
    <qemu:arg value='device.virtio-disk1.config-wce=off'/>
    <qemu:arg value='-set'/>
    <qemu:arg value='device.virtio-disk1.x-data-plane=on'/>
</qemu:commandline>
</domain>

gstanden@vmem1:~$

Dump the xml of the source and target machine and do a linux diff to see the changes as shown below. Machine ID, MAC addresses and have all be changed in the cloned guest.

gstanden@vmem1:~$ virsh dumpxml oracle632 > oracle632.xml.diff
gstanden@vmem1:~$ virsh dumpxml oracle631 > oracle631.xml.diff
gstanden@vmem1:~$ diff oracle631.xml.diff oracle632.xml.diff
2,3c2,3
<   <name>oracle631</name>
<   <uuid>911e6458-20ba-11e4-a4f1-8b953fd3315f</uuid>
---
>   <name>oracle632</name>
>   <uuid>a55c4432-8f82-547d-ecc0-6d07a40d68d0</uuid>
29c29
<       <source file='/var/lib/libvirt/images/oracle631-0.img'/>
---
>       <source file='/var/lib/libvirt/images/oracle632-0.img'/>
35c35
<       <source file='/var/lib/libvirt/images/oracle631-1.u00.img'/>
---
>       <source file='/var/lib/libvirt/images/oracle632-1.u00.img'/>
54c54
<       <mac address='52:54:00:75:2b:37'/>
---
>       <mac address='52:54:00:a4:7a:0a'/>
59d58
<       <target dev='s3'/>
64c63
<       <mac address='52:54:00:03:40:a1'/>
---
>       <mac address='52:54:00:ec:cd:15'/>
69d67
<       <target dev='t3'/>
74c72
<       <mac address='52:54:00:e9:04:1d'/>
---
>       <mac address='52:54:00:7f:65:e2'/>
79d76
<       <target dev='w3'/>
gstanden@vmem1:~$

The OpenvSwitch sections will have to be edited manually as shown below since virt-clone has not changed the settings on the clone.

    <interface type='bridge'>
      <mac address='52:54:00:a4:7a:0a'/>
      <source bridge='sw1'/>
      <virtualport type='openvswitch'>
        <parameters interfaceid='b046f2fe-5f84-4c5a-9f00-7bcc642ba4b1'/>
      </virtualport>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>
    <interface type='bridge'>
      <mac address='52:54:00:ec:cd:15'/>
      <source bridge='sw2'/>
      <virtualport type='openvswitch'>
        <parameters interfaceid='827841a4-2e7d-40b5-ac86-5161e19f546b'/>
      </virtualport>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
    </interface>
    <interface type='bridge'>
      <mac address='52:54:00:7f:65:e2'/>
      <source bridge='sw3'/>
      <virtualport type='openvswitch'>
        <parameters interfaceid='2ae36348-31d7-462b-b3f4-a67d302773ee'/>
      </virtualport>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>
    </interface>

Change the settings so that the section above looks like the following as shown below, and then save the XML file.

    <interface type='bridge'>
      <mac address='52:54:00:a4:7a:0a'/>
      <source bridge='sw1'/>
      <virtualport type='openvswitch'/>

<target dev='s5'/>

      <model type='virtio'/>
    </interface>
    <interface type='bridge'>
      <mac address='52:54:00:ec:cd:15'/>
      <source bridge='sw2'/>
      <virtualport type='openvswitch'/>

<target dev='t5'/>

      <model type='virtio'/>
    </interface>
    <interface type='bridge'>
      <mac address='52:54:00:7f:65:e2'/>
      <source bridge='sw3'/>
      <virtualport type='openvswitch'/>

<target dev='w5'/>

<model type='virtio'/>
</interface>

Dump the XML file and note that KVM has updated the entries as shown below with additional lines of XML code.

There is some work that needs to be done on the ethernet interfaces, MAC addresses, and the SCST iSCSI SAN connectivity as indicated below.

Bringing up interface Auto_eth1: Device has different MAC address than expected, ignoring.
[FAILED]
Bringing up interface Auto_eth2: Device has different MAC address than expected, ignoring.
[FAILED]
Bringing up interface eth0: Device eth0 does not seem to be present, delaying initialization.
[FAILED]
Starting auditd: [ OK ]
Starting portreserve: [ OK ]
Starting system logger: [ OK ]
Starting irqbalance: [ OK ]

Starting iscsi: iscsiadm: Could not login to [iface: default, target: iqn.2014-08.oracle.asm.storage:luns, portal: 10.207.40.74,3260].

iscsiadm: initiator reported error (8 - connection timed out)
iscsiadm: Could not login to [iface: default, target: iqn.2014-08.oracle.asm.storage:luns, portal: 10.207.41.74,3260].
iscsiadm: initiator reported error (8 - connection timed out)
iscsiadm: Could not log into all portals

The /etc/udev/rules.d/70-persistent-net.rules file needs to be edited to use the new MAC addresses as shown below.

[root@oracle631 ~]# cat /etc/udev/rules.d/70-persistent-net.rules
# This file was automatically generated by the /lib/udev/write_net_rules
# program, run by the persistent-net-generator.rules rules file.
#
# You can modify it, as long as you keep each rule on a single
# line, and change only the value of the NAME= key.

# PCI device 0x1af4:0x1000 (virtio-pci)

SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="52:54:00:75:2b:37", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0"

# PCI device 0x1af4:0x1000 (virtio-pci)

SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="52:54:00:03:40:a1", ATTR{type}=="1", KERNEL=="eth*", NAME="eth1"

# PCI device 0x1af4:0x1000 (virtio-pci)

SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="52:54:00:e9:04:1d", ATTR{type}=="1", KERNEL=="eth*", NAME="eth2"

# PCI device 0x1af4:0x1000 (virtio-pci)

SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="52:54:00:a4:7a:0a", ATTR{type}=="1", KERNEL=="eth*", NAME="eth3"

# PCI device 0x1af4:0x1000 (virtio-pci)

SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="52:54:00:7f:65:e2", ATTR{type}=="1", KERNEL=="eth*", NAME="eth4"

# PCI device 0x1af4:0x1000 (virtio-pci)

SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="52:54:00:ec:cd:15", ATTR{type}=="1", KERNEL=="eth*", NAME="eth5"

[root@oracle631 ~]#

The current XML files for oracle631 source and oracle632 target can be dumped and used to determine what changes need to be made to the above /etc/udev/rules.d/70-persistent-net.rules file as shown below.

gstanden@vmem1:~$ virsh dumpxml oracle631 > oracle631.xml.diff
gstanden@vmem1:~$ virsh dumpxml oracle632 > oracle632.xml.diff
gstanden@vmem1:~$ diff oracle631.xml.diff oracle632.xml.diff
2,3c2,3
<   <name>oracle631</name>
<   <uuid>911e6458-20ba-11e4-a4f1-8b953fd3315f</uuid>
---
>   <name>oracle632</name>
>   <uuid>a55c4432-8f82-547d-ecc0-6d07a40d68d0</uuid>
29c29
<       <source file='/var/lib/libvirt/images/oracle631-0.img'/>
---
>       <source file='/var/lib/libvirt/images/oracle632-0.img'/>
35c35
<       <source file='/var/lib/libvirt/images/oracle631-1.u00.img'/>
---
>       <source file='/var/lib/libvirt/images/oracle632-1.u00.img'/>
54c54
<       <mac address='52:54:00:75:2b:37'/> <--Old MAC Address
---
>       <mac address='52:54:00:a4:7a:0a'/> <--New MAC Address
57c57
<         <parameters interfaceid='b046f2fe-5f84-4c5a-9f00-7bcc642ba4b1'/>
---
>         <parameters interfaceid='0a50175c-9b92-4c19-a5fb-f07e873b4321'/>
59c59
<       <target dev='s3'/>
---
>       <target dev='s5'/>
64c64
<       <mac address='52:54:00:03:40:a1'/> <--Old MAC Address
---
>       <mac address='52:54:00:ec:cd:15'/> <--New MAC Address
67c67
<         <parameters interfaceid='827841a4-2e7d-40b5-ac86-5161e19f546b'/>
---
>         <parameters interfaceid='cc66c6a4-56dd-4a04-8970-ecb6ce1d74a0'/>
69c69
<       <target dev='t3'/>
---
>       <target dev='t5'/>
74c74
<       <mac address='52:54:00:e9:04:1d'/> <--Old MAC Address
---
>       <mac address='52:54:00:7f:65:e2'/> <--New MAC Address
77c77
<         <parameters interfaceid='2ae36348-31d7-462b-b3f4-a67d302773ee'/>
---
>         <parameters interfaceid='f11d944c-e07c-4d02-b221-78467b0cf15c'/>
79c79
<       <target dev='w3'/>
---
>       <target dev='w5'/>
gstanden@vmem1:~$

The edited /etc/udev/rules.d/70-persistent-net.rules file is edited to use the new MAC addresses only as shown below.

# SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="52:54:00:75:2b:37", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0"

# PCI device 0x1af4:0x1000 (virtio-pci)

# SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="52:54:00:03:40:a1", ATTR{type}=="1", KERNEL=="eth*", NAME="eth1"

# PCI device 0x1af4:0x1000 (virtio-pci)

# SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="52:54:00:e9:04:1d", ATTR{type}=="1", KERNEL=="eth*", NAME="eth2"

# PCI device 0x1af4:0x1000 (virtio-pci)

SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="52:54:00:a4:7a:0a", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0"

# PCI device 0x1af4:0x1000 (virtio-pci)

SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="52:54:00:ec:cd:15", ATTR{type}=="1", KERNEL=="eth*", NAME="eth1"

# PCI device 0x1af4:0x1000 (virtio-pci)

SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="52:54:00:7f:65:e2", ATTR{type}=="1", KERNEL=="eth*", NAME="eth2"

[root@oracle631 ~]#

The following changes are made to the interface configuration files in /etc/sysconfig/network-scripts directory as shown below to update the MAC addresses similar to what was done for the 70-persistent-net.rules file.

[root@oracle631 network-scripts]# cat ifcfg-eth0 | grep HWADDR
# virt-clone oracle631 HWADDR=52:54:00:75:2B:37
HWADDR=52:54:00:A4:7A:0A
[root@oracle631 network-scripts]# cat ifcfg-Auto_eth1 | grep HWADDR
# virt-clone oracle631 HWADDR=52:54:00:03:40:A1
HWADDR=52:54:00:EC:CD:15
[root@oracle631 network-scripts]# cat ifcfg-Auto_eth2 | grep HWADDR
# virt-clone oracle631 HWADDR=52:54:00:E9:04:1D
HWADDR=52:54:00:7F:65:E2

[root@oracle631 network-scripts]#

After these changes have been made reboot the oracle632 clone KVM guest. As shown below, guests oracle651 the SCST iSCSI Linux SAN and oracle632 the KVM guest clone of oracle631 are the only KVM guests running.

gstanden@vmem1:~$ virsh -c qemu:///system list
Id    Name                           State
----------------------------------------------------
2     oracle651                      running
3     oracle632                      running

gstanden@vmem1:~$

This is the output of "crsctl stat res -t" on the oracle632 KVM clone (still has "oracle631.vmem.org" FQDN)

[root@oracle631 ~]# crsctl stat res -t
--------------------------------------------------------------------------------

Name           Target  State        Server                   State details

--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.DATA.dg
               ONLINE ONLINE       oracle631                STABLE
ora.LISTENER.lsnr
               ONLINE ONLINE       oracle631                STABLE
ora.RECO.dg
               ONLINE ONLINE       oracle631                STABLE
ora.SYSTEMDG.dg
               ONLINE ONLINE       oracle631                STABLE
ora.asm
               ONLINE ONLINE       oracle631                Started,STABLE
ora.ons
               OFFLINE OFFLINE      oracle631                STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.cssd
      1        ONLINE ONLINE       oracle631                STABLE
ora.diskmon
      1        OFFLINE OFFLINE                               STABLE
ora.evmd
      1        ONLINE ONLINE       oracle631                STABLE
ora.prod.db
      1        ONLINE ONLINE       oracle631                Open,STABLE
--------------------------------------------------------------------------------
[root@oracle631 ~]#

Shutdown oracle632 KVM clone and verify that the two servers are fully equivalent and are both able to fully start the Oracle ASM and RDBMS stacks. After oracle632 has shutdown, startup oracle631 and verify that the output of "crsctl stat res -t" also shows fully operatioin Oracle ASM and RDBMS stacks.

If the eth0 interface on the oracle631 KVM source guest was set to DHCP, it may have acquired a different IP address because the KVM target guest oracle632 may have leased the original IP address from DHCP that oracle631 originally had. If so, go into the GUI and edit "Network Connections" from the oracle631 desktop to change eth0 to be a Manual IP address. After making this change using the GUI tools, while connected as "root" to oracle631, do an "ifdown" and an "ifup" to put the new Manual IP address on eth0. Finally, it may be necessary to use srvctl to stop and start the listener if the IP address on eth0 was changed and was referenced in the listener.ora file, as shown below.

[grid@oracle631 ~]$ lsnrctl status

LSNRCTL for Linux: Version 12.1.0.2.0 - Production on 16-AUG-2014 20:56:46

Copyright (c) 1991, 2014, Oracle. All rights reserved.

Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=10.207.39.72)(PORT=1532)))
TNS-12541: TNS:no listener
TNS-12560: TNS:protocol adapter error
TNS-00511: No listener
   Linux Error: 111: Connection refused
Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=EXTPROC1532)))
TNS-12541: TNS:no listener
TNS-12560: TNS:protocol adapter error
TNS-00511: No listener
   Linux Error: 2: No such file or directory
[grid@oracle631 ~]$ srvctl start listener
[grid@oracle631 ~]$ lsnrctl status

LSNRCTL for Linux: Version 12.1.0.2.0 - Production on 16-AUG-2014 20:57:14

Copyright (c) 1991, 2014, Oracle. All rights reserved.

Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=10.207.39.72)(PORT=1532)))
STATUS of the LISTENER
------------------------
Alias                     LISTENER
Version                   TNSLSNR for Linux: Version 12.1.0.2.0 - Production
Start Date                16-AUG-2014 20:57:00
Uptime                    0 days 0 hr. 0 min. 14 sec
Trace Level               off
Security                  ON: Local OS Authentication
SNMP                      OFF
Listener Parameter File   /u00/app/12.1.0/product/grid/network/admin/listener.ora
Listener Log File         /u00/app/12.1.0/diag/tnslsnr/oracle631/listener/alert/log.xml
Listening Endpoints Summary...
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=10.207.39.72)(PORT=1532)))
(DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=EXTPROC1532)))
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=127.0.1.1)(PORT=1532)))
Services Summary...
Service "+ASM" has 1 instance(s).
Instance "+ASM", status READY, has 1 handler(s) for this service...
The command completed successfully
[grid@oracle631 ~]$

However, because oracle631 and oracle632 are identical the cannot currently be brought up at the same time it is now time to shutdown oracle631 now that it has been verified that both boxes can bring up the Oracle ASM and Oracle RDBMS stacks.

Currently, both KVM guests have the same FQDN hostname, namely, "oracle631.vmem.org". To prepare this KVM guest to be FQDN "oracle632.vmem.org" the Oracle Dataguard Migration Standby for oracle631, it will be necessary to do several things as shown below.

Detach the SCST iSCSI Linux SAN oracle651 from the oracle632 KVM clone

Rename the FQDN hostname of oracle632 KVM clone to "oracle632.vmem.org"

Change the IP addresses of all the ethernet interfaces

Update the IP addresses (or FQDN if used) values in $GRID_HOME/network/admin/listener.ora

Update the IP addresses (or FQDN if used) values in $GRID_HOME/network/admin/tnsnames.ora

Delete the database from server configuration files ("dbca" GUI tool)

Delete the ASM instance from server configuration files ("deinstall" CLI tool)

Install a new ASM instance with new SCST iSCSI Linux SAN 4K native LUNs of it's own

Use RMAN Active Duplicate to create the Oracle Dataguard Standby Migration instance.

Set the login to the SCST iSCSI Linux SAN to "manual" as shown below so that the Oracle ASM instance and Oracle RDBMS will not have access to the LUNs of the oracle631 source server as shown below.

[root@oracle631 ~]# iscsiadm -m node -T iqn.2014-08.oracle.asm.storage:luns -p 10.207.40.74 --op update -n node.startup -v manual

[root@oracle631 ~]# iscsiadm -m node -T iqn.2014-08.oracle.asm.storage:luns -p 10.207.41.74 --op update -n node.startup -v manual

[root@oracle631 ~]

Change the hostname of the oracle632 KVM guest to match its' name as shown below. Npte that the edit of /etc/hosts requires an IP changes as well.

[root@oracle631 ~]# vi /etc/sysconfig/network

[root@oracle631 ~]# cat /etc/sysconfig/network

NETWORKING=yes
HOSTNAME=oracle632.vmem.org
NTPSERVERARGS=iburst
[root@oracle631 ~]# vi /etc/hosts

[root@oracle631 ~]# cat /etc/hosts

127.0.0.1    localhost localhost.localdomain localhost4 localhost4.localdomain4
::1          localhost localhost.localdomain localhost6 localhost6.localdomain6
127.0.1.1    oracle632
10.207.39.76 oracle632.vmem.org    oracle632

[root@oracle631 ~]#

Next, set the IP addresses of the ethernet interfaces to a new set of matched manual values as indicated below. The GUI was used to reset all of the below interfaces to Manual setting to the ".76" IP addresses, which will take effect on reboot as shown below after reboot. Notice that both the linux "grid" and "oracle" user have no processes running. The server has been completely detached from the SCST iSCSI Linux SAN, there are no LUNs listed in /dev/mapper for ASM, and the hostname and IP addresses have been updated to new values.

[root@oracle632 ~]# ifconfig
eth0      Link encap:Ethernet  HWaddr 52:54:00:A4:7A:0A  
          inet addr:10.207.39.76  Bcast:10.207.39.255  Mask:255.255.255.0
          inet6 addr: fe80::5054:ff:fea4:7a0a/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:321 errors:0 dropped:0 overruns:0 frame:0
          TX packets:237 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:30466 (29.7 KiB)  TX bytes:29669 (28.9 KiB)

eth1      Link encap:Ethernet  HWaddr 52:54:00:EC:CD:15  
          inet addr:10.207.40.76  Bcast:10.207.40.255  Mask:255.255.255.0
          inet6 addr: fe80::5054:ff:feec:cd15/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1 errors:0 dropped:0 overruns:0 frame:0
          TX packets:21 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:42 (42.0 b)  TX bytes:1526 (1.4 KiB)

eth2      Link encap:Ethernet  HWaddr 52:54:00:7F:65:E2  
          inet addr:10.207.41.76  Bcast:10.207.41.255  Mask:255.255.255.0
          inet6 addr: fe80::5054:ff:fe7f:65e2/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:10 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 b)  TX bytes:656 (656.0 b)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:16 errors:0 dropped:0 overruns:0 frame:0
          TX packets:16 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:1342 (1.3 KiB)  TX bytes:1342 (1.3 KiB)

[root@oracle632 ~]# ls -lrt /dev/mapper
total 0
crw-rw---- 1 root root 10, 236 Aug 16 21:40 control
lrwxrwxrwx 1 root root       7 Aug 16 21:40 vg_oracle631-lv_swap -> ../dm-1
lrwxrwxrwx 1 root root       7 Aug 16 21:40 vg_oracle631-lv_root -> ../dm-0
lrwxrwxrwx 1 root root       7 Aug 16 21:40 vg_u00-lv_oracle -> ../dm-2
[root@oracle632 ~]# ps -ef | grep grid
root      3095  2890  0 21:42 pts/0    00:00:00 grep grid
[root@oracle632 ~]# ps -ef | grep oracle
root      3097  2890  0 21:42 pts/0    00:00:00 grep oracle
[root@oracle632 ~]#

[root@oracle631 ~]#

Now startup oracle631 source KVM guest to verify continued correct startup of Oracle ASM and RDBMS stacks.

[root@oracle631 ~]# crsctl stat res -t
--------------------------------------------------------------------------------
Name           Target  State        Server                   State details       
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.DATA.dg
               ONLINE  ONLINE       oracle631                STABLE
ora.LISTENER.lsnr
               ONLINE  ONLINE       oracle631                STABLE
ora.RECO.dg
               ONLINE  ONLINE       oracle631                STABLE
ora.SYSTEMDG.dg
               ONLINE  ONLINE       oracle631                STABLE
ora.asm
               ONLINE  ONLINE       oracle631                Started,STABLE
ora.ons
               OFFLINE OFFLINE      oracle631                STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.cssd
      1        ONLINE  ONLINE       oracle631                STABLE
ora.diskmon
      1        OFFLINE OFFLINE                               STABLE
ora.evmd
      1        ONLINE  ONLINE       oracle631                STABLE
ora.prod.db
      1        ONLINE  ONLINE       oracle631                Open,STABLE
--------------------------------------------------------------------------------
[root@oracle631 ~]#

Next connect to the oracle632 database as "oracle" using the following commands as shown below and start the "dbca" tool.

gstanden@vmem1:~$ ssh -Y -C oracle@10.207.39.76
oracle@10.207.39.76's password:
Last login: Wed Aug 13 14:39:36 2014 from 10.207.39.1
[oracle@oracle632 ~]$ dbca

Choose "Delete Database" --> "sid" (in this case "prod") --> Answer "Yes" to warning about deleting database including datafiles. Note: because ASM is completely disconnected from the oracle632 KVM guest, the dbca delete will not be able to delete anything except configuration files. It is safe to answer "Yes" to this warning. Next it will give the popup warning that the datafiles could not be accessed and that the database is not running. Only the configuration files will be deleted. Answer "OK" to this advice also to delete the service for the database. Now it reports database deletion completed, and answer "No" to the question of whether to perform another operation. The work with dbca is complete.

Next deconfigure the Oracle Grid Infrastructure 12c and the Oracle 12c ASM configuration. This is done using the "deinstall" command as the "grid" Linux user owner of the GI and ASM install as shown below.

[grid@oracle632 deinstall]$ ./deinstall
Checking for required files and bootstrapping ...
Please wait ...
Location of logs /u00/app/oraInventory/logs/

############ ORACLE DECONFIG TOOL START ############

######################### DECONFIG CHECK OPERATION START #########################
## [START] Install check configuration ##

Checking for existence of the Oracle home location /u00/app/12.1.0/product/grid
Oracle Home type selected for deinstall is: Oracle Grid Infrastructure for a Standalone Server
Oracle Base selected for deinstall is: /u00/app/12.1.0
Checking for existence of central inventory location /u00/app/oraInventory
Checking for existence of the Oracle Grid Infrastructure home /u00/app/12.1.0/product/grid

## [END] Install check configuration ##

Traces log file: /u00/app/oraInventory/logs//crsdc_2014-08-16_10-06-43PM.log

Network Configuration check config START

Network de-configuration trace file location: /u00/app/oraInventory/logs/netdc_check2014-08-16_10-06-43-PM.log

Specify all Oracle Restart enabled listeners that are to be de-configured. Enter .(dot) to deselect all. [LISTENER]: <-- Accept Default

Network Configuration check config END

Asm Check Configuration START

ASM de-configuration trace file location: /u00/app/oraInventory/logs/asmcadc_check2014-08-16_10-08-37-PM.log

Automatic Storage Management (ASM) instance is detected in this Oracle home /u00/app/12.1.0/product/grid.
ASM Diagnostic Destination : /u00/app/12.1.0
ASM Diskgroups :
ASM diskstring : <Default>
Diskgroups will not be dropped

 If you want to retain the existing diskgroups or if any of the information detected is incorrect, you can modify by entering 'y'. Do you  want to modify above information (y|n) [n]:  <-- Accept Default

Database Check Configuration START

Database de-configuration trace file location: /u00/app/oraInventory/logs/databasedc_check2014-08-16_10-10-24-PM.log

Database Check Configuration END

######################### DECONFIG CHECK OPERATION END #########################

####################### DECONFIG CHECK OPERATION SUMMARY #######################
Oracle Grid Infrastructure Home is: /u00/app/12.1.0/product/grid
The following nodes are part of this cluster: null
The cluster node(s) on which the Oracle home deinstallation will be performed are:null
Oracle Home selected for deinstall is: /u00/app/12.1.0/product/grid
Inventory Location where the Oracle home registered is: /u00/app/oraInventory
Following Oracle Restart enabled listener(s) will be de-configured: LISTENER
ASM instance will be de-configured from this Oracle home
Do you want to continue (y - yes, n - no)? [n]: y <--Answer "y"

A log of this session will be written to: '/u00/app/oraInventory/logs/deinstall_deconfig2014-08-16_10-06-42-PM.out'

Any error messages from this session will be written to: '/u00/app/oraInventory/logs/deinstall_deconfig2014-08-16_10-06-42-PM.err'

######################## DECONFIG CLEAN OPERATION START ########################
Database de-configuration trace file location: /u00/app/oraInventory/logs/databasedc_clean2014-08-16_10-11-18-PM.log
ASM de-configuration trace file location: /u00/app/oraInventory/logs/asmcadc_clean2014-08-16_10-11-18-PM.log
ASM Clean Configuration START
ASM Clean Configuration END

Network Configuration clean config START

Network de-configuration trace file location: /u00/app/oraInventory/logs/netdc_clean2014-08-16_10-11-21-PM.log

De-configuring Oracle Restart enabled listener(s): LISTENER

De-configuring listener: LISTENER
    Stopping listener: LISTENER
    Warning: Failed to stop listener. Listener may not be running.
    Deleting listener: LISTENER
    Listener deleted successfully.
Listener de-configured successfully.

De-configuring Listener configuration file...
Listener configuration file de-configured successfully.

De-configuring Naming Methods configuration file...
Naming Methods configuration file de-configured successfully.

De-configuring Local Net Service Names configuration file...
Local Net Service Names configuration file de-configured successfully.

De-configuring backup files...
Backup files de-configured successfully.

The network configuration has been cleaned up successfully.

Network Configuration clean config END

---------------------------------------->

Run the following command as the root user or the administrator on node "oracle632".

/tmp/deinstall2014-08-16_10-06-20PM/perl/bin/perl -I/tmp/deinstall2014-08-16_10-06-20PM/perl/lib -I/tmp/deinstall2014-08-16_10-06-20PM/crs/install /tmp/deinstall2014-08-16_10-06-20PM/crs/install/roothas.pl -force  -deconfig -paramfile "/tmp/deinstall2014-08-16_10-06-20PM/response/deinstall_OraGI12Home1.rsp"

Press Enter after you finish running the above commands

<----------------------------------------

Since the command that has to be run as root is a long command that likely spans more than one line, it is recommended to create an executable file "deconfig-gi-asm.sh" and run the command from that file, as root, after verifying that the command is one single line with no returns at the line break, as shown below.

[root@oracle632 oracle-build-scripts]# pwd
/root/oracle-build-scripts
[root@oracle632 oracle-build-scripts]# cat deconfig-gi-asm.sh

/tmp/deinstall2014-08-16_10-06-20PM/perl/bin/perl -I/tmp/deinstall2014-08-16_10-06-20PM/perl/lib -I/tmp/deinstall2014-08-16_10-06-20PM/crs/install /tmp/deinstall2014-08-16_10-06-20PM/crs/install/roothas.pl -force  -deconfig -paramfile "/tmp/deinstall2014-08-16_10-06-20PM/response/deinstall_OraGI12Home1.rsp"

[root@oracle632 oracle-build-scripts]# ls -lrt deconfig-gi-asm.sh
-rwxr-xr-x 1 root root 310 Aug 16 22:15 deconfig-gi-asm.sh
[root@oracle632 oracle-build-scripts]# id
uid=0(root) gid=0(root) groups=0(root)
[root@oracle632 oracle-build-scripts]#

Go ahead and run the script as root as shown below. The deconfiguration step runs. Some commands fail because these are steps which are designed to shutdown GI and processes, and since they are not currently running, they cannot be stopped, thus the commands fail. The overall result however is reported as success.

[root@oracle632 oracle-build-scripts]# ./deconfig-gi-asm.sh | tee deconfig-gi-asm.log
Using configuration parameter file: /tmp/deinstall2014-08-16_10-06-20PM/response/deinstall_OraGI12Home1.rsp
CRS-4639: Could not contact Oracle High Availability Services
CRS-4000: Command Stop failed, or completed with errors.
CRS-4639: Could not contact Oracle High Availability Services
CRS-4000: Command Delete failed, or completed with errors.
CLSU-00100: operating system function: opendir failed with error data: 2
CLSU-00101: operating system error message: No such file or directory
CLSU-00103: error location: scrsearch1
CLSU-00104: additional error information: cant open scr home dir scls_scr_getval
CRS-4639: Could not contact Oracle High Availability Services
CRS-4000: Command Stop failed, or completed with errors.
2014/08/16 22:19:17 CLSRSC-337: Successfully deconfigured Oracle Restart stack

[root@oracle632 oracle-build-scripts]#

Now return to the main deinstall window and resume the deinstall after successful run of root script.
\

######################### DECONFIG CLEAN OPERATION END #########################

####################### DECONFIG CLEAN OPERATION SUMMARY #######################
ASM instance was de-configured successfully from the Oracle home
Following Oracle Restart enabled listener(s) were de-configured successfully: LISTENER
Oracle Restart was already stopped and de-configured on node "oracle632"
Oracle Restart is stopped and de-configured successfully.
#######################################################################

############# ORACLE DECONFIG TOOL END #############

Using properties file /tmp/deinstall2014-08-16_10-06-20PM/response/deinstall_2014-08-16_10-06-42-PM.rsp
Location of logs /u00/app/oraInventory/logs/

############ ORACLE DEINSTALL TOOL START ############

####################### DEINSTALL CHECK OPERATION SUMMARY #######################
A log of this session will be written to: '/u00/app/oraInventory/logs/deinstall_deconfig2014-08-16_10-06-42-PM.out'

Any error messages from this session will be written to: '/u00/app/oraInventory/logs/deinstall_deconfig2014-08-16_10-06-42-PM.err'

######################## DEINSTALL CLEAN OPERATION START ########################
## [START] Preparing for Deinstall ##
Setting LOCAL_NODE to oracle632
Setting CRS_HOME to true
Setting oracle.installer.invPtrLoc to /tmp/deinstall2014-08-16_10-06-20PM/oraInst.loc
Setting oracle.installer.local to false

## [END] Preparing for Deinstall ##

Setting the force flag to false
Setting the force flag to cleanup the Oracle Base
Oracle Universal Installer clean START

Detach Oracle home '/u00/app/12.1.0/product/grid' from the central inventory on the local node : Done

Delete directory '/u00/app/12.1.0/product/grid' on the local node : Done

The Oracle Base directory '/u00/app/12.1.0' will not be removed on local node. The directory is not empty.

Oracle Universal Installer cleanup was successful.

Oracle Universal Installer clean END

## [START] Oracle install clean ##

Clean install operation removing temporary directory '/tmp/deinstall2014-08-16_10-06-20PM' on node 'oracle632'

## [END] Oracle install clean ##

######################### DEINSTALL CLEAN OPERATION END #########################

####################### DEINSTALL CLEAN OPERATION SUMMARY #######################
Successfully detached Oracle home '/u00/app/12.1.0/product/grid' from the central inventory on the local node.
Successfully deleted directory '/u00/app/12.1.0/product/grid' on the local node.
Oracle Universal Installer cleanup was successful.

Oracle deinstall tool successfully cleaned up temporary directories.
#######################################################################

############# ORACLE DEINSTALL TOOL END #############

The GI and ASM deinstall is completed successfully. Using "virsh console oracle632" reboot the oracle632 KVM guest and verify that console output is clean and shows no Oracle startup or shutdown activity as shown below.

[root@oracle632 ~]# shutdown -h now

Broadcast message from root@oracle632.vmem.org
(/dev/ttyS0) at 22:27 ...

The system is going down for halt NOW!
Stopping certmonger: [ OK ]
Stopping atd: [ OK ]
Stopping cups: [ OK ]
Stopping abrt daemon: [ OK ]
Stopping sshd: [ OK ]
Shutting down postfix: [ OK ]
Stopping mcelog
Stopping crond: [ OK ]
Stopping automount: [ OK ]
Stopping acpi daemon: [ OK ]
Stopping HAL daemon: [ OK ]
Shutting down ntpd: [ OK ]
Stopping block device availability: Deactivating block devices:
[UMOUNT]: unmounting vg_u00-lv_oracle (dm-2) mounted on /u00... done
[SKIP]: unmount of vg_oracle631-lv_root (dm-0) mounted on /
[LVM]: deactivating Volume Group vg_u00... done
[ OK ]
Stopping NetworkManager daemon: [ OK ]
Stopping system message bus: [ OK ]
Stopping multipathd daemon: [ OK ]
Stopping rpcbind: [ OK ]
Stopping auditd: type=1305 audit(1408246073.447:16): audit_pid=0 old=1928 auid=4294967295 ses=4294967295 res=1
[ OK ]
type=1305 audit(1408246073.554:17): audit_enabled=0 old=1 auid=4294967295 ses=4294967295 res=1
Shutting down system logger: [ OK ]
Shutting down interface Auto_eth1: [ OK ]
Shutting down interface Auto_eth2: [ OK ]
Shutting down interface eth0: [ OK ]
Shutting down loopback interface: [ OK ]
ip6tables: Setting chains to policy ACCEPT: filter [ OK ]
ip6tables: Flushing firewall rules: [ OK ]
ip6tables: Unloading modules: [ OK ]
iptables: Setting chains to policy ACCEPT: filter [ OK ]
iptables: Flushing firewall rules: [ OK ]
iptables: Unloading modules: [ OK ]
Stopping monitoring for VG vg_oracle631: 2 logical volume(s) in volume group "vg_oracle631" unmonitored
[ OK ]
Stopping monitoring for VG vg_u00: [ OK ]
Sending all processes the TERM signal... [ OK ]
Sending all processes the KILL signal... [ OK ]
Saving random seed: [ OK ]
Syncing hardware clock to system time [ OK ]
Turning off swap: [ OK ]
Turning off quotas: [ OK ]
Unmounting file systems: [ OK ]
init: Re-executing /sbin/init
EXT4-fs (dm-0): re-mounted. Opts: (null)
Halting system...
kvm: exiting hardware virtualization
ACPI: Preparing to enter system sleep state S5
Power down.

gstanden@vmem1:~$

Startup oracle632 and monitor console output using "virsh console oracle632". The server should startup normally. The cloned KVM guest oracle632 is now ready for configuration as a Oracle Dataguard Standby Migration KVM guest server.

RMAN Active Duplicate Script

DUPLICATE TARGET DATABASE



FOR STANDBY



FROM ACTIVE DATABASE



DORECOVER



SPFILE



SET cluster_database='false'



SET db_unique_name='suntest2'



SET log_archive_dest_2='SERVICE=suntest ASYNC VALID_FOR=(ONLINE_LOGFILES,PRIMARY_ROLE) DB_UNIQUE_NAME=suntest'



SET fal_server='suntest'



SET db_recovery_file_dest='+FRAT'



SET db_create_online_log_dest_1='+FRAT'



SET db_create_online_log_dest_2='+FRAT'



SET db_file_name_convert='+APPA','+APPA'



NOFILENAMECHECK;

Subpages (1): RMAN Script

Comments

nandydandyoracle	Search this site
Sitemap