In last article[click here] - we learnt how to remove quorum disk from the Exadata system, now here we would see it we add and configure that.
Before we are adding quorum disk configuration in the system, you need to have IB switches interface names, ASM binary owner and group handy as that is required to create the config.
[root@exa01dbadm01 oracle.SupportTools]# /opt/oracle.SupportTools/quorumdiskmgr --create --config --owner=oragrid --group=asmadmin --network-iface-list="ib0, ib1"
[Info] Successfully created iface exadata_ib0 with iface.net_ifacename ib0
[Info] Successfully created iface exadata_ib1 with iface.net_ifacename ib1
[Success] Successfully created quorum disk configurations
[root@exa01dbadm01 oracle.SupportTools]#Do the same as above on node2 as well.
[root@exa01dbadm02 oracle.SupportTools]# /opt/oracle.SupportTools/quorumdiskmgr --create --config --owner=oragrid --group=asmadmin --network-iface-list="ib0, ib1"
[Info] Successfully created iface exadata_ib0 with iface.net_ifacename ib0
[Info] Successfully created iface exadata_ib1 with iface.net_ifacename ib1
[Success] Successfully created quorum disk configurations
[root@exa01dbadm02 oracle.SupportTools]#Check if the quoum disk configuration is created as below from node1 and node2.
[root@exa01dbadm01 oracle.SupportTools]# /opt/oracle.SupportTools/quorumdiskmgr --list --config
Owner: oragrid
Group: asmadmin
ifaces: exadata_ib1 exadata_ib0
Initiatior name: iqn.1988-12.com.oracle:192.168.10.1
[root@exa01dbadm01 oracle.SupportTools]#[root@exa01dbadm02 oracle.SupportTools]# /opt/oracle.SupportTools/quorumdiskmgr --list --config
Owner: oragrid
Group: asmadmin
ifaces: exadata_ib1 exadata_ib0
Initiatior name: iqn.1988-12.com.oracle:192.168.10.3
[root@exa01dbadm02 oracle.SupportTools]#Now, create quorum disk target for DATAC1 diskgroup which is visible to both compute nodes, before it you need to identify IB interface IPs of both nodes as that is required here.
Run the command on node1 and node2.
[root@exa01dbadm01 oracle.SupportTools]# /opt/oracle.SupportTools/quorumdiskmgr --create --target --asm-disk-group=datac1 --visible-to="192.168.10.1, 192.168.10.2, 192.168.10.3, 192.168.10.4"
[Success] Created logical volume /dev/VGExaDb/LVDbVdexa01dbadm01DATAC1.
[Success] Created backstore QD_DATAC1_exa01dbadm01.
[Success] Created target iqn.2015-05.com.oracle:qd--datac1--exa01dbadm01.
[root@exa01dbadm01 oracle.SupportTools]#[root@exa01dbadm02 oracle.SupportTools]# /opt/oracle.SupportTools/quorumdiskmgr --create --target --asm-disk-group=datac1 --visible-to="192.168.10.1, 192.168.10.2, 192.168.10.3, 192.168.10.4"
[Success] Created logical volume /dev/VGExaDb/LVDbVdexa01dbadm02DATAC1.
[Success] Created backstore QD_DATAC1_exa01dbadm02.
[Success] Created target iqn.2015-05.com.oracle:qd--datac1--exa01dbadm02.
[root@exa01dbadm02 oracle.SupportTools]#Now you can list the quorum disk targets on node1 and node2 as below to validate.
[root@exa01dbadm01 oracle.SupportTools]# /opt/oracle.SupportTools/quorumdiskmgr --list --target
Name: iqn.2015-05.com.oracle:qd--datac1--exa01dbadm01
Host name: exa01dbadm01
ASM disk group name: DATAC1
Visible to: iqn.1988-12.com.oracle:192.168.10.1, iqn.1988-12.com.oracle:192.168.10.2, iqn.1988-12.com.oracle:192.168.10.3, iqn.1988-12.com.oracle:192.168.10.4
Discovered by:
[root@exa01dbadm01 oracle.SupportTools]#[root@exa01dbadm02 oracle.SupportTools]# /opt/oracle.SupportTools/quorumdiskmgr --list --target
Name: iqn.2015-05.com.oracle:qd--datac1--exa01dbadm02
Host name: exa01dbadm02
ASM disk group name: DATAC1
Visible to: iqn.1988-12.com.oracle:192.168.10.1, iqn.1988-12.com.oracle:192.168.10.2, iqn.1988-12.com.oracle:192.168.10.3, iqn.1988-12.com.oracle:192.168.10.4
Discovered by:
[root@exa01dbadm02 oracle.SupportTools]#Create quorum disk device as below on node1 and node2.
[root@exa01dbadm01 oracle.SupportTools]# /opt/oracle.SupportTools/quorumdiskmgr --create --device --target-ip-list="192.168.10.1, 192.168.10.2, 192.168.10.3, 192.168.10.4"
[Success] Successfully created all device(s) from target(s) on machine with IP address 192.168.10.1
[Success] Successfully created all device(s) from target(s) on machine with IP address 192.168.10.2
[Success] Successfully created all device(s) from target(s) on machine with IP address 192.168.10.3
[Success] Successfully created all device(s) from target(s) on machine with IP address 192.168.10.4
[root@exa01dbadm01 oracle.SupportTools]#[root@exa01dbadm02 oracle.SupportTools]# /opt/oracle.SupportTools/quorumdiskmgr --create --device --target-ip-list="192.168.10.1, 192.168.10.2, 192.168.10.3, 192.168.10.4"
[Success] Successfully created all device(s) from target(s) on machine with IP address 192.168.10.1
[Success] Successfully created all device(s) from target(s) on machine with IP address 192.168.10.2
[Success] Successfully created all device(s) from target(s) on machine with IP address 192.168.10.3
[Success] Successfully created all device(s) from target(s) on machine with IP address 192.168.10.4
[root@exa01dbadm02 oracle.SupportTools]#Now you can list the quorum disk device on node1 and node2 to validate.
[root@exa01dbadm01 oracle.SupportTools]# /opt/oracle.SupportTools/quorumdiskmgr --list --device
Device path: /dev/exadata_quorum/QD_DATAC1_exa01dbadm01
Host name: exa01dbadm01
ASM disk group name: DATAC1
Size: 128 MB
Device path: /dev/exadata_quorum/QD_DATAC1_exa01dbadm02
Host name: exa01dbadm02
ASM disk group name: DATAC1
Size: 128 MB
[root@exa01dbadm01 oracle.SupportTools]#[root@exa01dbadm02 oracle.SupportTools]# /opt/oracle.SupportTools/quorumdiskmgr --list --device
Device path: /dev/exadata_quorum/QD_DATAC1_exa01dbadm02
Host name: exa01dbadm02
ASM disk group name: DATAC1
Size: 128 MB
Device path: /dev/exadata_quorum/QD_DATAC1_exa01dbadm01
Host name: exa01dbadm01
ASM disk group name: DATAC1
Size: 128 MB
[root@exa01dbadm02 oracle.SupportTools]#Now, check in the ASM instance if quorum disk devices are visible to ASM to be used for the DATAC1 diskgroup as below, we can see that quorum disk devices are available as CANDIDATE disks that we can add in to the diskgroup.
SQL> l
1* SELECT inst_id, label, path, mode_status, header_status FROM gv$asm_disk WHERE path LIKE '/dev/exadata_quorum/%'
SQL> /
INST_ID LABEL PATH MODE_ST HEADER_STATU
---------- ------------------------------- ---------------------------------------------------------------------- ------- ------------
1 QD_DATAC1_exa01dbadm02 /dev/exadata_quorum/QD_DATAC1_exa01dbadm02 ONLINE CANDIDATE
1 QD_DATAC1_exa01dbadm01 /dev/exadata_quorum/QD_DATAC1_exa01dbadm01 ONLINE CANDIDATE
2 QD_DATAC1_exa01dbadm02 /dev/exadata_quorum/QD_DATAC1_exa01dbadm02 ONLINE CANDIDATE
2 QD_DATAC1_exa01dbadm01 /dev/exadata_quorum/QD_DATAC1_exa01dbadm01 ONLINE CANDIDATE
SQL>Add the quorum devices in to the diskgroup as below.
SQL> ALTER DISKGROUP datac1 ADD QUORUM FAILGROUP exa01dbadm01 DISK '/dev/exadata_quorum/QD_DATAC1_exa01dbadm01'
QUORUM FAILGROUP exa01dbadm02 DISK '/dev/exadata_quorum/QD_DATAC1_exa01dbadm02'; 2
Diskgroup altered.
SQL>After disk addition, we can see that now these are parts of diskgroup and status is MEMBER.
SQL> SELECT inst_id, label, path, mode_status, header_status
FROM gv$asm_disk WHERE path LIKE '/dev/exadata_quorum/%'; 2
INST_ID LABEL PATH MODE_ST HEADER_STATU
---------- ------------------------------- ---------------------------------------------------------------------- ------- ------------
1 QD_DATAC1_exa01dbadm02 /dev/exadata_quorum/QD_DATAC1_exa01dbadm02 ONLINE MEMBER
1 QD_DATAC1_exa01dbadm01 /dev/exadata_quorum/QD_DATAC1_exa01dbadm01 ONLINE MEMBER
2 QD_DATAC1_exa01dbadm02 /dev/exadata_quorum/QD_DATAC1_exa01dbadm02 ONLINE MEMBER
2 QD_DATAC1_exa01dbadm01 /dev/exadata_quorum/QD_DATAC1_exa01dbadm01 ONLINE MEMBER
SQL>Finally, replaced/moved the voting disks back to DATAC1 diskgroup and five copies of files are now available.
[root@exa01dbadm02 oracle.SupportTools]# crsctl query css votedisk
## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
1. ONLINE cf9c32d1c0c34fafbf47caa1b77821b0 (o/10.0.0.5;10.0.0.6/DATAC1_CD_02_exad02cel01) [DATAC1]
2. ONLINE e2e61cd646224f08bf2c4c47e719e8fd (o/10.0.0.3;10.0.0.4/DATAC1_CD_05_exad02cel02) [DATAC1]
3. ONLINE 7691da5101a34f95bfc56c3bec4e681b (o/10.0.0.1;10.0.0.2/DATAC1_CD_03_exad02cel03) [DATAC1]
4. ONLINE 3dea97bdc5aa4f61bf0355abe31c1361 (/dev/exadata_quorum/QD_DATAC1_exa01dbadm02) [DATAC1]
5. ONLINE cc5671a086eb4f38bf5c690666099d66 (/dev/exadata_quorum/QD_DATAC1_exa01dbadm01) [DATAC1]
Located 5 voting disk(s).
[root@exa01dbadm02 oracle.SupportTools]#After CRS restart : I bounced the CRS on both nodes to check if things are coming up normally.
[root@exa01dbadm02 oracle.SupportTools]# crsctl query css votedisk
## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
1. ONLINE cf9c32d1c0c34fafbf47caa1b77821b0 (o/10.0.0.5;10.0.0.6/DATAC1_CD_02_exad02cel01) [DATAC1]
2. ONLINE e2e61cd646224f08bf2c4c47e719e8fd (o/10.0.0.3;10.0.0.4/DATAC1_CD_05_exad02cel02) [DATAC1]
3. ONLINE 7691da5101a34f95bfc56c3bec4e681b (o/10.0.0.1;10.0.0.2/DATAC1_CD_03_exad02cel03) [DATAC1]
4. ONLINE 3dea97bdc5aa4f61bf0355abe31c1361 (/dev/exadata_quorum/QD_DATAC1_exa01dbadm02) [DATAC1]
5. ONLINE cc5671a086eb4f38bf5c690666099d66 (/dev/exadata_quorum/QD_DATAC1_exa01dbadm01) [DATAC1]
Located 5 voting disk(s).
[root@exa01dbadm02 oracle.SupportTools]#Our missing quorum disk issue stands fixed here......!!
Hope it helps, thanks for reading, please subscribe to this blog to stay updated with latest news on Oracle Cloud Infrastructure and Oracle Autonomous Database Cloud Services and new articles.
Twitter : https://twitter.com/rajsoft8899
Linkedin : https://www.linkedin.com/in/raj-kumar-kushwaha-5a289219/
Facebook : https://www.facebook.com/rkushawaha
No comments:
Post a Comment