It was a challenging day to to troubleshoot the issue of cluster configuration failure while setting up 2 Node Oracle 11gR2(11.2.0.4) RAC cluster on Windows Server 2012 R2 operating system for one of our customer. At every attempt Grid Confuguration for cluster was getting failed while Installing Oracle Grid Infrastructure software for cluster.
InstallAction log did't have much clue about the GI configuration failure but rootcrs_racnode1.log located at $GRID_HOME\cfgtoollogs\rootcrs_racnode1.log file provided the root cause of cluster configuration failure which reveals that "The driver is not currently installed on this node."
Below is the entries from rootcrs_racnode1.log log file from Node 1.
From Log file : C:\app\11.2.0\grid\cfgtoollogs\rootcrs_racnode1.log
------------------------------------------------------
2016-06-09 08:22:36: The 'ROOTCRS_ACFSINST' is either in START/FAILED state
2016-06-09 08:22:36: Executing 'E:\app\11.2.0\grid\bin\acfsroot.bat install'
2016-06-09 08:22:36: Executing cmd: E:\app\11.2.0\grid\bin\acfsroot.bat install
2016-06-09 08:22:37: Command output:
> ACFS-9300: ADVM/ACFS distribution files found.
> ACFS-9307: Installing requested ADVM/ACFS software.
> acfsinstall: ACFS-09420: The driver is not currently installed on this node.
> acfsinstall: ACFS-09411: CreateService succeeded.
> acfsinstall: CLSU-00100: Operating System function: StartDriver failed with error data: 31
> acfsinstall: CLSU-00101: Operating System error message: A device attached to the system is not functioning.
> acfsinstall: CLSU-00103: error location: StartDriver_
> acfsinstall: CLSU-00104: additional error information: W
> acfsinstall: ACFS-09419: StartService failed.
> acfsinstall: ACFS-09401: Failed to install the driver.
>
> ACFS-9340: failed to install OKS driver.
> acfsinstall: ACFS-09420: The driver is not currently installed on this node.
> acfsinstall: ACFS-09411: CreateService succeeded.
> acfsinstall: CLSU-00100: Operating System function: StartDriver failed with error data: 1068
> acfsinstall: CLSU-00101: Operating System error message: The dependency service or group failed to start.
> acfsinstall: CLSU-00103: error location: StartDriver_
> acfsinstall: CLSU-00104: additional error information: J
> acfsinstall: ACFS-09419: StartService failed.
> acfsinstall: ACFS-09401: Failed to install the driver.
>
> ACFS-9340: failed to install ADVM driver.
> acfsinstall: ACFS-09420: The driver is not currently installed on this node.
> acfsinstall: ACFS-09411: CreateService succeeded.
> acfsinstall: CLSU-00100: Operating System function: StartDriver failed with error data: 1068
> acfsinstall: CLSU-00101: Operating System error message: The dependency service or group failed to start.
> acfsinstall: CLSU-00103: error location: StartDriver_
> acfsinstall: CLSU-00104: additional error information: ]
> acfsinstall: ACFS-09419: StartService failed.
> acfsinstall: ACFS-09401: Failed to install the driver.
>
> ACFS-9340: failed to install ACFS driver.
> ACFS-9310: ADVM/ACFS installation failed.
>End Command output
2016-06-09 08:22:37: E:\app\11.2.0\grid\bin\acfsroot.bat install ... failed
2016-06-09 08:22:37: USM driver install status is 0
2016-06-09 08:22:37: USM driver install actions failed
2016-06-09 08:22:37: Running as user Administrator: E:\app\11.2.0\grid\bin\cluutil -ckpt -oraclebase E:\app\Administrator -writeckpt -name ROOTCRS_ACFSINST -state FAIL
2016-06-09 08:22:37: s_run_as_user2: Running E:\app\11.2.0\grid\bin\cluutil -ckpt -oraclebase E:\app\Administrator -writeckpt -name ROOTCRS_ACFSINST -state FAIL
2016-06-09 08:22:37: E:\app\11.2.0\grid\bin\cluutil successfully executed
2016-06-09 08:22:37: Succeeded in writing the checkpoint:'ROOTCRS_ACFSINST' with status:FAIL
2016-06-09 08:22:37: CkptFile: E:\app\Administrator\Clusterware\ckptGridHA_win1.xml
2016-06-09 08:22:37: Sync the checkpoint file 'E:\app\Administrator\Clusterware\ckptGridHA_win1.xml'
Solutions :
=========
As per the logs pasted above, I came about an unpublished BUG 17927204 - ACFS SUPPORT FOR WINDOWS 2012R2 in Oracle Grid Infrastructure version 11.2.0.4 itself so in order to resolve the cluster configuration issue I downloaded the one-off patch (p22839608_112040_MSWIN-x86-64) from MOS to be applied on #GRID_HOME binaries on both nodes in the cluster. And in order to apply the Patch mentioned above, you will have to download relevant Opatch utility (p6880880_112000_MSWIN-x86-64) from MOS.
Please refer MOS Doc ID 1987371.1 for its details.
Once you have downloaded both patches mentioned above please follow the below steps for successful Grid Infrastructure Installation.
1 - Clean the currently failed run of GI from both nodes (it includes deinstall of GI and removal of all related entries from Windows Registry.)
Click Here to see how to clean failed Grid Installation.
2 - Bounce the nodes once you are done with step 1.
3 - Run the GI Installer (setup.exe) for GI installation and choose to install Grid Infrastructure software only.
Note :- In step 3 you will have to install GI software on each individual nodes in the cluster as it won't do GI installation on remote nodes at once.
4 - Once the GI software(Software only) is installed on both the nodes then replace(or rename the existing Opatch directory in Grid Home. e.g. Opatch_old) Opatch folder in Grid Home with the one you have downloaded above (p6880880_112000_MSWIN-x86-64) on both nodes.
5 - Verify the Opatch utility is working fine with : opatch.exe lsinv command and apply the one-off patch(p22839608_112040_MSWIN-x86-64) on Grid Home at both nodes.
6 - After successful application of Opatch on both nodes, Start the Grid Infrastructure Configuration for the cluster as mentioned below.
Navigate to and run $GRID_HOME/crs/config/config.bat
Note : - It will launch GUI of Grid Infrastructure Configuration and proceed as per its steps gets prompted on interface. Please note that if your GI configuration still fails at 33% then don't just cancel the installation, just click OK on the error prompt and click on "Retry" to continue with the installation.
Hope it wold help to resolve the issue.
No comments:
Post a Comment