Home » Server Options » RAC & Failsafe » 11gr2, AIX 7.1, root.sh failed on node 2 (AIX 7.1, 11gr2)
11gr2, AIX 7.1, root.sh failed on node 2 [message #623966] Tue, 16 September 2014 21:24 Go to next message
trantuananh24hg
Messages: 742
Registered: January 2007
Location: Ha Noi, Viet Nam
Senior Member
Hi all!

I were on installation of Grid Infrastructure 11gr2 on AIX 7.1 yesterday.
At the end of root.sh requirement, it was sucessful on node 1, but failed on node 2, I could not wrote down on notepad as description brief full, then I write some errors as following

CRS-2673: Attempting to stop 'ora.cssdmonitor' on 'mbfdb'
CRS-2677: Stop of 'ora.cssdmonitor' on 'mbfdb' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'mbfdb'
CRS-2677: Stop of 'ora.cssd' on 'mbfdb' succeeded
CRS-2673: Attempting to stop 'ora.gpnpd' on 'mbfdb'
CRS-2677: Stop of 'ora.gpnpd' on 'mbfdb' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'mbfdb'
CRS-2677: Stop of 'ora.gipcd' on 'mbfdb' succeeded
CRS-2673: Attempting to stop 'ora.mdnsd' on 'mbfdb'
CRS-2677: Stop of 'ora.mdnsd' on 'mbfdb' succeeded
Initial cluster configuration failed. See /mbfdbapp/app/11.2.0/grid/cfgtoollogs/crsconfig/rootcrs_mbfdb.log for details


Tracked with occsdOut.log, I saw 3 Ora errors ORA-15032 ORA-15063 ORA-15080, especially ORA-15080: synchronous I/O operation to a disk failed. There are many reasons arround this error, such as "UNABLE TO CREATE 4K SECTOR SIZE ASM DISK: ASM see 512K logical sector but actually 4096"; "Failed on change asm_diskstring"; "different between c-major/minor of 2 nodes when seen on LUNs";

On AIX, I must done something before installation of Grid Infrastructure 11gr2
- Disable HAMCP
- Change grid:asmadmin, 660 on those disks for OCR ( /dev/rhdisk5,6,7,8 )
- Set reserve_lock/reserve_policy to none, clear PVID
- Check by dd command out on thoes disks

When reinstall GI, the installation stop at "Saving ora Inventory" and did not return "Run root.sh on both of node" box. So, I decided to stop there, post here, hope some-one had got experience to help to solve this error.

Thank you!

[Updated on: Tue, 16 September 2014 21:28]

Report message to a moderator

Re: 11gr2, AIX 7.1, root.sh failed on node 2 [message #624198 is a reply to message #623966] Thu, 18 September 2014 23:05 Go to previous messageGo to next message
trantuananh24hg
Messages: 742
Registered: January 2007
Location: Ha Noi, Viet Nam
Senior Member
I had solved by myself, then, I re-write explaination once again
Re: 11gr2, AIX 7.1, root.sh failed on node 2 [message #624710 is a reply to message #624198] Wed, 24 September 2014 21:35 Go to previous message
trantuananh24hg
Messages: 742
Registered: January 2007
Location: Ha Noi, Viet Nam
Senior Member
I've just finished configuration and installation Oracle Rac 11gr2 on AIX (64) yesterday.

First time, I should reply with those occured errors in the question topic, they were mistake operations between System Administrator who deployed AIX Operating System and me.

Second, I describe brief as following, including solution:

- As far as you know, there are many storages certificated by Oracle, some of them are issued of using shared_file_system; NAS instead of raw-devices for ASM. We got 2 storages, one is EMP, one is IBM. The EMP storage should be configured to used HACMP (PowerHA) but not also raw-devices, our SA had not got upgraded information from Oracle, he did done brought LUNs from EMP Storage to Servers. The Server, of course, has got funny UPID (Unique Physical ID) but not PVID right. I have got not any idea for, and I say: Nothing to do with it, do not try. Why?

+ Whenever you configure raw-device for ASM, example: /dev/rhdiskX, you must compare them between each node. Any wrong information cause failures Grid installation.
+ How to compare? For each them, example: /dev/rhdisk7, you should enable PVID and compare:
root@node1# chdev -l hdiskn -a pv=yes
root@node2# chdev -l hdiskn -a pv=yes
root@node1# lspv -E -l hdisk7 --> You will see PVID such as: 0009005fb9c23648
root@node2# lspv -E -l hdisk7 --> You will see PVID such as: 0009005fb9c23648
Right, the PVID are same on 2 nodes, it seem got a sucessful Grid installation, but not. Why? Oracle recommend clear PVID when take a complete comparasion. No, for the second time of installation, I took carefully re-compare again. Amazing, the PVID on 2 nodes were not same, it was 0009005fc9d23649.

- Said I above, I did not have got any idea for, I'm not storage administrator, I'm just only DBA. And then, I suggest my SA configured IBM storage for Grid using LUNs, EMP for HACMP using IBM clusterware separately.

With the oraInventory file were not able upgrade cause hang installation, it's just simply by edit file, add those line: location of oraInventory in /etc/oraInst.loc, group (example: oinstall).


Should I write a complete tasks for Oracle RAC 11gr2 installation without image?

[Updated on: Wed, 24 September 2014 21:49]

Report message to a moderator

Previous Topic: RAC 11g new install...newbie question
Next Topic: Convert Single Instance to RAC
Goto Forum:
  


Current Time: Sun Sep 27 19:14:37 CDT 2020