Home » Server Options » RAC & Failsafe » ASM diskgroup working in 1st instance, but not in 2nd instance (RAC 11gr2, Linux x86)
ASM diskgroup working in 1st instance, but not in 2nd instance [message #626893] Tue, 04 November 2014 13:02 Go to next message
ezhou
Messages: 3
Registered: November 2014
Location: Canada
Junior Member
Hi,

I encountered an issue, when converting single instance to RAC. problem is ASM.
1st instance everything works fine, GI/database.

Following 4 DGs mount and work fine in 1st node.
DATA,
ARCH,
REDO,
OCTVG

when tried to bring up 2nd node asm, from asm trace log, found ARCH not able to mount,
Errors in file /prod01/home/dba/oracle/diag/asm/+asm/+ASM2/trace/+ASM2_ora_14376.trc:
ORA-15196: invalid ASM block header [kfc.c:26076] [obj_kfbl] [8] [0] [3 != 8]
ORA-15196: invalid ASM block header [kfc.c:26076] [obj_kfbl] [8] [0] [3 != 8]
ERROR: cache failed to read group=1(ARCH) fn=8 blk=0 from disk(s): 0(ARCH_0000)
ORA-15196: invalid ASM block header [kfc.c:26076] [obj_kfbl] [8] [0] [3 != 8]
ORA-15196: invalid ASM block header [kfc.c:26076] [obj_kfbl] [8] [0] [3 != 8]
NOTE: cache initiating offline of disk 0 group ARCH
NOTE: process _user14376_+asm2 (14376) initiating offline of disk 0.3916206202 (ARCH_0000) with mask 0x7e in group 1
WARNING: Disk 0 (ARCH_0000) in group 1 in mode 0x7f is now being taken offline on ASM inst 2
NOTE: initiating PST update: grp = 1, dsk = 0/0xe96c907a, mask = 0x6a, op = clear


DATA mounted, but missing some datafiles checked from asmcmd. which included spfile and datafiles, so that database couldn't be started up. I copied those files from another path, I thought be reflected in 2nd node automatically since it's SAN shared storage.
I rebooted 2nd node, scan disks, oracleasm used, still the same.

Just wonder if anyone had similar situation.

Thank you in advance,
Emily
Re: ASM diskgroup working in 1st instance, but not in 2nd instance [message #626896 is a reply to message #626893] Tue, 04 November 2014 16:34 Go to previous messageGo to next message
babuknb
Messages: 1734
Registered: December 2005
Location: NJ
Senior Member

Emily,

Technically you don't need to copy as its shared on both node and your able to mount diskgroup from first node. I would suggest you to check SAN Admin for disk status on second node.

In addition, how big your diskgroup and complete version of your database ? 11.2.x.x ?

thanks
Re: ASM diskgroup working in 1st instance, but not in 2nd instance [message #627023 is a reply to message #626896] Thu, 06 November 2014 10:54 Go to previous messageGo to next message
ezhou
Messages: 3
Registered: November 2014
Location: Canada
Junior Member
Thank you for reply.

DATA 500G
ARCH 100G

SAN in 2nd node looks no problem, double checked with admin.

This morning, I totally restarted 2nd node, worse, asm not able to start.
consistently in instance recovery mode, and finally give up, instance shutdown.
1st node still perfectly run.


11.2.0.0.0 I changed to this. after installation GI 11.2, the compatibility was 10.0.0.0.
this is actually 2nd pair cluster, I refreshed database from another RAC to here.
I used SAN snapshot, then dd command to copy DATA/ARCH to 2nd RAC SAN LUNs. then change dbname and recovery database in 1st node, everything fine.
After 1st node done, tried to join 2nd node into cluster, then found out 2nd node database couldn't be started up, because 2nd node couldn't find spfiles and datafiles in DATA. still doesn't make sense to me. mount to 1st node ok, means the DG shouldn't be problem, how come same stuff mount to 2nd node, then problem.
Re: ASM diskgroup working in 1st instance, but not in 2nd instance [message #629560 is a reply to message #627023] Wed, 10 December 2014 21:10 Go to previous message
trantuananh24hg
Messages: 742
Registered: January 2007
Location: Ha Noi, Viet Nam
Senior Member
@ezhou:

How about permission from node 2?
Example:
# chown -R grid:asmadmin /dev/cciss/c0t0d4
# chmod 660 /dev/cciss/c0t0d4
which c0t0d4 is a usable partition for ARC.

How about oracleasm configure and init in node 2?

[Updated on: Wed, 10 December 2014 21:11]

Report message to a moderator

Previous Topic: Noob Installing RAC 11.2.0.4.0 RHEL6.5
Next Topic: What Oracle Version is Suitable
Goto Forum:
  


Current Time: Sun Sep 27 19:18:34 CDT 2020