Mar 25, 2016

ORA-00600: error in Queue Table SYS.SYS$SERVICE_METRICS_TAB

I found below ORA-600 in my RAC production database of version 11.2.0.4+PSU7. From the Oracle documents, I did the following and floating of ORA-600 is stopped completly.

Verify the following:

- alert_PROD1.log
- PROD1_q002_7930002.trc


"alert_PROD1.log" shows an ORA-00600 [kwqitnmphe:ltbagi] here:
---
Fri Mar 25 13:43:49 2016
Errors in file /u01/app/oracle/diag/rdbms/prod/PROD1/trace/PROD1_q000_10682516.trc (incident=40305):
ORA-00600: internal error code, arguments: [kwqitnmphe:ltbagi], [1], [0], [], [], [], [], [], [], [], [], []
Incident details in: /u01/app/oracle/diag/rdbms/prod/PROD1/incident/incdir_40305/PROD1_q000_10682516_i40305.trc
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
..............................
..............................


Verify the "PROD1_q000_10682516.trc" and "PROD1_q000_10682516_i40305.trc" files.


Trace file /u01/app/oracle/diag/rdbms/prod/PROD1/trace/PROD1_q000_10682516.trc
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Data Mining and Real Application Testing options
ORACLE_HOME = /u01/app/oracle/11.2.0/db_home
System name: AIX
Node name: rac01
Release: 1
Version: 7
Machine: 00F9D98B4C00
Instance name: PROD1
Redo thread mounted by this instance: 1
Oracle process number: 38
Unix process pid: 10682516, image: oracle@rac01 (Q000)


*** 2016-03-25 13:43:49.018
*** SESSION ID:(579.19) 2016-03-25 13:43:49.018
*** CLIENT ID:() 2016-03-25 13:43:49.018
*** SERVICE NAME:(SYS$BACKGROUND) 2016-03-25 13:43:49.018
*** MODULE NAME:(Streams) 2016-03-25 13:43:49.018
*** ACTION NAME:(QMON Slave) 2016-03-25 13:43:49.018

Incident 40305 created, dump file: /u01/app/oracle/diag/rdbms/prod/PROD1/incident/incdir_40305/PROD1_q000_10682516_i40305.trc
ORA-00600: internal error code, arguments: [kwqitnmphe:ltbagi], [1], [0], [], [], [], [], [], [], [], [], []

kwqitmmit during kwqitmmit:time mgr IOT proc: Error 600 in Queue Table SYS.SYS$SERVICE_METRICS_TAB
kwqicaclcur: Error 600
Cursor Session Number : 580
Cursor Session Serial : 3
Cursor Pin Number : 24585
kwqitmmit during kwqitmmit:end proc: Error 600 in Queue Table SYS.SYS$SERVICE_METRICS_TAB
kwqitptm: error 600 encountered during serving 13593

*** 2016-03-25 13:43:51.148
ORA-00600: internal error code, arguments: [kwqitnmphe:ltbagi], [1], [0], [], [], [], [], [], [], [], [], []
OPIRIP: Uncaught error 447. Error stack:
ORA-00447: fatal error in background process
ORA-00600: internal error code, arguments: [kwqitnmphe:ltbagi], [1], [0], [], [], [], [], [], [], [], [], []


Action plan:

1) If you are upgrading :

Drop the queue table -

exec dbms_aqadm.drop_queue_table( queue_table=>'SYS.SYS$SERVICE_METRICS_TAB', force => TRUE)

select object_name, object_type from dba_objects where object_name like '%SYS$SERVICE_%'; ----> no rows should be returned.

Follow the below steps to recreate them.

SQL>STARTUP UPGRADE

SQL>@$ORACLE_HOME/rdbms/admin/catproc.sql

SQL>@$ORACLE_HOME/rdbms/admin/utlrp.sql

SQL>@$ORACLE_HOME/rdbms/admin/utlrp.sql

select object_name, object_type,status from dba_objects where object_name like '%SYS$SERVICE_%';

If the script fails to create the queue, please let us know so that we may recreate it manually.

2) If it is comming in production, then take downtime and do the following:

 Drop the queue table -

exec dbms_aqadm.drop_queue_table( queue_table=>'SYS.SYS$SERVICE_METRICS_TAB', force => TRUE)

select object_name, object_type from dba_objects where object_name like '%SYS$SERVICE_%'; ----> no rows should be returned.

Follow the below steps to recreate them.

SQL>STARTUP

SQL>alter system enable restricted session;

SQL>@$ORACLE_HOME/rdbms/admin/catproc.sql

SQL>@$ORACLE_HOME/rdbms/admin/utlrp.sql

SQL>@$ORACLE_HOME/rdbms/admin/utlrp.sql

SQL>alter system disable restricted session;

Thanks .

No comments:

Post a Comment

Translate >>