Unable to start Oracle Enterprise Manager (OEM) using emctl command:
Found : OC4J Configuration issue while start
-- This document also useful to recreate OEM repository in Cluster database (more than one node RAC)
Fix / Solution:
There are couple of reason for the above issue.
Fix-1:
Crosscheck your hostname configuration details. Don't put any complexity for host name configuration. Chek where your /etc/hosts filename is like beolw or not ( with order):
$ cat /etc/hsosts
127.0.0.1 loopback localhost # loopback (lo0) name/address
10.99.1.1 prod-db1
OR
$ cat /etc/hsosts
127.0.0.1 loopback localhost # loopback (lo0) name/address
10.99.1.1 prod-db1 mycompany.com
I found some recommendations like below from oracle.com comunity:
They really create a directory whose name is composed of hostname (or IP) and the SID!
So it breaks whenever you change the hostname. Furthermore this strongly depends on the entries in /etc/hosts which typicall look like so: 10.193.63.76 strongbow.odi.ch strongbow.
Oracle will use the first entry after the IP address (of probably the first ethernet device) as the name.
So whenever IP or hostname changes you need to alter this line.
If you have the FQDN / Hostname problem change the order in your /etc/hosts file
If oracle use the Hostname change from FQDN to Hostname after the IP address
10.193.63.76 strongbow.odi.ch strongbow => 10.193.63.76 strongbow strongbow.odi.ch strongbow
If oracle use the FQDN change from Hostname to FQDN after the IP address
10.193.63.76 strongbow strongbow.odi.ch => 10.193.63.76 strongbow.odi.ch strongbow
Fix:2 : Recreate the OEM repository
Caution: Do it only in fully OFF peak hour / down time only.
Pre-requisites:
1) You should have following user passwords:
SYS, DBSNMP, SYSMAN
2) You schould have ORACLE_HOME address
3) Export ORACLE_SID and ORACLE_UNQNAME
4) If you want to configure mail alerts in future, collect SMTP server IP address and one mail address atleast.
When you use "emctl status dbconsole" command, you may have the below issue.
$ emctl status dbconsole
OC4J Configuration issue. /home/oracle/app/oracle/product/11.2.0/dbhome_1/oc4j/j2ee/OC4J_DBConsole_prod-db1_FINPROD not found.
-- Recreate the OEM respository.
When you use "emca -config dbcontrol db -repos recreate" command following output will come.
$ emca -config dbcontrol db -repos recreate
STARTED EMCA at Oct 24, 2014 2:23:59 PM
EM Configuration Assistant, Version 11.2.0.3.0 Production
Copyright (c) 2003, 2011, Oracle. All rights reserved.
Enter the following information:
Database SID: FINPROD
Listener port number: 1521
Listener ORACLE_HOME [ /home/oracle/app/oracle/product/11.2.0/dbhome_1 ]: /home/oracle/app/oracle/product/11.2.0/dbhome_1
Password for SYS user:
Password for DBSNMP user:
Password for SYSMAN user:
Email address for notifications (optional):
Outgoing Mail (SMTP) server for notifications (optional):
-----------------------------------------------------------------
You have specified the following settings
Database ORACLE_HOME ................ /home/oracle/app/oracle/product/11.2.0/dbhome_1
Local hostname ................ prod-db1
Listener ORACLE_HOME ................ /home/oracle/app/oracle/product/11.2.0/dbhome_1
Listener port number ................ 1521
Database SID ................ FINPROD
Email address for notifications ...............
Outgoing Mail (SMTP) server for notifications ...............
-----------------------------------------------------------------
----------------------------------------------------------------------
WARNING : While repository is dropped the database will be put in quiesce mode.
----------------------------------------------------------------------
Do you wish to continue? [yes(Y)/no(N)]: Y
Oct 24, 2014 2:24:45 PM oracle.sysman.emcp.EMConfig perform
INFO: This operation is being logged at /home/oracle/app/oracle/cfgtoollogs/emca/FINPROD/emca_2014_10_24_14_23_58.log.
Oct 24, 2014 2:24:46 PM oracle.sysman.emcp.EMReposConfig invoke
INFO: Dropping the EM repository (this may take a while) ...
Oct 24, 2014 2:24:47 PM oracle.sysman.emcp.EMReposConfig invoke
INFO: Repository successfully dropped
Oct 24, 2014 2:24:47 PM oracle.sysman.emcp.EMReposConfig createRepository
INFO: Creating the EM repository (this may take a while) ...
Oct 24, 2014 2:41:45 PM oracle.sysman.emcp.EMReposConfig invoke
INFO: Repository successfully created
Oct 24, 2014 2:41:51 PM oracle.sysman.emcp.EMReposConfig uploadConfigDataToRepository
INFO: Uploading configuration data to EM repository (this may take a while) ...
Oct 24, 2014 2:43:52 PM oracle.sysman.emcp.EMReposConfig invoke
INFO: Uploaded configuration data successfully
Oct 24, 2014 2:44:00 PM oracle.sysman.emcp.util.DBControlUtil secureDBConsole
INFO: Securing Database Control (this may take a while) ...
Oct 24, 2014 2:44:25 PM oracle.sysman.emcp.util.DBControlUtil secureDBConsole
INFO: Database Control secured successfully.
Oct 24, 2014 2:44:25 PM oracle.sysman.emcp.util.DBControlUtil startOMS
INFO: Starting Database Control (this may take a while) ...
Oct 24, 2014 2:45:30 PM oracle.sysman.emcp.EMDBPostConfig performConfiguration
INFO: Database Control started successfully
Oct 24, 2014 2:45:30 PM oracle.sysman.emcp.EMDBPostConfig performConfiguration
INFO: >>>>>>>>>>> The Database Control URL is https://prod-db1:1158/em <<<<<<<<<<<
Oct 24, 2014 2:45:33 PM oracle.sysman.emcp.EMDBPostConfig invoke
WARNING:
************************ WARNING ************************
Management Repository has been placed in secure mode wherein Enterprise Manager data will be encrypted. The encryption key has been placed
in the file: /home/oracle/app/oracle/product/11.2.0/dbhome_1/prod-db1_FINPROD/sysman/config/emkey.ora. Ensure this file is backed up as the
encrypted data will become unusable if this file is lost.
***********************************************************
Enterprise Manager configuration completed successfully
FINISHED EMCA at Oct 24, 2014 2:45:33 PM
-- Now recheck the status of dbcolsole
$ emctl status dbconsole
Oracle Enterprise Manager 11g Database Control Release 11.2.0.3.0
Copyright (c) 1996, 2011 Oracle Corporation. All rights reserved.
https://prod-db1:1158/em/console/aboutApplication
Oracle Enterprise Manager 11g is running.
------------------------------------------------------------------
Logs are generated in directory /home/oracle/app/oracle/product/11.2.0/dbhome_1/prod-db1_FINPROD/sysman/log
OEM Directory structure for Oracle 11g and more:
Use below links for details information:
http://docs.oracle.com/cd/E11882_01/server.112/e25494/dbcontrol.htm
Deleting and Creating Database Control Objects:
=====================================
For an Oracle RAC database ( more than One node ):
Option 1: Deleting Database Control Configuration Files Using EMCA Scripts
$ $ORACLE_HOME/bin/emca -deconfig dbcontrol db –cluster
Option 2: Deleting Database Control Configuration Files and Repository Objects Using An EMCA Script
$ $ORACLE_HOME/bin/emca -deconfig dbcontrol db -repos drop –cluster
Option 3: Deleting A Database Control Repository Using EMCA Scripts
$ $ORACLE_HOME/bin/emca -repos drop -cluster
Option 4: Deleting Database Control Configuration Files Manually
In addition to using EMCA scripts to delete Database Control objects, you can delete objects manually, as follows:
Remove the following directories from your file system:
$ $ORACLE_HOME/hostname_sid
$ $ORACLE_HOME/oc4j/j2ee/OC4J_DBConsole_hostname_sid
Option 5: Deleting the Database Control Repository Using RepManager
You can delete the Database Control repository using RepManager. This option is not as complete as the other options. You may find that dropping the repository using the command line options is a better solution. Also note that RepManager cannot be used to create a Database Control Repository.
Run the following command:
$ $ORACLE_HOME/sysman/admin/emdrep/bin/RepManager hostname listener_port sid -action drop
Creating Database Control Objects:
===========================
Option 1: Creating the DB Control Configuration Files
$ $ORACLE_HOME/bin/emca -config dbcontrol db -cluster
Option 2: Creating the Database Control Repository Objects and Configuration Files
$ $ORACLE_HOME/bin/emca -config dbcontrol db -repos create –cluster
Option 3: Creating the Database Control Repository Objects and Configuration Files For a Cloned Database Home
$ $ORACLE_HOME/bin/emca -config dbcontrol db -repos recreate -cluster
Recreating Or Reconfiguring Database Control:
====================================
Option 1: Recreating the Database Control Configuration Files Only (Leaving Repository Intact)
$ $ORACLE_HOME/bin/emca -config dbcontrol db -cluster
Option 2: Recreating the Database Control Configuration Files and Repository
$ $ORACLE_HOME/bin/emca -config dbcontrol db -repos recreate -cluster
Note: Export ORACLE_SID and RDBMS home in your connecting/ oprating node.
Below error message during OEM repository creation:
$ emca -config dbcontrol db -repos recreate
.................
.................
Mar 14, 2016 11:49:08 AM oracle.sysman.emcp.EMConfig perform
INFO: This operation is being logged at /u01/app/oracle/cfgtoollogs/emca/UAT/emca_2016_03_14_11_47_54.log.
Mar 14, 2016 11:49:09 AM oracle.sysman.emcp.ParamsManager checkListenerStatusForDBControl
WARNING: ORA-12541: TNS:no listener
Mar 14, 2016 11:49:09 AM oracle.sysman.emcp.EMConfig perform
SEVERE:
Database connection through listener failed. Fix the error and run EM Configuration Assistant again.
Some of the possible reasons may be:
1) Listener port 1251 provided is incorrect. Provide the correct port.
2) Listener is not up. Start the Listener.
3) Database service UAT is not registered with listener. Register the database service.
4) Listener is up on physical host and ORACLE_HOSTNAME environment variable is set to virtual host. Unset ORACLE_HOSTNAME environment variable.
5) Listener is up on virtual host. Set environment variable ORACLE_HOSTNAME=<virtual host>.
6) /etc/hosts does not have correct entry for hostname.
Refer to the log file at /u01/app/oracle/cfgtoollogs/emca/UAT/emca_2016_03_14_11_47_54.log for more details.
Could not complete the configuration. Refer to the log file at /u01/app/oracle/cfgtoollogs/emca/UAT/emca_2016_03_14_11_47_54.log for more details.
$
Solution:
check the following points
1)-Ensure that the Database is up and running by executing the below commands:
<ORACLE_HOME>/bin/sqlplus "/ as sysdba"
SQL> select status from v$instance;
2)-Ensure that the Listener servicing the Database is up and running by executing the below command:
<ORACLE_HOME>/bin/lsnrctl status <listener name>
3)-SYS should be able to establish connection to the Database both via the listener and also the bequeath adaptor.
<ORACLE_HOME>/bin/sqlplus sys as sysdba
or
<ORACLE_HOME>/bin/sqlplus "sys as sysdba"
and
<ORACLE_HOME>/bin/sqlplus sys/<Password>@<TNS Connect descriptor>
If above connection fails take corrective actions to get the connection working from sqlplus.
4)-Password file should be correctly configured for SYS user connection.
Verify the initialization parameter REMOTE_LOGIN_PASSWORDFILE is set to EXCLUSIVE by executing the below command.
SQL> show parameter REMOTE_LOGIN_PASSWORDFILE;
5)-DBSNMP user must exist in the database and should be able to connect to the database.
<ORACLE_HOME>/bin/sqlplus dbsnmp/<password>
Following this, I fixed the issue. Crosscheck all.
This above simple solution fix my issue. You can try in the same way. If you have other issues, please send mail / post in comment box.
Thank you. Cheers !!!
Found : OC4J Configuration issue while start
-- This document also useful to recreate OEM repository in Cluster database (more than one node RAC)
Fix / Solution:
There are couple of reason for the above issue.
Fix-1:
Crosscheck your hostname configuration details. Don't put any complexity for host name configuration. Chek where your /etc/hosts filename is like beolw or not ( with order):
$ cat /etc/hsosts
127.0.0.1 loopback localhost # loopback (lo0) name/address
10.99.1.1 prod-db1
OR
$ cat /etc/hsosts
127.0.0.1 loopback localhost # loopback (lo0) name/address
10.99.1.1 prod-db1 mycompany.com
I found some recommendations like below from oracle.com comunity:
They really create a directory whose name is composed of hostname (or IP) and the SID!
So it breaks whenever you change the hostname. Furthermore this strongly depends on the entries in /etc/hosts which typicall look like so: 10.193.63.76 strongbow.odi.ch strongbow.
Oracle will use the first entry after the IP address (of probably the first ethernet device) as the name.
So whenever IP or hostname changes you need to alter this line.
If you have the FQDN / Hostname problem change the order in your /etc/hosts file
If oracle use the Hostname change from FQDN to Hostname after the IP address
10.193.63.76 strongbow.odi.ch strongbow => 10.193.63.76 strongbow strongbow.odi.ch strongbow
If oracle use the FQDN change from Hostname to FQDN after the IP address
10.193.63.76 strongbow strongbow.odi.ch => 10.193.63.76 strongbow.odi.ch strongbow
Fix:2 : Recreate the OEM repository
Caution: Do it only in fully OFF peak hour / down time only.
Pre-requisites:
1) You should have following user passwords:
SYS, DBSNMP, SYSMAN
2) You schould have ORACLE_HOME address
3) Export ORACLE_SID and ORACLE_UNQNAME
4) If you want to configure mail alerts in future, collect SMTP server IP address and one mail address atleast.
When you use "emctl status dbconsole" command, you may have the below issue.
$ emctl status dbconsole
OC4J Configuration issue. /home/oracle/app/oracle/product/11.2.0/dbhome_1/oc4j/j2ee/OC4J_DBConsole_prod-db1_FINPROD not found.
-- Recreate the OEM respository.
When you use "emca -config dbcontrol db -repos recreate" command following output will come.
$ emca -config dbcontrol db -repos recreate
STARTED EMCA at Oct 24, 2014 2:23:59 PM
EM Configuration Assistant, Version 11.2.0.3.0 Production
Copyright (c) 2003, 2011, Oracle. All rights reserved.
Enter the following information:
Database SID: FINPROD
Listener port number: 1521
Listener ORACLE_HOME [ /home/oracle/app/oracle/product/11.2.0/dbhome_1 ]: /home/oracle/app/oracle/product/11.2.0/dbhome_1
Password for SYS user:
Password for DBSNMP user:
Password for SYSMAN user:
Email address for notifications (optional):
Outgoing Mail (SMTP) server for notifications (optional):
-----------------------------------------------------------------
You have specified the following settings
Database ORACLE_HOME ................ /home/oracle/app/oracle/product/11.2.0/dbhome_1
Local hostname ................ prod-db1
Listener ORACLE_HOME ................ /home/oracle/app/oracle/product/11.2.0/dbhome_1
Listener port number ................ 1521
Database SID ................ FINPROD
Email address for notifications ...............
Outgoing Mail (SMTP) server for notifications ...............
-----------------------------------------------------------------
----------------------------------------------------------------------
WARNING : While repository is dropped the database will be put in quiesce mode.
----------------------------------------------------------------------
Do you wish to continue? [yes(Y)/no(N)]: Y
Oct 24, 2014 2:24:45 PM oracle.sysman.emcp.EMConfig perform
INFO: This operation is being logged at /home/oracle/app/oracle/cfgtoollogs/emca/FINPROD/emca_2014_10_24_14_23_58.log.
Oct 24, 2014 2:24:46 PM oracle.sysman.emcp.EMReposConfig invoke
INFO: Dropping the EM repository (this may take a while) ...
Oct 24, 2014 2:24:47 PM oracle.sysman.emcp.EMReposConfig invoke
INFO: Repository successfully dropped
Oct 24, 2014 2:24:47 PM oracle.sysman.emcp.EMReposConfig createRepository
INFO: Creating the EM repository (this may take a while) ...
Oct 24, 2014 2:41:45 PM oracle.sysman.emcp.EMReposConfig invoke
INFO: Repository successfully created
Oct 24, 2014 2:41:51 PM oracle.sysman.emcp.EMReposConfig uploadConfigDataToRepository
INFO: Uploading configuration data to EM repository (this may take a while) ...
Oct 24, 2014 2:43:52 PM oracle.sysman.emcp.EMReposConfig invoke
INFO: Uploaded configuration data successfully
Oct 24, 2014 2:44:00 PM oracle.sysman.emcp.util.DBControlUtil secureDBConsole
INFO: Securing Database Control (this may take a while) ...
Oct 24, 2014 2:44:25 PM oracle.sysman.emcp.util.DBControlUtil secureDBConsole
INFO: Database Control secured successfully.
Oct 24, 2014 2:44:25 PM oracle.sysman.emcp.util.DBControlUtil startOMS
INFO: Starting Database Control (this may take a while) ...
Oct 24, 2014 2:45:30 PM oracle.sysman.emcp.EMDBPostConfig performConfiguration
INFO: Database Control started successfully
Oct 24, 2014 2:45:30 PM oracle.sysman.emcp.EMDBPostConfig performConfiguration
INFO: >>>>>>>>>>> The Database Control URL is https://prod-db1:1158/em <<<<<<<<<<<
Oct 24, 2014 2:45:33 PM oracle.sysman.emcp.EMDBPostConfig invoke
WARNING:
************************ WARNING ************************
Management Repository has been placed in secure mode wherein Enterprise Manager data will be encrypted. The encryption key has been placed
in the file: /home/oracle/app/oracle/product/11.2.0/dbhome_1/prod-db1_FINPROD/sysman/config/emkey.ora. Ensure this file is backed up as the
encrypted data will become unusable if this file is lost.
***********************************************************
Enterprise Manager configuration completed successfully
FINISHED EMCA at Oct 24, 2014 2:45:33 PM
-- Now recheck the status of dbcolsole
$ emctl status dbconsole
Oracle Enterprise Manager 11g Database Control Release 11.2.0.3.0
Copyright (c) 1996, 2011 Oracle Corporation. All rights reserved.
https://prod-db1:1158/em/console/aboutApplication
Oracle Enterprise Manager 11g is running.
------------------------------------------------------------------
Logs are generated in directory /home/oracle/app/oracle/product/11.2.0/dbhome_1/prod-db1_FINPROD/sysman/log
OEM Directory structure for Oracle 11g and more:
Use below links for details information:
http://docs.oracle.com/cd/E11882_01/server.112/e25494/dbcontrol.htm
Deleting and Creating Database Control Objects:
=====================================
For an Oracle RAC database ( more than One node ):
Option 1: Deleting Database Control Configuration Files Using EMCA Scripts
$ $ORACLE_HOME/bin/emca -deconfig dbcontrol db –cluster
Option 2: Deleting Database Control Configuration Files and Repository Objects Using An EMCA Script
$ $ORACLE_HOME/bin/emca -deconfig dbcontrol db -repos drop –cluster
Option 3: Deleting A Database Control Repository Using EMCA Scripts
$ $ORACLE_HOME/bin/emca -repos drop -cluster
Option 4: Deleting Database Control Configuration Files Manually
In addition to using EMCA scripts to delete Database Control objects, you can delete objects manually, as follows:
Remove the following directories from your file system:
$ $ORACLE_HOME/hostname_sid
$ $ORACLE_HOME/oc4j/j2ee/OC4J_DBConsole_hostname_sid
Option 5: Deleting the Database Control Repository Using RepManager
You can delete the Database Control repository using RepManager. This option is not as complete as the other options. You may find that dropping the repository using the command line options is a better solution. Also note that RepManager cannot be used to create a Database Control Repository.
Run the following command:
$ $ORACLE_HOME/sysman/admin/emdrep/bin/RepManager hostname listener_port sid -action drop
Creating Database Control Objects:
===========================
Option 1: Creating the DB Control Configuration Files
$ $ORACLE_HOME/bin/emca -config dbcontrol db -cluster
Option 2: Creating the Database Control Repository Objects and Configuration Files
$ $ORACLE_HOME/bin/emca -config dbcontrol db -repos create –cluster
Option 3: Creating the Database Control Repository Objects and Configuration Files For a Cloned Database Home
$ $ORACLE_HOME/bin/emca -config dbcontrol db -repos recreate -cluster
Recreating Or Reconfiguring Database Control:
====================================
Option 1: Recreating the Database Control Configuration Files Only (Leaving Repository Intact)
$ $ORACLE_HOME/bin/emca -config dbcontrol db -cluster
Option 2: Recreating the Database Control Configuration Files and Repository
$ $ORACLE_HOME/bin/emca -config dbcontrol db -repos recreate -cluster
Note: Export ORACLE_SID and RDBMS home in your connecting/ oprating node.
Below error message during OEM repository creation:
$ emca -config dbcontrol db -repos recreate
.................
.................
Mar 14, 2016 11:49:08 AM oracle.sysman.emcp.EMConfig perform
INFO: This operation is being logged at /u01/app/oracle/cfgtoollogs/emca/UAT/emca_2016_03_14_11_47_54.log.
Mar 14, 2016 11:49:09 AM oracle.sysman.emcp.ParamsManager checkListenerStatusForDBControl
WARNING: ORA-12541: TNS:no listener
Mar 14, 2016 11:49:09 AM oracle.sysman.emcp.EMConfig perform
SEVERE:
Database connection through listener failed. Fix the error and run EM Configuration Assistant again.
Some of the possible reasons may be:
1) Listener port 1251 provided is incorrect. Provide the correct port.
2) Listener is not up. Start the Listener.
3) Database service UAT is not registered with listener. Register the database service.
4) Listener is up on physical host and ORACLE_HOSTNAME environment variable is set to virtual host. Unset ORACLE_HOSTNAME environment variable.
5) Listener is up on virtual host. Set environment variable ORACLE_HOSTNAME=<virtual host>.
6) /etc/hosts does not have correct entry for hostname.
Refer to the log file at /u01/app/oracle/cfgtoollogs/emca/UAT/emca_2016_03_14_11_47_54.log for more details.
Could not complete the configuration. Refer to the log file at /u01/app/oracle/cfgtoollogs/emca/UAT/emca_2016_03_14_11_47_54.log for more details.
$
Solution:
check the following points
1)-Ensure that the Database is up and running by executing the below commands:
<ORACLE_HOME>/bin/sqlplus "/ as sysdba"
SQL> select status from v$instance;
2)-Ensure that the Listener servicing the Database is up and running by executing the below command:
<ORACLE_HOME>/bin/lsnrctl status <listener name>
3)-SYS should be able to establish connection to the Database both via the listener and also the bequeath adaptor.
<ORACLE_HOME>/bin/sqlplus sys as sysdba
or
<ORACLE_HOME>/bin/sqlplus "sys as sysdba"
and
<ORACLE_HOME>/bin/sqlplus sys/<Password>@<TNS Connect descriptor>
If above connection fails take corrective actions to get the connection working from sqlplus.
4)-Password file should be correctly configured for SYS user connection.
Verify the initialization parameter REMOTE_LOGIN_PASSWORDFILE is set to EXCLUSIVE by executing the below command.
SQL> show parameter REMOTE_LOGIN_PASSWORDFILE;
5)-DBSNMP user must exist in the database and should be able to connect to the database.
<ORACLE_HOME>/bin/sqlplus dbsnmp/<password>
Following this, I fixed the issue. Crosscheck all.
This above simple solution fix my issue. You can try in the same way. If you have other issues, please send mail / post in comment box.
Thank you. Cheers !!!
No comments:
Post a Comment