Configuring three IPs for SCAN listener in Oracle 11gR2

How to configure SCAN listener with DNS?

About SCAN( Single Client Access Name) Listener in Oracle 11gR2 RAC:
Single Client Access Name (SCAN) is a new Oracle Real Application Clusters (RAC) 11g Release 2 feature that provides a single name for clients to access Oracle Databases running in a cluster. The benefit is that the client’s connect information does not need to change if you add or remove nodes in the cluster. Having a single name to access the cluster allows clients to use the EZConnect client and the simple JDBC thin URL to access any database running in the cluster, independently of which server(s) in the cluster the database is active. SCAN provides load balancing and failover for client connections to the database. The SCAN works as a cluster alias for databases in the cluster.

Why three IPs to be configured for SCAN listener through DNS (Domain Name Server):

If we configure 3 IPs for SCAN listener through DNS, then in case of any failure on any SCAN IP, then fail-over will happen to other running IP. Another benefit, any client access should resolved throgh DNS also. 

Before Configuration:

$ srvctl config scan
SCAN name: prddbscan, Network: 1/101.10.1.1/255.255.255.192/en0
SCAN VIP name: scan1, IP: /prddbscan/101.10.1.4
$

Starting Configuration:
Step:1  - add three IPs in DNS , e.g.,

101.10.1.5            hrdbscan.hrapplication.com
101.10.1.6            hrdbscan.hrapplication.com
101.10.1.7            hrdbscan.hrapplication.com

Step:2 - Stop all node scan listeners

$ srvctl stop scan_listener

Step:3 - Create the below file in /etc location with adding domain name

# vi /etc/resolv.conf
search domain hrapplication.com
nameserver      101.10.9.9

Note : Name of the DNS/ AD server is "hrapplication.com" and IP is 10.10.9.9

Step: 4 - Verify with nslookup in all nodes - all node should show configured three IPs

# nslookup hrdbscan.hrapplication.com
Server:         101.10.9.9
Address:        101.10.9.9#53

Name:   hrdbscan.hrapplication.com
Address: 101.10.1.7
Name:   hrdbscan.hrapplication.com
Address: 101.10.1.6
Name:   hrdbscan.hrapplication.com
Address: 101.10.1.5

Note: If your DNS server does not return a set of 3 IPs as shown in figure 3 or does not round-robin, ask your network administrator to enable such a setup. DNS using a round-robin algorithm on its own does not ensure failover of connections. However, the Oracle Client typically handles this. It is therefore recommended that the minimum version of the client used is the Oracle Database 11g Release 2 client.

Step: 5 - modify scan
#./srvctl modify scan -n hrdbscan.hrapplication.com
#./srvctl modify scan_listener -u

-- again verify

# ./srvctl config scan 

Step: 6 - start the scan listener

#./srvctl start scan_listener
#./srvctl status scan_listener

Step: 7 - Now stop cluster services and start it again to effect

./crsctl stop crs -- one by one node

./crsctl start crs

./crsctl stat res -t
./crsctl check crs

Step: 8 - check the services

./crsctl stat res -t

HOW CONNECTION LOAD BALANCING WORKS USING SCANFor clients connecting using Oracle SQL*Net 11g Release 2, three IP addresses will be received by the client by resolving the SCAN name through DNS as discussed. The client will then go through the list it receives from the DNS and try connecting through one of the IPs received. If the client receives an error, it will try the other addresses before returning an error to the user or application. This is similar to how client connection failover works in previous releases when an address list is provided in the client connection string.

When a SCAN Listener receives a connection request, the SCAN Listener will check for the least loaded instance providing the requested service. It will then re-direct the connection request to the local listener on the node where the least loaded instance is running. Subsequently, the client will be given the address of the local listener. The local listener will finally create the connection to the database instance.

 

This document may help you. You can refer Oracle support documents for more clarification.

Case -1: Unable to start database instances after starting all cluster services:

When I tried to start database instances after starting all node cluster services, I found below error.

SQL> startup nomount;
ORA-00119: invalid specification for system parameter REMOTE_LISTENER
ORA-00132: syntax error or unresolved network name 'hrapplication.com:1521'

After some verification, I found some body disbaled below options during some RAM upgrations time.

# vi /etc/resolv.conf
#search domain hrapplication.com
#nameserver      101.10.9.9

I uncommented like below and then started my databases services and found ok.

# vi /etc/resolv.conf
search domain hrapplication.com
nameserver      101.10.9.9

Converting shared server architecture to dedicated architecture in Oracle

shared server architecture vs dedicated architecture in Oracle

About Share server architecture :
When client load causes a strain on memory and other system resources, database administrators can alleviate load issues by starting shared server resources. The shared server architecture enables a database server to allow many client processes to share very few server processes, so the number of users that can be supported is increased. With the shared server architecture, many client processes connect to a dispatcher. The dispatcher directs multiple incoming network session requests to a common queue. An idle shared server process from a shared pool of server processes picks up a request from the queue. This means a small pool of server processes can serve a large number of clients. This is useful when a system is overloaded or has limited memory.


About Dedicated architecture:
A server process that is dedicated to one client connection. Contrast with shared server.

Note: UGA is resides in SGA in case of Share server but it resides in PGA in case of dedicated architecture. Usually in case of huge no. of user connections are controlled with shared-server architecture but typical customaized applications (OLTP) are always configuring with dedicated architecture now a days.

If you system is confugured with shared-server, then follow the bellow steps to convert into dedicated:

Step: 1 :Do the following in mount stage:

*.dispatchers='(PROTOCOL=TCP)(dispatchers=0) (SERVICE=MYDB)'
alter system set shared_servers=0;
alter system set max_shared_servers=0;
alter system set shared_server_sessions=0;
alter system set max_dispatchers=0;

Step:2 : Restart the database

Step:3 : Use (server=dedicated) in client tnsnames.ora

For details follow the bellow link from Oracle site.
http://docs.oracle.com/cd/B28359_01/server.111/b28310/manproc001.htm#ADMIN11166

Compress table/ table-partition / tablespace using Oracle 11g

DBMS_COMPRESSION Example

With starting Oracle 11g compression concept, this question may arise that, how to estimate the size of tables after compression. In addition to my previous posting that, I have therefore prepared this little script that calls DBMS_COMPRESSION. Typically, the documentation also gives an example.

Please notice that you can use that script already on a non-Exadata Oracle Database (from 11.2 on)  also to get an estimation about that, not only about BASIC and OLTP compression. The estimation is quite good but takes much space – about as much as the tables you estimate are in size. Therefore, it may be advisable to use a scratch tablespace only for that purpose and drop it afterwards.

-- Create table

create table hr.bigtab as
select * from dba_objects;  

Note : you can append same data for 100 or 1000 times with a loop to create actually a big table.

-- Check the tablespace

select * from all_tables where table_name='BIGTAB' 

-- Can be moved to any tablespace if created in SYSTEM.

alter table BIGTAB move tablespace HR_TBLSPC;

-- Gather table stats 

exec dbms_stats.gather_table_stats('HR','BIGTAB');

-- Now compress the table

alter table owner.table_name compress | nocompress;
OR
ALTER TABLE owner.table_name 
MODIFY PARTITION partition_name COMPRESS FOR ALL OPERATIONS;

alter table hr.bigtab compress;

-- Find compression statistics

set serveroutput on
declare
 v_blkcnt_cmp     pls_integer;
 v_blkcnt_uncmp   pls_integer;
 v_row_cmp        pls_integer;
 v_row_uncmp      pls_integer;
 v_cmp_ratio      number;
 v_comptype_str   varchar2(60);
begin
 dbms_compression.get_compression_ratio(
 scratchtbsname   => upper('HR_TBLSPC'),
 ownname          => 'HR',
 tabname          => upper('BIGTAB'),
 partname         => NULL,
 comptype         => dbms_compression.comp_for_query_high,
 blkcnt_cmp       => v_blkcnt_cmp,
 blkcnt_uncmp     => v_blkcnt_uncmp,
 row_cmp          => v_row_cmp,
 row_uncmp        => v_row_uncmp,
 cmp_ratio        => v_cmp_ratio,
 comptype_str     => v_comptype_str);
 dbms_output.put_line('Estimated Compression Ratio: '||to_char(v_cmp_ratio));
 dbms_output.put_line('Blocks used by compressed sample: '||to_char(v_blkcnt_cmp));
 dbms_output.put_line('Blocks used by uncompressed sample: '||to_char(v_blkcnt_uncmp));
end;
/

output:

Compression Advisor self-check validation successful. select count(*) on both Uncompressed and EHCC Compressed format = 1000001 rows
Estimated Compression Ratio: 12.3
Blocks used by compressed sample: 1199
Blocks used by uncompressed sample: 14842

PL/SQL procedure successfully completed

Compress on Partitioned Table:

The following examples show the various compression options applied at table and partition level.

-- Table compression.
CREATE TABLE sample_tab_1 (
  id            NUMBER(10)    NOT NULL,
  description   VARCHAR2(50)  NOT NULL,
  created_date  DATE          NOT NULL
)
COMPRESS FOR ALL OPERATIONS;

-- Partition-level compression.
CREATE TABLE sample_tab_2 (
  id            NUMBER(10)    NOT NULL,
  description   VARCHAR2(50)  NOT NULL,
  created_date  DATE          NOT NULL
)
PARTITION BY RANGE (created_date) (
  PARTITION sample_tab_q1 VALUES LESS THAN (TO_DATE('01/01/2014', 'DD/MM/YYYY')) COMPRESS,
  PARTITION sample_tab_q2 VALUES LESS THAN (TO_DATE('01/04/2014', 'DD/MM/YYYY')) COMPRESS FOR DIRECT_LOAD OPERATIONS,
  PARTITION sample_tab_q3 VALUES LESS THAN (TO_DATE('01/07/2014', 'DD/MM/YYYY')) COMPRESS FOR ALL OPERATIONS,
  PARTITION sample_tab_q4 VALUES LESS THAN (MAXVALUE) NOCOMPRESS
);Table-level compression settings are reflected in the COMPRESSION and COMPRESS_FOR columns of the [DBA|ALL|USER]_TABLES views.

SELECT table_name, compression, compress_for FROM user_tables;

TABLE_NAME                     COMPRESS COMPRESS_FOR
------------------------------ -------- ------------------
sample_TAB_1                     ENABLED  FOR ALL OPERATIONS
sample_TAB_2

2 rows selected.

SQL>Tables defined with partition-level compression and no table-level compression display NULL values in these columns.

Partition-level compression settings are reflected in the COMPRESSION and COMPRESS_FOR columns of the [DBA|ALL|USER]_TAB_PARTITIONS views.

SELECT table_name, partition_name, compression, compress_for FROM user_tab_partitions;

output:

TABLE_NAME                     PARTITION_NAME                 COMPRESS COMPRESS_FOR

------------------------------ ------------------------------ -------- ------------------

sample_TAB_2                     sample_TAB_Q1                    ENABLED  DIRECT LOAD ONLY

sample_TAB_2                     sample_TAB_Q2                    ENABLED  DIRECT LOAD ONLY

sample_TAB_2                     sample_TAB_Q3                    ENABLED  FOR ALL OPERATIONS

sample_TAB_2                     sample_TAB_Q4                    DISABLED

4 rows selected.

SQL>

Tablespace compression:

Default compression settings can be specified at the tablespace level using the CREATE TABLESPACE and ALTER TABLESPACE commands. The current settings are displayed in the DEF_TAB_COMPRESSION and COMPRESS_FOR columns of the DBA_TABLESPACES view.

CREATE TABLESPACE test_tbs
  DATAFILE '/u02/oradata/datafiles/prod/test_ts01.dbf'
  SIZE 1M
  DEFAULT COMPRESS FOR ALL OPERATIONS;

SELECT def_tab_compression, compress_for
FROM   dba_tablespaces
WHERE  tablespace_name = 'TEST_TBS';

DEF_TAB_ COMPRESS_FOR
-------- ------------------
ENABLED  FOR ALL OPERATIONS

1 row selected.

SQL>

ALTER TABLESPACE test_tbs DEFAULT NOCOMPRESS;

SELECT def_tab_compression, compress_for
FROM   dba_tablespaces
WHERE  tablespace_name = 'TEST_TBS';

DEF_TAB_ COMPRESS_FOR
-------- ------------------
DISABLED

1 row selected.

SQL>

DROP TABLESPACE test_tbs INCLUDING CONTENTS AND DATAFILES;

When compression is specified at multiple levels, the most specific setting is always used. As such, partition settings always override table settings, which always override tablespace settings.

The restrictions associated with table compression include:

• Compressed tables can only have columns added or dropped if the COMPRESS FOR ALL OPERATIONS option was used.
• Compressed tables must not have more than 255 columns.
• Compression is not applied to lob segments.
• Table compression is only valid for heap organized tables, not index organized tables.
• The compression clause cannot be applied to hash or hash-list partitions. Instead, they must inherit their compression settings from the tablespace, table or partition settings.
• Table compression cannot be specified for external or clustered tables.

Compression Advisor:

Overview:

Compression Advisor provides an estimate of the compression ratio that can be realized through the use of the Oracle Advanced Compression option. This estimate is based on analysis of a sample of data and provides a good estimate of the actual results you may obtain once you implement the OLTP Table compression feature in your environment.

The Compression Advisor PL/SQL package you use is dependent upon which Oracle Database release you currently have deployed. Those customers that want to use Compression Advisor with Oracle Database 9i Release 2 through Oracle Database 11g Release 1 will use the DBMS_COMP_ADVISOR package available for download below. Customers that want to use Compression Advisor with Oracle Database 11g Release 2 through Oracle Database 12c will use the DBMS_COMPRESSION package that is included with the database.

This package can be used on Oracle Databases running Oracle Database 9i Release 2 through 11g Release 1. A compression advisor (DBMS_COMPRESSION) is included with Oracle Database 11g Release 2 and Oracle Database 12c.

Using Compression Advisor:

This procedure can be used with Oracle Database 9i Release 2 through Oracle Database 11g Release 1. Running this procedure will create tables in the default tablespace of the user running the procedure. While these tables will get dropped at the end of the procedure they will consume space while the procedure runs. Oracle recommends creating a tablespace specifically for storing these tables and assigning it as the default tablespace to the user running the procedure. The DBMS_COMP_ADVISOR advisor package is only available as a free download.

Compression Advisor consists of the DBMS_COMP_ADVISOR package containing the following procedure:

 getratio(
    ownername            IN     varchar2,
    tabname                IN     varchar2,
    sampling_percent   IN     number
  );

where

- 'ownername' is the schema that the table belongs to
- 'tabname' is name of the table for which compression ratio is to be estimated
- 'sampling_percent' is any value between 0.000001 and 99

The output of this procedure is the estimated compression ratio.

Example:

SQL>  set serveroutput on
SQL>  exec dbms_comp_advisor.getratio('SH','SALES',10);

Sampling table: SH.SALES
Sampling percentage: 10%
Expected Compression ratio with Advanced Compression option: 2.96

PL/SQL procedure successfully completed.

Note:  Compression Advisor concepts collected from Oracle magazine.

Start/ Stop / Relocate SCAN listener in Oracle 11gR2 RAC

Start/ Stop / Relocate SCAN listener in Oracle 11gR2 RAC

1) Check listener status ( login to grid home)

a) Check the cluster resource status
$ crsctl stat res -t

verify the output for listener
--------------------------------------------------------------------------------
NAME           TARGET  STATE        SERVER                   STATE_DETAILS    
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.DATA.dg
               ONLINE  ONLINE       node1                                      
               ONLINE  ONLINE       node2                                      
ora.FRA.dg
               ONLINE  ONLINE       node1                                      
               ONLINE  ONLINE       node2                                      
ora.LISTENER.lsnr
               ONLINE  ONLINE       node1                                      
               ONLINE  ONLINE       node2                                      
ora.asm
               ONLINE  ONLINE       node1                    Started          
               ONLINE  ONLINE       node2                    Started          
ora.gsd
               OFFLINE OFFLINE      node1                                      
               OFFLINE OFFLINE      node2                                      
ora.net1.network
               ONLINE  ONLINE       node1                                      
               ONLINE  ONLINE       node2                                      
ora.ons
               ONLINE  ONLINE       node1                                      
               ONLINE  ONLINE       node2                                      
ora.registry.acfs
               ONLINE  ONLINE       node1                                      
               ONLINE  ONLINE       node2                                      
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
      1        ONLINE  ONLINE       node2                                      
ora.LISTENER_SCAN2.lsnr
      1        ONLINE  ONLINE       node1                                      
ora.LISTENER_SCAN3.lsnr
      1        ONLINE  ONLINE       node1                                      
ora.cvu
      1        ONLINE  ONLINE       node1                                      
ora.node1.vip
      1        ONLINE  ONLINE       node1                                      
ora.node2.vip
      1        ONLINE  ONLINE       node2                                      
ora.PROD.db
      1        ONLINE  ONLINE       node1                    Open              
      2        ONLINE  ONLINE       node2                    Open              
ora.oc4j
      1        ONLINE  ONLINE       node1                                      
ora.scan1.vip
      1        ONLINE  ONLINE       node2                                      
ora.scan2.vip
      1        ONLINE  ONLINE       node1                                      
ora.scan3.vip
      1        ONLINE  ONLINE       node1

b) Check the scan listener status

$ srvctl status scan_listener

SCAN Listener LISTENER_SCAN1 is enabled
SCAN listener LISTENER_SCAN1 is running on node node2
SCAN Listener LISTENER_SCAN2 is enabled
SCAN listener LISTENER_SCAN2 is running on node node1
SCAN Listener LISTENER_SCAN3 is enabled
SCAN listener LISTENER_SCAN3 is running on node node1

c) Check the listener home. That sholud run in grid home:

LSNRCTL for IBM/AIX RISC System/6000: Version 11.2.0.3.0 - Production on 13-DEC-2014 14:02:27
Copyright (c) 1991, 2011, Oracle.  All rights reserved.
Connecting to (ADDRESS=(PROTOCOL=tcp)(HOST=)(PORT=1521))
STATUS of the LISTENER
------------------------
Alias                     LISTENER
Version                   TNSLSNR for IBM/AIX RISC System/6000: Version 11.2.0.3.0 - Production
Start Date                19-AUG-2014 21:46:52
Uptime                    115 days 16 hr. 15 min. 34 sec
Trace Level               off
Security                  ON: Local OS Authentication
SNMP                      ON
Listener Parameter File   /u01/app/grid/network/admin/listener.ora
Listener Log File         /u01/app/oracle/diag/tnslsnr/node1/listener/alert/log.xml
Listening Endpoints Summary...
  (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=LISTENER)))
  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=10.20.30.40)(PORT=1521)))
  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=10.20.30.41)(PORT=1521)))
Services Summary...
Service "+ASM" has 1 instance(s).
  Instance "+ASM1", status READY, has 1 handler(s) for this service...
Service "PROD" has 1 instance(s).
  Instance "PROD1", status READY, has 1 handler(s) for this service...
Service "PRODXDB" has 1 instance(s).
  Instance "PROD1", status READY, has 1 handler(s) for this service...
The command completed successfully

2) Start scan listener:

a) Start scan listener

$ srvctl start scan_listener

b) If lsnrctl status showing rdbms home, then do the following

$ lsnrctl stop
$ export ORACLE_HOME=/u01/app/grid
$ lsnrctl start

3) Relocate SCAN listener

a) To relocate
When you find all 3 scan listeners are running on single node, then you may relocate any one of the listener

$ srvctl relocate scan_LISTENER -i 1 -n node2

b) Check current status after Relocate SCAN_LISTENER:

bash-3.2$ srvctl status scan_listener
SCAN Listener LISTENER_SCAN1 is enabled
SCAN listener LISTENER_SCAN1 is running on node node2
SCAN Listener LISTENER_SCAN2 is enabled
SCAN listener LISTENER_SCAN2 is running on node node1
SCAN Listener LISTENER_SCAN3 is enabled
SCAN listener LISTENER_SCAN3 is running on node node1

c) inherit status

$ ps -ef|grep inherit
  oracle 49741838  9633998   0 14:10:00  pts/0  0:00 grep inherit
  oracle 18547030        1   0 13:26:56      -  0:00 /u01/app/grid/bin/tnslsnr LISTENER_SCAN1 -inherit
  oracle 31588762        1   0 13:20:20      -  0:14 /u01/app/grid/bin/tnslsnr LISTENER -inherit


So What is difference is between relocating the SCAN using srvctl relocate scan and SCAN_LISTENER  by using srvctl relocate scan_listener command?

Regarding questions; The difference between a SCAN VIP and a normal RAC VIP, is that the RAC VIP has a node it want’s to run on and each node has one (whereas you only have 3 SCANs). If it fails over to another node, the normal VIP exists, but does not accept connections, whereas the SCAN is not fix to a node and can run on any node in the cluster (and will accept connections anytime).

Now that this works, the SCAN VIP will always move with the SCAN listener (otherwise it would not make any sense). Hence there is really no difference in moving the SCAN VIP (because this will trigger a relocate of the listener) or to move the SCAN_Listener (since this will move the VIP it depends on).

3) Checking SCAN IPs

$ srvctl config scan

SCAN name: scandb.production.com, Network: 1/10.20.30.0/255.255.255.192/en8
SCAN VIP name: scan1, IP: /scandb.production.com/10.20.30.42
SCAN VIP name: scan2, IP: /scandb.production.com/10.20.30.43
SCAN VIP name: scan3, IP: /scandb.production.com/10.20.30.44

Configuring ssh key gen between database servers ( Linux / Aix, Oracle Method)

User Equivalence (Key-Based Authentication) Configuration on Linux/AIX

Basic Checks:

Turn on the SSH service and make sure it starts automatically on reboot.

# service sshd start
# chkconfig sshd onThe SSH service is configured using the "/etc/ssh/sshd_config" file. Configuration changes have to be followed by a restart of the service.

# service sshd restart
# # or
# service sshd reloadFirewall
The server must have the TCP port 22 open. This can be achieved by adding the following entry to the type of firewall script described here.

# Open port for NTP server.
iptables -A INPUT -p tcp --dport 22 -j ACCEPT

Step:1  : Check communication between all servers.

e.g., log in to 160.100.1.1 and do a ping. It should be succeded.

Step:2  : Add host names into /etc/hosts file of all required servers.

e.g. Server-1 IP : 160.100.1.2
Server-1 IP : 160.100.1.3

After entry:

$ cat /etc/hosts  (on 160.100.1.1)

160.100.2  prod-db2
160.100.1  prod-db3

$ cat /etc/hosts  (on 160.100.1.2)

160.100.1  prod-db3
160.100.2  prod-db2


Step: 3 : Find path of "sshUserSetup.sh" to generate ssh keys.

I found above file in 160.100.1.2.

/u01/SOFTWARE/database/sshsetup

Step: 4 : run sshUserSetup.sh

e.g.,
$ ./sshUserSetup.sh -user oracle -hosts "prod-db3 prod-db2" -advanced -noPromptPassphrase

If we pass in the basic parameters it will configure user equivalence for us.

Output:
$ ./sshUserSetup.sh -user oracle -hosts "prod-db3 prod-db2" -advanced -noPromptPassphrase
The output of this script is also logged into /tmp/sshUserSetup_2014-12-09-18-17-42.log
Hosts are prod-db3 prod-db2
user is oracle
Platform:- Linux 
Checking if the remote hosts are reachable
PING prod-db3 (160.100.1.1) 56(84) bytes of data.
64 bytes from prod-db3 (160.100.1.1): icmp_seq=1 ttl=64 time=0.032 ms
64 bytes from prod-db3 (160.100.1.1): icmp_seq=2 ttl=64 time=0.043 ms
64 bytes from prod-db3 (160.100.1.1): icmp_seq=3 ttl=64 time=0.033 ms
64 bytes from prod-db3 (160.100.1.1): icmp_seq=4 ttl=64 time=0.037 ms
64 bytes from prod-db3 (160.100.1.1): icmp_seq=5 ttl=64 time=0.041 ms

--- prod-db3 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4000ms
rtt min/avg/max/mdev = 0.032/0.037/0.043/0.005 ms
PING prod-db2 (160.100.1.2) 56(84) bytes of data.
64 bytes from prod-db2 (160.100.1.2): icmp_seq=1 ttl=64 time=0.336 ms
64 bytes from prod-db2 (160.100.1.2): icmp_seq=2 ttl=64 time=0.467 ms
64 bytes from prod-db2 (160.100.1.2): icmp_seq=3 ttl=64 time=0.296 ms
64 bytes from prod-db2 (160.100.1.2): icmp_seq=4 ttl=64 time=1.16 ms
64 bytes from prod-db2 (160.100.1.2): icmp_seq=5 ttl=64 time=0.600 ms

--- prod-db2 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4003ms
rtt min/avg/max/mdev = 0.296/0.573/1.169/0.317 ms
Remote host reachability check succeeded.
The following hosts are reachable: prod-db3 prod-db2.
The following hosts are not reachable: .
All hosts are reachable. Proceeding further...
firsthost prod-db3
numhosts 2
The script will setup SSH connectivity from the host prod-db3 to all
the remote hosts. After the script is executed, the user can use SSH to run
commands on the remote hosts or copy files between this host prod-db3
and the remote hosts without being prompted for passwords or confirmations.

NOTE 1:
As part of the setup procedure, this script will use ssh and scp to copy files between the local host and the remote hosts. Since the script does not store passwords, you may be prompted for the passwords during the execution of the script whenever ssh or scp is invoked.

NOTE 2:
AS PER SSH REQUIREMENTS, THIS SCRIPT WILL SECURE THE USER HOME DIRECTORY
AND THE .ssh DIRECTORY BY REVOKING GROUP AND WORLD WRITE PRIVILEDGES TO THESE directories.

Do you want to continue and let the script make the above mentioned changes (yes/no)?
yes

The user chose yes
User chose to skip passphrase related questions.
Creating .ssh directory on local host, if not present already
Creating authorized_keys file on local host
Changing permissions on authorized_keys to 644 on local host
Creating known_hosts file on local host
Changing permissions on known_hosts to 644 on local host
Creating config file on local host
If a config file exists already at /home/oracle/.ssh/config, it would be backed up to /home/oracle/.ssh/config.backup.
Creating .ssh directory and setting permissions on remote host prod-db3
THE SCRIPT WOULD ALSO BE REVOKING WRITE PERMISSIONS FOR group AND others ON THE HOME DIRECTORY FOR oracle. THIS IS AN SSH REQUIREMENT.
The script would create ~oracle/.ssh/config file on remote host prod-db3. If a config file exists already at ~oracle/.ssh/config, it would be backed up to ~oracle/.ssh/config.backup.
The user may be prompted for a password here since the script would be running SSH on host prod-db3.
Warning: Permanently added 'prod-db3,160.100.1.1' (RSA) to the list of known hosts.
oracle@prod-db3's password: 
Done with creating .ssh directory and setting permissions on remote host prod-db3.
Creating .ssh directory and setting permissions on remote host prod-db2
THE SCRIPT WOULD ALSO BE REVOKING WRITE PERMISSIONS FOR group AND others ON THE HOME DIRECTORY FOR oracle. THIS IS AN SSH REQUIREMENT.
The script would create ~oracle/.ssh/config file on remote host prod-db2. If a config file exists already at ~oracle/.ssh/config, it would be backed up to ~oracle/.ssh/config.backup.
The user may be prompted for a password here since the script would be running SSH on host prod-db2.
Warning: Permanently added 'prod-db2,160.100.1.2' (RSA) to the list of known hosts.
oracle@prod-db2's password: 
Done with creating .ssh directory and setting permissions on remote host prod-db2.
Copying local host public key to the remote host prod-db3
The user may be prompted for a password or passphrase here since the script would be using SCP for host prod-db3.
oracle@prod-db3's password: 
Done copying local host public key to the remote host prod-db3
Copying local host public key to the remote host prod-db2
The user may be prompted for a password or passphrase here since the script would be using SCP for host prod-db2.
oracle@prod-db2's password: 
Done copying local host public key to the remote host prod-db2
Creating keys on remote host prod-db3 if they do not exist already. This is required to setup SSH on host prod-db3.

Creating keys on remote host prod-db2 if they do not exist already. This is required to setup SSH on host prod-db2.

Updating authorized_keys file on remote host prod-db3
Updating known_hosts file on remote host prod-db3
Updating authorized_keys file on remote host prod-db2
Updating known_hosts file on remote host prod-db2
cat: /home/oracle/.ssh/known_hosts.tmp: No such file or directory
cat: /home/oracle/.ssh/authorized_keys.tmp: No such file or directory
SSH setup is complete.

------------------------------------------------------------------------
Verifying SSH setup
===================
The script will now run the date command on the remote nodes using ssh
to verify if ssh is setup correctly. IF THE SETUP IS CORRECTLY SETUP,
THERE SHOULD BE NO OUTPUT OTHER THAN THE DATE AND SSH SHOULD NOT ASK FOR
PASSWORDS. If you see any output other than date or are prompted for the
password, ssh is not setup correctly and you will need to resolve the
issue and set up ssh again.
The possible causes for failure could be:
1. The server settings in /etc/ssh/sshd_config file do not allow ssh
for user oracle.
2. The server may have disabled public key based authentication.
3. The client public key on the server may be outdated.
4. ~oracle or ~oracle/.ssh on the remote host may not be owned by oracle.
5. User may not have passed -shared option for shared remote users or
may be passing the -shared option for non-shared remote users.
6. If there is output in addition to the date, but no password is asked,
it may be a security alert shown as part of company policy. Append the
additional text to the <OMS HOME>/sysman/prov/resources/ignoreMessages.txt file.
------------------------------------------------------------------------
--prod-db3:--
Running /usr/bin/ssh -x -l oracle prod-db3 date to verify SSH connectivity has been setup from local host to prod-db3.
IF YOU SEE ANY OTHER OUTPUT BESIDES THE OUTPUT OF THE DATE COMMAND OR IF YOU ARE PROMPTED FOR A PASSWORD HERE, IT MEANS SSH SETUP HAS NOT BEEN SUCCESSFUL. Please note that being prompted for a passphrase may be OK but being prompted for a password is ERROR.
Tue Dec  9 18:18:10 IST 2014
------------------------------------------------------------------------
--prod-db2:--
Running /usr/bin/ssh -x -l oracle prod-db2 date to verify SSH connectivity has been setup from local host to prod-db2.
IF YOU SEE ANY OTHER OUTPUT BESIDES THE OUTPUT OF THE DATE COMMAND OR IF YOU ARE PROMPTED FOR A PASSWORD HERE, IT MEANS SSH SETUP HAS NOT BEEN SUCCESSFUL. Please note that being prompted for a passphrase may be OK but being prompted for a password is ERROR.
Tue Dec  9 18:19:59 IST 2014
------------------------------------------------------------------------
------------------------------------------------------------------------
Verifying SSH connectivity has been setup from prod-db3 to prod-db3
IF YOU SEE ANY OTHER OUTPUT BESIDES THE OUTPUT OF THE DATE COMMAND OR IF YOU ARE PROMPTED FOR A PASSWORD HERE, IT MEANS SSH SETUP HAS NOT BEEN SUCCESSFUL.
bash: -c: line 0: unexpected EOF while looking for matching `"'
bash: -c: line 1: syntax error: unexpected end of file
------------------------------------------------------------------------
------------------------------------------------------------------------
Verifying SSH connectivity has been setup from prod-db3 to prod-db2
IF YOU SEE ANY OTHER OUTPUT BESIDES THE OUTPUT OF THE DATE COMMAND OR IF YOU ARE PROMPTED FOR A PASSWORD HERE, IT MEANS SSH SETUP HAS NOT BEEN SUCCESSFUL.
bash: -c: line 0: unexpected EOF while looking for matching `"'
bash: -c: line 1: syntax error: unexpected end of file
------------------------------------------------------------------------


You should now be able to SSH and SCP between servers without entering passwords.

Note:

Remember to restart or reload the service after making any changes.

# service sshd reload