How to apply PSU patch and what are the pre-requisite parameter we have to take care before going for a patch.
Important- $ORACLE_HOME -> Before going for PSU patch we have to check $ORACLE_HOME must have minimum 2GB of space.
Here- prdrms is the server name and prdrms is the database name as well.
Important- $ORACLE_HOME -> Before going for PSU patch we have to check $ORACLE_HOME must have minimum 2GB of space.
Here- prdrms is the server name and prdrms is the database name as well.
Ø
Login to the server prdrms server and sudo to oracle,
validate the instance name and set the profile as below :-
prdrms:/home1/mukhers3> sudo su - oracle
prdrms:/apps/oracle> ps -aef|grep pmon
prdrms:/apps/oracle> .
/prdrms/admin/profile/oracle.profile
oracle 21889166 1
0 Feb 26 -
0:20 ora_pmon_PRDRMS
Ø
Check the Operating System version from the
below command
prdrms /prdrms/admin/profile=>function aixversion {
OSLEVEL=$(oslevel
-s)
AIXVERSION=$(echo
"scale=1; $(echo $OSLEVEL | cut -d'-' -f1)/1000" | bc)
AIXTL=$(echo
$OSLEVEL | cut -d'-' -f2 | bc)
AIXSP=$(echo $OSLEVEL
| cut -d'-' -f3 | bc)
echo "AIX
${AIXVERSION} - Technology Level ${AIXTL} - Service Pack ${AIXSP}"
}
aixversion
prdrms /prdrms/admin/profile=>aixversion
AIX 6.1 - Technology Level 8 - Service Pack 3
prdrms /prdrms/admin/profile=>
Check the operating system bit from the below command.
prdrms /prdrms/admin/profile=>getconf KERNEL_BITMODE
64
Ø Check
the database version as below :-
SQL> select * from v$version ;
BANNER
----------------------------------------------------------------------------
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit
Production
PL/SQL Release 11.2.0.3.0 - Production
CORE
11.2.0.3.0 Production
TNS for IBM/AIX RISC System/6000: Version 11.2.0.3.0 -
Production
NLSRTL Version 11.2.0.3.0 – Production
Ø
Go to MOS (mysupport.oracle.com-Patches and Upgrades
section) and download the relevant patch set selecting the appropriate
Operating System as well as Database version by searching it with patch number.
Download the exact PSU patchset from Supoort.Oracle.com.In this case the
patchset is p19769496_112030_AIX64-5L.zip.
Ø
Psftp/winscp the patchset to the prdrms server under /prdrms/oracle/temp.
Ø
Unzip the patchset prdrms /prdrms/oracle/temp=>unzip p19769496_112030_AIX64-5L.zip
Ø
After unzipping it will create a folder similar
to below.
drwxrwxr-x 15 oracle
oradba 4096 Dec 10 20:57
19769496
Ø
Export PATH as below :-
export PATH=$ORACLE_HOME/OPatch:$PATH
Ø
Check perl version and check that whether the
minimum perl required for the patch is installed
by using perl –v
Ø
Check that the proper Oracle home is getting
pointed and binaries are getting invoked from proper location
prdrsm /prdrms/oracle/temp=>which opatch
/prdrms/oracle/product/11.2.0/OPatch/opatch
prdrms /apps/oracle=>which unzip
/prdrms/oracle/product/11.2.0/bin/unzip
Ø
Check that whether minimum opatch utility is installed
as per Patch Read me document.
prdrms /prdrms/oracle/temp=>opatch
version
OPatch Version: 11.2.0.3.6
OPatch succeeded.
Oracle recommends that you use the latest
released OPatch version for 11.2, which is available for download from My
Oracle Support patch 6880880by
selecting the 11.2.0.0.0 release.
Ø
Check what are the patches installed in the
database from operating system level:-
prdrms /prdrms/oracle/temp=>opatch lsinventory
Oracle Interim Patch Installer version 11.2.0.3.6
Copyright (c) 2013, Oracle Corporation. All rights reserved.
Oracle Home :
/prdrms/oracle/product/11.2.0
Central Inventory : /prdrms/oracle/oraInventory
from : /prdrms/oracle/product/11.2.0/oraInst.loc
OPatch version :
11.2.0.3.6
OUI version :
11.2.0.3.0
Log file location :
/prdrms/oracle/product/11.2.0/cfgtoollogs/opatch/opatch2015-02-26_16-57-23PM_1.log
Lsinventory Output file location : /prdrms/oracle/product/11.2.0/cfgtoollogs/opatch/lsinv/lsinventory2015-02-26_16-57-23PM.txt
--------------------------------------------------------------------------------
Installed Top-level Products (1):
Oracle Database 11g 11.2.0.3.0
There are 1 product(s) installed in this Oracle Home.
Interim patches (1) :
Patch 17540582 : applied on Mon Jan 20 10:55:24 PST 2014
Unique Patch ID:
16985513
Patch description:
"Database Patch Set Update : 11.2.0.3.9 (17540582)"
Created on 7 Jan
2014, 04:32:31 hrs PST8PDT
Sub-patch 16902043;
"Database Patch Set Update : 11.2.0.3.8 (16902043)"
Sub-patch 16619892;
"Database Patch Set Update : 11.2.0.3.7 (16619892)"
Sub-patch 16056266;
"Database Patch Set Update : 11.2.0.3.6 (16056266)"
Sub-patch 14727310;
"Database Patch Set Update : 11.2.0.3.5 (14727310)"
Sub-patch 14275605;
"Database Patch Set Update : 11.2.0.3.4 (14275605)"
Sub-patch 13923374;
"Database Patch Set Update : 11.2.0.3.3 (13923374)"
Sub-patch 13696216;
"Database Patch Set Update : 11.2.0.3.2 (13696216)"
Sub-patch 13343438;
"Database Patch Set Update : 11.2.0.3.1 (13343438)"
Bugs fixed:
--------------------------------------------------------------------------------
OPatch succeeded.
One off Patch Conflict Detection and
Resolution
Ø
Determine whether any currently installed one-off
patches conflict with the PSU patch as follows:
prdrms /prdrms/oracle/temp=>opatch prereq
CheckConflictAmongPatchesWithDetail -phBaseDir ./19769496 -invPtrLOc $ORACLE_HOME/oraInst.loc
Oracle Interim Patch Installer version 11.2.0.3.6
Copyright (c) 2013, Oracle Corporation. All rights reserved.
PREREQ session
Oracle Home :
/prdrms/oracle/product/11.2.0
Central Inventory : /prdrms/oracle/oraInventory
from :
/prdrms/oracle/product/11.2.0/oraInst.loc
OPatch version :
11.2.0.3.6
OUI version :
11.2.0.3.0
Log file location :
/prdrms/oracle/product/11.2.0/cfgtoollogs/opatch/opatch2015-02-19_12-08-36PM_1.log
Invoking prereq "checkconflictamongpatcheswithdetail"
Prereq "checkConflictAmongPatchesWithDetail"
passed.
OPatch succeeded.
Execute the below check conflict command from outside of the
patch directory if you want to use the central inventory.
prdrms /prdrms/oracle/temp=>opatch prereq CheckConflictAmongPatchesWithDetail
-phBaseDir ./19769496
Oracle Interim Patch Installer version 11.2.0.3.6
Copyright (c) 2013, Oracle Corporation. All rights reserved.
PREREQ session
Oracle Home :
/prdrms/oracle/product/11.2.0
Central Inventory : /prdrms/oracle/oraInventory
from :
/prdrms/oracle/product/11.2.0/oraInst.loc
OPatch version :
11.2.0.3.6
OUI version :
11.2.0.3.0
Log file location :
/prdrms/oracle/product/11.2.0/cfgtoollogs/opatch/opatch2015-02-19_12-08-45PM_1.log
Invoking prereq
"checkconflictamongpatcheswithdetail"
Prereq "checkConflictAmongPatchesWithDetail"
passed.
OPatch succeeded.
Ø
Check if there is sufficient space in the oracle
home directory as PSU patches normally consume 2 GB of space in the oracle
home.
prdrms /prdrms/oracle/temp=>df -g $ORACLE_HOME
Filesystem GB
blocks Free %Used Iused %Iused Mounted on
/dev/prdrms_oraclelv
30.00 16.87 44%
48963 2% /prdrms/oracle
Ø
Shutdown the listener and database.
prdrms /apps/oracle=>lsnrctl stop
LISTENER_PRDRMS
SQL> shu immediate;
Ø
Ensure that no oracle process is running from
O/s end and no jobs is running like below :-
ps –aef|grep –i oracle
$ jobs
If any jobs/processes arerunning,
pleasekill those processes using kill -9 process id and stop the ongoing jobs.
If any applications jobs are running, coordinate with ITAS DBA team.
Ø
Comment all the crontab entries relevant to the
database.
Ø
Take a backup of Oracle Home directory and
oracle inventory file in a mount point where there is sufficient amount of
space.
prdrms /prdrms/oracle/product/11.2.0=>cd /prdrms/oracle/product
prdrms /prdrms/oracle/product=>ls -ltr
total 8
drwxr-xr-x 80
oracle oradba 4096 Feb 12 03:04 11.2.0
prdrms /prdrms/oracle/product=>tar -cvf /prdrms/tmp002/11.2.0_PRDRMS.tar
11.2.0
prdrms /prdrms/oracle/product=>cat
$ORACLE_HOME/oraInst.loc
inventory_loc=/prdrms/oracle/oraInventory
inst_group=oradba
prdrms /prdrms/oracle/product=>cd /prdrms/oracle
prdrms /prdrms/oracle=>tar -cvf
/prdrms/tmp002/oraInventory_PRDRMS.tar oraInventory
Ø
Go to the patch directory and apply the patch
after confirm no database process is running from the Oracle Home
prdrms /prdrms/oracle/temp/19769496=>pwd
/prdrms/oracle/temp/19769496
prdrms /prdrms/oracle/temp/19769496=> opatch apply
OPatch detected non-cluster Oracle Home from the inventory
and will patch the local system only.Please shutdown Oracle instances running
out of this ORACLE_HOME on the local system.
(Oracle Home = '/prdrms/oracle/product/11.2.0')
[Feb 26,
2015 5:51:07 PM] User Responded with:
Y
Press Enter when
asked email and select ‘YES’ about security updates when asked
Ø
It was noticed that opatch failed with the error
message “Copy failed from '/prdrms/oracle/temp/19769496/18031683/files/lib/libnnz11.so'
to '/prdrms/oracle/product/11.2.0/lib/libnnz11.so'...” .Do you want to proceed?
[y|n]
Start to wait for user-input at Thu Feb 26
17:51:38 PST 2015
Finish waiting for user-input at Thu Feb 26
17:51:44 PST 2015
User Responded with: N
OUI-67124:ApplySession failed in system
modification phase... 'ApplySession::apply failed: Copy failed from
'/prdrms/oracle/temp/19769496/18031683/files/lib/libnnz11.so' to
'/prdrms/oracle/product/11.2.0/lib/libnnz11.so'...
‘Restoring "/prdrms/oracle/product/11.2.0" to the state prior to
running NApply...
[Feb 26, 2015 5:51:49 PM] OPatch failed to restore the files from
backup area. Not running "make".
[Feb 26, 2015 5:51:49 PM] OUI-67124:
NApply was not
able to restore the home. Please invoke
the following scripts:
-
restore.[sh,bat]
- make.txt (Unix
only)
to restore the
ORACLE_HOME. They are located under
"/prdrms/oracle/product/11.2.0/.patch_storage/NApply/2015-02-26_17-49-26PM"
Ø
Oracle failed to restore Oracle home, hence
instructed us to run restore.sh as above.
Ø
Restore .sh was executed as below :-
=~=~=~=~=~=~=~=~=~=~=~= PuTTY log
2015.02.26 17:53:19 =~=~=~=~=~=~=~=~=~=~=~=
./restore.sh
This script is going to restore the Oracle
Home to the previous state.
It does not perform any of the following:
- Running init/pre/post scripts
- Oracle binary re-link
- Customized steps performed manually by
user
Please use this script with supervision
from Oracle Technical Support.
About to modify Oracle Home(
/prdrms/oracle/product/11.2.0 )
Do you want to proceed? [Y/N]
Y
User responded with : Y
Restore script completed.
Ø
After running restore.sh,Oracle home has been
relinked as below :-
prdrms /prdrms/oracle/product/11.2.0/.patch_storage/NApply/2015-02-26_17-49-26PM=>$ORACLE
_HOME/bin/relink all;
writing relink log to:
/prdrms/oracle/product/11.2.0/install/relink.log
Ø
Once relinking is completed , opatch is applied
again and it is completed successfully
as below :-
prdrms /prdrms/oracle/temp/19769496=> opatch apply
[Feb 26,
2015 6:02:32 PM]
--------------------------------------------------------------------------------
[Feb 26, 2015 6:02:32 PM]
OUI-67008:OPatch Session completed with warnings.
We can ignore the warning as it is specific to some AIX
environments.
Ø
Once the patch apply has finished perform the
following post patching activities.
Bring the database up.
cd $ORACLE_HOME/rdbms/admin
sqlplus /nolog
SQL> CONNECT / AS SYSDBA
SQL> STARTUP
SQL> @catbundle.sql psu apply
SQL> @utlrp.sql
Ø
Bounce the database again and open it, start the
listener.
Ø
Check the alert log for any error/warning
messages after database is started.
Post Patching Steps
1) Check that no object owner by “SYS” is in invalid state
as below:-
SQL> select OWNER,OBJECT_NAME,OBJECT_TYPE from
dba_objects where OWNER = 'SYS' and status <>'VALID';
If any object is invalid, compile them.
2) Validate the PSU level from O/S level and database level
by below query:-
set linesize 120
col instance_name format a15 heading
"Instance|Name"
col host_name
format a15 heading "Hostname"
col version
format a10 heading "Version"
col comments
format a20 heading "Patch|Applied"
col id heading "Patch|Number"
col to_char(r.action_time,'DD-Mon-YYYY@HH24:mi:ss') format a20 heading "Date"
rem ******************
prompt MISC: APPLIED CPU HISTORY BY DATE
select distinct i.instance_name, i.host_name, i.version,
r.comments, r.id, to_char(r.action_time,'DD-Mon-YYYY@HH24:mi:ss')
from v$instance i, sys.registry$history r
where r.comments like 'CPU%' or r.comments like 'PSU%'
order by to_char(r.action_time,'DD-Mon-YYYY@HH24:mi:ss');
/
!$ORACLE_HOME/OPatch/opatch lsinventory -invPtrLoc
$ORACLE_HOME/oraInst.loc |grep "Patch" | grep "applied on"