Thursday, November 18, 2010

RHEL6 64-bit and Oracle 11g R2

ok so RH Enterprise Linux 6 has been released. I'm installing it to test an oracle 11g installation. included below is the kickstart script used.

BIG CAUTION: ORACLE 11g (nor any other oracle versions afaik) is not certified (yet) to run on RHEL6. SO DON'T PUT THIS ON PRODUCTION.

first issue encountered is the installation over NFS. my NFS RHEL6 sources are on anAIX 5L machine. and anaconda won't mount the export.

i resorted to using an RHEL5U5 system to host the ISOs.

here's the directory structure:

/apps/erik/RHEL6/

|-- i386
|   |-- images
|   |   |-- README
|   |   |-- TRANS.TBL
|   |   |-- install.img
|   |   |-- product.img
|   |   `-- pxeboot
|   |-- rhel-server-6.0-i386-dvd.iso
|   |-- rhel-server-supplementary-6.0-i386-dvd.iso
|   `-- sha256sum.txt
`-- x86_64
    |-- images
    |   |-- README
    |   |-- TRANS.TBL
    |   |-- efiboot.img
    |   |-- efidisk.img
    |   |-- install.img
    |   |-- product.img
    |   `-- pxeboot
    |-- rhel-server-6.0-x86_64-dvd.iso
    |-- rhel-server-supplementary-6.0-x86_64-dvd.iso
    `-- sha256sum.txt

the "images" directory is copied from the server iso image. the nfs based installationwon't proceed without that directory. appears to be a mandatory requirement now for the NFS install.

kickstart file used:

#START of ks file
# Kickstart file automatically generated by anaconda.

#version=RHEL6
install
nfs --server=LXISAP0050 --dir=/apps/erik/RHEL6/x86_64
lang en_US.UTF-8
keyboard us
network --device eth0 --bootproto static --ip 10.a.b.c --netmask 255.255.255.0 --gateway 10.a.6.1 --nameserver 10.a.8.11 --hostname LX0130.local
#r00tpassw0rd
rootpw  --iscrypted $1$ib6Rr/$wuTR7.6MF4DalT4WdirT90
# System services
services --disabled="sendmail,xfs,bluetooth,kudzu,firstboot,cups,ip6tables"
# Reboot after installation
reboot
firewall --disabled
authconfig --enableshadow --passalgo=sha512 --enablefingerprint
selinux --enforcing
timezone --utc Asia/Singapore
bootloader --location=mbr --driveorder=cciss/c0d0 --append="crashkernel=auto crashkernel=auto rhgb quiet" 
clearpart --linux --initlabel --drives=cciss/c0d0
part /boot --fstype=ext4 --asprimary --size=200
part pv.None --grow --size=200
volgroup vg00 --pesize=16384 pv.None
logvol / --fstype=ext4 --name=lvsys00 --vgname=vg00 --size=1024
logvol swap --name=lvsys01 --vgname=vg00 --size=8192
logvol /usr --fstype=ext4 --name=lvsys02 --vgname=vg00 --size=2048
logvol /var --fstype=ext4 --name=lvsys03 --vgname=vg00 --size=1024
logvol /opt --fstype=ext4 --name=lvsys04 --vgname=vg00 --size=1024
#this will produce error during oracle install
logvol /tmp --fstype ext4 --name=lvsys05 --vgname=vg00 --size=2048 --fsoptions=noexec,nodev
logvol /home --fstype ext4 --name=lvsys06 --vgname=vg00 --size=1024
logvol /apps --fstype ext4 --name=lvora00 --vgname=vg00 --size=2048 --grow


%packages
@Base
@Core
@additional-devel
@base
@compat-libraries
@console-internet
@core
@hardware-monitoring
@legacy-software-support
binutils
compat-db gcc
compat-gcc-34-c++
compat-glibc
compat-glibc-headers
compat-glibc.i386
compat-libgcc-296
compat-libstdc++-296
compat-libstdc++-33
dejavu-lgc-fonts
device-mapper
device-mapper-event
device-mapper-multipath
e2fsprogs
elfutils-libelf
elfutils-libelf-devel
gcc
gcc-c++
glibc
glibc-common
glibc-devel
glibc-headers
kernel
libX11-devel
libXext
libXp
libXt
libXtst
libXxf86dga
libXxf86misc
libaio
libaio-devel
libdmx
libgcc
libgcc.i386
libstdc++
libstdc++-devel
make
ncompress
ncurses
ncurses.i386
ntp
openmotif
openssh-clients
openssh-server
pdksh
postfix
pyOpenSSL
sg3_utils
strace
sysstat
unixODBC
unixODBC-devel
vnc-server
xorg-x11-deprecated-libs
xorg-x11-deprecated-libs-devel
xorg-x11-utils
xterm
yum
-bluez-gnome
-bluez-utils
-firstboot-tui
-ftp
-irda-utils
-nano
-notification-daemon
-openswan
-pango
-ppp
-rdist
-rp-pppoe
-rsh
-setserial
-talk
-usbutils
-wireless-tools
-wpa_supplicant
%end

%post --log=/root/ks-post.log
{
#~-> Remove GRUB's splash image
/bin/ed < User limits
ed /etc/security/limits.conf << EOF
$
.a
oracle       soft    nofile          65536
oracle       hard    nofile          101062
oracle       soft    nproc           16384
oracle       hard    nproc           101062
.
w
EOF

#kernel params
/bin/cat < /etc/sysctl.conf
net.ipv4.ip_forward = 0
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.default.accept_source_route = 0
kernel.sysrq = 0
kernel.core_uses_pid = 1
net.ipv4.ip_local_port_range = 9000 65500
net.ipv4.tcp_syncookies = 1
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.all.rp_filter = 1
net.ipv4.icmp_echo_ignore_broadcasts = 1
net.ipv4.icmp_ignore_bogus_error_responses = 1
net.ipv4.tcp_fin_timeout = 30
net.ipv4.tcp_keepalive_time = 1800
net.ipv4.tcp_sack = 0
net.ipv4.tcp_timestamps = 0
net.ipv4.tcp_window_scaling = 0
fs.file-max = 6815744
kernel.shmall = 10523004
kernel.shmmax = 6465333657
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
net.core.rmem_default=262144
net.core.wmem_default=262144
net.core.rmem_max=4194304
net.core.wmem_max=1048576
fs.aio-max-nr = 1048576
EOF_sysctl

/bin/chown root:root /etc/sysctl.conf
/bin/chmod 0600 /etc/sysctl.conf
/sbin/sysctl -p

# create group
/usr/sbin/groupadd -g 306 dba
/bin/mkdir --mode=0750 /home/dba
chgrp dba /home/dba
# temp password - oracl3pass
/usr/sbin/useradd -c 'oracle user' -d /home/dba/oracle -u 306 -p '$1$diNEr/$gu0FHftsYmIm/T6PT9yIN/' -g dba -G dba   oracle


# oracle user .bash_profile
cat > /home/dba/oracle/.bash_profile << EOF
# .bash_profile

# Get the aliases and functions
if [ -f ~/.bashrc ]; then
      . ~/.bashrc
fi

# User specific environment and startup programs
export ORACLE_BASE=/apps/oracle
export ORACLE_HOME=\${ORACLE_BASE}/product/11g/
export ORA_CRS_HOME=/apps/crs
export ORACLE_PATH=\${ORACLE_BASE}/common/oracle/sql:.:\${ORACLE_HOME}/rdbms/admin
export CV_JDKHOME=/usr/local/java
# Each RAC node must have a unique ORACLE_SID. (i.e. orcl1, orcl2,...)
export ORACLE_SID=ISITEST
export PATH=.:\${JAVA_HOME}/bin:${PATH}:$HOME/bin:\${ORACLE_HOME}/bin
export PATH=\${PATH}:/usr/bin:/bin:/usr/bin/X11:/usr/local/bin
export PATH=\${PATH}:\${ORACLE_BASE}/common/oracle/bin
export ORACLE_TERM=xterm
export TNS_ADMIN=\${ORACLE_HOME}/network/admin
export ORA_NLS10=\${ORACLE_HOME}/nls/data
export LD_LIBRARY_PATH=\${ORACLE_HOME}/lib
export LD_LIBRARY_PATH=\${LD_LIBRARY_PATH}:\${ORACLE_HOME}/oracm/lib
export LD_LIBRARY_PATH=\${LD_LIBRARY_PATH}:/lib:/usr/lib:/usr/local/lib
export CLASSPATH=\${ORACLE_HOME}/JRE
export CLASSPATH=\${CLASSPATH}:\${ORACLE_HOME}/jlib
export CLASSPATH=\${CLASSPATH}:\${ORACLE_HOME}/rdbms/jlib
export CLASSPATH=\${CLASSPATH}:\${ORACLE_HOME}/network/jlib
export THREADS_FLAG=native
export TEMP=/tmp
export TMPDIR=/tmp
alias ls="ls -FA"
set -o vi
EOF
# end oracle profile
chown -R oracle.dba /apps
%end
} 1>/root/ks-post.log 2>&1
##END of kickstart file


AFTER RHEL6 Install

[root@LX0130 /]# uname -a
Linux LX0130.isap.asia.cib 2.6.32-71.el6.x86_64 #1 SMP Wed Sep 1 01:33:01 EDT 2010 x86_64 x86_64 x86_64 GNU/Linux

[root@LX0130 /]# cat /etc/redhat-release
Red Hat Enterprise Linux Server release 6.0 (Santiago)

[root@LX0130 /]# df -Ph | column -t
Filesystem                Size   Used  Avail  Use%  Mounted   on
/dev/mapper/vg00-lvsys00  1008M  203M  754M   22%   /
tmpfs                     1.9G   0     1.9G   0%    /dev/shm
/dev/cciss/c0d0p1         194M   25M   159M   14%   /boot
/dev/mapper/vg00-lvsys06  1008M  34M   924M   4%    /home
/dev/mapper/vg00-lvsys04  1008M  34M   924M   4%    /opt
/dev/mapper/vg00-lvsys05  2.0G   68M   1.9G   4%    /tmp
/dev/mapper/vg00-lvsys02  2.0G   1.5G  471M   76%   /usr
/dev/mapper/vg00-lvsys03  1008M  92M   866M   10%   /var
/dev/mapper/vg00-lvora00  51G    180M  49G    1%    /apps


the kickstart install sets an oracle user and needed environment for
an 11g installation. my oracle11g sources are hosted on an AIX 5L machine
(UX0027). installation will be done via vnc session.

the usual nfs mounting procedure won't work. it spits out an error:

mount.nfs   UX0027:/export/isapsys/oracle11g /mnt/nfs   -v
mount.nfs: timeout set for Tue Nov 16 14:36:19 2010
mount.nfs: trying text-based options 'vers=4,addr=UX0027,clientaddr=LX013'
mount.nfs: mount(2): Remote I/O error
mount.nfs: Remote I/O error

the fix is to use v3 or v2 option:

[root@LX0130 ~]# mount.nfs   UX0027:/export/isapsys/oracle11g /mnt/nfs -o v3
[root@LX0130 ~]# ls /mnt/nfs
AIX  AIXclient  Linux64  Linuxclient  linux.x64_11gR1_database_1013.zip
[root@LX0130 ~]#

Installation of Oracle 11g R2

in a vnc session, running runInstaller encountered a permission denied
error due to the mount optins of /tmp:

/dev/mapper/vg00-lvsys05 on /tmp type ext4 (rw,noexec,nodev)

to fix (just for the install)

[root@LX0130 ~]# mount -o exec,remount /tmp
[root@LX0130 ~]# mount | grep /tmp
/dev/mapper/vg00-lvsys05 on /tmp type ext4 (rw,nodev)

install then started but encountered another error during a pre-req
check - there are "missing" packages:

libaio-0.3.105
compat-libstdc++-33-3.2.3
libaio-devel-0.3.105
libgcc-3.4.6
libstdc++-3.4.6
unixODBC-2.2.11
unixODBC-devel-2.2.11
pdksh-5.2.14

these packages are old versions. RHEL6 ships with the following:

libaio-0.3.107-10.el6.x86_64
libaio-0.3.107-10.el6.i686
libaio-devel-0.3.107-10.el6.x86_64
libaio-devel-0.3.107-10.el6.i686
compat-libstdc++-33-3.2.3-69.el6.x86_64
compat-libstdc++-33-3.2.3-69.el6.i686
libgcc-4.4.4-13.el6.x86_64
libgcc-4.4.4-13.el6.i686
unixODBC-2.2.14-11.el6.x86_64
unixODBC-devel-2.2.14-11.el6.x86_64

(these packages will be installed by the kickstart file above).

pdksh is no longer part of rhel6. i did install the following:

ksh-20100621-2.el6.x86_64
mksh-39-4.el6.x86_64

So i chose "Ignore All" and continued the installation. it finished with
no additional complaints.




[oracle@LX0130 ~]$ env | grep ORA
ORA_CRS_HOME=/apps/crs
ORACLE_PATH=/apps/oracle/common/oracle/sql:.:/apps/oracle/product/11g//rdbms/admin
ORA_NLS10=/apps/oracle/product/11g//nls/data
ORACLE_SID=ISITEST
ORACLE_BASE=/apps/oracle
ORACLE_TERM=xterm
ORACLE_HOME=/apps/oracle/product/11g/
[oracle@LX0130 ~]$ which sqlplus
/apps/oracle/product/11g/bin/sqlplus
[oracle@LX0130 ~]$ sqlplus -v

SQL*Plus: Release 11.2.0.1.0 Production

[oracle@LX0130 ~]$

hth.

update (in reference of comments below from Bob):

i did a re-install. this time with a database created. it proceeded with no problems. haven't tried doing it manually AFTER oracle SW only installation.





4 comments:

  1. Thanks for the information, very useful.

    Did you manage to create a database though? It just hangs at 26% for me. Similar with a listener, I can't close the netca gui afterwards. odd.

    ReplyDelete
  2. Thanks Bob for the info. I first received the error Error in CreateOUIProcess(): 13 : Permission denied
    when executing the ./runInstaller .Then I took your advise for remounting /tmp: mount -o exec,remount /tmp
    , but then the processed just died and I received no error message what so ever. Any advise?
    SFF

    ReplyDelete
  3. Thanks for the note about pdksh. I waslooking for this, and knew that it was replaced with mksh, but had to convince the DBA of this ;)

    ReplyDelete
  4. Thanks for the pdksh tip!

    ReplyDelete