When you have many development servers or cloud servers which runs Oracle software and then you know that configuration management of those servers is very important and must be repeatable. When you configure and install all that software by hand, then you also know that the chance is high that those servers are not exactly the same. This can lead to many configuration issues, tons of documentation pages and days for a new environment is released.
With Puppet of puppetlabs you can automate installations and configuration of all these servers and check the status or read the configuration of the servers. Puppet is open source and it works perfectly with VMware or cloud providers like amazon. There is also Puppet Enterprise which is free up to 10 nodes.
If you want to learn Puppet you can follow this guide and download the puppet test VirtualBox or VMware image.
On github I made a puppet wls module which can retrieve all the Oracle WebLogic software, domains + weblogic artifacts, running nodemanager + weblogic servers data and displays these values as facts in the Puppet Dashboard.
Second part of the puppet module are scripts to automatically install the following software.
- Oracle WebLogic 11g (10.3.x ) and WebLogic 12c
- Oracle Oracle Service Bus 11g ( OSB 10.3.x )
- OEPE Oracle Enterprise Pack for Eclipse
- BSU patch for WebLogic
Third part is the configuration of WebLogic domains like
- WebLogic Domain or with OSB
- Configure and start NodeManager
- Startup WebLogic Servers
- Create File Persistence
- Create JDBC Persistence
- Create JMS Server
- Create JMS Module
- Create JMS SubDeployment
- Create JMS Connection Factory
- Create JMS ( Distributed ) Queue or Topic
- Create SAF agents
- Create SAF Remote Destinations
- Create SAF Imported Destinations
- Create SAF Objects
- Create Foreign Server
- Create Foreign Server entries
the WLST module ( define) is very flexible and can be used to add your own WebLogic scripts. ( see the last part of this blogpost )
Let’s start how the Oracle Middleware facts looks like in the Puppet DashBoard. Each puppet node send its run reports + facts to the puppet master and are displayed in the dashboard.
Here you have an overview of all the servers.
With the overall times of the runs and the classes ( a configuration set ) installed on this server
On the same page, the Oracle WebLogic facts are displayed. For this I read the oraInst.loc + oracle inventory for all the installed Oracle software. The bea homelist for all the middleware homes and config.xml and sub files of the WebLogic Domains. Plus as an extra I monitor the running nodemanager and weblogic processes.
Click here to see the Facter code
I will use these facts to know if the software is already installed. Else each run will take 5 min , now only 20 seconds ( in a vmware image ) .
A new OSB installation on a clean server with a fully configured and running OSB Domain takes me on VMware 4 minutes.
Next step is to take a look how we can define a class and add this class to a server.
I start with a site.pp which imports the nodes and class templates.
# site.pp # import "templates" import "nodes" $defaultJDK = '7u7' $defaultFullJDK = 'jdk1.7.0_07'
Nodes.pp which all my servers and environment settings
# nodes.pp
#
node default {
}
node basenode {
}
node devel inherits basenode {
$my_zone = "devel"
include wls1036osb
}
node prod inherits basenode {
$my_zone = "prod"
}
node 'devagent1.alfa.local' inherits devel {
}
node 'win7user.localdomain' inherits prod {
include wls12
}
node 'hudson.alfa.local' {
include hudson
}
The templates.pp with only the hudson.class, to see it all you can take a look at this link
# templates.pp
#
include jdk7
include wls
class hudson{
jdk7::install7{'jdk7_husdon':
version => $defaultJDK,
x64 => "true",
}
$osMdwHome = "/opt/oracle/wls/wls11g"
$osWlHome = "/opt/oracle/wls/wls11g/wlserver_10.3"
$osTemplate = "osb"
$osDomainPath = "/opt/oracle/wls/wls11g/admin"
$user = "oracle"
$group = "dba"
$oepeFile = "oepe-indigo-all-in-one-11.1.1.8.0.201110211138-linux-gtk-x86_64.zip"
# set the defaults
Wls::Installwls {
version => '1036',
versionJdk => $defaultJDK,
user => $user,
group => $group,
}
Wls::Installosb {
mdwHome => $osMdwHome,
wlHome => $osWlHome,
fullJDKName => $defaultFullJDK,
user => $user,
group => $group,
}
# install
wls::installwls{'11gPS5_hudson':
require => Jdk7::Install7['jdk7_husdon'],
}
# download oepe to hudson server
if ! defined(File["/install/${oepeFile}"]) {
file { "/install/${oepeFile}":
source => "puppet:///modules/wls/${oepeFile}",
require => Wls::Installwls['11gPS5_hudson'],
ensure => present,
mode => 0775,
}
}
# extract oepe in middleware home
if ! defined(Exec["extract ${oepeFile}"]) {
exec { "extract ${oepeFile}":
command => "unzip -n /install/${oepeFile} -d ${osMdwHome}/oepe11.1.1.8",
require => File["/install/${oepeFile}"],
creates => "${osMdwHome}/oepe11.1.1.8",
path => "/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:",
user => $user,
group => $group,
logoutput => true,
}
}
#install OSB with OEPE
wls::installosb{'osbPS5_oepe':
osbFile => 'ofm_osb_generic_11.1.1.6.0_disk1_1of1.zip',
oepeHome => 'oepe11.1.1.8',
require => Exec["extract ${oepeFile}"],
}
}
To make this work you need to download and add the oracle software, under the files folder. The puppet scripts(defines) will download these files from the puppet master. I won’t add this software to this module.
for example I added the following software wls1036_generic.jar , ofm_osb_generic_11.1.1.6.0_disk1_1of1.zip , oepe. bsu patches.
Besides the oracle software we also need templates files ( located at the template folder ) for the WebLogic configuration or silent installations. These files are not in the files folder because I added some placeholders to these files and these are changed at runtime.
The lib/facter folder contains the files for facter and can be read in the puppet dashboard or by the manifests and the functions scripts.
The lib/puppet/parser/functions folder contains the files which can read the facts and are used by the manifests files
The manifests folder contains the scripts which do all the work.
At last we take a look at the definitions in the templates.pp and the manifests files.
First I will show you how you can create an OSB domain with this puppet module,
We define the global variables for the right operating systems, The default for the Installosb define and at last execute the installosb task.
# set the general defaults
case $operatingsystem {
centos, redhat, OracleLinux, ubuntu, debian: {
$osMdwHome = "/opt/oracle/wls/wls11g"
$osWlHome = "/opt/oracle/wls/wls11g/wlserver_10.3"
$osTemplate = "osb"
$osDomainPath = "/opt/oracle/wls/wls11g/admin"
$user = "oracle"
$group = "dba"
}
windows: {
$osMdwHome = "c:/opt/oracle/wls/wls11g"
$osWlHome = "c:/oracle/wls/wls11g/wlserver_10.3"
$osTemplate = "osb"
$osDomainPath = "c:/oracle/wls/wls11g/admin"
$user = "Administrator"
$group = "Administrators"
$serviceName = "C_oracle_wls_wls11g_wlserver_10.3"
}
}
# set the default for Installosb
Wls::Installosb {
mdwHome => $osMdwHome,
wlHome => $osWlHome,
fullJDKName => $defaultFullJDK,
user => $user,
group => $group,
}
# install OSB domain
wls::wlsdomain{
'osbDomain':
wlsTemplate => $osTemplate,
domain => 'osbDomain',
domainPath => $osDomainPath,
adminListenPort => '9001',
nodemanagerPort => '5556',
require => Wls::Nodemanager['nodemanager11g'];
}
Next we can add a DataSource to the WebLogic domain, First we set the defaults for all the wlstexec executes
# default parameters for the wlst scripts
Wls::Wlstexec {
wlsDomain => "${osDomainPath}/osbDomain",
wlHome => $osWlHome,
fullJDKName => $defaultFullJDK,
user => $user,
group => $group,
address => "localhost",
wlsUser => "weblogic",
password => "weblogic1",
}
# create jdbc datasource for osb_server1
wls::wlstexec {
'createJdbcDatasourceHr':
wlstype => "jdbc",
wlsObjectName => "hrDS",
script => 'createJdbcDatasource.py',
port => '9001',
params => ["dsName = 'hrDS'",
"jdbcDatasourceTargets = 'AdminServer,osb_server1'",
"dsJNDIName = 'jdbc/hrDS'",
"dsDriverName = 'oracle.jdbc.xa.client.OracleXADataSource'",
"dsURL = 'jdbc:oracle:thin:@master.alfa.local:1521/XE'",
"dsUserName = 'hr'",
"dsPassword = 'hr'",
"datasourceTargetType = 'Server'",
"globalTransactionsProtocol = 'xxxx'"
],
require => Wls::Wlstexec['startOSBAdminServer'];
}
The WLST template for creating a JDBC datasource, the params variable entries will be injected into this template script.
wlsUser = '<%= @wlsUser %>'
password = '<%= @password %>'
machine = '<%= @address %>'
portNumber = '<%= @port %>'
#dsName, jdbcDatasourceTargets, dsJNDIName,dsDriverName,dsURL,dsUserName,dsPassword,datasourceTargetType,globalTransactionsProtocol
<% params.each do |param| -%>
<%= param %>
<% end -%>
connect(wlsUser,password,'t3://'+machine+':'+portNumber)
print "datasource " +dsName
datasourceTargets=String(jdbcDatasourceTargets).split(",")
for datasourceTarget in datasourceTargets:
print "Datasource Target",datasourceTarget
print "dsJNDIName " +dsJNDIName
print "dsDriverName " +dsDriverName
print "dsUserName " +dsUserName
edit()
startEdit()
try:
cd('/')
cmo.createJDBCSystemResource(dsName)
cd('/JDBCSystemResources/' + dsName + '/JDBCResource/' + dsName)
cmo.setName(dsName)
cd('/JDBCSystemResources/' + dsName + '/JDBCResource/' + dsName + '/JDBCDataSourceParams/' + dsName )
set('JNDINames',jarray.array([String('jdbc/' + dsName )], String))
cd('/JDBCSystemResources/' + dsName + '/JDBCResource/' + dsName + '/JDBCDriverParams/' + dsName )
cmo.setUrl(dsURL)
cmo.setDriverName( dsDriverName )
cmo.setPassword(dsPassword)
cd('/JDBCSystemResources/' + dsName + '/JDBCResource/' + dsName + '/JDBCConnectionPoolParams/' + dsName )
cmo.setTestTableName('SQL SELECT 1 FROM DUAL')
cd('/JDBCSystemResources/' + dsName + '/JDBCResource/' + dsName + '/JDBCDriverParams/' + dsName + '/Properties/' + dsName )
cmo.createProperty('user')
cd('/JDBCSystemResources/' + dsName + '/JDBCResource/' + dsName + '/JDBCDriverParams/' + dsName + '/Properties/' + dsName + '/Properties/user')
cmo.setValue(dsUserName)
if globalTransactionsProtocol != "xxxx":
cd('/JDBCSystemResources/' + dsName + '/JDBCResource/' + dsName + '/JDBCDataSourceParams/' + dsName)
cmo.setGlobalTransactionsProtocol('None')
cd('/SystemResources/' + dsName )
targetList = []
for datasourceTarget in datasourceTargets:
targetList.append( ObjectName('com.bea:Name=' + datasourceTarget + ',Type='+datasourceTargetType) )
set('Targets',jarray.array(targetList, ObjectName))
save()
activate()
except:
print "Unexpected error:", sys.exc_info()[0]
undo('true','y')
stopEdit('y')
And the WLST define manifest which does all the work, It first checks it the datasource is already defined on the WebLogic Domain.
define wls::wlstexec ($wlsDomain = undef,
$wlstype = undef,
$wlsObjectName = undef,
$wlHome = undef,
$fullJDKName = undef,
$script = undef,
$address = "localhost",
$port = '7001',
$wlsUser = "weblogic",
$password = "weblogic1",
$user = 'oracle',
$group = 'dba',
$params = undef,
) {
# if these params are empty always continue
if $wlsDomain == undef or $wlstype == undef or wlsObjectName == undef {
$continue = true
} else {
# check if the object already exists on the weblogic domain
$found = artifact_exists($wlsDomain ,$wlstype,$wlsObjectName )
if $found == undef {
$continue = true
} else {
if ( $found ) {
notify {"wls::wlstexec ${title} already exists":}
$continue = false
} else {
$continue = true
}
}
}
if ( $continue ) {
$javaCommand = "java weblogic.WLST"
case $operatingsystem {
centos, redhat, OracleLinux, ubuntu, debian: {
$execPath = "/usr/java/${fullJDKName}/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:"
$path = '/install/'
$JAVA_HOME = "/usr/java/${fullJDKName}"
Exec { path => $execPath,
user => $user,
group => $group,
logoutput => true,
}
File {
ensure => present,
replace => 'yes',
mode => 0555,
owner => $user,
group => $group,
}
}
windows: {
$execPath = "C:\\oracle\\${fullJDKName}\\bin;C:\\unxutils\\bin;C:\\unxutils\\usr\\local\\wbin;C:\\Windows\\system32;C:\\Windows"
$path = "c:/temp/"
$JAVA_HOME = "c:\\oracle\\${fullJDKName}"
Exec { path => $execPath,
logoutput => true,
}
File { ensure => present,
replace => 'yes',
mode => 0555,
}
}
}
# the py script used by the wlst
# if ! defined(File["${path}${title}${script}"]) {
file { "${path}${title}${script}":
path => "${path}${title}${script}",
content => template("wls/${script}.erb"),
}
# }
case $operatingsystem {
centos, redhat, OracleLinux, ubuntu, debian: {
exec { "execwlst ${title}${script}":
command => "${javaCommand} ${path}${title}${script}",
environment => ["CLASSPATH=${wlHome}/server/lib/weblogic.jar",
"JAVA_HOME=${JAVA_HOME}",
"CONFIG_JVM_ARGS=-Djava.security.egd=file:/dev/./urandom"],
require => File["${path}${title}${script}"],
}
# if ! defined(Exec["rm ${path}${title}${script}"]) {
exec { "rm ${path}${title}${script}":
command => "rm -I ${path}${title}${script}",
require => Exec["execwlst ${title}${script}"],
}
# }
}
windows: {
exec { "execwlst ${title}${script}":
command => "C:\\Windows\\System32\\cmd.exe /c ${javaCommand} ${path}${title}${script}",
environment => ["CLASSPATH=${wlHome}\\server\\lib\\weblogic.jar",
"JAVA_HOME=${JAVA_HOME}"],
require => File["${path}${title}${script}"],
}
if ! defined(Exec["rm ${path}${title}${script}"]) {
exec { "rm ${path}${id}${script}":
command => "C:\\Windows\\System32\\cmd.exe /c delete ${path}${title}${script}",
require => Exec["execwlst ${title}${script}"],
}
}
}
}
}
}
All this can be hard in the beginning ( you must learn some ruby and know your way in puppet ) but when you can configure servers in just 5 minutes and the task can be repeated many times then you get a very great ROI.
Next step is retrieving software like EARs and OSB projects from Nexus and deploy this to the servers and make it fully Windows compatible .
