Deploying an Oracle WebLogic Domain on a Kubernetes cluster using Oracle WebLogic Server Kubernetes Operator

At the Oracle Partner PaaS Summer Camp IX 2019 in Lisbon, held at the end of August, I followed a 5 day during workshop called “Modern Application Development with Oracle Cloud”. In this workshop, on day 4, the topic was “WebLogic on Kubernetes”.
_{[https://paascommunity.com/2019/09/02/oracle-paas-summer-camp-2019-results-become-a-trained-certified-oracle-cloud-platform-expert/]}

At the Summer Camp we used a free Oracle Cloud trial account.

On day 4, I did a hands-on lab in which an Oracle WebLogic Domain was deployed on an Oracle Container Engine for Kubernetes (OKE) cluster using Oracle WebLogic Server Kubernetes Operator.

In this article, I will describe the steps that I went through to get an Oracle WebLogic Domain running on a three-node Kubernetes cluster instance (provisioned by OKE) on Oracle Cloud Infrastructure (OCI) in an existing OCI tenancy.

Oracle Container Engine for Kubernetes (OKE)

Oracle Container Engine for Kubernetes (OKE) is a fully-managed, scalable, and highly available service that you can use to deploy your containerized applications to the cloud. Use OKE when your development team wants to reliably build, deploy, and manage cloud-native applications. You specify the compute resources that your applications require, and OKE provisions them on Oracle Cloud Infrastructure (OCI) in an existing OCI tenancy.

Container Engine for Kubernetes uses Kubernetes – the open-source system for automating deployment, scaling, and management of containerized applications across clusters of hosts. Kubernetes groups the containers that make up an application into logical units (called pods) for easy management and discovery. Container Engine for Kubernetes uses versions of Kubernetes certified as conformant by the Cloud Native Computing Foundation (CNCF).

You can access Container Engine for Kubernetes to define and create Kubernetes clusters using the Console and the REST API. You can access the clusters you create using the Kubernetes command line (kubectl), the Kubernetes Dashboard, and the Kubernetes API.

Container Engine for Kubernetes is integrated with Oracle Cloud Infrastructure Identity and Access Management (IAM), which provides easy authentication with native Oracle Cloud Infrastructure identity functionality.
_{[https://docs.cloud.oracle.com/iaas/Content/ContEng/Concepts/contengoverview.htm]}

Oracle Cloud Infrastructure (OCI)

Oracle Cloud Infrastructure is a set of complementary cloud services that enable you to build and run a wide range of applications and services in a highly available hosted environment. Oracle Cloud Infrastructure offers high-performance compute capabilities (as physical hardware instances) and storage capacity in a flexible overlay virtual network that is securely accessible from your on-premises network.
_{[https://docs.cloud.oracle.com/iaas/Content/GSG/Concepts/baremetalintro.htm]}

In order to get an Oracle WebLogic Domain running on a Kubernetes cluster instance (provisioned by OKE) on Oracle Cloud Infrastructure (OCI) in an existing OCI tenancy, a number of steps have to be taken.

Also, some tools (like kubectl) will be used. At the Summer Camp our instructors provided us with a VirtualBox appliance for this.

For OKE cluster creation, I used the Quick Create feature, which uses default settings to create a quick cluster with new network resources as required. This approach is the fastest way to create a new cluster. If you accept all the default values, you can create a new cluster in just a few clicks. New network resources for the cluster are created automatically, along with a node pool and three worker nodes. All the nodes will be deployed in different Availability Domains to ensure high availability.

These are the steps that I took:

• Creating a Policy (allow service OKE to manage all-resources in tenancy)
• Creating an OKE (Oracle Container Engine for Kubernetes) cluster, including network resources (VCN, Subnets, Security lists, etc.) and a Node Pool (with three nodes)
• Configuring OCI CLI
• Creating an OCI CLI config file using a setup dialog
• Uploading the public key of the API signing key pair
• Setting up the RBAC policy for the OKE cluster
• Installing and configuring Oracle WebLogic Server Kubernetes Operator
• Installing the “operator” with a Helm chart
• Installing and configuring Traefik with a Helm chart
• Deploying a WebLogic domain
• Opening the Oracle WebLogic Server Administration Console
• Testing the demo web application
• Opening the Kubernetes Web UI (Dashboard)

In the overview below you can see a number of these steps:

I logged in to my Oracle Cloud environment.

Then the Infrastructure Classic Console page is opened were you can check the overall status of your purchased services and manage your accounts or subscriptions.

I clicked on “Compute”.

In the page “Service: Oracle Cloud Infrastructure”, I clicked on button “Open Service Console”.

In the OCI console, the Oracle Cloud Infrastructure Compute page was opened, which lets you provision and manage compute hosts, known as instances.

Creating a Policy

In the OCI console, I opened the navigation menu and clicked on Governance and Administration | Identity | Policies.

There I selected in the left-hand side menu the “root” compartment for my account, and I clicked on button “Create Policy”.

I entered the following values:

Name A unique name for the policy. The name must be unique across all policies in your tenancy. You cannot change this later.	WebLogick8sPolicy
Description A friendly description.	Allow OKE to manage resources
Policy Versioning	KEEP POLICY CURRENT This ensures that the policy stays current with any future changes to the service’s definitions of verbs and resources.
Statement	allow service OKE to manage all-resources in tenancy
Tags	– I didn’t apply tags.

I clicked on button “Create”.

Creating an OKE (Oracle Container Engine for Kubernetes) cluster

In the OCI console, I opened the navigation menu and clicked on Solutions and Platform | Developer Services | Container Clusters (OKE).

On the Cluster List page, I clicked on button “Create Cluster”.

I entered the following values:

Name The name of the new cluster.	clusterAMISMLA1
Kubernetes Version The Kubernetes version that runs on the master nodes and worker nodes of the cluster.	v1.13.5 I selected the latest version.
Quick or Custom Create	QUICK CREATE This creates a new cluster with default settings, along with new network resources for the new cluster. The Create Virtual Cloud Network panel shows the network resources that will be created for you by default, namely a VCN, two load balancer subnets, and three worker node subnets.
Create Virtual Cloud Network:
Compartment The compartment in which the network resources will be created.	marclameriks1(root)
Resource Creation	1 VCN, 1 service lb subnet and 1 worker node subset
Private or Public	PRIVATE The Kubernetes worker nodes that are created will be hosted in private subnet(s).
Create Node Pool: The Create Node Pool panel shows the fixed properties of the first node pool in the cluster that will be created for you. The Create Node Pool panel also contains some node pool properties that you can change.
Name The name of the node pool.	pool1
Compartment The compartment in which the node pool will be created (always the same as the one in which the new network resources will reside).	amismarclameriks1(root)
Version The version of Kubernetes that will run on each worker node in the node pool (always the same as the version specified for the master nodes).	V1.13.5
Image The image to use on each node in the node pool.	Oracle-Linux-7.6
Shape The shape to use for each node in the node pool. The shape determines the number of CPUs and the amount of memory allocated to each node. The list shows only those shapes available in your tenancy that are supported by Container Engine for Kubernetes.	VM.Standard2.1
Number of nodes The number of nodes in the node pool.	3
Public SSH Key	– I left this field empty.
Kubernetes Labels	Key: name Value: pool1
Additional Add Ons:
Kubernetes Dashboard	Enabled
Tiller (Helm)	Enabled

I clicked on button “Create”, to create the new network resources and the new cluster.

I clicked on the link “clusterAMISMLA1” to show the cluster details.

So, by selecting the Quick Create, the new cluster (clusterAMISMLA1) with default settings was created, along with new network resources (namely a VCN, two load balancer subnets, and three worker node subnets).

Configuring OCI CLI

Before the OCI CLI can be used, some necessary information has to be collected using the OCI console:

• User OCID

In the OCI console I clicked on my OCI user name and selected ‘User Settings”.

On the User Details page, under tab “User Information”, you can find the User OCID. I clicked on link “Copy” and pasted it temporary to a text editor.

OCID: ocid1.user.oc1..aaaaaaaa111…111q

• Tenancy OCID

In the OCI console, I opened the navigation menu and clicked on Governance and Administration | Administration | Tenancy Details.

On the Tenancy Details page, under tab “Tenancy Information”, you can find the Tenancy OCID. I clicked on link “Copy” and pasted it temporary to a text editor.

OCID: ocid1.tenancy.oc1..aaaaaaaa111…111q

• Region

On the Tenancy Details page, under “Regions” you can find the current Region Identifier (in my case: eu-frankfurt-1).

Command Line Interface (CLI)

The CLI is a tool that lets you work with most of the available services in Oracle Cloud Infrastructure. The CLI provides the same core functionality as the Console, plus additional commands. The CLI’s functionality and command help are based on the service’s API.
_{[https://docs.cloud.oracle.com/iaas/Content/GSG/Tasks/gettingstartedwiththeCLI.htm]}

If you need to install OCI CLI then follow the documentation:
https://docs.cloud.oracle.com/iaas/Content/API/SDKDocs/cliinstall.htm

Before using the CLI, you must create a config file that contains the required credentials for working with Oracle Cloud Infrastructure. You can create this file using a setup dialog or manually using a text editor.
_{[https://docs.cloud.oracle.com/iaas/Content/API/SDKDocs/cliinstall.htm]}

Creating an OCI CLI config file using a setup dialog

To have the CLI walk me through the first-time setup process, I used the oci setup config command. The command prompts you for the information required for the config file and the API public/private keys. The setup dialog generates an API key pair and creates the config file.

At the Summer Camp, our instructors provided us with a VirtualBox appliance, that contained the necessary tools.

I opened a Terminal and typed the following command to create a valid OCI CLI config file. I provided the information asked by the wizard (including a new passphrase for my private key):

oci setup config

With the following output:

This command provides a walkthrough of creating a valid CLI config file.

The following links explain where to find the information required by this
script:

User OCID and Tenancy OCID:

https://docs.cloud.oracle.com/Content/API/Concepts/apisigningkey.htm#Other

Region:

https://docs.cloud.oracle.com/Content/General/Concepts/regions.htm

General config documentation:

https://docs.cloud.oracle.com/Content/API/Concepts/sdkconfig.htm

Enter a location for your config [/home/oracle/.oci/config]:
Enter a user OCID: ocid1.user.oc1..aaaaaaaa111…111q
Enter a tenancy OCID: ocid1.tenancy.oc1..aaaaaaaa111…111q
Enter a region (e.g. ap-mumbai-1, ap-seoul-1, ap-tokyo-1, ca-toronto-1, eu-frankfurt-1, eu-zurich-1, sa-saopaulo-1, uk-london-1, us-ashburn-1, us-gov-ashburn-1, us-gov-chicago-1, us-gov-phoenix-1, us-langley-1, us-luke-1, us-phoenix-1): eu-frankfurt-1
Do you want to generate a new RSA key pair? (If you decline you will be asked to supply the path to an existing key.) [Y/n]: Y
Enter a directory for your keys to be created [/home/oracle/.oci]:
Enter a name for your key [oci_api_key]:
Public key written to: /home/oracle/.oci/oci_api_key_public.pem
Enter a passphrase for your private key (empty for no passphrase):
Private key written to: /home/oracle/.oci/oci_api_key.pem
Fingerprint: 1c:7b:111..111:a1
Config written to /home/oracle/.oci/config

If you haven’t already uploaded your public key through the console,
follow the instructions on the page linked below in the section ‘How to
upload the public key’:

https://docs.cloud.oracle.com/Content/API/Concepts/apisigningkey.htm#How2

I checked the files in directory /home/oracle/.oci:

• config

[DEFAULT]
user=ocid1.user.oc1..aaaaaaaa111…111q
fingerprint=1c:7b:111..111:a1
key_file=/home/oracle/.oci/oci_api_key.pem
tenancy=ocid1.tenancy.oc1..aaaaaaaa111…111q
region=eu-frankfurt-1

• oci_api_key.pem

—–BEGIN RSA PRIVATE KEY—–
MIIEpAIBAAKCAQEAxfonSva57bhbf+l8WWkHjcwOXnqpUF11bN+zW+uid/N6shHu
…
pcQuX0PgnyZVy8UEp13+zMz3nflbZRxdp/KcJj+OmzG4aoGEeDFxCQ==
—–END RSA PRIVATE KEY—–

• oci_api_key_public.pem

—–BEGIN PUBLIC KEY—–
MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAxfonSva57bhbf+l8WWkH
jcwOXnqpUF11bN+zW+uid/N6shHunleyss2I6JHsLnTqqdLIgXTTL2NpJd2b1vIj
iI/XJ9WacEq5GZB6FDDUpIOge15kVU/f1jPkJv1xiWQNi/SgllI49GggrRdKlf2a
6VI4e8jbBN+Ej7uA83FQ1PgC0GU9H0NX7H1z2e3xcRiXca0hBfnXUh0yobzKyb/d
sL0+6xsdPJoLP0HP3nIXGuml6oASaDY4Q4VKBkgag8Pmre9q+UKYt8UOcPxx3W7F
Ug3uAjQRQBtqYPu0mlezCcfnmq3h/ILRgHd7KAJZH+6hzJqhcrjry8+fn0PD7IC+
0wIDAQAB
—–END PUBLIC KEY—–

Uploading the public key of the API signing key pair

The final step to complete the OCI CLI setup is to upload the freshly generated public key through the console.

In the OCI console I clicked on my OCI user name and selected “User Settings”. On the User Details page, under tab “User Information”, I clicked on button “Add Public Key” and copied the content of the oci_api_key_public.pem file into the PUBLIC KEY text area. Then I clicked on button “Add”.

The key was uploaded and its fingerprint is displayed in the list.

In the OCI console, I opened the navigation menu and clicked on Solutions and Platform | Developer Services | Container Clusters (OKE). I clicked on the link “clusterAMISMLA1”.

Next, I clicked on button “Access Kubeconfig”. A dialog popped up which contained the customized OCI command that I needed to execute to create a Kubernetes configuration file.

In the VirtualBox Appliance, I opened a Terminal and typed the following commands, to create a kubeconfig file for my cluster:

mkdir -p $HOME/.kube

oci ce cluster create-kubeconfig --cluster-id ocid1.cluster.oc1.eu-frankfurt-1.aaaaaaaaa111...111g --file $HOME/.kube/config --region eu-frankfurt-1

With the following output:

New config written to the Kubeconfig file /home/oracle/.kube/config

I checked the files in directory /home/oracle/.kube:

• config

apiVersion: v1
clusters:
– cluster:
certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURqRENDQW5TZ0F3SUJBZ0lVTC9ueDVGeHJqYXpHSHk1NlV0VS9JK2…
TkQgQ0VSVElGSUNBVEUtLS0tLQo=
server: https://csdsylemnrg.eu-frankfurt-1.clusters.oci.oraclecloud.com:6443
name: cluster-csdsylemnrg
contexts:
– context:
cluster: cluster-csdsylemnrg
user: user-csdsylemnrg
name: context-csdsylemnrg
current-context: context-csdsylemnrg
kind: “”
users:
– name: user-csdsylemnrg
user:
token: eyJoZWFkZXIiOnsiQXV0aG9yaXphdGlvbiI6WyJTaWduYXR1cmUgYWxnb3JpdGhtPVwicnNhLXNoYTI1NlwiLGhlYWRlcn…
aW10ZGdlM3Rhb2p3bWNzZHN5bGVtbnJnL2t1YmVjb25maWcvY29udGVudCJ9

Use kubeconfig files to organize information about clusters, users, namespaces, and authentication mechanisms. The kubectl command-line tool uses kubeconfig files to find the information it needs to choose a cluster and communicate with the API server of a cluster.
By default, kubectl looks for a file named config in the $HOME/.kube directory. You can specify other kubeconfig files by setting the KUBECONFIG environment variable or by setting the –kubeconfig flag.
_{[https://kubernetes.io/docs/concepts/configuration/organize-cluster-access-kubeconfig/]}

If you need to install kubectl then follow the documentation:
https://kubernetes.io/docs/tasks/tools/install-kubectl/

In order to check that kubectl was working correctly, I used the get nodes command:

kubectl get nodes

With the following output:

NAME STATUS ROLES AGE VERSION
10.0.10.2 Ready node 25m v1.13.5
10.0.11.2 Ready node 25m v1.13.5
10.0.12.2 Ready node 25m v1.13.5

Setting up the RBAC policy for the OKE cluster

In order to have permission to access the Kubernetes cluster, I needed to authorize my OCI account as a cluster-admin on the OKE cluster.

I used the following command (which required my User OCID), to create a ClusterRoleBinding for a particular ClusterRole:

kubectl create clusterrolebinding my-cluster-admin-binding --clusterrole=cluster-admin --user=ocid1.user.oc1..aaaaaaaa111...111q

With the following output:

clusterrolebinding “my-cluster-admin-binding” created

For more information about Role-based access control (RBAC) and ClusterRoleBinding, see for example:

https://kubernetes.io/docs/reference/access-authn-authz/rbac/#rolebinding-and-clusterrolebinding

Now, my OCI OKE environment was ready to deploy my WebLogic domain.

Installing and configuring Oracle WebLogic Server Kubernetes Operator

An operator is an application-specific controller that extends Kubernetes to create, configure, and manage instances of complex applications. The Oracle WebLogic Server Kubernetes Operator (the “operator”) follows the standard Kubernetes operator pattern and simplifies the management and operation of WebLogic domains and deployments.

You can have one or more operators in your Kubernetes cluster that manage one or more WebLogic domains each. A Helm chart is provided to manage the installation and configuration of the operator.

_{[https://oracle.github.io/weblogic-kubernetes-operator/userguide/introduction/introduction/]}

Installing the “operator” with a Helm chart

First, clone the Oracle WebLogic Server Kubernetes Operator git repository to your desktop.

git clone https://github.com/oracle/weblogic-kubernetes-operator.git -b 2.0

In order to use Helm to install and manage the operator, you need to ensure that the service account that Tiller uses has the cluster-admin role. The default would be default in namespace kube-system.

I used the following command, to give that service account the necessary permissions:

cat << EOF | kubectl apply -f -
> apiVersion: rbac.authorization.k8s.io/v1
> kind: ClusterRoleBinding
> metadata:
>   name: helm-user-cluster-admin-role
> roleRef:
>   apiGroup: rbac.authorization.k8s.io
>   kind: ClusterRole
>   name: cluster-admin
> subjects:
> - kind: ServiceAccount
>   name: default
>   namespace: kube-system
> EOF

With the following output:

clusterrolebinding “helm-user-cluster-admin-role” created

Kubernetes distinguishes between the concept of a user account and a service account for a number of reasons. The main reason is that user accounts are for humans while service accounts are for processes, which run in pods. Oracle WebLogic Server Kubernetes Operator also requires service accounts. If a service account is not specified, it defaults to default (for example, the namespace’s default service account). If you want to use a different service account, then you must create the operator’s namespace and the service account before installing the “operator” Helm chart.

I used the following command, to create the operator’s namespace:

kubectl create namespace sample-weblogic-operator-ns

With the following output:

namespace “sample-weblogic-operator-ns” created

I used the following command, to create the service account:

kubectl create serviceaccount -n sample-weblogic-operator-ns sample-weblogic-operator-sa

With the following output:

serviceaccount “sample-weblogic-operator-sa” created

I used the following commands, to install the “operator” Helm chart:

cd /u01/content/weblogic-kubernetes-operator/

helm install kubernetes/charts/weblogic-operator \
>   --name sample-weblogic-operator \
>   --namespace sample-weblogic-operator-ns \
>   --set image=oracle/weblogic-kubernetes-operator:2.0 \
>   --set serviceAccount=sample-weblogic-operator-sa \
>   --set "domainNamespaces={}"

Remark about the values:

• name: name of the resource
• namespace: where the operator gets deployed
• image: the prebuilt WebLogic Operator 2.0 image. Available on public Docker hub.
• serviceAccount: service account required to run “operator”
• domainNamespaces: namespaces where WebLogic domains get deployed in order to control. Note WebLogic domain is not yet deployed so this value will be updated when namespaces created for WebLogic deployment.

With the following output:

NAME: sample-weblogic-operator
LAST DEPLOYED: Thu Aug 29 07:20:02 2019
NAMESPACE: sample-weblogic-operator-ns
STATUS: DEPLOYED

RESOURCES:
==> v1/ClusterRoleBinding
NAME AGE
sample-weblogic-operator-ns-weblogic-operator-clusterrolebinding-nonresource 1s
sample-weblogic-operator-ns-weblogic-operator-clusterrolebinding-discovery 1s
sample-weblogic-operator-ns-weblogic-operator-clusterrolebinding-auth-delegator 1s
sample-weblogic-operator-ns-weblogic-operator-clusterrolebinding-general 1s

==> v1beta1/Deployment
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
weblogic-operator 1 1 1 0 1s

==> v1/Pod(related)
NAME READY STATUS RESTARTS AGE
weblogic-operator-848cd6c4cb-rpf9v 0/1 ContainerCreating 0 1s

==> v1/Secret
NAME TYPE DATA AGE
weblogic-operator-secrets Opaque 0 2s

==> v1/ConfigMap
NAME DATA AGE
weblogic-operator-cm 2 2s

==> v1/RoleBinding
NAME AGE
weblogic-operator-rolebinding-namespace 1s
weblogic-operator-rolebinding 1s

==> v1/Service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
internal-weblogic-operator-svc ClusterIP 10.96.61.254 8082/TCP 1s

==> v1/ClusterRole
NAME AGE
sample-weblogic-operator-ns-weblogic-operator-clusterrole-nonresource 1s
sample-weblogic-operator-ns-weblogic-operator-clusterrole-namespace 1s
sample-weblogic-operator-ns-weblogic-operator-clusterrole-operator-admin 1s
sample-weblogic-operator-ns-weblogic-operator-clusterrole-domain-admin 1s
sample-weblogic-operator-ns-weblogic-operator-clusterrole-general 1s

==> v1/Role
weblogic-operator-role 1s

I used the following command, to list the Pods:

kubectl get pods -n sample-weblogic-operator-ns

With the following output:

NAME READY STATUS RESTARTS AGE
weblogic-operator-848cd6c4cb-rpf9v 1/1 Running 0 1m

I used the following command, to list the “operator” release:

helm list sample-weblogic-operator

With the following output:

NAME REVISION UPDATED STATUS CHART NAMESPACE
sample-weblogic-operator 1 Thu Aug 29 07:20:02 2019 DEPLOYED weblogic-operator-2 sample-weblogic-operator-ns

So, now the Oracle WebLogic Server Kubernetes Operator had been installed.

Installing and configuring Traefik with a Helm chart

The Oracle WebLogic Server Kubernetes Operator supports three load balancers: Traefik, Voyager, and Apache.

I installed Traefik to provide load balancing for my WebLogic cluster.
_{[https://docs.traefik.io/]}

I used the following commands, to install the Traefik Helm chart:

cd /u01/content/weblogic-kubernetes-operator/

helm install stable/traefik \
> --name traefik-operator \
> --namespace traefik \
> --values kubernetes/samples/charts/traefik/values.yaml  \
> --set "kubernetes.namespaces={traefik}" \
> --set "serviceType=LoadBalancer"

With the following output:

NAME: traefik-operator
LAST DEPLOYED: Thu Aug 29 07:23:26 2019
NAMESPACE: traefik
STATUS: DEPLOYED

RESOURCES:
==> v1/ConfigMap
NAME DATA AGE
traefik-operator 1 1s
traefik-operator-test 1 1s

==> v1/ServiceAccount
NAME SECRETS AGE
traefik-operator 1 1s

==> v1/ClusterRoleBinding
NAME AGE
traefik-operator 1s

==> v1beta1/Ingress
NAME HOSTS ADDRESS PORTS AGE
traefik-operator-dashboard traefik.example.com 80 1s

==> v1/Secret
NAME TYPE DATA AGE
traefik-operator-default-cert Opaque 2 2s

==> v1/ClusterRole
NAME AGE
traefik-operator 1s

==> v1/Service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
traefik-operator-dashboard ClusterIP 10.96.46.6 80/TCP 1s
traefik-operator LoadBalancer 10.96.253.105 80:30542/TCP,443:30638/TCP 1s

==> v1/Deployment
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
traefik-operator 1 1 1 0 1s

==> v1/Pod(related)
NAME READY STATUS RESTARTS AGE
traefik-operator-7f9c8ff8b8-cqxcf 0/1 ContainerCreating 0 1s

NOTES:

1. Get Traefik’s load balancer IP/hostname:

NOTE: It may take a few minutes for this to become available.

You can watch the status by running:

$ kubectl get svc traefik-operator –namespace traefik -w

Once ‘EXTERNAL-IP’ is no longer ”:

$ kubectl describe svc traefik-operator –namespace traefik | grep Ingress | awk ‘{print $3}’

2. Configure DNS records corresponding to Kubernetes ingress resources to point to the load balancer IP/hostname found in step 1

I used the following command, to list the Services:

kubectl get services -n traefik

With the following output:

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
traefik-operator LoadBalancer 10.96.253.105 111.11.11.1 80:30542/TCP,443:30638/TCP 35s
traefik-operator-dashboard ClusterIP 10.96.46.6 80/TCP 35s

Remark about the EXTERNAL-IP of the traefik-operator service:
The EXTERNAL-IP is the Public IP address of the load balancer that I will be using to open the Oracle WebLogic Administration Console.

To print only the Public IP address, you can execute the following command:

kubectl describe svc traefik-operator --namespace traefik | grep Ingress | awk '{print $3}'

With the following output:

111.11.11.1

I used the following command, to lists all of the releases:

helm list

With the following output:

NAME REVISION UPDATED STATUS CHART NAMESPACE
sample-weblogic-operator 1 Thu Aug 29 07:20:02 2019 DEPLOYED weblogic-operator-2 sample-weblogic-operator-ns
traefik-operator 1 Thu Aug 29 07:23:26 2019 DEPLOYED traefik-1.77.1 traefik

Finally, I used the following command (using the EXTERNAL-IP address from the result above), to hit the Traefik’s dashboard:

curl -H 'host: traefik.example.com' http://111.11.11.1

With the following output:

<a href=”/dashboard/”>Found</a>.

Deploying a WebLogic domain

For deploying a WebLogic domain some steps have to be taken.

• Preparing the Kubernetes cluster to run WebLogic domains

I used the following command, to create the WebLogic domain namespace:

kubectl create namespace sample-domain1-ns

With the following output:

namespace “sample-domain1-ns” created

I used the following command, to create a Kubernetes secret containing the Administration Server boot credentials:

kubectl -n sample-domain1-ns create secret generic sample-domain1-weblogic-credentials \
>   --from-literal=username=weblogic \
>   --from-literal=password=welcome1

With the following output:

secret “sample-domain1-weblogic-credentials” created

I used the following command, to label the Secret:

kubectl label secret sample-domain1-weblogic-credentials \
>   -n sample-domain1-ns \
>   weblogic.domainUID=sample-domain1 \
>   weblogic.domainName=sample-domain1

The following labels are used:

Label key	Label Value
weblogic.domainUID	sample-domain1
weblogic.domainName	sample-domain1

With the following output:

secret “sample-domain1-weblogic-credentials” labeled

• Updating Traefik load balancer and Oracle WebLogic Server Kubernetes Operator configuration

Once you have your domain namespace (of the WebLogic domain that is not yet deployed), you have to update the load balancer’s and operator’s configuration about where the domain will be deployed.

I used the following commands, to upgrade the “operator” Helm chart (with the domain namespace, I created earlier):

cd /u01/content/weblogic-kubernetes-operator/

helm upgrade \
>   --reuse-values \
>   --set "domainNamespaces={sample-domain1-ns}" \
>   --wait \
>   sample-weblogic-operator \
>   kubernetes/charts/weblogic-operator

With the following output:

Release “sample-weblogic-operator” has been upgraded. Happy Helming!
LAST DEPLOYED: Thu Aug 29 07:30:08 2019
NAMESPACE: sample-weblogic-operator-ns
STATUS: DEPLOYED

RESOURCES:
==> v1/Secret
NAME TYPE DATA AGE
weblogic-operator-secrets Opaque 1 10m

==> v1/ConfigMap
NAME DATA AGE
weblogic-operator-cm 3 10m

==> v1/ClusterRoleBinding
NAME AGE
sample-weblogic-operator-ns-weblogic-operator-clusterrolebinding-discovery 10m
sample-weblogic-operator-ns-weblogic-operator-clusterrolebinding-nonresource 10m
sample-weblogic-operator-ns-weblogic-operator-clusterrolebinding-auth-delegator 10m
sample-weblogic-operator-ns-weblogic-operator-clusterrolebinding-general 10m

==> v1/Role
NAME AGE
weblogic-operator-role 10m

==> v1/RoleBinding
NAME AGE
weblogic-operator-rolebinding-namespace 3s
weblogic-operator-rolebinding 10m

==> v1/Service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
internal-weblogic-operator-svc ClusterIP 10.96.61.254 8082/TCP 10m

==> v1/ClusterRole
NAME AGE
sample-weblogic-operator-ns-weblogic-operator-clusterrole-namespace 10m
sample-weblogic-operator-ns-weblogic-operator-clusterrole-general 10m
sample-weblogic-operator-ns-weblogic-operator-clusterrole-nonresource 10m
sample-weblogic-operator-ns-weblogic-operator-clusterrole-operator-admin 10m
sample-weblogic-operator-ns-weblogic-operator-clusterrole-domain-admin 10m

==> v1beta1/Deployment
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
weblogic-operator 1 1 1 1 10m

==> v1/Pod(related)
NAME READY STATUS RESTARTS AGE
weblogic-operator-848cd6c4cb-rpf9v 1/1 Running 0 10m

I used the following commands, to upgrade the Traefik Helm chart (with the domain namespace, I created earlier):

cd /u01/content/weblogic-kubernetes-operator/

helm upgrade \
>   --reuse-values \
>   --set "kubernetes.namespaces={traefik,sample-domain1-ns}" \
>   --wait \
>   traefik-operator \
>   stable/traefik

With the following output:

Release “traefik-operator” has been upgraded. Happy Helming!
LAST DEPLOYED: Thu Aug 29 07:30:30 2019
NAMESPACE: traefik
STATUS: DEPLOYED

RESOURCES:
==> v1/ConfigMap
NAME DATA AGE
traefik-operator 1 7m
traefik-operator-test 1 7m

==> v1/ClusterRole
NAME AGE
traefik-operator 7m

==> v1/Service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
traefik-operator-dashboard ClusterIP 10.96.46.6 80/TCP 7m
traefik-operator LoadBalancer 10.96.253.105 111.11.11.1 80:30542/TCP,443:30638/TCP 7m

==> v1/Deployment
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
traefik-operator 1 1 1 1 7m

==> v1beta1/Ingress
NAME HOSTS ADDRESS PORTS AGE
traefik-operator-dashboard traefik.example.com 80 7m

==> v1/Pod(related)
NAME READY STATUS RESTARTS AGE
traefik-operator-7f9c8ff8b8-cqxcf 0/1 Terminating 0 7m
traefik-operator-7fdb7f5c99-m6sbq 1/1 Running 0 27s

==> v1/Secret
NAME TYPE DATA AGE
traefik-operator-default-cert Opaque 2 7m

==> v1/ClusterRoleBinding
NAME AGE
traefik-operator 7m

==> v1/ServiceAccount
NAME SECRETS AGE
traefik-operator 1 7m

NOTES:

1. Get Traefik’s load balancer IP/hostname:

NOTE: It may take a few minutes for this to become available.

You can watch the status by running:

$ kubectl get svc traefik-operator –namespace traefik -w

Once ‘EXTERNAL-IP’ is no longer ”:

$ kubectl describe svc traefik-operator –namespace traefik | grep Ingress | awk ‘{print $3}’

2. Configure DNS records corresponding to Kubernetes ingress resources to point to the load balancer IP/hostname found in step 1

• Deploying a WebLogic domain on Kubernetes

To deploy a WebLogic domain, you need to create a domain resource definition which contains the necessary parameters for the “operator” to start the WebLogic domain properly.

I used the following command, to download the domain resource yaml file and saved it as /u01/domain.yaml:
_{[https://raw.githubusercontent.com/nagypeter/weblogic-operator-tutorial/master/k8s/domain_short.yaml]}

curl -LSs https://raw.githubusercontent.com/nagypeter/weblogic-operator-tutorial/master/k8s/domain_short.yaml >/u01/domain.yaml

The file domain.yaml has the following content:

# Copyright 2017, 2019, Oracle Corporation and/or its affiliates. All rights reserved.

# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl.
#
# This is an example of how to define a Domain resource. Please read through the comments which explain
# what updates are needed.
#
apiVersion: “weblogic.oracle/v2”
kind: Domain
metadata:
# Update this with the `domainUID` of your domain:
name: sample-domain1
# Update this with the namespace your domain will run in:
namespace: sample-domain1-ns
labels:
weblogic.resourceVersion: domain-v2
# Update this with the `domainUID` of your domain:
weblogic.domainUID: sample-domain1

spec:
# This parameter provides the location of the WebLogic Domain Home (from the container’s point of view).
# Note that this might be in the image itself or in a mounted volume or network storage.
domainHome: /u01/oracle/user_projects/domains/sample-domain1

# If the domain home is inside the Docker image, set this to `true`, otherwise set `false`:
domainHomeInImage: true

# Update this with the name of the Docker image that will be used to run your domain:
#image: “YOUR_OCI_REGION_CODE.ocir.io/YOUR_TENANCY_NAME/weblogic-operator-tutorial:latest”
#image: “fra.ocir.io/johnpsmith/weblogic-operator-tutorial:latest”
image: “iad.ocir.io/weblogick8s/weblogic-operator-tutorial-store:1.0”

# imagePullPolicy defaults to “Always” if image version is :latest
imagePullPolicy: “Always”

# If credentials are needed to pull the image, uncomment this section and identify which
# Secret contains the credentials for pulling an image:
#imagePullSecrets:
#- name: ocirsecret

# Identify which Secret contains the WebLogic Admin credentials (note that there is an example of
# how to create that Secret at the end of this file)
webLogicCredentialsSecret:
# Update this with the name of the secret containing your WebLogic server boot credentials:
name: sample-domain1-weblogic-credentials

# If you want to include the server out file into the pod’s stdout, set this to `true`:
includeServerOutInPodLog: true

# If you want to use a mounted volume as the log home, i.e. to persist logs outside the container, then
# uncomment this and set it to `true`:
# logHomeEnabled: false
# The in-pod name of the directory to store the domain, node manager, server logs, and server .out
# files in.
# If not specified or empty, domain log file, server logs, server out, and node manager log files
# will be stored in the default logHome location of /shared/logs//.
# logHome: /shared/logs/domain1

# serverStartPolicy legal values are “NEVER”, “IF_NEEDED”, or “ADMIN_ONLY”
# This determines which WebLogic Servers the Operator will start up when it discovers this Domain
# – “NEVER” will not start any server in the domain
# – “ADMIN_ONLY” will start up only the administration server (no managed servers will be started)
# – “IF_NEEDED” will start all non-clustered servers, including the administration server and clustered servers up to the replica count
serverStartPolicy: “IF_NEEDED”
# restartVersion: “applicationV2”
serverPod:
# an (optional) list of environment variable to be set on the servers
env:
– name: JAVA_OPTIONS
value: “-Dweblogic.StdoutDebugEnabled=false”
– name: USER_MEM_ARGS
value: “-Xms64m -Xmx256m ”
# nodeSelector:
# licensed-for-weblogic: true

# If you are storing your domain on a persistent volume (as opposed to inside the Docker image),
# then uncomment this section and provide the PVC details and mount path here (standard images
# from Oracle assume the mount path is `/shared`):
# volumes:
# – name: weblogic-domain-storage-volume
# persistentVolumeClaim:
# claimName: domain1-weblogic-sample-pvc
# volumeMounts:
# – mountPath: /shared
# name: weblogic-domain-storage-volume

# adminServer is used to configure the desired behavior for starting the administration server.
adminServer:
# serverStartState legal values are “RUNNING” or “ADMIN”
# “RUNNING” means the listed server will be started up to “RUNNING” mode
# “ADMIN” means the listed server will be start up to “ADMIN” mode
serverStartState: “RUNNING”
adminService:
channels:
# Update this to set the NodePort to use for the Admin Server’s default channel (where the
# admin console will be available):
– channelName: default
nodePort: 30701
# Uncomment to export the T3Channel as a service
#- channelName: T3Channel
# serverPod:
# nodeSelector:
# wlservers2: true
# managedServers:
# – serverName: managed-server1
# serverPod:
# nodeSelector:
# wlservers1: true
# – serverName: managed-server2
# serverPod:
# nodeSelector:
# wlservers1: true
# – serverName: managed-server3
# serverPod:
# nodeSelector:
# wlservers2: true
# clusters is used to configure the desired behavior for starting member servers of a cluster.
# If you use this entry, then the rules will be applied to ALL servers that are members of the named clusters.
clusters:
– clusterName: cluster-1
serverStartState: “RUNNING”
replicas: 2
# The number of managed servers to start for any unlisted clusters
# replicas: 1
#
# configOverrides: jdbccm
# configOverrideSecrets: [dbsecret]

I used the following command, to create the Domain:

kubectl apply -f /u01/domain.yaml

With the following output:

domain “sample-domain1” created

I used the following command, to list the Pods:

kubectl get pods -n sample-domain1-ns

With the following output:

NAME READY STATUS RESTARTS AGE
sample-domain1-introspect-domain-job-qkh82 0/1 ContainerCreating 0 28s

Here you see the Pod, being the introspect domain job which needs to be running first.
_{[https://oracle.github.io/weblogic-kubernetes-operator/userguide/managing-domains/configoverrides/#internal-design-flow]}

I used the following command (running it periodically), to list the Pods (with output format wide):

kubectl get pods -n sample-domain1-ns -o wide

After some while, with the following output:

NAME READY STATUS RESTARTS AGE IP NODE
sample-domain1-admin-server 1/1 Running 0 3m 10.244.2.7 10.0.10.2
sample-domain1-managed-server1 1/1 Running 0 3m 10.244.2.8 10.0.10.2
sample-domain1-managed-server2 0/1 ContainerCreating 0 3m 10.0.12.2

And in the end, with the following output:

NAME READY STATUS RESTARTS AGE IP NODE
sample-domain1-admin-server 1/1 Running 0 6m 10.244.2.7 10.0.10.2
sample-domain1-managed-server1 1/1 Running 0 5m 10.244.2.8 10.0.10.2
sample-domain1-managed-server2 1/1 Running 0 5m 10.244.1.4 10.0.12.2

In the end (after I checked periodically), there are three running pods. The whole Domain deployment may take up to 2-3 minutes depending on the compute shapes.

In order to access any application or the Administration Console deployed on WebLogic, you have to configure Traefik Ingress. The load balancer is already assigned during the previous step “Installing and configuring Traefik with a Helm chart”.

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
traefik-operator LoadBalancer 10.96.253.105 111.11.11.1 80:30542/TCP,443:30638/TCP 35s

I used the following command, to create the Ingress:

cat << EOF | kubectl apply -f -
> apiVersion: extensions/v1beta1
> kind: Ingress
> metadata:
>   name: traefik-pathrouting-1
>   namespace: sample-domain1-ns
>   annotations:
>     kubernetes.io/ingress.class: traefik
> spec:
>   rules:
>   - host:
>     http:
>       paths:
>       - path: /
>         backend:
>           serviceName: sample-domain1-cluster-cluster-1
>           servicePort: 8001
>       - path: /console
>         backend:
>           serviceName: sample-domain1-admin-server
>           servicePort: 7001          
> EOF

Remark about Ingress:
Ingress exposes HTTP and HTTPS routes from outside the cluster to services within the cluster. Traffic routing is controlled by rules defined on the Ingress resource.
An Ingress can be configured to give Services externally-reachable URLs, load balance traffic, terminate SSL / TLS, and offer name based virtual hosting. An Ingress controller is responsible for fulfilling the Ingress, usually with a load balancer, though it may also configure your edge router or additional frontends to help handle the traffic.
_{[https://kubernetes.io/docs/concepts/services-networking/ingress/#what-is-ingress]}

Remark about Ingress rules:
Each HTTP rule contains the following information:

• An optional host. If no host is specified, the rule applies to all inbound HTTP traffic through the IP address specified. If a host is provided (for example, foo.bar.com), the rules apply to that host.
• A list of paths (for example, /testpath), each of which has an associated backend defined with a serviceName and servicePort. Both the host and path must match the content of an incoming request before the load balancer directs traffic to the referenced Service.
• A backend is a combination of Service and port names as described in the Service doc. HTTP (and HTTPS) requests to the Ingress that matches the host and path of the rule are sent to the listed backend.

A default backend is often configured in an Ingress controller to service any requests that do not match a path in the spec.
_{[https://kubernetes.io/docs/concepts/services-networking/ingress/#ingress-rules]}

Default Backend
An Ingress with no rules sends all traffic to a single default backend. The default backend is typically a configuration option of the Ingress controller and is not specified in your Ingress resources.
If none of the hosts or paths match the HTTP request in the Ingress objects, the traffic is routed to your default backend.
_{[https://kubernetes.io/docs/concepts/services-networking/ingress/#default-backend]}

In the table below we can see, that as a simple solution, path routing is configured, which will route the external traffic through Traefik to the Domain cluster address (to reach a demo web application at the root context path) or the Oracle WebLogic Server Administration Console.

Host	Path	Backend		Route to
		Service	Port
<no host is specified>	/	sample-domain1-cluster-cluster-1	8001	A demo web application
<no host is specified>	/console	sample-domain1-admin-server	7001	Oracle WebLogic Server Administration Console

With the following output:

ingress “traefik-pathrouting-1” created

Once the Ingress has been created, you can construct the URL of the Oracle WebLogic Server Administration Console based on the following pattern:

http://EXTERNAL-IP/console

The EXTERNAL-IP was determined during Traefik install (see above).

Opening the Oracle WebLogic Server Administration Console

I opened a browser and logged in to the Oracle WebLogic Server Administration Console (where I entered the admin user credentials weblogic/welcome1), via URL:

http://111.11.11.1/console

Next, on the left, in the Domain Structure, I clicked on “Environment”.

So, the Domain (named: sample-domain1) has 1 running Administration Server (named: admin-server) and 2 running Managed Servers (named: managed-server1 and managed-server2). The Managed Servers are configured to be part of a WebLogic Server cluster (named: cluster-1). A cluster is a group of WebLogic Server instances that work together to provide scalability and high-availability for applications.

Remark about the use of Oracle WebLogic Server Administration Console:
The use of Oracle WebLogic Server Administration Console is just viewing purposes because the Domain configuration is persisted in the Pod, which means that after a restart the original values (baked into the image) will be used again.

Testing the demo web application

I opened a browser and started the demo web application, via URL:

http://111.11.11.1/opdemo/?dsname=testDatasource

I refreshed the page several times and noticed the hostname changed.

The hostname reflects the Managed Server’s name which responds to the request. So, I saw the load balancing between the two Managed Servers in action.

Opening the Kubernetes Web UI (Dashboard)

Dashboard is a web-based Kubernetes user interface. You can use Dashboard to deploy containerized applications to a Kubernetes cluster, troubleshoot your containerized application, and manage the cluster resources. You can use Dashboard to get an overview of applications running on your cluster, as well as for creating or modifying individual Kubernetes resources (such as Deployments, Jobs, DaemonSets, etc).
_{[https://kubernetes.io/docs/tasks/access-application-cluster/web-ui-dashboard/]}

You can access Dashboard using the kubectl command-line tool by running the following command:

kubectl proxy

Kubectl will make Dashboard available at

http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/.

The UI can only be accessed from the machine where the command is executed.
_{[https://kubernetes.io/docs/tasks/access-application-cluster/web-ui-dashboard/#command-line-proxy]}

I used the following command, to access Dashboard:

kubectl proxy

Next, I opened a browser and started Dashboard, via URL:

http://localhost:8001/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/

I selected the Kubeconfig option, clicked on “Choose kubeconfig file”, and selected the kubeconfig file that was created earlier (in my case: /home/oracle/.kube/config). Next, I clicked on button “SIGN IN”.

I clicked on Cluster | Nodes:

Here again you can see that the Kubernetes cluster instance consists of three nodes.

I changed the namespace to sample-domain1-ns and clicked on Workloads | Pods:

Here again you can see that there is one Administration Server and two Managed Servers.

So now it’s time to conclude this article. I described the steps that I went through to get an Oracle WebLogic Domain running on a three-node Kubernetes cluster instance (provisioned by Oracle Container Engine for Kubernetes (OKE)) on Oracle Cloud Infrastructure (OCI) in an existing OCI tenancy.
The Oracle WebLogic Server Kubernetes Operator (which is an application-specific controller that extends Kubernetes) was used because it simplifies the management and operation of WebLogic domains and deployments.

In a next article I will describe how I made several changes to the configuration of the WebLogic domain, for example scaling up the number of managed servers.