Three services have been deployed to my Kubernetes cluster running on a public cloud environment. These services expose port for three underlying Pods and six or so containers. The service make the Pods accessible within the cluster – but not to the outside world, the public internet.
For a number of reasons – including scarcity of public IP addresses, cost of public cloud load balancer instances, decoupling between (the structure/naming/location/availability of) internal resources and the outside world – we increasingly tend to make use of an Ingress Controller – a single gateway on the Kubernetes cluster that handles incoming traffic and performing routing, load balancing, SSL termination, (virtual) host name recognition and even help with A/B tests, canary deploys and circuit breaking.
Quoting from DZone:
The most flexible option, although often the most confusing for new users, is the Ingress Controller. An Ingress Controller can sit in front of many services within our cluster, routing traffic to them and depending on the implementation, can also add functionality like SSL termination, path rewrites, or name-based virtual hosts. From an architecture perspective, ingress controllers are generally an automation layer integrated with a backend proxy service, and can generally operate at both layer 4 and layer 7.
Several Ingress Controller implementations are available, for example those based on Nginx, Traefik, HA Proxy, Kong. Regardless of the implementation we pick, the Kubernetes Ingress resource is used to define how incoming traffic on a specific path or sent to a specific host should be routed to a service. In this article I will discuss the Traefik implementation – which I happily resorted to when I could not get the Nginx implementation to properly rewrite URL paths. Traefik served me well.
The visualization of what this article describes:
To be clear: the Ingress Controller is not a special type of resource. What is indicated as Ingress Controller is actually a combination of a Service (of type Load Balancer that gets assigned a Public IP address by the Oracle Cloud Infrastructure or whatever provider you are using) with a Deployment of a container and a Service Account plus ClusterRole plus ClusterRoleBinding.
The three Ingress resources in this image each specify for a Service in the soaring-clouds namespace how traffic arriving from the public internet should be – at a certain URL path and possibly directed to a specific host or carrying special headers – should be routed by the ingress controller to the service (possibly adding headers, rewriting paths etc).
Sometimes working with Kubernetes can be a struggle or feel overly complex. Working with the Traefik Ingress Controller and the little Ingress resources is actually quite pleasant and straightforward. The allocation of a cloud load balancer and public IP address on my cloud provider environment (Oracle Cloud Infrastructure using the OKE service) was fully automated and pretty smooth. It took very little time or effort to be able to expose my services (and thereby my Pods) to the public internet in a controlled manner with friendly urls.
The steps for setting up the Traefik Ingress Controller and subsequently configuring the Ingress resources are outlined next. I made very good use of this article (Teaching How to Get Started with Oracle Container Engine for Kubernetes (OKE) on RedThunder Blog) to get started in no time.
Deploying the Traefik Ingress Controller
The deployment of the Traefik Ingress Controller – a collection of five Kubernetes resources – can be done using a single yml file.
Note: In this case, Traefik is not yet configured for handling HTTPS traffic.
The yml file I used for deploying the ingress controller:
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: traefik-ingress-controller
rules:
- apiGroups:
- ""
resources:
- services
- endpoints
- secrets
verbs:
- get
- list
- watch
- apiGroups:
- extensions
resources:
- ingresses
verbs:
- get
- list
- watch
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: traefik-ingress-controller
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: traefik-ingress-controller
subjects:
- kind: ServiceAccount
name: traefik-ingress-controller
namespace: kube-system
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: traefik-ingress-controller
namespace: kube-system
---
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
name: traefik-ingress-controller
namespace: kube-system
labels:
k8s-app: traefik-ingress-lb
spec:
replicas: 1
selector:
matchLabels:
k8s-app: traefik-ingress-lb
template:
metadata:
labels:
k8s-app: traefik-ingress-lb
name: traefik-ingress-lb
spec:
serviceAccountName: traefik-ingress-controller
terminationGracePeriodSeconds: 60
containers:
- image: traefik
name: traefik-ingress-lb
ports:
- name: http
containerPort: 80
- name: admin
containerPort: 8080
args:
- --api
- --kubernetes
- --logLevel=INFO
---
kind: Service
apiVersion: v1
metadata:
name: traefik-ingress-service
namespace: kube-system
spec:
selector:
k8s-app: traefik-ingress-lb
ports:
- protocol: TCP
port: 80
name: web
- protocol: TCP
port: 8080
name: admin
type: LoadBalancer
---
apiVersion: v1
kind: Service
metadata:
name: traefik-web-ui
namespace: kube-system
spec:
selector:
k8s-app: traefik-ingress-lb
ports:
- port: 80
targetPort: 8080
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: traefik-web-ui
namespace: kube-system
annotations:
kubernetes.io/ingress.class: traefik
spec:
rules:
- host: localhost
http:
paths:
- backend:
serviceName: traefik-web-ui
servicePort: 80
The resulting resources can be inspected using kubectl or in the Kubernetes Dashboard:
After kubectl sent the instructions from this file to the K8S cluster and the service was created, a load balancer with public IP address was assigned by cloud infrastructure to the traefik-ingress-service. From that moment on, requests from anywhere on the internet could be sent to that public IP address to be handled by the Traefik Ingress Controller.
Configuring Ingress Resources to route traffic to My Services
The Ingress Controller – my public gateway into the K8S instance on Oracle Kubernetes Engine – is running. However, it has no routing configuration – no instructions to send any requests anywhere. So this is just one step of the way.
The second step is the definition of Ingress resources. Each Ingress defines rules for routing requests on a specific path or for a specific (virtual) host to one or more services. In my case, I have a Deployment called soaring-webshop-portal-deployment. It runs one or more Pod instances, exposing two ports. These two ports are exposed by a service called soaring-webshop-portal. I want these service ports to be accessible publicly – which has to be done through the Ingress Controller.
I have created an Ingress resource to take care of this. It specifies that requests (arriving at the Public IP address assigned to the Traefik Ingress Controller) with URL path /soaring/portal, are to be routed to the webui on port service soaring-webshop-portal. Requests for path /soaring/reloadportal are to be forwarded to the reloader port on the same service. In both cases, the path prefix – /soaring/portal and /soaring/reloadportal respectively – are to be stripped from the url for the rerouted requests. This means that when the requests arrive at the service and the pod and the application inside the pod handling the request the URL path in the application will see starts with whatever follows the path prefix. (I could not get this stripping to work with NGINX, that’s how I arrived at Traefik Ingress Controller).
This path prefix stripping is specified through an annotation in the Ingress resource: traefik.frontend.rule.type: PathPrefixStrip.
The full ingress definition is shown here:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: soaring-portal-ingress
namespace: soaring-clouds
annotations:
kubernetes.io/ingress.class: traefik
traefik.frontend.rule.type: PathPrefixStrip
spec:
rules:
- http:
paths:
- path: /soaring/portal
backend:
serviceName: soaring-webshop-portal
servicePort: webui
- path: /soaring/reloadportal
backend:
serviceName: soaring-webshop-portal
servicePort: reloader
Resources
Teaching How to Get Started with Oracle Container Engine for Kubernetes (OKE) on RedThunder Blog – this article
Ingress Controllers for Kubernetes – https://dzone.com/articles/ingress-controllers-for-kubernetes
Docs on Traefik Kubernetes Ingress provider – including all available annotations – https://docs.traefik.io/configuration/backends/kubernetes/
