First Steps with Prometheus and Grafana on Kubernetes on Windows

Preparation: before I started with Prometheus, I installed VirtualBox, Minikube with Tiller on my Windows 10 laptop – as described in this article: https://technology.amis.nl/2018/08/13/running-kubernetes-1-10-using-minikube-on-windows-10-adding-kubectl-and-helm-tiller/.

I then increased the Base Memory setting for the minikube vm to 8192 MB (the default 2048 MB is not sufficient)

After making this change – which can only be done when the VM is not running – I restarted minikube.

Then:

git clone https://github.com/coreos/prometheus-operator

kubectl apply -f bundle.yaml

The Kubernetes Dashboard shows what has been created as a result:

Note: installing the Prometheus Operator with helm – the more fancy option:

helm init --client-only
helm repo add coreos https://s3-eu-west-1.amazonaws.com/coreos-charts/stable/
helm upgrade --install --namespace=monitoring -f values.yaml kube-prometheus coreos/kube-prometheus

did not work, because of this failure: Error: open values.yaml: The system cannot find the file specified..

First steps with Prometheus on Kubernetes

One thing we can easily use Prometheus for, is for monitoring the Kubernetes Cluster itself. The CoreOS Kube Prometheus package offers that functionality: it leverages Prometheus through the Prometheus Operator to monitor relevant metrics exposed by Kubernetes.

Kube Prometheus can be installed using this command:

helm install coreos/kube-prometheus --name kube-prometheus --set global.rbacEnable=true --namespace monitoring

Runing this command resulted in a new namespace - monitoring - and a number of Kubernetes artifacts - some of which are of a type defined by the Prometheus Operator:

To get live insight in what Prometheus is scraping from Kubernetes, we can use some of the UIs available. In order to access them on my Windows laptop, I have to execute the following commands in as many command windows:

kubectl port-forward -n monitoring prometheus-kube-prometheus-0 9090 &

kubectl port-forward -n monitoring alertmanager-kube-prometheus-0 9093 &

and after finding the name for the Grafana pod using:

 kubectl get  pods --selector=app=kube-prometheus-grafana -n  monitoring

We can also forward the port for the Grafana dashboard:

kubectl port-forward -n monitoring kube-prometheus-grafana-57d5b4d79f-48v66 3000 &

The user interfaces for Prometheus, the Alert Manager and the Grafana Dashboard UI are now available like this:

An overview of some of the Scrape targets configured by Kube Prometheus:

resulting in a large number of metrics on the cluster nodes, the Pods and containers and many more Kubernetes artifacts.

Here is a Grafana overview for the single node in the Kubernetes cluster (the VirtualBox VM running minikube):

The next step would be to deploy my own instrumented applications to the Kubernetes cluster or to use the Prometheus instance running in my minikube to scrape metrics from targets outside the cluster.

Resources

Article Deploy Prometheus on Kubernetes- https://www.cloudtechnologyexperts.com/deploy-prometheus-on-kubernetes/ (March 2018)

Article Running Kubernetes 1.10 using MiniKube on Windows 10 (adding kubectl and helm/tiller) – https://technology.amis.nl/2018/08/13/running-kubernetes-1-10-using-minikube-on-windows-10-adding-kubectl-and-helm-tiller/ (August 2018)

CoreOS Prometheus Operator on GitHub – https://github.com/coreos/prometheus-operator 

CoreOS Prometheus Getting Started – https://coreos.com/operators/prometheus/docs/latest/user-guides/getting-started.html