How to success get Kubernetes provider metric by hawklur?

Hi all,
I add kubernetes provider in manageiq and enable C&U role, but the metric is nil.
In the evm.log show

[----] E, [2017-10-30T03:52:41.825814 #16603:9fd134] ERROR -- : MIQ(ManageIQ::Providers::Kubernetes::ContainerManager::MetricsCapture#perf_collect_metrics) Hawkular metrics service unavailable: Failed to o
pen TCP connection to (Connection refused - connect(2) for "" port 5000)

It need hawuklar, but there is not enough document for it, there only openshift document.


@bazulay @simon3z


I do not know of any supported way to install Hawkualr + Heapster on Kubernetes, The methods I know assume you have roles and permissions created when installing openshift.

But you can install Hawkular + Heapster on a Kubernetes cluster. The setup we use in ManageIQ metrics is Heapster with source kubernetes.summary_api and sink hawkular-metrics.

a. Do you have Hawkular + Heapster configured and you want to access this metrics via ManageIQ ?
b. Do you need help setting up Hawkualr + Heapster on your Kubernetes cluster ?

Hi, Yaacov_Zamir
a. No, I don’t have hawkular and heapster configured.
b. Yes, I need help.

Thanks your help.

Do you need this for production or for proof of concept ?

We do not support metrics on k8s for production

If we are on none production land :slight_smile: -

Hawkular is a big thing ( uses Cassandra ) that has a lot of features you may not need …

@MilkBotttle : Do you need Hawkular, or you can settle for something smaller with less features ? ( e.g. no persistence, no alerts, only metrics for ManageIQ )

A smaller metrics engine like prometheus or mohawk may be easier to install, but will have less features.

@abonas : Do you know who can help with installing a TSDB (Hawkular/Prometheus/Mohawk) on k8s, all the methods I know relay on secrets / rolls installed using the openshift-ansible playbook that install openshift ?

We do not have any ready made template for that, it will be time consuming …

@theute afaik there’s no doc/template for k8s, is that right?

Correct, only for OpenShift


I use for proof of concept.
The other metric method is fine.



I do not have a k8s setup, so I can not at this time check if it works … I will spin up one, and try to help.


The easiest way ( that I know of ) is to install Heapster with Mohawk.

Below is a link to a Openshift yaml file for Heapster + Mohawk:

Changes needed:

  • remove the template part, k8s does not have templates :slight_smile:
  • remove the route , k8s does not have routes
  • create a secret with two files tls.key and tls.crt and use it instead of the openshift secret.

If you can translate it to a k8s yaml file, please send me a link to the translation :slight_smile:


I used minikube, credentials and routing may different when using a “real” k8s deploy.
I used my key and cert to deploy Mohawk, you may want to use your credential files for that.

a. Spinning up minikube:

~]# minikube start --vm-driver=kvm

b. Creating the Mahawk + Heapster pods:
The k8s list file I used is ( note: the credential files are missing ):

~]# kubectl create -f Projects/mohawk-container/mohawk-k8s.yaml

c. Check that the pods and endpoints are up:

~]# kubectl get pods --all-namespaces
NAMESPACE     NAME                          READY     STATUS    RESTARTS   AGE
kube-system   heapster-jgzvx                1/1       Running   0          41m
kube-system   kube-addon-manager-minikube   1/1       Running   0          58m
kube-system   kube-dns-6fc954457d-m5rkf     3/3       Running   0          58m
kube-system   kubernetes-dashboard-qxd4j    1/1       Running   0          58m
kube-system   mohawk-b2qhn                  1/1       Running   0          41m

d. Check endpoints:

~]# kubectl get ep --all-namespaces
NAMESPACE     NAME                      ENDPOINTS                     AGE
default       kubernetes                 49m
kube-system   kube-controller-manager   <none>                        49m
kube-system   kube-dns        ,   49m
kube-system   kube-scheduler            <none>                        49m
kube-system   kubernetes-dashboard               49m
kube-system   mohawk                         32m

e. Get the k8s token:
Use the correct secret name.

~]# kubectl get secret default-token-f59kh -o yaml | grep token | head -n 1 - |  cut -d' ' -f 4 | base64 -d -
eyJhbG ... YkwIpww

f. Set provider and metrics on ManageIQ:

provider url: the default endpoint, in this case
provider port: the default endpoint port, in this case 8443
provider token: the token

metrics url: the same as the provider
mertics port: the mohawk endpoint port 30064

In this setup use ssl-without-validation.