RHEVM Hang after adding Manageiq


#1

Dear Expert,

I have added RHEVM with Manageiq, after adding Manageiq RHEVM started to hang. CPU utilization goes 97% continuously.
07:13:58 PM CPU %user %nice %system %iowait %steal %idle
07:13:59 PM all 96.01 0.00 1.50 0.75 0.00 1.75
07:14:00 PM all 94.76 0.00 2.74 0.75 0.00 1.75
07:14:01 PM all 93.97 0.00 3.77 0.50 0.00 1.76
Average: all 94.92 0.00 2.67 0.67 0.00 1.75

Also I could see below error in engine.log. I have increased Heap size upto 4g still same issue.

014-09-24 19:05:08,576 INFO [org.ovirt.engine.core.utils.LocalConfig] (ServerService Thread Pool – 39) Value of property “ENGINE_JVM_ARGS” is " -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath="/var/log/ovirt-engine/dump"".


#2

What version of rhev m is this and how much memory does the rhev manager have?


#3

RHEVM 3.4 version. Memory allocated is 11GB.


#4

Is any one could add RHEVM with Manageiq ?? Please help


#5

@chandrahasa01,

Could you provide all of the environmental details regarding your setup?

Such as:

  • of Hosts

  • of Clusters

  • # of VMs

Hardware per host and hardware for the RHEVM machine. (I assume its bare metal or is it virtual?)

In my cluster I’m seeing an 1.8GB of Resident memory on my ovirt-engine process.

Also what are your Miq appliance roles and # of appliances?


#6

Host : 5
Cluster : 1
VMS : around 100

RHEVM hosted on Vmware Virtual machine. 5 CPU and 15 GB


#7

Hi,

Is any one could add RHEV Manager with ManageIq.

Chandra


#8

Chandrahasa01

You have news ?

I begin using ManageIQ with my RHEV Manager and I have same problem.

Regards,
Tiago Menezes


#9

RHEV-M seems to consume a lot of heap when connected to manageiq, which seems to lead to the process basically being stuck doing only garbage collecting. You can try increasing ENGINE_HEAP_MIN and ENGINE_HEAP_MAX in ovirt-engine.conf (8GB stabilized it for me). Heap consumption should be reduced in rhev 3.5+…


#10

The HEAP size can indeed be changed, as described in the previous comment.
From 3.5.1 (to be released in ~2 weeks), the default will be set to max(1G, 25% * RAM). However, it seems like it only applies automatically if you setup reports as well.
The Bug is:
Bug 1190466 - HEAP_MAX default value as 1G must be changed

In 3.6.0 it will change also for regular setups:
Bug 1185411 - ENGINE_HEAP_MAX default value as 1G must be changed


#11

dsundqvist and ovedo.

Thanks for helping.

I change /usr/share/ovirt-engine/services/ovirt-engine/ovirt-engine.conf :

ENGINE_HEAP_MIN=8g
ENGINE_HEAP_MAX=16g
ENGINE_PERM_MIN=2g
ENGINE_PERM_MAX=4g

Work fine.

Regards,

Tiago Menezes


#12

In addition you should look at rhev-m AuditLogAgingThreshold .We reduced that from default 30 days to 3 days and it reduced the load on rhev-m a lot. Indeed CFME is looking at /api/events on a (very) regular basis, and producing an xml file of 500k + events every minutes consumes a lot of resources on rhev-m side !