New guest detection

wpoteat · January 7, 2015, 4:46pm

I would like to see what method is used to detect the creation of a new guest vm. Is this there a bus message on the host that can be tapped into? does it cover different fabrics?

I am intending to use this to send information upstream in the entitlement realm. Any assitance would be greatly appreciated.

Thanks

Fryguy · January 7, 2015, 9:09pm

When a Provider is added, ManageIQ monitors for new VMs automatically, however the information we get is from the provider’s perspective, and may not have detail. So, SmartState Analysis (aka fleecing or scanning) returns that detailed information, and can be set up on a schedule, when new VMs are detected, or by handling the event for when scanning completes in automate or policy.

Fleecing is available for VMware, RHEVM/oVirt, and for OpenStack (images only there).

wpoteat · January 7, 2015, 9:39pm

How does ManageIQ monitor for new VMs? Are the guest/host system ids available?

Is it a bus message? Is there a variation for each fabric?

At this point I just need that much.

geertj · January 7, 2015, 10:57pm

The exact mechanics depend on the provider. For VMware there is an event bus we can connect to. For AWS we use the AWS Config service.

wpoteat · January 8, 2015, 1:27pm

Where can I go to see the details of that? Is there documentation on the events on the bus? Is there information on the AWS?

Thanks

ewannema · January 8, 2015, 2:40pm

Are you interested in the VMware event bus or what you get in ManageIQ?

If you are looking at ManageIQ, you can examine the /var/www/miq/vmdb/log/automate.log file and see events coming in. When you see an event type you want to do something with, just make sure there is an instance at the path in the log file that points to a method. That method can take action or just log information.

In the log snippet below you can see that there is a VDP event that nothing is set up to handle. If I wanted to do something with it I could create the instance that the system could not find and take action.

Following Relationship [miqaedb:/System/Event/com.vmware.vdp2.integrity#create]
Updated namespace [miqaedb:/System/Event/com.vmware.vdp2.integrity#create  ManageIQ/System]
Instance [/ManageIQ/System/Event/com.vmware.vdp2.integrity] not found in MiqAeDatastore - trying [.missing]

wpoteat · January 9, 2015, 2:27pm

Ultimately I am interested in letting a package called virt-who to be able to detect the creation, removal, or migration of a VM. virt-who will then alert Satellite real time to report the system id’s of the guest and host. I am hoping that virt-who can monitor the setup to get this data.

I am short on details at this juncture. Maybe there is a better way to think of it. I just need the system id’s so I can establish their relationship in another system.

And I cannot wait for the polling to happen after the fact.

If someone can help me [point me] to hook into a vmware bus running on a separate system, that would be great.

jrafanie · January 9, 2015, 10:12pm

@wpoteat I don’t know how to actually do it but I believe the various events from external systems like ovirt/rhevm/vcenter/amazon are handled or ignored(see filtered_events) via the event_handling yaml. I would think you’d want to write a policy(event/condition/action) that when specific events occur, you’d want pass along the identity information to satellite.

The alternative would be to hook into each management system separately and monitor them like we already do in manageiq.

@Fryguy @blomquisg is this right?

wpoteat · January 12, 2015, 1:38pm

@jrafanie @Fryguy

Other [VMware] sources tell me that guest detection of its host is a security no-no. Does this mean that the events are unlikely to contain the host information as they guest would initiate the event? Are most if not all scenarios in ManageIQ such that the guest reports itself, but the relationship to its hypervisor is always waiting for the poll of the network?

jrafanie · January 12, 2015, 3:07pm

@wpoteat I’m not sure what you mean. Most vendors including VMware provide the information as to the guest vm <-> hypervisor in their inventory information. Granted, sometimes it’s not immediately available. I don’t know that the raw event information always excludes the hypervisor/host information but it’s easily discovered by a followup “refresh” of the vm’s relationships.

wpoteat · January 12, 2015, 3:55pm

That is the rub. The people using it and those planning around it, would really like to have it readily available.

Do any of the fabrics that MangeIQ runs with do immediate availability? Is there someone who can give me some example events so I can at least get an idea if those events have enough data?

Even if the answer is to have a process on my end that monitors the message traffic and the messages have just enough info, it may be enough.

Thanks

Fryguy · January 12, 2015, 4:15pm

@wpoteat All events goes through automate, so I believe an automate method could be written in the right place to do whatever is needed. If you hook the vm_created event, then you will get the basic information about the vm in the event payload. If you need more information, then you’d have to wait for the EMS refresh to complete and then send that information. I don’t believe there are events for refresh completing yet. cc @gmccullough

Here is an example event payload for a VmStartedEvent from VMware (in Ruby yaml format from our database). We parse most of that information into an EmsEvent record, so in automate can just use that, and you probably don’t need the raw event, however it should be available.

--- !ruby/hash:VimHash
key: !ruby/string:VimString
  str: '43829772'
  xsiType: :SOAP::SOAPInt
  vimType: 
chainId: !ruby/string:VimString
  str: '43829771'
  xsiType: :SOAP::SOAPInt
  vimType: 
createdTime: !ruby/string:VimString
  str: '2014-07-13T12:14:07.812734Z'
  xsiType: :SOAP::SOAPDateTime
  vimType: 
userName: !ruby/string:VimString
  str: jfrey
  xsiType: :SOAP::SOAPString
  vimType: 
datacenter: !ruby/hash:VimHash
  name: !ruby/string:VimString
    str: Mahwah
    xsiType: :SOAP::SOAPString
    vimType: 
  datacenter: !ruby/string:VimString
    str: datacenter-1841
    xsiType: :ManagedObjectReference
    vimType: :Datacenter
  __iv__@xsiType: :DatacenterEventArgument
  __iv__@vimType: 
computeResource: !ruby/hash:VimHash
  name: !ruby/string:VimString
    str: Production Cluster
    xsiType: :SOAP::SOAPString
    vimType: 
  computeResource: !ruby/string:VimString
    str: domain-c1357
    xsiType: :ManagedObjectReference
    vimType: :ClusterComputeResource
  __iv__@xsiType: :ComputeResourceEventArgument
  __iv__@vimType: 
host: !ruby/hash:VimHash
  name: !ruby/string:VimString
    str: testvc.manageiq.com
    xsiType: :SOAP::SOAPString
    vimType: 
  host: !ruby/string:VimString
    str: host-1921
    xsiType: :ManagedObjectReference
    vimType: :HostSystem
  __iv__@xsiType: :HostEventArgument
  __iv__@vimType: 
vm: !ruby/hash:VimHash
  name: !ruby/string:VimString
    str: jfrey-prod
    xsiType: :SOAP::SOAPString
    vimType: 
  vm: !ruby/string:VimString
    str: vm-125
    xsiType: :ManagedObjectReference
    vimType: :VirtualMachine
  path: !ruby/string:VimString
    str: ! '[STARWND1] jfrey-prod/jfrey-prod.vmx'
    xsiType: :xsd:string
    vimType: 
  __iv__@xsiType: :VmEventArgument
  __iv__@vimType: 
fullFormattedMessage: !ruby/string:VimString
  str: jfrey-prod on host testvc.manageiq.com in Production Cluster
    is starting
  xsiType: :SOAP::SOAPString
  vimType: 
changeTag: !ruby/string:VimString
  str: ''
  xsiType: :SOAP::SOAPString
  vimType: 
template: !ruby/string:VimString
  str: 'false'
  xsiType: :SOAP::SOAPBoolean
  vimType: 
eventType: VmStartingEvent
__iv__@xsiType: :VmStartingEvent
__iv__@vimType:

Fryguy · January 12, 2015, 4:30pm

If you wanted to hook the VMware event bus directly, it’s not exactly trivial, but it could be done. You would need to access their API directly for events.

https://pubs.vmware.com/vsphere-50/index.jsp#com.vmware.wssdk.pg.doc_50/PG_Ch15_Alarms.17.1.html

This is what our event monitor uses already:

https://github.com/ManageIQ/manageiq/blob/master/lib/VMwareWebService/MiqVimEventHistoryCollector.rb
https://github.com/ManageIQ/manageiq/blob/master/lib/VMwareWebService/MiqVimEventMonitor.rb

wpoteat · January 12, 2015, 5:38pm

@Fryguy Is there connection info floating around in the project? Config file?

gmccullough · January 12, 2015, 5:47pm

Wanted to clarify one point around the vm_create event mentioned above. This policy event is not raised when we capture the corresponding provider event since there is no record in the vms table to work against. Instead this policy event is raised in a post-create action for records in the vms table as part of the EMS refresh logic. Therefore the data and associations collected from an EMS refresh would be available at the time the vm_create event is raised.

Fryguy · January 12, 2015, 6:18pm

@gmccullough But could it be raised to automate and access the raw data?

Fryguy · January 12, 2015, 6:19pm

When you add an EMS, you specify the connection info. @wpoteat Have you played around with ManageIQ to add an EMS and see what it brings back?

wpoteat · January 12, 2015, 6:29pm

No, not yet. In the end I think directly connecting to the vmware bus will be the most fruitful. I was asking if there was connection info to that bus in the ManageIQ config files. It is unlikely we would mandate ManageIQ for the customers in this case.

A bus monitor will likely run on our end in a python daemon. I was hoping for the expected connection parameters.

Perhaps the queue name and routing key?

Fryguy · January 12, 2015, 7:00pm

It’s not an MQ bus, if that’s what you’re thinking. It’s just an API on which you request events. The links to our code and the VMware SDK API will be your best bet for getting something like that written.

wpoteat · January 13, 2015, 12:47pm

@Fryguy So its still a pull scenario.

ManagerIQ polls the VMWare setup on a set frequency to get new guest VM’s. It uses the API to batch pull the events and then processes them.

Is that about right?