Refactoring EMS Connections

Microsoft’s SCVMM exposes their event stream from a completely different server from where the general API is accessible. This presents a major problem for integrating with SCVMM: our provider integrations only accept a single IP address for each provider.

For other providers, there has been several “nice to have” requests to allow separation of various components behind different IP addresses. For SCVMM, it’s the first time it’s actually required.

One way I’ve thought of to solve this problem is to refactor the was we store provider connections. Currently, we store the IP address on the Ext_Management_Systems table. Then, we allow multiple authentication records to reference each individual provider record.

Instead, we could introduce a new model for Ext_Management_System_Connections that would store the list of possible connection objects available for the EMS. We’d likely need the concept of a “default” connection, as well as a way to select specific connections for different purposes.

However, we can’t simply remove the association from Authentication to Ext_Management_System. There are other types of authentications that are stored for Ext_Management_Systems that are used outside the context of a connection (e.g., key pairs for launched instances).

See below for a simple model diagram:

                         Current Model                
+-----------------------+ 1    * +----------------+   
| Ext_Management_System <--------+ Authentication |   
|                       |        |                |   
+-----------------------+        +----------------+   
                      New Suggested Model             
                      * +----------------+ 1          
              +---------+ EMS_Connection <---+        
              |         |                |   |        
              |         +----------------+   |        
            1 |                              | *      
 +------------+----------+          +--------+-------+
 | Ext_Management_System |          | Authentication |
 |                       |          |                |
 |                       |          |                |
 |                       |          |                |
 |                       |1       * |                |
 |                       +----------+                |
 +-----------------------+          +----------------+

In the Microsoft System Center scenario there are 2 completely separate providers, SCOM and SCVMM, each with their own API. For the purposes of our integration, there is no overlap in functionality between the two. In the new suggested model above which System Center product is represented by Ext_Management_System? Or does it represent a generic System Center EMS and we differentiate with instances of Ext_Management_System_Connection?

This might be an opportune time to consider further subclassing the Ext_Management_System. In scenarios like Microsoft System Center where there are multiple separate providers we currently have functionality specific to one provider and not the other in the EMS, e.g. power operations are exclusive to SCVMM yet are in the EMS_Microsoft class. This might defeat the purpose of adding the Ext_Management_System_Connections class but i think it is something that we should discuss at this time.

To your question, I think that the ExtManagementSystem still represents the generic System Center EMS.

I’d definitely shy away from subclassing ExtManagementSystem.

If we have unique entities for SCOM and SCVMM which are subclasses of ExtManagementSystem, then it implies that we will have distinct records in the ext_management_systems table (through single table inheritance).

This poses a couple of problems:

  1. In one case, we could try to jam both ext_management_system records into the same UI for adding a Microsoft provider. I think it’s important to keep the Microsoft provider under one proverbial “roof” in the UI. But, this would take a extraordinary amount of refactoring in the UI/Controller layer to make the Provider UI work with two separate ExtManagementSystem records.

  2. We could decide to split this up and present the different aspects of the Microsoft provider into separate Provider entries (from the user perspective). So, adding a Microsoft provider really entails adding more than one provider. But, to an end user, it’s really just different aspects of one provider.

And, I think that’s what Microsoft is really doing here. They’ve separated different aspects of their services into different API endpoints. This is not entirely different from how Openstack works. But, openstack provides a nice single service to discover all of the other services. And, we don’t have to maintain that ourselves.

Unfortunately, Microsoft hasn’t provided the same thing here. In this case, it looks like we’re going to have to build that knowledge directly into the Ems_Microsoft.

So, thinking about this some more … maybe there’s a single Ems_Microsoft and it have various EMS Services (SCOM and SCVMM). And, EMS_Microsoft has the smarts to know which service provides which features that ManageIQ depends on. For instance, when EMS Refresh asks Ems_Microsoft to gather all of the VMs from the API, Ems_Microsoft knows that Inventory operations are handled through the SCVMM service. When the user clicks the button to start a VM, the controller asks Ems_Microsoft to start a particular VM, Ems_MIcrosoft knows that VM power ops are handled throught the SCVMM service. And, when the event catcher asks Ems_Microsoft to gather events, Ems_Microsoft knows to use the SCOM service.

This way, instead of there simply being separate connections (IP address, port, etc…). Now, there’s a way we can possibly separate out some of the operations into services. And, the services know how to handle inventory, power ops, and event gathering. Also, services know what IP address, port, and any special authentication they require to operate.

How does that sound?

This sounds like a good way to go, creating services achieves the same outcome that I was looking for by subclassing Ems_Microsoft (which now that i think about it some more will create separate provider objects which is not what we want). However, this sounds like a huge amount of refactoring and will affect delivery of the next phase of the MSFT integration.

This is something I’ve expressed the need for in the past.

I think my view seems to match yours. Essentially, the Ext_management_System defines the scope of management (the view if the set of objects that can be managed) and, what you call EMS_Connections define various services that can operate in that scope. In concept, it’s a more generalized version of the OpenStack model - without the unifying Identity service.

In my view, an EMS_Connection would encapsulate more than the connection information, it would also provide a means to instantiate an API handle to access the given service for the given scope.

So, the Ext_management_System defines the managed environment, and one or more EMS_Connections define the means through which said environment can be managed.

Does that seem to correspond to your vision of this model?

There is an interim solution we could implement specifically for the System Center integration that would tide us over until the larger scale redesign is in place. It has its warts which I will elaborate on but it should work.

SCVMM could be used as a conduit for events to stream from SCOM to CF. This means we can connect to the same provider IP address using the same credentials that we used to gather inventory. SCVMM would make the connection to SCOM, not CF
A big concern is around the load this will place on the SCVMM server. We can increase the quantity of compute resources we require to compensate but I think load testing would still have to be carried out. This might also introduce a latency that will adversely affect EMS refreshes.

There are significant limitations around being granted permission to do “second hop” remoting using powershell (appliance → SCVMM → SCOM) so the only way to be able to make this work is to change the contents of the powershell script to store the SCOM credentials and servername after the file has been read from disk and before it is encoded and transmitted. This poses obvious security vulnerabilities that might make this a non starter but given that this will not be run over the internet maybe it will be OK.


@bronaghs, is that something that we’d do through SmartProxy? I don’t know much about that end of things.

I would definitely worry about the security vulnerabilities you point out. However, we might be able to get past those by asking the user to enter the SCOM credentials directly into ManageIQ. Then, we could simply pass those values along to the power shell script.

Nonetheless, I’m definitely not a huge fan of the extra hop idea. There’s a lot that can fail there that’s out of our purview. And, debugging such an issue could be a real mess.

I would rather solve this in our app directly. It would give us more control over the situation directly.

My $0.02

@blomquisg - No, we would not leverage the SmartProxy. In fact, we direct the SCVMM server to execute the cmdlets on the SCOM box by passing the parameter ‘-ComputerName SCOMBOX’ to the powershell cmdlet. It is common practice to use this parameter in powershell. Any errors returned to us from running powershell cmdlets (single or multi hop) are printed in our log files, I’m not sure what kind of mess we cannot anticipate ahead of time given that SCVMM and SCOM already maintain a persistent connection.
The question is do the cons outweigh the pros for using this as an interim solution. I guess we should wait and see what the level of effort is for the redesign.