Revamp user experience for Automate/Policy


#1

I have developed a few custom workflows for customers and every time this ended with mixed feelings. They were all impressed by the extensibility of the product, being able to be integrated with the inhouse tools. But they were also worried about the complexity of the development : no designer, only the Automation Engine tree and Ruby code.

And to be honest, it is the same for me. Designing workflows with automation.log is not what I would call user friendly. Of course, I now have a library of recipes that I import in the Automation Engine to bootstrap my development. And I have created my own conventions that enforce a good level of consistency across all my classes and methods.

What I would expect from a Cloud Management Platform is a strong set of graphical designers that are linked by conventions, so that one could interact from command line also. Let’s take an example: you want to create a new service catalog item. The service implementation would require a few actions in a specific order, also called a workflow. People would thus expect to have a workflow designer that allows to pick actions from a library, as well as decision points based on the environment or action result. Some of the actions could be forms in order for the user to provide information. People would then expect to have a form designer that does ask only label, description and type for each element. The data from the form would then be available to the actions. And of course, for advanced users, actions could be created/modified to fit their needs: Ruby would be a good choice.

Regarding the workflow and form designers, my first thought is that Business Process Model and Annotation, aka. BPMN, could be used for modelization. Additionally, Business Process Execution Language, aka. BPEL, would provide execution description. And in the community, jBPM provides tools to work with them. I don’t know exactly how they could be integrated or even if that would be a good choice, but they could be a source of inspiration for ManageIQ tooling.

Continuing with Policy, it is easier to use/extend than Automate, but again could be more user friendly. Once again, I don’t know how to integrate it or if it is relevant but a Business Rules Management System, aka. BRMS, could improve rules definition. In the community, Drools is well known and provides a human readable language, that one can extend with a Domain Specific Language (DSL), and a Complex Event Processing (CEP) module (Drools Fusion), that could be used to filter/aggregate events. And it is well integrated with jBPM…

As these tools do not have the look-and-feel of ManageIQ, they could be separated in a « CloudForms Studio » that would be dedicated to bending CloudForms to the customers needs. And indeed, very few users edit the Policy and Automation engines. Most of them just want to manage resources.


#2

It seems your post got cut off, but even with the preamble, I like the premise. Automate is pretty nasty to work with as is and could use a lot of UX love.


#3

Yep. Somehow, I published only a teaser :wink:


#4

Maybe a first step could be providing a tool to generate the YAML files of a custom domain. This would require development guidelines for AE Datastore, such as « whenever you add a feature to an object type, say an infrastructure VM, it must be composed of a class that defines the attributes, and the states. This class must have an instance called default and each state must point to a method of the same name as the state. ». [1]

Then the tool would enforce those guidelines by providing a feature called Add a feature to this object type. And the code would create the subtree for the class and methods. The user would then be able to click on a state of the workflow and edit the method.

It would even be better if the user had a autocompletion feature in the editor that present the available objects that he can handle in the context of the method. But defining the context of the method might be a bit tricky.

And of course, this tool would version the domain subtree with Git and propose to upload the code in the database. AFAIK, it is already possible to import a domain with a rake command, so it would just mean create a zip file with the domain subtree, upload it to the appliance (requires SSH access) and call this rake command ; or even more simple: create a REST API call to do it.

And last but not least, the tool would provide a nice and shiny web interface to model the workflows. Remember jBPM workflow editor :wink:

[1] I know it’s not a real guideline. A state can also be a relationship :slight_smile:


#5

Just bumping the post to get an opportunity to have some insight on what’s going on in this area.

I have posted a question on rollback and I feel that it should be treated more globally in workflow management. When following a path to perform a bunch of actions, we should be able to go backwards and undo the changes. We should also be able to resume a workflow once the issue has be solved.

What are your thoughts ?


#6

This is an interesting point you raise as I too used to feel the same way. That was until I got a deep understanding of some competition (vCAC). The experience in other products is quite often blurred, the other vendors show how easy not how difficult it can be. As an example, I would urge you to bake off against another product creating a simple multi tiered service, with 2 or more different VM’s and a generic service to copy a file between the two vms deployed. In MIQ I agreed we create ruby code to perform the copy, but in vCAC they create JavaScript (wheres the difference?), they also need to wire input and output attributes to form the dialog, the dialog and workflow have to be wired into the vCAC service catalog, and believe me this is not small task, and then they have to add the machine template blue prints. Its actually quite a bit task to achieve what we can do in less than 10 minutes.

So the designer replaces some if statements, but then we are state machines with added functionality around re-entracy and states that vCO workflows that do have a designer cannot leverage.

I think you need to compare apples with apples, of which vCAC and ManageIQ are not that comparable.

I am not saying we would not value form a having a designer, I would love one too, but in the mean time you should be able to show the value of MIQ and its ease without the need for the designer.

One thing the other products have done is taken the workflows and productised them, this limits the amount of interaction with the workflow designers that are cumbersome and add little value anyway.

John Mark, is there something we could put up on ManageIQ.org that would allow submission of workflow/statemachine ideas and vote for them? If we could understand which tasks are so common that everyone is using the same state machines over and over again, this may help some drive in developers to productise those state machines : example - Rename a VM, if this is being used a lot, then maybe instead of having a custom button call the workflow to rename a vm, we could productise it and have it off the Configuration menu as a core function.

disclaimer - the use of the word “productise” was used on the sense of having a developer turn someones script into something that works for all, in no way did I mean make it commercial :wink:

Thanks


#7

Well, I agree with the fact that ManageIQ should work without a designer, but Automation Engine development guidelines would help having a more homogeneous and consistent code repository, and thus productise snippets.


#8

@jhardy I think the biggest takeaway here is that it takes far too much time to really grasp how to work with the automate model. I am not saying we need to make it so simple that people end up with even more complex scenarios but until this “clicks” in your head, its hard to know whats up from down. I think that at the very least being able to do the visual UX changes just for the StateMachines should be a priority. Being able to double click on a “State” that takes you to the instance, that you can then edit the code. This alone would be a major breakthrough on how customers would perceive this. So basically, imagine flipping the current way the StateMachines are shown sideways, that give you a easier linear visual representation of whats happening. When I first layed eyes on ManageIQ/Cloudforms a few years ago it was very overwhelming to reverse trace what things were doing, this change alone would give us a huge leg up for customers to overcome this initial learning curve.


#9

@jhardy - Interesting idea. We would basically need a submission system for that. Things that immediately come to mind are GitHub issues and this forum, albeit with its own category. Problem is, we would need to seed it with a few examples so that others would get the idea.

Maybe this is something we can work on in conjunction with the Depot - or perhaps that’s version 2.1 of the Depot.


#10

@jsimonelli - How would you handle parameterized relationships ? Those where you rely on ${#var}… Following them seems quite difficult :wink:


#11

@fdupont - hey now, I was just trying to tackle the low hanging fruit… It would still be a hairy process to show the parametized relationships. Unless we just keep them as they are just a few boxes… Either way, you would still have to sort of reverse engineer what they are doing.

Unless we would do something that if you double clicked on the first box it would ask you which parameter to enter so it knows what to “follow” through the rabbit hole. Aghhh, makes me cringe just thinking about that.

let’s first focus on baby steps.


#12

@jsimonelli

This does sound like low hanging fruit for the UI. Can you paste in a screen shot or two to describe this clearly and I can whip up a new issue on github to get it done.

Thanks, Dan


#13

@dclarizio - Something like that ?


#14

@dclarizio at the very least if we start with what @fdupont suggested would already be a HUGE improvement in the way we “navigate” things.

With that said. What I wish we could do in the future is something along the lines of actually change the instance view of statemachines like the following.

Change the view to use something more familiar like a flowchart of each state.

StateMachine Flowchart

Add/Edit Schema

You could then hover in between each state to add to the schema to add a step in the state machine

Add/Edit/View Existing State

If you want to modify a specific state in the statemachine, simply double click the state to be modified.

Add/Edit/View StateMachine State Options

Which would then populate the state’s options.

Add/Edit/View Instance Options

Once you double click on the specific state it calls, then it would follow it and take you to the instance being called. Each object shown here would be able to be clicked, it would either bring up another view, or simply just option an Options table on the side of the current view to allow you to modify each options available to that object. (i.e. collect, would just bring up a textbox on the side so you can view current settings and modify it) I think it is important that when you hover over some of these objects it should either expand or allow you to see a hover message of the values, once clicked you can modify at your hearts content.

View/Edit Method

Finally, if you clicked on the instance’s method, it would simply go to the usual Method screen

Conclusion

The point is that it is always difficult to get people to wrap their head around how the whole automate process works. I am sure that in the event we start incorporating some of the other tools within the realm of Red Hat it would bring on some of these features. Until then I feel like at the very least being able to easily navigate down each rabbit hole is critical for the product to be easily adaptable without the current steep learning curve it requires.


#15

@jsimonelli - Awesome job. Really nice mashups.


#16

@jsimonelli +100 great mashups