Some of the ideas we have regarding automate database under version control are
Currently the AUTOMATE models are stored in Postgres in 7 different ActiveRecord classes
MiqAeDomain (Is the highest level namespace stored as MiqAeNamespace)
MiqAeNamespace (Folders to separate out the classes)
MiqAeClass (stores the schema which is a collection of MiqAeFields)
MiqAeInstance (stores the instance properties and values are stored in MiqAeValue)
MiqAeMethod (stores the method properties, script, input parameters (collection via MiqAeField))
MiqAeField (stores the class schema and method input parameters)
MiqAeValue (stores the instance values per instance/method based on the MiqAeField)
Whenever changes are made to the Automate Model the changes are committed into the DB and its not easy to do version control on the Automate Model.
In Anand release we have split up the Automate Model into Domains so that each domain can be owned by different groups (Community,Vendor,Customer,Site). Each domain has a priority when the Automate Engine executes a model it will search for instances/methods in the similar location (namespace+class) in other domains and pick up the one with the highest priority.
Another change that was implemented in Anand was the splitting of the automation_base.xml into separate single YAML files for each of the domain,namespace,class,instance and method. The field and value are directly stored in the YAML and don’t have a representation on the disk.
The Automate model files are currently stored in the filesystem but are imported at startup into the Postgres DB or via an explicit Export feature under Automate Explorer.
We are planning on using version control since the model files are now on the filesystem split up as individual YAML or .rb files. We can make each domain to be part of a version control system like GIT.
Here are some of the points to ponder as we make this transition
Advantages of Storing Automate in DB
- Active Record search/add/update/delete features
- Multi Zone Synchronization
- Column searches of data is easier
Advantages of GIT for AUTOMATE
Challenges of implementing GIT especially in the AUTOMATE use case
Files would have to be read(cached) when doing attribute specific searches
Need a GIT client in each of the zones to synchronize the changes (git clone)
Support Active Record style functions
Remove all the ID specific functions
Since the YAML file has the field and the values inside of it the MiqAeValue and MiqAeField classes would become irrelevant, so any methods calling into these classes would have to be updated
At a high level the goal is to do parallel development till we have all the functionality in the GIT Automate classes and then swap out the
with the YAML equivalents
The YAML classes would inherit from the MiqAeGit class which will provide all the ActiveRecord functions for Automate that are in use throughout the product in many different places.
There would be some modeling needed to store the GIT REPO information
Each domain would be stored in a separate repository.
The domain names and the GIT URL/credentials will be stored in a Relational Database (Postgres)
For the editable domains we would keep a history of all the commits
The Administrator would have to arbitrate between the list of available commits from different automate editors and which ones to be used by different automation request based on some criteria (production/QA).
The commit to use per domain would be stored in the $evm.root object so that the Automate engine can use the right commit to process a request and can support multiple commits.
Automate stores relationships which are a kin to the path in a filesystem. These are edited by the user so it is possible that these could be in mixed case. Automate internally ignores the case of the domain, namespace,class,instances, methods and fields in schemas and methods. Since the filesystem can be case insensitive or case sensitive, git would have to be configured to be case insensitive.
ignorecase = true
This should be done whenever we create a new domain. to keep this setting local to the automate domain and not for every git repo on the system.
Will the Automate Explorer be able to show the different commits and allow users to make changes to previous commits (amend commits inside of Automate Explorer)
There might be other things I might have missed, this is just a collection of thoughts on this topic.