Creating a Pull Request for enhancements to ManageIQ Automate Domain


#1

We typically have a use case to edit or add automate classes or instances into the ManageIQ domain when we are introducing a new provider or adding a new feature. After we make the changes to the Automate model we want to commit the changes to the git repository to create a pull request.

When we have to edit existing instances/classes or add new instances/classes to an Automate Domain like ManageIQ we have 2 options

  1. Edit the Model using the Automate Explorer
  2. Make the changes directly by manually editing the YAML files on the disk

Option 1 requires that you unlock the ManageIQ domain before you make the changes and lock it back after you have made the changes.

Option 2 is prone to editing errors and also flags unnecessary changes when trying to keep the files in sync on the disk with the data in the DB.

If we use Option 1 from above we will make all the changes using the Automate Explorer, this might be a slow process but it ensures that the data can stay in sync and prevents user errors. After we are satisfied with the changes we can export the ManageIQ domain to the disk so we can commit the changes to the repository.

Steps

  1. Unlock the ManageIQ domain
  2. Make the changes using the Automate Explorer
  3. Test all your changes
  4. Lock the ManageIQ domain
  5. Export the ManageIQ domain
    bin/rake evm:automate:export DOMAIN=ManageIQ EXPORT_DIR=/your_src_dir/manageiq/db/fixtures/ae_datastore/ OVERWRITE=true
  6. git diff (Should only list your changes)
  7. Commit your changes to build the PR

If you use Option 2 where you have manually edited files on the disk, the steps would be
(1) Make manual changes to the YAML files
(2) Import it into the ManageIQ domain
(3) Test and make sure your changes work
(4) Export the ManageIQ domain using
bin/rake evm:automate:export DOMAIN=ManageIQ EXPORT_DIR=/your_src_dir/manageiq/db/fixtures/ae_datastore/ OVERWRITE=true
(6) git diff (should only show your changes)
(7) Check in the changes into the git repository

Step (4) is very critical because it uses the consistent YAML export package to create the model files on the disk. So things like

  • New Lines (Carriage Return, Line Feeds)
  • Double Quotes versus Single Quotes
  • Other escape sequences
  • Inconsistent names, the filename on disk can be different from the internal name stored in the YAML files.

Are all consistent, when you hand edit these files you might not see these nuances.

To unlock the ManageIQ domain use the following command from a rails console
MiqAeDomain.where(:name => ‘ManageIQ’).first.update_attributes(:system => false)

To lock the ManageIQ domain use the following command from a rails console
MiqAeDomain.where(:name => ‘ManageIQ’).first.update_attributes(:system => true)

Please note that after you do a round trip of the ManageIQ domain you would still see that db/fixtures/ae_datastore/ManageIQ/domain.yaml would be different because it has the tenant_id which is different from appliance to appliance. The tenant_id is used only when a customer does a backup/restore, when doing normal imports the tenant_id is ignored and we use the actual tenant_id from the appliance where the import is being run.


#2

This is very helpful :+1:. I normally took option 2. Yes step (4) is important although I often forgot. When I took option1 I directly modified the lock flag through NaviCat, but this is only a personal preference.


#3

Wow, this would be really useful.

I have always used option 1 but have not tried the step exporting the domain with the rake export command. What I have done is to export it into a ZIP file via UI then extract the file and copy the changes into the source directory. The rake command would be much easier in this case.

Nice written!


#4

The __domain__.yaml file from the ManageIQ folder should never be included in the PR.


#5

Starting with the E release the system attribute has been removed from MiqAeDomain and source has been added in its place. To unlock a system domain you would have to change source to user and to lock it back use system

To Unlock the ManageIQ domain use the following from a rails console

MiqAeDomain.find_by_name(‘ManageIQ’).update_attributes(:source => ‘user’)

To lock the ManageIQ domain use the following from a rails console

MiqAeDomain.find_by_name(‘ManageIQ’).update_attributes(:source => ‘system’)