[SOLVED] Git-flow for Automate Code


#1

Hello,
I am trying to upgrade our workflow to develop automate code to a git-flow workflow (https://www.atlassian.com/git/tutorials/comparing-workflows/gitflow-workflow) and I was wondering if someone already did it and can point me to some resources on their solution.

I have a rough idea on how it could work, but it would require quite a bit of scripting to do it, therefore I thought I’d ask the community first, to validate my assumptions and get another opinion. The main problems I am facing are:

  1. There is no way to export or import on a more granular level than a single automate domain.
  2. Since higher priority domains overwrite everything from lower priority domains, any domain modeling a feature-branch MUST only contain changed files.
  3. I cannot use the git-import feature for feature-branches, because I need to import a dirty working copy.

A bit of Background:
At the moment we have a DEV-Cloudforms and a PROD-Cloudforms appliance. Every week we do an export from DEV, commit the whole export to git and import the same commit PROD.
All the development happens in the WebUI and we want to have the possibility to do small changes in the UI (espacially changes in dialogs, etc.)

The problems:

  • At the moment git is just a documentation tool. There is no save way to contribute via git and not have that change overwritten or producing a merge-conflict.
  • All the code is pushed to PROD all the time, If a change is not yet ready for production, it has do be manually disabled in code ( e.g. with an if-statement) before the export and enabled again after the export. => Doing a production deployment at the wrong time could break everything
  • This process is against all coding best practices :slight_smile:

Implementation:

  • Have a complicated git repo, with at least 3 types of branches.
    • release-branches: one for PROD and one for DEV
    • helper-branches: this is where automated commits go
    • feature-branches: this is where humans work
  • Export everything from DEV
    • do really small commits (e.g. for each class) to the helper-branches
    • at this point all the manual changes are in the helper-branches
  • merge the helper-branches to the relese branches
  • Import PROD
    • release-branches should work fine
    • import feature-branches, after removing all unchanged files
  • do_magic()
    • create new branches as needed and prune old ones
    • automatically maintain all the branches
    • keep all the merges free from any possible conflicts

As I said, the goal for this post is to validate my assumptions and to see if someone has already done something similar already. The rest is just background information and my thoughts on the topic.


#2

Not exactly the same but we are using git repo with branch too.
We have one Prod manageiq that rely on prod branch, and one dev manageiq which use development branch. When we are ready to push new functionalities, we merge branch and got a git import.
You can also do webhook in order to trigger the git import on your dev server when someone do a push.


#3

Hi @buc
Currently we are using such branching model, but big problem for us is that configuration for domain stored with code:

If it wasn’t for this problem, we would work in the following way:

  1. in my own tenant on the development appliance I connect a feature-branch domain from the repository.
  2. I’m working on it.
  3. Doing PR to the devel branch, which is connected on the same appliance, but applies to all tenants.
  4. After running the tests, the code goes to the master branch.

Devel-branch and Master-branch domains have the same configuration in _domain_.yaml file, but feautre-branch domains have different configuration for tenant_id. For this reason, we do cherry-picks from feature-domains to devel-domain.

Storing configuration with code is known anti-pattern and I think we should pay attention of MIQ developers on this.


#4

@LorkScorguar: That is exactly what I plan to do with the release-branches.

@igortiunov: I was not aware that automate domains can be scoped to apply to just one tenant (reference: https://pemcg.gitbooks.io/mastering-automation-in-cloudforms-4-2-and-manage/content/tenancy_and_automate/chapter.html). It is definitely a much cleaner solution, than my approach.

Regarding the __domain__.yaml:
I guess it would break the import if you just add it to .gitignore or something similar? :slight_smile:
However the rake-import-task has an option to specify the tenant (https://github.com/rhtconsulting/cfme-rhconsulting-scripts/blob/master/rhconsulting_miq_ae_datastore.rake). It looks like it does, what you want and having that function in the git import is probably just a matter of exposing the parameter(?)

Thanks for the feedback


#5

@buc
ssh on appliance is not a solution. I am using domain refresh via API: http://manageiq.org/docs/reference/latest/api/reference/automate_domains

I am sure that the domain configuration should not be together with the code. This configuration need to specify during the import from the git repository or at the time of updating from the repository.