Getting to a standard Rails directory structure in ManageIQ repo

Developers
#1

Problem Statement

In the ManageIQ Repo, the main Rails app lives inside the vmdb directory. This causes all sorts of issues with SaaS tools, such as Gemnasium, that expect the Rails app to live in the root of the repo. Changing our repo to be more standard will also make it easier for Ruby on Rails developers to join the community.

Suggested Approach

In the root directory of our repo, we have the following directories:

  • build
  • ca
  • host
  • lib
  • system
  • vmdb

At a high-level, we should create a number of new repos and move all subdirectories except vmdb out.

Step 1 - Moving directories into new repos.

  • manageiq_appliance repo should house build, ca, and system directories
  • manageiq_smart_proxy repo should house host directory
  • manageiq_lib repo should house lib directory

Step 2 - Standardization

  • Move the contents of the vmdb directory to live at the root of the repo
  • Remove the empty vmdb directory

Step 3 - Gemification

  • Make gems out of everything in manageiq_lib
  • Once manageiq_lib repo is fully gemified, it should be empty - so remove it

Issues

  • How will build process work?
  • How will developers be impacted?
  • How will automated tests be impacted?
#2

From a development perspective, I would expect moving directories to multiple repos would make running from source more difficult.

#3

As a first pass, it might be nice to become more gem friendly in lib.

  1. move the files around to be in actual packages. The current directory structure does not align as a gem.
  2. make the directory structure similar to what is in a gem.
  3. change the code to not have all these requires with ../../../ in them.
  4. point to the lib directory from the Gemfile.

The directory structure would probably be:

lib/appliance_console.rb # temporary shim, only 1 require statement
lib/appliance_console/lib/appliance_console.rb # gem file to require all sub classes
lib/appliance_console/lib/appliance_console/*.rb # rest of the gook
lib/appliance_console/test/ #tests

All over our code, we would change the require '../../../lib/applinace_console' to a simple require 'appliance_console'

Next step is to move tests and determine dependencies.
Then it would be a trivial step to move to “gems”. do note, all the code is in the current repo.

vmdb/Gemfile:

gem 'appliance_console', :path => '../lib/appliance_console'

Impact is mostly positive

It is a little frustrating to find methods for me right now.
Some of them are hiding in a manageiq specific gem. and it takes me a while to track it down. Think the solution may be as simple as grouping all of the manageiq specific gems in the Gemfile.

Tests will be quicker and simpler to run

Less cognitive load since the structre is like every other gem known to man.

I think a lot of developers will have a locally modified Gemfile. With a ton of path statements in there. (You do this when you are making changes across files and want to point to local changes in multiple gems)

Metrics will be split up into each area. So we can make progress with code coverage and quality separately. this is very nice.

@rpo

I agree with the difficulties of having code span across repos. I guess the goal is to have local code local. Seaching across repos is a pain.

Gemfile and path: is quite easy and alleviates most of the pain with modifying files in multiple repos.

But dependencies across multiple repos could be tricky, so we may want to go the submodules route. having a directory at the top level being its own gem.

#4

@Fryguy @tenderlove @matthewd @blomquisg @jrafanie @abellotti @kbrock @rpo @gmccullough @dclarizio - Let’s restart this discussion as I really want to get this done this summer.

#5

@chessbyte - done :smile: And, I barely lifted a finger

#6

Nice, and there’s still time left in the summer to fix the chaos we just caused.