Proposal: the "ManageIQ Extensions Depot"

Developers
#1

Given that ManageIQ isn’t just a user tool but an extensible platform, I’d like to propose that we create an area on manageiq.org that allows community members to create and publish their extensions, as well as navigate others’ extensions and use them.

I think it makes sense to have the marketplace be GitHub-driven, given that our main repository and issue tracking it done on GitHub.com. This allows us to use existing tooling and makes the most sense from the perspective of project and community agility.

I’d like to propose the following:

  • create repositories for individual projects around extending ManageIQ
  • create HTML page that functions as an interface to the extension repositories
       - allows people to find and navigate extension projects
       - gives directions for how to create your own project
  • Create approval process for repos from community members and find a way to wrap them under the ManageIQ account
       - There can be a “free-for-all” category where repos stay under individuals’ accounts, but we push links to them into the marketplace
       - There can be a category that denotes some type of approval or endorsement from the community, and those repos are forked/cloned/whatever under the ManageIQ account

If you have specific thoughts on the best way to do this, add your comments here. We can start developing the HTML interface in the manageiq.org repo, which will start out as essentially a list of links with basic descriptions, plus directions on creating a repository and submitting it for approval.

Looking forward to it!

  • John Mark
    ManageIQ Community Leader
1 Like
#2

Hi John Mark,

From my perspective, from the user’s point of view, there are a few things that are important:

  1. I can see what modules are popular & well rated
  2. I can search modules to find those which do what I need
  3. I want to be able to comment on a module or give feedback to the module author when something isn’t as I expected (whether it’s a missing feature or a bug)

From the point of view of the module author:

  1. I want to be able to use my preferred toolchain (if that’s github, I have my project in github)
  2. I want to have an opportunity to promote my module on manageiq.org (some way to announce them, have the module appear in a specific category, a way to have the project “promoted” or “featured”)
  3. I would like to have a direct relationship with my users (comments & dialog, etc)
  4. I want to be able to manage versions (module version 0.2 corresponds to ManageIQ version 7.6 or whatever, when a new version comes out, providing a way for users to upgrade the module)

Finally, from the point of view of ManageIQ:

  1. I want to ensure that modules are open source
  2. I want to have a quality standard that modules must meet - nothing malicious or obviously broken/untested
  3. I would like to have a process be mostly automated for handling updates, creating the marketplace page, search, etc

Certainly, you can do something simple with a yaml file on manageiq.org listing repositories where extensions can be found, which we parse & mark up as the extensions marketplace. We would be delegating everything to do with managing the extensions to their authors (comments, bugs, etc). We would not have any popularity or rating stats, and no easy way to make people aware of updates which are available.

The simplest way to do this, since this is a Middleman site, would be to use the Middleman Extensions module as a means to do that, we might also consider oo-index from OpenShift Origin, which is a similar “sit on GitHub” interface

There are a few other options which might make sense - something forge-based, but which delegates the source code/issue handling and just provides the “extensions” front end (comments, ratings, lifecycle management, download stats). I guess it comes down to how quickly we want this functionality availmable, and how important it is to have popularity, ratings, comments, etc - which I think are important to both users and module creators - and how much of the work we want to do ourselves versus building on existing projects.

Thanks,
Dave.

#3

I fully agree with this approach.

  • create repositories for individual projects around extending ManageIQ

The repositories wouldn’t have to be limited to being official ManageIQ repositories.

I believe it would be better if they weren’t limited to the official repositories, actually. Using user namespaced repositories on GitHub (or possibly anywhere else) would probably be pretty powerful.

Ruby Gems are done in this manner (although most are on GitHub just naturally).

  • create HTML page that functions as an interface to the extension repositories
  • allows people to find and navigate extension projects

Makes sense. The extensions have to be found somehow.

I’m thinking something similar to:
http://directory.middlemanapp.com/

  1. Browser-based, client-side search via JS
  2. Categories / tags (there shouldn’t be too many)
  3. Very basic extension data on the page; extended info would be in a subpage (either hosted on GitHub or integrated on ManageIQ.org)
  • gives directions for how to create your own project

This would be a link to a developer document. Content would be required.

Create approval process for repos from community members and find a way to wrap them under the ManageIQ account

This would be a pull request to a directory YAML file. As long as the extension follows guidelines, the pull request for it would be accepted.

  • There can be a category that denotes some type of approval or endorsement from the community, and those repos are forked/cloned/whatever under the ManageIQ account
  • There can be a “free-for-all” category where repos stay under
    individuals’ accounts, but we push links to them into the marketplace

Perhaps we want a top-level category as “verified” and “unverified” or something to that effect. (Not these particular words, but the concept.) Other words might be “reviewed” & “unreviewed”; “approved” & “upcoming”; etc.

…However, I don’t think we would have many extensions to begin with, so I think it makes sense to review each before accepting anyway.

It might make sense to just highlight a certain few instead, like “featured”.

Suggested implementation

Centralized YAML file in the manageiq.org repository; it contains all the information for the extensions. New ones would be added via pull requests. The file would be in manageiq.org/data/extensions.yml most likely. The documentation would mention this and provide a link to edit the file and make a pull request on GitHub.

The ManageIQ website repo would understand this file and present the page, once we figure out the detail of the data (and I write a bit of code to handle it).

Moving forward (requirements)

To rapidly develop this solution, the following is needed:

  1. Several example extensions (minimum: 3; hopefully more)
  2. Data we’d like to present on the page for each extension; so far I’d suggest the following (and please provide more):
  • Name
  • Version
    • Link to each version
    • (versions and links could be automated if GitHub is assumed)
  • Project link
  • Summary
  • Featured status
  • License
  • Link to source code
  1. Starter set of categories
  2. Common installation instructions (on another page)
  3. “Developing extensions” documentation

If it is at all possible to implement a version by the end of this month, this information must be decided upon by the end of this week. (It could be modified throughout the following weeks, but it needs to be pretty solid before next week.)

Even still, the end of the month is a pretty aggressive timeline.

#4
  1. Are there really going to be so many extensions that this is needed?
  2. How is it going to be tracked? (Hint: Views are not the correct metric. Downloads are not the correct metric, either.)

I want to be able to comment on a module or give feedback to the module author when something isn’t as I expected (whether it’s a missing feature or a bug)

  1. Commenting might be useful (or might not), but it would require a lot more implementation (spam checking, people monitoring the comments, etc.)
  2. Missing features and bugs could be filed in an issue tracker. Issue tracking can be linked from the page.

Regarding comments: It could be possible to spin-up another Discourse instance to link in for commenting per extension, if there are enough extensions to be useful. Or we could re-use this same instance if there’s a way to have two content embeds (although I don’t think there is currently…)

Alternatively, we could set up a Juvia instance to handle the comments… but that’s yet-another-piece-of-infrastructure.

…but, realistically, I don’t think comments are really useful, especially during launch.

I want to ensure that modules are open source

We need to add “license” and a link to the source code as fields in the data.

I would like to have a process be mostly automated for handling updates, creating the marketplace page, search, etc

It could be possible to require people to use GitHub for extensions, and have integration for GitHub releases, if we want to have all versions for all extensions automatically updated (without requiring pull requests, that is). Requiring this would make things a lot easier.

(There could also be limited support for non-GitHub too, where all links are specified and versions are handled manually.)

#5

I think we may be able to learn from the ruby and chef communities here.

Both are for the most part hosted in github (or similar git hosting service).
An interface for ruby gems is http://rubygems.org/ which is more of a reference, and https://www.ruby-toolbox.com/ that aims to help users determine which gem is the best one to use.

The interface for chef is http://community.opscode.com/cookbooks

I’m sure we can some ideas on how we want to handle finding widgets, choosing widgets, and allowing users to know which fork is officially supported.

#6

+1 on doing it like other communities. In addition to being tested and worked out, there is also a familiarity people may have that would make adoption easier. There are a lot of repos for cookbooks…here’s one for travis-ci, for example: https://github.com/travis-ci/travis-cookbooks

Another thing I like is how Sublime Text manages their plugins. Each plugin is it’s own repo and anyone can have a repo. They are presented by a tool called Package Manager: https://sublime.wbond.net

I can envision that lots of smaller repos could be made to handle a specific feature, and other repos could be made to aggregate those. With the domain support in automate, it’s possible that each feature could be in its own domain.

#7

New development! Chef has released their collaborative site as an open source project. You can see the code here - https://github.com/opscode/supermarket

and see the implementation here - https://supermarket.getchef.com/

I think it would be good to evaluate this for our use.

#8

Bumping this back up, as there is new activity to report.

The name will be “ManageIQ Extensions Depot”

The repo is here: http://github.com/manageiq/manageiq_depot

This is the format of individual extensions in the repo:

  • /images directory - where you place screenshots
  • /scripts directory - where you place, you guessed it, code
  • content.md - descriptive text and instructions for whoever will want to use it
  • metadata.yaml - this contains the following definitions:
    • author: username (do we want definitions file to map users to names?)
    • collaborator: comma,delimited,text
    • date: yyyy-mm-dd
    • name: free text
    • slug: text_no_spaces (how it will appear in a URL, if we ever need it)
    • tags: comma,delimited,text
    • description:
      this is free form text that features paragraphs and whatever else is necessary to give a brief description that will be displayed on the main index page
    • miq_ver: manageiq_release
    • dependencies: comma,delimited,list

We’re looking to mock up an index page very soon. If you’d like to help us by adding your extensions, it would be much appreciated!

Simply fork the repo and issue a pull request when you’re ready to add your extensions.

-JM

3 Likes