[Proposal] [Provider] Yarn


Hi community :grinning:

How do you think about create a Yarn Provider?
I would like to be able to create a new provider. Is there anyway to do it?

I wanna share with you this proposal document
Proposal Document

I look forward to your feedback !!!
Thanks @sergio_ocon for support

[Proposal] [Provider] HDFS

Hi Alberto,

Just to clarify, what you are proposing is to manage the infrastructure that is needed to create a hadoop cluster, and that is managed using a connection to yarn.

I understand that there is also the need to get full metrics to add value to the provider.

Is this a generic provider or there is a need for a different provider per vendor supporting yarn?



Hi Sergio,

Yeahh, I wanna manage a Hadoop Cluster where is minimal configuration is HDFS provider and Yarn Porvider.

Yarn provider is generic for each vendor (MPR, Pivotal, Horton Works, Cloudera…) they are installed with Apache Yarn project.


Is authentication shared? Is there any relationship between them?

I mean, in AWS when you authenticate you get access to all the providers within (EC2, EBS, etc).



Yes normally there is not authentication or is managed by Kerberos etc…



What do you think @gtanzillo, would it make sense to try this?


Hello @aljesusg

Thanks for offering to do this. @blomquisg, @Fryguy and I are still working on a provider incubation process that we will be publishing when complete.

In the meantime, I suggest you start working on this provider in your own repo. If you have any questions, @blomquisg should be able to help.

When this provider will become capable enough to be included in the ManageIQ Build, @Fryguy and @blomquisg will work with you on that. In so doing, you will help us crystallize the provider incubation process!



Great new !!! thanks @chessbyte :grin:


Fog-hadoop Gem in Fog organization https://github.com/fog/fog-hadoop