Why does miq use it's own worker manage system instead of sidekiq?


#1

what are the benefits?


#2

Hi @adam,

There are a number of reasons.

  1. The worker system pre-dates Sidekiq.

  2. Not all workers do Actor-model like tasks. For example, the ScheduleWorker doesn’t process queue items. For these purposes, we could probably switch to a worker process tool like the god gem. Sidekiq could then be another worker process. So, for the purpose of discussion, let’s limit the discussion to just the queue based workers where Sidekiq could make sense.

  3. The queue based work items are not structured like Actor model tasks. There would be a significant rewrite to move everything to that format.

  4. The queue based work items are not thread safe. Sidekiq has a pool of workers running in threads, so all of that would need to be tested and probably would need many concurrency fixes.

  5. The MiqQueue class couples both message routing (roles, zones, etc) and message processing. Sidekiq is just message processing. There is ongoing work at the moment to decouple the MiqQueue, but it is very difficult. Coincidentally, I recently spoke with Mike Perham at the last RailsConf about what it would take to get ManageIQ onto Sidekiq, and this point was the one we discovered together that I hadn’t realized beforehand.

I hope that answers your question!

Jason


#3

Hey. Greatly appreciated.

I have learned a lot from the answers.

Thank you so much.