Vim method missing

casius · July 3, 2016, 9:52am

Hi,

After upgraded from Boninik to Darga-1 appliance, I tested on my old adddisk_to_vm code which I took reference from the CloudFormPOC that worked well previously.

In this case, when I worked on negative testing, upon executing the following codes, I got a “VimFault”“method_missing” that was triggered by "vm_storage"error:
prov = $evm.root[‘mig_provision’]
vm = prov.vm
vim_vm = vm.object_send( ‘instance_eval’, ‘with_provider_object( | vimVM | return vimVM }’ )
vim_vm.addDisk( “[#{vm.storage_name}]”, size*1024)

I appreciate if someone could point me to how vm.addDisk works? Is the request tap on VMware API? Do we need VDDK for this to work? Thanks.

Cheers,
Casius.

jprause · August 3, 2016, 1:31pm

@blomquisg can you review this question from @casius and forward to a SME if necessary.

gmccullough · August 3, 2016, 4:25pm

@casius

I would suggest using the add_disk method that is exposed from automate instead of the method you are using.

github.com

ManageIQ/manageiq/blob/be21d059f104be93b39061b0918b608fd77e65d0/lib/miq_automation_engine/service_models/miq_ae_service_manageiq-providers-vmware-infra_manager-vm.rb#L11


module MiqAeMethodService
class MiqAeServiceManageIQ_Providers_Vmware_InfraManager_Vm < MiqAeServiceManageIQ_Providers_InfraManager_Vm
  def set_number_of_cpus(count, options = {})
    sync_or_async_ems_operation(options[:sync], "set_number_of_cpus", [count])
  end


  def set_memory(size_mb, options = {})
    sync_or_async_ems_operation(options[:sync], "set_memory", [size_mb])
  end


  def add_disk(disk_name, disk_size_mb, options = {})
    sync_or_async_ems_operation(options[:sync], "add_disk", [disk_name, disk_size_mb])
  end


  def create_snapshot(name, desc = nil, memory = false)
    snapshot_operation(:create_snapshot, :name => name, :description => desc, :memory => !!memory)
  end


  def remove_all_snapshots
    snapshot_operation(:remove_all_snapshots)
  end

You may also want to test against Darga-2

casius · August 5, 2016, 3:17am

Hi @gmccullough. May I know how can this method be called from MIQ lifecycle in “Automate”? Thanks!

gmccullough · August 8, 2016, 6:15pm

Assuming you have a handle to the VM object like you show above:

prov = $evm.root['mig_provision']
vm = prov.vm

The call would be:

vm.add_disk( "[#{vm.storage_name}]", size*1024)

Just wanted to point out that this is just supported on VMware provider VMs at the moment.

casius · August 10, 2016, 3:59am

@gmccullough, thanks for your response. At the moment, when I initiated vm.addDisk(“storage_name”, size), the ruby method add_disk() in apps/models/vm_or_template/operations/configuration.rb will be called. As I am new to ruby on rails, I am wondering how the methods in lifecycle state could correctly locate the right add_disk method. I do also noticed that the example given used vm.add_disk() instead of vm_addDisk(). Is this the one that make the difference when we call a method in /lib over one in /app/…/… ? Thanks.

gmccullough · August 10, 2016, 6:30pm

The initial logic you reference is directly accessing the VimVM object which is calling the addDisk method here:

github.com

ManageIQ/manageiq/blob/910137d371fc97d87cb6464f430d08ecaab6098f/gems/pending/VMwareWebService/MiqVimVm.rb#L734


#    file names will be generated as appropriate.
#
# Options available:
# thin_provisioned - if new disk is thin (grow on demand, allowing storage overcommitment) or thick (pre-allocated)
# dependent       - if new disk is dependent (usual one) or not (meaning no delta file and no live snapshots of running VM)
# persistent      - if new disk should really save changes or discard them on VM poweroff
#
# P.S. Good overview of use cases for dependent / independent and persistent / nonpersistent disks
# can be found at http://cormachogan.com/2013/04/16/what-are-dependent-independent-disks-persistent-and-non-persisent-modes/
#
def addDisk(backingFile, sizeInMB, label = nil, summary = nil, options = {})
  # Remove nil keys if any, since the next line may not work
  options.reject! { |_k, v| v.nil? }
  # Merge default values:
  # - persistent is set to true to be backward compatible
  # - thin_provisioned is set to false explicitly since we call to_s on it further, so nil will not work for us
  options = {:persistent => true, :thin_provisioned => false}.merge(options)
  ck, un = available_scsi_units.first
  raise "addDisk: no SCSI controller found" unless ck


  vmConfigSpec = VimHash.new("VirtualMachineConfigSpec") do |vmcs|

The add_disk call from the automate service model will call this add_disk method:

github.com

ManageIQ/manageiq/blob/910137d371fc97d87cb6464f430d08ecaab6098f/app/models/vm_or_template/operations/configuration.rb#L80


def disconnect_floppies
  raw_disconnect_floppies
end


def raw_add_disk(disk_name, disk_size_mb, options = {})
  raise _("VM has no EMS, unable to add disk") unless ext_management_system
  run_command_via_parent(:vm_add_disk, :diskName => disk_name, :diskSize => disk_size_mb,
      :thinProvisioned => options[:thin_provisioned], :dependent => options[:dependent], :persistent => options[:persistent])
end


def add_disk(disk_name, disk_size_mb, options = {})
  raw_add_disk(disk_name, disk_size_mb, options)
end


def raw_attach_volume(volume_id, device = nil)
  raise _("VM has no EMS, unable to attach volume") unless ext_management_system
  run_command_via_parent(:vm_attach_volume, :volume_id, :device)
end


def attach_volume(volume_id, device = nil)
  raw_attach_volume(volume_id, device)

Which then calls raw_add_disk and routes the call to vm_add_disk on the provider. Ultimately they both end up calling the same MiqVimVM.rb method.

The big difference is that the sync_or_async_ems_operation method from automate will ensure the work is routed to the proper worker and zone where the appliance is configured to talk to the provider.

Also the object_send( 'instance_eval' logic is hacking around the automate service model and is not a recommended approach.

casius · August 17, 2016, 5:41am

Hi @gmccullough,

I tested on the suggested method add_disk(). I observed that when disks was not successfully added in VMware, errors were not shown on MIQ. In fact, no exceptions were thrown in MIQ, which may not be ideal as errors would then only be discovered much later. Thanks for your response.

Cheers,
Casius.

gmccullough · August 17, 2016, 12:45pm

The add_disk method takes an optional third parameter where you can specify a synchronous flag for the operation.

You might want to try running with that flag set to true:

vm.add_disk( "[#{vm.storage_name}]", size*1024, :sync => true)

casius · November 7, 2016, 10:10am

Hi @gmccullough,

Currently add_disk method adds a new independent disk. I am wondering to create a disk where diskMode is “non-independent”? I tried the following method but it does not work:
vm.add_disk( “[#{vm.storage_name}]”, size*1024, {:sync=> true, :dependent => true} )?

Your help is much appreciated. Thanks!

Cheers,
Casius.

gmccullough · November 8, 2016, 2:38am

The automate add_disk method is only passing along disk_name and disk_size as you can see here:

github.com

ManageIQ/manageiq/blob/29257412638651dd3d1832ccbc4a6ce10e70e8ad/lib/miq_automation_engine/service_models/miq_ae_service_manageiq-providers-vmware-infra_manager-vm.rb#L11-L13


def add_disk(disk_name, disk_size_mb, options = {})
  sync_or_async_ems_operation(options[:sync], "add_disk", [disk_name, disk_size_mb])
end

However, the backend method supports passing a few additional properties: :thinProvisioned, :dependent and :persistent

See here:

github.com

ManageIQ/manageiq/blob/29257412638651dd3d1832ccbc4a6ce10e70e8ad/app/models/vm_or_template/operations/configuration.rb#L74-L78


def raw_add_disk(disk_name, disk_size_mb, options = {})
  raise _("VM has no EMS, unable to add disk") unless ext_management_system
  run_command_via_parent(:vm_add_disk, :diskName => disk_name, :diskSize => disk_size_mb,
      :thinProvisioned => options[:thin_provisioned], :dependent => options[:dependent], :persistent => options[:persistent])
end

Please open a git issue and we can work on passing the additional hash values to support setting the other fields.

switchboard.op · May 18, 2017, 7:05pm

Is there a way to call that back-end method from the Automate Engine?

I’m new to MIQ and don’t quite grasp when or how the method that calls raw_add_disk() would be accessed if not from the AE.

I am trying to thinly provision a disk from an Automate Method and it looks like Issue #12488 hasn’t see any movement.

gmccullough · May 18, 2017, 7:47pm

@switchboard.op Thanks for the update, the issue was actually resolved but the PR did not get linked to the issue but to a bugzilla ticket so there was no update.

I have associated the issue to merged PR #14350 and closed the issue.

Thanks again.

switchboard.op · May 18, 2017, 8:16pm

Thank you for the info @gmccullough!

Unfortunately it looks like the version of CloudForms we’re running is based on Darga and doesn’t have these changes rolled in.

Thankfully it’s only three lines.

switchboard.op · May 18, 2017, 9:43pm

Instead of pulling in upstream code and doing a rake compile, I just called sync_or_async_ems_operation() directly:

# Get vm object
vm = $evm.root['vm']

# Get the size for the new disk from the root object
size = $evm.root['dialog_size'].to_i
size_in_mb = size * 1024

# Add disk to a VM
unless size.zero?
  $evm.log(:info, "Creating a new #{size}GB disk on Storage: #{vm.storage_name}")

  # In CFME 5.8 this should work
  # vm.add_disk("[#{vm.storage_name}]", size_in_mb, thin_provisioned: true)

  # This is a hack to make thin provisioning from AE work in CFME 5.7
  vm.sync_or_async_ems_operation(true, "add_disk", ["[#{vm.storage_name}]", size_in_mb, thin_provisioned: true])
end

exit MIQ_OK