VMWare provider does not work in Kasparov 2

Hi, I am looking for an advice regarding our problem.
We recently tried to upgrade from Ivanchuk (ivanchuk-3.20200226215823_dbb486d) to Kasparov 2 (kasparov-2.20210716154634_ddad0d6)
Everything seems to be working fine but our VMWare providers cannor refresh any data from VMWare with the following error:

[----] E, [2021-09-17T17:20:40.511472 #185236:2aac4e64290c] ERROR – : MIQ(ManageIQ::Providers::Vmware::InfraManager::Inventory::Collector#vim_collector) EMS: [VMware HCI vamgkb200], id: [1] Refresh failed
[----] E, [2021-09-17T17:20:40.511752 #185236:2aac4e64290c] ERROR – : [RuntimeError]: unknown VMODL type VsanDatastoreInfo Method:[block (2 levels) in class:LogProxy]
[----] E, [2021-09-17T17:20:40.512750 #185236:2aac4e64290c] ERROR – : /opt/manageiq/manageiq-gemset/gems/rbvmomi-3.0.0/lib/rbvmomi/type_loader.rb:105:in make_type' /opt/manageiq/manageiq-gemset/gems/rbvmomi-3.0.0/lib/rbvmomi/type_loader.rb:67:in block in get’
/usr/share/ruby/monitor.rb:235:in mon_synchronize' /opt/manageiq/manageiq-gemset/gems/rbvmomi-3.0.0/lib/rbvmomi/type_loader.rb:65:in get’
/opt/manageiq/manageiq-gemset/gems/rbvmomi-3.0.0/lib/rbvmomi/deserialization.rb:87:in deserialize' /opt/manageiq/manageiq-gemset/gems/rbvmomi-3.0.0/lib/rbvmomi/deserialization.rb:124:in block in traverse_data’
/opt/manageiq/manageiq-gemset/gems/rbvmomi-3.0.0/lib/rbvmomi/deserialization.rb:106:in each' /opt/manageiq/manageiq-gemset/gems/rbvmomi-3.0.0/lib/rbvmomi/deserialization.rb:106:in traverse_data’
/opt/manageiq/manageiq-gemset/gems/rbvmomi-3.0.0/lib/rbvmomi/deserialization.rb:90:in deserialize' /opt/manageiq/manageiq-gemset/gems/rbvmomi-3.0.0/lib/rbvmomi/deserialization.rb:119:in block in traverse_data’
/opt/manageiq/manageiq-gemset/gems/rbvmomi-3.0.0/lib/rbvmomi/deserialization.rb:106:in each' /opt/manageiq/manageiq-gemset/gems/rbvmomi-3.0.0/lib/rbvmomi/deserialization.rb:106:in traverse_data’
/opt/manageiq/manageiq-gemset/gems/rbvmomi-3.0.0/lib/rbvmomi/deserialization.rb:90:in deserialize' /opt/manageiq/manageiq-gemset/gems/rbvmomi-3.0.0/lib/rbvmomi/deserialization.rb:119:in block in traverse_data’
/opt/manageiq/manageiq-gemset/gems/rbvmomi-3.0.0/lib/rbvmomi/deserialization.rb:106:in each' /opt/manageiq/manageiq-gemset/gems/rbvmomi-3.0.0/lib/rbvmomi/deserialization.rb:106:in traverse_data’
/opt/manageiq/manageiq-gemset/gems/rbvmomi-3.0.0/lib/rbvmomi/deserialization.rb:90:in deserialize' /opt/manageiq/manageiq-gemset/gems/rbvmomi-3.0.0/lib/rbvmomi/deserialization.rb:119:in block in traverse_data’
/opt/manageiq/manageiq-gemset/gems/rbvmomi-3.0.0/lib/rbvmomi/deserialization.rb:106:in each' /opt/manageiq/manageiq-gemset/gems/rbvmomi-3.0.0/lib/rbvmomi/deserialization.rb:106:in traverse_data’
/opt/manageiq/manageiq-gemset/gems/rbvmomi-3.0.0/lib/rbvmomi/deserialization.rb:90:in deserialize' /opt/manageiq/manageiq-gemset/gems/rbvmomi-3.0.0/lib/rbvmomi/connection.rb:70:in block in parse_response’
/opt/manageiq/manageiq-gemset/gems/rbvmomi-3.0.0/lib/rbvmomi/connection.rb:70:in map' /opt/manageiq/manageiq-gemset/gems/rbvmomi-3.0.0/lib/rbvmomi/connection.rb:70:in parse_response’
/opt/manageiq/manageiq-gemset/gems/rbvmomi-3.0.0/lib/rbvmomi/connection.rb:92:in call' /opt/manageiq/manageiq-gemset/gems/rbvmomi-3.0.0/lib/rbvmomi/basic_types.rb:213:in _call’
/opt/manageiq/manageiq-gemset/gems/rbvmomi-3.0.0/lib/rbvmomi/basic_types.rb:76:in block (2 levels) in init' /opt/manageiq/manageiq-gemset/bundler/gems/manageiq-providers-vmware-42e38d3d8685/app/models/manageiq/providers/vmware/infra_manager/inventory/collector.rb:174:in wait_for_updates’
/opt/manageiq/manageiq-gemset/bundler/gems/manageiq-providers-vmware-42e38d3d8685/app/models/manageiq/providers/vmware/infra_manager/inventory/collector.rb:118:in monitor_updates' /opt/manageiq/manageiq-gemset/bundler/gems/manageiq-providers-vmware-42e38d3d8685/app/models/manageiq/providers/vmware/infra_manager/inventory/collector.rb:106:in targeted_refresh’
/opt/manageiq/manageiq-gemset/bundler/gems/manageiq-providers-vmware-42e38d3d8685/app/models/manageiq/providers/vmware/infra_manager/inventory/collector.rb:62:in vim_collector' /opt/manageiq/manageiq-gemset/bundler/gems/manageiq-providers-vmware-42e38d3d8685/app/models/manageiq/providers/vmware/infra_manager/inventory/collector.rb:48:in block in vim_collector_thread’

We tried to investigate but so far with no luck.
Anyone experienced such problem or has an idea how to resolve it?
Thanks a lot!

Hey @anders_ulrich just circling back on this, the root cause of this is using VMware’s gem against a vSphere version which it doesn’t support (namely 7.0U1 or U2).

We have since pushed 7.0U1/U2 to our ManageIQ/rbvmomi2 in Add support for vSphere 7.0U2 by agrare · Pull Request #749 · ManageIQ/manageiq-providers-vmware · GitHub so this should be resolved by that fix.

Hi @agrare,

we are having the same problem here with Kasparov and vCenter 7.0.3 version: kasparov-2.20210716154634_ddad0d6
I have 2 questions, first will this fix help with 7.0.3 version, and second are there any guidelines how to implement this fix?

Best regards,
Matko

Hi @mmirak yes this will also work with 7.0U3 (well technically we added support for U3 here Add support for vSphere 7.0U3 by agrare · Pull Request #7 · ManageIQ/rbvmomi2 · GitHub )

As for applying the fix, you can use the morphy-1 release when it comes out (we just tagged it) or you can use the morphy1-beta1 pre-release build.

Hi @agrare thank you for your response. So this means there will be no backporting onto Lasker or Kasparov 2?

Also, because we need to export DB from old and import to the new miq version, is it dangerous to jump from Kasparov directly to Morphy, skipping the Lasker on the way?

EDIT:
I have tested the new official Morphy version, first vm provision try didn’t worked but second(after the reboot) had.