Hi, our MIQ instance is unable for the last few days to read new VM instances or tenants from our company openstack provider. evm.log is filled with error from refresher job:
[----] E, [2020-11-20T10:48:09.800307 #10278:4da5f8] ERROR – : MIQ(ManageIQ::Providers::Openstack::CloudManager::Refresher#refresh) EMS: [openstack.kb.cz], id: [7] Unable to perform refresh for the following targets:
[----] E, [2020-11-20T10:48:09.800382 #10278:4da5f8] ERROR – : MIQ(ManageIQ::Providers::Openstack::CloudManager::Refresher#refresh) — ManageIQ::Providers::Openstack::CloudManager [openstack.kb.cz] id [7]
[----] I, [2020-11-20T10:48:09.875858 #10278:4da5f8] INFO – : MIQ(ManageIQ::Providers::Openstack::CloudManager::Refresher#refresh) Refreshing all targets…Complete
[----] E, [2020-11-20T10:48:09.876152 #10278:4da5f8] ERROR – : MIQ(MiqQueue#deliver) Message id: [97700615], Error: [no implicit conversion of nil into String]
[----] E, [2020-11-20T10:48:09.876275 #10278:4da5f8] ERROR – : [ManageIQ::Providers::BaseManager::Refresher::PartialRefreshError]: no implicit conversion of nil into String Method:[block (2 levels) in class:LogProxy]
[----] E, [2020-11-20T10:48:09.876368 #10278:4da5f8] ERROR – : /var/www/miq/vmdb/app/models/manageiq/providers/base_manager/refresher.rb:67:inrefresh' /var/www/miq/vmdb/app/models/manageiq/providers/base_manager/refresher.rb:11:in
refresh’
/var/www/miq/vmdb/app/models/ems_refresh.rb:103:inblock in refresh' /var/www/miq/vmdb/app/models/ems_refresh.rb:102:in
each’
/var/www/miq/vmdb/app/models/ems_refresh.rb:102:inrefresh' /var/www/miq/vmdb/app/models/miq_queue.rb:479:in
block in dispatch_method’
/usr/local/lib/ruby/2.5.0/timeout.rb:93:inblock in timeout' /usr/local/lib/ruby/2.5.0/timeout.rb:33:in
block in catch’
/usr/local/lib/ruby/2.5.0/timeout.rb:33:incatch' /usr/local/lib/ruby/2.5.0/timeout.rb:33:in
catch’
/usr/local/lib/ruby/2.5.0/timeout.rb:108:intimeout' /var/www/miq/vmdb/app/models/miq_queue.rb:477:in
dispatch_method’
/var/www/miq/vmdb/app/models/miq_queue.rb:454:inblock in deliver' /var/www/miq/vmdb/app/models/user.rb:290:in
with_user_group’
/var/www/miq/vmdb/app/models/miq_queue.rb:454:indeliver' /var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:104:in
deliver_queue_message’
…
I have no idea where to look. Fog.log doesn’t help and from that stacktrace I am not even sure if it is about openstack API problem or something local on MIQ cluster.
I have also found a similar trace here https://access.redhat.com/solutions/5295101 - but I can’t view details (not having redhat account)
Could anyone point me to the right direction please?
Thank you