Openstack heat, VMs are not assigned to service


#1

(on capablanca rc2)
When I deploy HEAT template, I don’t get assigned the new VMs to the service. Under VMs, there is only ‘No records found’

Ii must be my fault (some other heat template does get VMs assigned). But only obvious error I saw in logs was:

fog.log
[----] E, [2015-12-03T05:17:33.682835 #3101:5ddd95c] ERROR -- : Q-task_id([service_template_provision_task_139]) excon.error     #<Excon::Errors::NotFound: Expected(200) <=> Actual(404 Not Found)
excon.error.response
  :body          => "404 Not Found\n\nThe resource could not be found.\n\n   "
  :headers       => {
    "Content-Length"         => "52"
    "Content-Type"           => "text/plain; charset=UTF-8"
    "Date"                   => "Thu, 03 Dec 2015 10:17:33 GMT"
    "X-Openstack-Request-Id" => "req-c1ce2f2e-743d-4a55-8a8e-420cae73f503"
  }
  :local_address => "172.16.92.63"
  :local_port    => 55622
  :reason_phrase => "Not Found"
  :remote_ip     => "172.16.47.1"
  :status        => 404
  :status_line   => "HTTP/1.1 404 Not Found\r\n"
>

[----] E, [2015-12-03T05:17:33.781764 #3101:5ddd95c] ERROR -- : Q-task_id([service_template_provision_task_139]) excon.error     #<Excon::Errors::NotFound: Expected(200) <=> Actual(404 Not Found)
excon.error.response
  :body          => "404 Not Found\n\nThe resource could not be found.\n\n   "
  :headers       => {
    "Content-Length"         => "52"
    "Content-Type"           => "text/plain; charset=UTF-8"
    "Date"                   => "Thu, 03 Dec 2015 10:17:33 GMT"
    "X-Openstack-Request-Id" => "req-30c72fad-d491-47fe-98dc-bca46b23ce63"
  }
  :local_address => "172.16.92.63"
  :local_port    => 55623
  :reason_phrase => "Not Found"
  :remote_ip     => "172.16.47.1"
  :status        => 404
  :status_line   => "HTTP/1.1 404 Not Found\r\n"
>

The annoying thing - I cant find req-UUID in openstack logs at all. Can someone point me to code/flow responsible to assigen VM started by heat with MiQ service? Or to better debug technique than tcpdump ip traffic, to see whitch service did return 404, and set verbose/debug log level for that service?


#2

tcpdump shows request (to heat port 8004):

GET /v1/f737e0ffcd1e42538bb7dc23941c89df/stacks///template

404 Not Found
The resource could not be found.

Replaying with curl, and adding valid stack name and uuid helps:

GET /v1/f737e0ffcd1e42538bb7dc23941c89df/stacks/jc2/f2bdc476-ae96-43f9-96f4-6430d2f2fdeb/template

{"outputs": {"instance_name": {"description": "Name of the instance", "value": {"get_attr": [ ...

I’m not sure what to do/try next. Hint?


#3

Looking at tcpdump closer, only a few tcpstreams from dump seems relevant.
TCP stream 10 and 11 - HTTP 404, so I guess MiQ tries to do some check. Hm, I have two old heat services, which were deleted at Openstack side. Maybe MiQ cleared (newer stored?) name and uuid for those two?
TCP stream 12 - HTTP POST, stack create, returns 204
TCP stream 13 - HTTP GET, with name uuid of the new stack, returns 200, and looks ok to me. No VM uuid there yet - stack is created, VMs are being created.
TCP stream 13 - AMQP notification, compute.instance.create.start and compute.instance.create.end. The VM uuid is here.

Should I assume the AMQP notification was not fully understood by the MiQ?


#4

@bill Can you look into this when you get a chance?