Openstack虛擬機器關機後無法啟動(start).md
阿新 • • 發佈:2018-12-13
Openstack虛擬機器關機後無法啟動(start)
#現象 執行啟動虛擬機器(nova start)之後,虛擬機器並未啟動,到該計算節點母機檢視/var/log/nova/nova-compute.log
日誌發現如下報錯
2018-09-30 02:31:27.083 4537 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2044, in reboot
2018-09-30 02:31:27.083 4537 TRACE oslo_messaging.rpc.dispatcher block_device_info)
2018-09-30 02:31:27.083 4537 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2156, in _hard_reboot
2018-09-30 02:31:27.083 4537 TRACE oslo_messaging.rpc.dispatcher vifs_already_plugged=True)
2018-09-30 02:31:27.083 4537 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py" , line 4429, in _create_domain_and_network
2018-09-30 02:31:27.083 4537 TRACE oslo_messaging.rpc.dispatcher power_on=power_on)
2018-09-30 02:31:27.083 4537 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4360, in _create_domain
2018-09-30 02:31:27.083 4537 TRACE oslo_messaging.rpc.dispatcher LOG.error( err)
2018-09-30 02:31:27.083 4537 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 85, in __exit__
2018-09-30 02:31:27.083 4537 TRACE oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2018-09-30 02:31:27.083 4537 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4350, in _create_domain
2018-09-30 02:31:27.083 4537 TRACE oslo_messaging.rpc.dispatcher domain.createWithFlags(launch_flags)
2018-09-30 02:31:27.083 4537 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 186, in doit
2018-09-30 02:31:27.083 4537 TRACE oslo_messaging.rpc.dispatcher result = proxy_call(self._autowrap, f, *args, **kwargs)
2018-09-30 02:31:27.083 4537 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 144, in proxy_call
2018-09-30 02:31:27.083 4537 TRACE oslo_messaging.rpc.dispatcher rv = execute(f, *args, **kwargs)
2018-09-30 02:31:27.083 4537 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 125, in execute
2018-09-30 02:31:27.083 4537 TRACE oslo_messaging.rpc.dispatcher six.reraise(c, e, tb)
2018-09-30 02:31:27.083 4537 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 83, in tworker
2018-09-30 02:31:27.083 4537 TRACE oslo_messaging.rpc.dispatcher rv = meth(*args, **kwargs)
2018-09-30 02:31:27.083 4537 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib64/python2.7/site-packages/libvirt.py", line 996, in createWithFlags
2018-09-30 02:31:27.083 4537 TRACE oslo_messaging.rpc.dispatcher if ret == -1: raise libvirtError ('virDomainCreateWithFlags() failed', dom=self)
2018-09-30 02:31:27.083 4537 TRACE oslo_messaging.rpc.dispatcher libvirtError: unsupported configuration: Unable to find security driver for label selinux
2018-09-30 02:31:27.083 4537 TRACE oslo_messaging.rpc.dispatcher
2018-09-30 02:32:14.535 4537 INFO nova.compute.resource_tracker [req-efb15755-2bc0-45d2-946c-ba01bf451836 - - - - -] Auditing locally available compute resources for node openstack-com04
分析解決
通過日誌直接分析問題
2018-09-30 02:31:27.083 4537 TRACE oslo_messaging.rpc.dispatcher if ret == -1: raise libvirtError ('virDomainCreateWithFlags() failed', dom=self)
2018-09-30 02:31:27.083 4537 TRACE oslo_messaging.rpc.dispatcher libvirtError: unsupported configuration: Unable to find security driver for label selinux
這兩行獲取到倆個資訊
withflags failed
說明flag可能有問題- 第二行不支援的安全驅動配置
排查flag,先看他是什麼東西
根據上面日誌資訊
vim /usr/lib64/python2.7/site-packages/libvirt.py +996
"""Launch a defined domain. If the call succeeds the domain moves from the
defined to the running domains pools.
If the VIR_DOMAIN_START_PAUSED flag is set, or if the guest domain
has a managed save image that requested paused state (see
virDomainManagedSave()) the guest domain will be started, but its
CPUs will remain paused. The CPUs can later be manually started
using virDomainResume(). In all other cases, the guest domain will
be running.
If the VIR_DOMAIN_START_AUTODESTROY flag is set, the guest
domain will be automatically destroyed when the virConnectPtr
object is finally released. This will also happen if the
client application crashes / loses its connection to the
libvirtd daemon. Any domains marked for auto destroy will
block attempts at migration, save-to-file, or snapshots.
If the VIR_DOMAIN_START_BYPASS_CACHE flag is set, and there is a
managed save file for this domain (created by virDomainManagedSave()),
then libvirt will attempt to bypass the file system cache while restoring
the file, or fail if it cannot do so for the given system; this can allow
less pressure on file system cache, but also risks slowing loads from NFS.
If the VIR_DOMAIN_START_FORCE_BOOT flag is set, then any managed save
file for this domain is discarded, and the domain boots from scratch. """
ret = libvirtmod.virDomainCreateWithFlags(self._o, flags)
if ret == -1: raise libvirtError ('virDomainCreateWithFlags() failed', dom=self)
通過註釋猜測到flag是一個虛擬機器狀態識別符號,如果是-1則擲出異常,-1一般是無效值,但這個虛擬機器原來狀態是好的,由此推測狀態被破壞
再看異常
2018-09-30 02:31:27.083 4537 TRACE oslo_messaging.rpc.dispatcher libvirtError: unsupported configuration: Unable to find security driver for label selinux
先看下libvirt中關於security driver
配置
grep security_driver /etc/libvirt/qemu.conf
# security_driver = [ "selinux", "apparmor" ]
# value of security_driver cannot contain "dac". The value "none" is
# a special value; security_driver can be set to that value in
#security_driver = "selinux"
可以看到都是註釋的,再看下虛擬機器xml檔案裡面有沒有包含sec(安全)與lable的欄位,虛擬機器配置檔案在
平臺 | 虛擬機器配置檔案路徑 |
---|---|
kvm | /etc/libvirt/qemu/ |
openstack | /var/lib/nova/instances/ |
檢視並對比配置檔案之後未發現與其他有正常伺服器有不同之處,沒有發現
檢視系統日誌
Sep 30 02:28:38 openstack-com04 journal: Module /usr/lib64/libvirt/connection-driver/libvirt_driver_lxc.so not accessible
Sep 30 02:28:38 openstack-com04 systemd: Started Virtualization daemon.
Sep 30 02:29:18 openstack-com04 journal: unsupported configuration: Unable to find security driver for label selinux
Sep 30 02:29:18 openstack-com04 journal: Unable to restore from managed state /var/lib/libvirt/qemu/save/instance-0000001f.save. Maybe the file is corrupted?
Sep 30 02:31:26 openstack-com04 journal: unsupported configuration: Unable to find security driver for label selinux
Sep 30 02:31:26 openstack-com04 journal: Unable to restore from managed state /var/lib/libvirt/qemu/save/instance-0000001f.save. Maybe the file is corrupted?
Sep 30 02:32:53 openstack-com04 kernel: device tapb4172400-9b entered promiscuous mode
kvm中.save
字尾的檔案是儲存虛擬機器狀態的檔案,從日誌可以看出,無法從這個狀態檔案恢復資訊,可以移除掉這個檔案直接重啟虛擬機器即可
mv /var/lib/libvirt/qemu/save/instance-0000001f.save /tmp/201809