[問題篇]VMWare搭建Openstack——Cinder建立擴充套件Volume的狀態Error的問題
阿新 • • 發佈：2019-01-10
這兩天安裝Cinder for icehouse，安裝過程比較順利，但是安裝完畢之後，在建立volume和檢視cinder list時，狀態為error.
[email protected]:~$ cinder list +--------------------------------------+--------+--------------+------+-------------+----------+--------------------------------------+ | ID | Status | Display Name | Size | Volume Type | Bootable | Attached to | +--------------------------------------+--------+--------------+------+-------------+----------+--------------------------------------+ | fe7f56d7-a319-4f67-b821-4904922889c7 | in-use | myVolume1 | 4 | None | false | 614a2641-1e8d-4442-9704-6ab62e3f39d5 | +--------------------------------------+--------+--------------+------+-------------+----------+--------------------------------------+
由於該問題，已經解決，目前檢視到的cinder list狀態是in-use，目前我曾經遇到過的關於建立volume的狀態（status)有四個：
1、正常建立，還沒有使用，狀態為：Available
2、正常建立，已經使用掛接，狀態為：in-use
3、建立失敗，狀態為：Error
4、建立失敗，狀態為：Creating。
對於Creating，有可能是你建立的Volume過大，可能等一些時間，但是我說的這個建立1GB的，也會出現一種處於Creating狀態，這肯定是有問題的。
對於狀態為Error時，我查看了控制節點的：sudo cat /var/log/cinder/cinder-scheduler.log日誌
我看到了兩個比較典型的錯誤，No Valid Host was Found
還有一種，提示volume service is down or disabled2015-06-12 23:10:15.319 10092 INFO oslo.messaging._drivers.impl_rabbit [req-202316a3-6086-41be-9608-92b52349f7ff - - - - -] Connected to AMQP server on 192.168.3.180:5672 2015-06-12 23:10:15.478 10092 INFO oslo.messaging._drivers.impl_rabbit [-] Connected to AMQP server on 192.168.3.180:5672 2015-06-12 23:17:46.610 10092 WARNING cinder.context [-] Arguments dropped when creating context: {'user': u'eca4528a0b5b46fd990b07ec5bee364a', 'tenant': u'b36c6f68c7bc45eab07b0b62ddc516a8', 'user_identity': u'eca4528a0b5b46fd990b07ec5bee364a b36c6f68c7bc45eab07b0b62ddc516a8 - - -'} 2015-06-12 23:17:46.694 10092 ERROR cinder.scheduler.flows.create_volume [req-00fd9195-c835-4bb1-9cf8-37a24d1700f1 eca4528a0b5b46fd990b07ec5bee364a b36c6f68c7bc45eab07b0b62ddc516a8 - - -] Failed to schedule_create_volume: No valid host was found.
2015-06-13 17:47:21.637 1060 AUDIT cinder.service [-] Starting cinder-scheduler node (version 2014.1.3)
2015-06-13 17:47:21.723 1060 ERROR oslo.messaging._drivers.impl_rabbit [req-c7a22eb6-7396-4c1b-99de-a51d90d9e502 - - - - -] AMQP server on 192.168.3.180:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 1 seconds.
2015-06-13 17:47:22.769 1060 INFO oslo.messaging._drivers.impl_rabbit [req-c7a22eb6-7396-4c1b-99de-a51d90d9e502 - - - - -] Reconnecting to AMQP server on 192.168.3.180:5672
2015-06-13 17:47:22.769 1060 INFO oslo.messaging._drivers.impl_rabbit [req-c7a22eb6-7396-4c1b-99de-a51d90d9e502 - - - - -] Delaying reconnect for 1.0 seconds...
2015-06-13 17:47:23.783 1060 ERROR oslo.messaging._drivers.impl_rabbit [req-c7a22eb6-7396-4c1b-99de-a51d90d9e502 - - - - -] AMQP server on 192.168.3.180:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 3 seconds.
2015-06-13 17:47:26.787 1060 INFO oslo.messaging._drivers.impl_rabbit [req-c7a22eb6-7396-4c1b-99de-a51d90d9e502 - - - - -] Reconnecting to AMQP server on 192.168.3.180:5672
2015-06-13 17:47:26.787 1060 INFO oslo.messaging._drivers.impl_rabbit [req-c7a22eb6-7396-4c1b-99de-a51d90d9e502 - - - - -] Delaying reconnect for 1.0 seconds...
2015-06-13 17:47:27.807 1060 ERROR oslo.messaging._drivers.impl_rabbit [req-c7a22eb6-7396-4c1b-99de-a51d90d9e502 - - - - -] AMQP server on 192.168.3.180:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 5 seconds.
2015-06-14 16:25:25.340 1060 INFO oslo.messaging._drivers.impl_rabbit [req-c7a22eb6-7396-4c1b-99de-a51d90d9e502 - - - - -] Reconnecting to AMQP server on 192.168.3.180:5672
2015-06-14 16:25:25.341 1060 INFO oslo.messaging._drivers.impl_rabbit [req-c7a22eb6-7396-4c1b-99de-a51d90d9e502 - - - - -] Delaying reconnect for 1.0 seconds...
2015-06-14 16:25:26.350 1060 INFO oslo.messaging._drivers.impl_rabbit [req-c7a22eb6-7396-4c1b-99de-a51d90d9e502 - - - - -] Connected to AMQP server on 192.168.3.180:5672
2015-06-14 16:25:26.855 1060 INFO oslo.messaging._drivers.impl_rabbit [-] Connected to AMQP server on 192.168.3.180:5672
2015-06-14 16:25:29.661 1060 WARNING cinder.context [-] Arguments dropped when creating context: {'user': None, 'tenant': None, 'user_identity': u'- - - - -'}
2015-06-14 16:26:08.255 1060 WARNING cinder.context [-] Arguments dropped when creating context: {'user': None, 'tenant': None, 'user_identity': u'- - - - -'}
2015-06-14 16:27:08.259 1060 WARNING cinder.context [-] Arguments dropped when creating context: {'user': None, 'tenant': None, 'user_identity': u'- - - - -'}
2015-06-14 16:28:08.258 1060 WARNING cinder.context [-] Arguments dropped when creating context: {'user': None, 'tenant': None, 'user_identity': u'- - - - -'}
2015-06-14 16:29:08.249 1060 WARNING cinder.context [-] Arguments dropped when creating context: {'user': None, 'tenant': None, 'user_identity': u'- - - - -'}
2015-06-14 16:30:04.562 1060 WARNING cinder.context [-] Arguments dropped when creating context: {'user': u'5bcdce96f7e749fbb157842c9eaf7765', 'tenant': u'7a9a86dfd657482a859688fb97e2375f', 'user_identity': u'5bcdce96f7e749fbb157842c9eaf7765 7a9a86dfd657482a859688fb97e2375f - - -'}
2015-06-14 16:30:04.653 1060 WARNING cinder.scheduler.host_manager [req-dcd56f67-84ab-4bab-ade1-614c6c06df16 5bcdce96f7e749fbb157842c9eaf7765 7a9a86dfd657482a859688fb97e2375f - - -] volume service is down or disabled. (host: cinder)
先稍微說一我的OpenStack實驗架構，一個控制節點，一個網路節點，一個計算節點，一個儲存（Cinder）節點
也就是說我的儲存節點的volume服務是有問題的，所以找不到儲存節點。那麼重啟儲存節點的
service cinder-volume restart經過上午搜尋，看到人家部落格介紹了又是以前我在Neutron碰到的問題，NTP的時間同步問題，當時間同步之後，問題解決。
[email protected]:~$ cinder service-list
+------------------+------------+------+---------+-------+----------------------------+
| Binary | Host | Zone | Status | State | Updated_at |
+------------------+------------+------+---------+-------+----------------------------+
| cinder-scheduler | controller | nova | enabled | up | 2015-06-14T08:46:58.000000 |
| cinder-volume | cinder | nova | enabled | down | 2015-06-14T08:33:18.000000 |
| cinder-volume | cinder | nova | enabled | up | 2015-06-14T08:46:52.000000 |
+------------------+------------+------+---------+-------+----------------------------+
但是又看到了新問題，我看到了兩個一樣的Cinder-volumn，我明明只有一個儲存節點，這個好處理，直接進入資料庫，將資訊刪除即可。
[email protected]:~$ sudo mysql -uroot -p#db4smtest#
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 52
Server version: 5.5.40-0ubuntu0.14.04.1 (Ubuntu)
Copyright (c) 2000, 2014, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> use cinder
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
mysql> show tables;
+--------------------------+
| Tables_in_cinder |
+--------------------------+
| backups |
| encryption |
| iscsi_targets |
| migrate_version |
| quality_of_service_specs |
| quota_classes |
| quota_usages |
| quotas |
| reservations |
| services |
| snapshot_metadata |
| snapshots |
| transfers |
| volume_admin_metadata |
| volume_glance_metadata |
| volume_metadata |
| volume_type_extra_specs |
| volume_types |
| volumes |
+--------------------------+
19 rows in set (0.00 sec)
mysql> desc services
-> ;
+-------------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-------------------+--------------+------+-----+---------+----------------+
| created_at | datetime | YES | | NULL | |
| updated_at | datetime | YES | | NULL | |
| deleted_at | datetime | YES | | NULL | |
| deleted | tinyint(1) | YES | | NULL | |
| id | int(11) | NO | PRI | NULL | auto_increment |
| host | varchar(255) | YES | | NULL | |
| binary | varchar(255) | YES | | NULL | |
| topic | varchar(255) | YES | | NULL | |
| report_count | int(11) | NO | | NULL | |
| disabled | tinyint(1) | YES | | NULL | |
| availability_zone | varchar(255) | YES | | NULL | |
| disabled_reason | varchar(255) | YES | | NULL | |
+-------------------+--------------+------+-----+---------+----------------+
12 rows in set (0.00 sec)
mysql> select host from services;
+------------+
| host |
+------------+
| controller |
| cinder |
| cinder |
+------------+
3 rows in set (0.00 sec)
mysql> select host,id from services;
+------------+----+
| host | id |
+------------+----+
| controller | 1 |
| cinder | 2 |
| cinder | 3 |
+------------+----+
3 rows in set (0.00 sec)
mysql> select host,id,updated_at from services;
+------------+----+---------------------+
| host | id | updated_at |
+------------+----+---------------------+
| controller | 1 | 2015-06-14 08:49:28 |
| cinder | 2 | 2015-06-14 08:49:32 |
| cinder | 3 | 2015-06-14 08:33:18 |
+------------+----+---------------------+
3 rows in set (0.00 sec)
mysql> delete from services where id=3;
Query OK, 1 row affected (0.01 sec)
這時候，在檢視Cinder的服務資訊
[email protected]:~$ source admin-openrc.sh
[email protected]:~$ cinder service-list
+------------------+------------+------+---------+-------+----------------------------+
| Binary | Host | Zone | Status | State | Updated_at |
+------------------+------------+------+---------+-------+----------------------------+
| cinder-scheduler | controller | nova | enabled | up | 2015-06-14T13:31:37.000000 |
| cinder-volume | cinder | nova | enabled | up | 2015-06-14T13:31:43.000000 |
+------------------+------------+------+---------+-------+----------------------------+
這時候，再建立Volume時，狀態就可以了.
期間還碰到了一個問題：使用cinder create命令建立volume時，這個volume一直處於creating的狀態，不論多長時間。
解決方法：仍然在資料庫中刪除，問題原因不詳。
有可能是建立volume時出錯了又沒有及時返回，導致一直處於creating狀態，對於這種卷，只能到資料庫裡去刪除了，也不用刪除資料庫的整條記錄，只改兩個欄位，先設定status=deleted，然後再設定deleted=1就可以了，這個volume就被刪除了。
mysql> select id,status,deleted from volumes;
+--------------------------------------+----------+---------+
| id | status | deleted |
+--------------------------------------+----------+---------+
| 125e2d2d-0e96-48cc-874a-1b92c95c2cae | deleted | 1 |
| 378f83a4-c72a-4ef8-aaae-557ce851146e | deleted | 1 |
| 50ae05fa-9e89-4512-84d5-64bb041b2d05 | deleted | 1 |
| 6c6d0513-8ca2-4fe6-9a9f-f412ae9e8cd9 | deleted | 1 |
| 7064d4cc-743b-4b02-9882-9af64b189e38 | deleted | 1 |
| 787d6ea0-fbb2-4e93-a8fa-11ca36503fdf | deleted | 1 |
| cfcffb39-ac03-4beb-af2e-eb6c1d698c33 | deleted | 1 |
| d0de1dab-2166-49c6-b3a2-965d28b61d59 | deleted | 1 |
| df92f31b-d1e5-42c4-86fb-02bdfb4d9858 | creating | 0 |
| f20bf6d1-cad1-42f8-9258-eac9cae273ac | deleted | 1 |
| f8ad5cdc-be86-40a8-9af1-99db1589c2e8 | deleted | 1 |
| fbf69bbc-5af5-41b4-a260-43c0ef1de5d9 | deleted | 1 |
+--------------------------------------+----------+---------+
12 rows in set (0.00 sec)
mysql> update volumes set status='deleted',deleted=1 where id=
-> 'df92f31b-d1e5-42c4-86fb-02bdfb4d9858';
Query OK, 1 row affected (0.00 sec)
Rows matched: 1 Changed: 1 Warnings: 0
mysql> select id,status,deleted from volumes;
+--------------------------------------+---------+---------+
| id | status | deleted |
+--------------------------------------+---------+---------+
| 125e2d2d-0e96-48cc-874a-1b92c95c2cae | deleted | 1 |
| 378f83a4-c72a-4ef8-aaae-557ce851146e | deleted | 1 |
| 50ae05fa-9e89-4512-84d5-64bb041b2d05 | deleted | 1 |
| 6c6d0513-8ca2-4fe6-9a9f-f412ae9e8cd9 | deleted | 1 |
| 7064d4cc-743b-4b02-9882-9af64b189e38 | deleted | 1 |
| 787d6ea0-fbb2-4e93-a8fa-11ca36503fdf | deleted | 1 |
| cfcffb39-ac03-4beb-af2e-eb6c1d698c33 | deleted | 1 |
| d0de1dab-2166-49c6-b3a2-965d28b61d59 | deleted | 1 |
| df92f31b-d1e5-42c4-86fb-02bdfb4d9858 | deleted | 1 |
| f20bf6d1-cad1-42f8-9258-eac9cae273ac | deleted | 1 |
| f8ad5cdc-be86-40a8-9af1-99db1589c2e8 | deleted | 1 |
| fbf69bbc-5af5-41b4-a260-43c0ef1de5d9 | deleted | 1 |
+--------------------------------------+---------+---------+
12 rows in set (0.00 sec)
mysql> exit