前言

之前在centos上安裝了一個版本，是使用的一鍵安裝，devstack來安裝的，這個安裝後很多細節都不太清楚，所以決定手動走一邊安裝過程。
我這裡使用的是一臺物理機進行的安裝，IP 為10.1.82.161，安裝的是最小集合，只安裝了keystone,nova,glance和dashboard
這裡我用的FEDORA21進行的安裝，參考的官方文件：
http://docs.openstack.org/kilo/install-guide/install/yum/content/
聽說用fuel可以快速安裝，之後有時間也會看下

準備工作

yum install ntp
# systemctl enable ntpd.service
 

# systemctl start ntpd.service
# ntpq -c peers
# ntpq -c assoc

修改 /etc/hosts

10.1.82.161 controller

設定hostname

hostnamectl set-hostname controller

安裝kilo的源

yum install http://rdo.fedorapeople.org/openstack-kilo/rdo-release-kilo.rpm

安裝資料庫mysql

# yum upgrade
# yum install mariadb mariadb-server MySQL-python
 

修改mysql的配置，（這裡我不太確定是否加里一個新的cnf檔案就會使用這個配置以及如何對應的,總之按照文件新建了配置檔案並進行了配置）

vim /etc/my.cnf.d/mariadb_openstack.cnf
[mysqld]
bind-address = 10.0.0.11

[mysqld]
default-storage-engine = innodb
innodb_file_per_table
collation-server = utf8_general_ci
init-connect = 'SET NAMES utf8'
character-set-server = utf8

設定資料庫開機啟動和初始密碼

# systemctl enable mariadb.service
# systemctl start mariadb.service

mysql_secure_installation

root 密碼我設定了 qwer1234
安裝rabbitmq並設定使用者和許可權

# yum install rabbitmq-server
# systemctl enable rabbitmq-server.service
# systemctl start rabbitmq-server.service

# rabbitmqctl add_user openstack RABBIT_PASS
# rabbitmqctl set_permissions openstack ".*" ".*" ".*"

keystone

安裝

$ mysql -u root -p
CREATE DATABASE keystone;
GRANT ALL PRIVILEGES ON keystone.* TO 'keystone'@'localhost' \
IDENTIFIED BY 'KEYSTONE_DBPASS';
GRANT ALL PRIVILEGES ON keystone.* TO 'keystone'@'%' \
IDENTIFIED BY 'KEYSTONE_DBPASS';

安裝軟體包

yum install openstack-keystone httpd mod_wsgi python-openstackclient memcached python-memcached

# systemctl enable memcached.service
# systemctl start memcached.service

生成一個隨機碼

openssl rand -hex 10 

7f0ccd900a0e81f0a949
編輯/etc/keystone/keystone.conf ，注意以下幾處的修改

[DEFAULT]
admin_token = 7f0ccd900a0e81f0a949

[database]
...
connection = mysql://keystone:[email protected]/keystone
[memcache]
...
servers = localhost:11211
[token]
...
provider = keystone.token.providers.uuid.Provider
driver = keystone.token.persistence.backends.memcache.Token
[revoke]
...
driver = keystone.contrib.revoke.backends.sql.Revoke
[DEFAULT]
...
verbose = True

同步資料庫資訊

#su -s /bin/sh -c "keystone-manage db_sync" 
我的環境keystone 執行這個命令無反應，使用下一行
keystone-manage db_sync

修改 the /etc/httpd/conf/httpd.conf
ServerName controller

新建 /etc/httpd/conf.d/wsgi-keystone.conf

Listen 5000
Listen 35357

<VirtualHost *:5000>
    WSGIDaemonProcess keystone-public processes=5 threads=1 user=keystone group=keystone display-name=%{GROUP}
    WSGIProcessGroup keystone-public
    WSGIScriptAlias / /var/www/cgi-bin/keystone/main
    WSGIApplicationGroup %{GLOBAL}
    WSGIPassAuthorization On
    LogLevel info
    ErrorLogFormat "%{cu}t %M"
    ErrorLog /var/log/httpd/keystone-error.log
    CustomLog /var/log/httpd/keystone-access.log combined
</VirtualHost>

<VirtualHost *:35357>
    WSGIDaemonProcess keystone-admin processes=5 threads=1 user=keystone group=keystone display-name=%{GROUP}
    WSGIProcessGroup keystone-admin
    WSGIScriptAlias / /var/www/cgi-bin/keystone/admin
    WSGIApplicationGroup %{GLOBAL}
    WSGIPassAuthorization On
    LogLevel info
    ErrorLogFormat "%{cu}t %M"
    ErrorLog /var/log/httpd/keystone-error.log
    CustomLog /var/log/httpd/keystone-access.log combined
</VirtualHost>

設定

mkdir -p /var/www/cgi-bin/keystone
curl http://git.openstack.org/cgit/openstack/keystone/plain/httpd/keystone.py?h=stable/kilo \
| tee /var/www/cgi-bin/keystone/main /var/www/cgi-bin/keystone/admin

設定許可權並設定開機啟動

# chown -R keystone:keystone /var/www/cgi-bin/keystone
# chmod 755 /var/www/cgi-bin/keystone/*
# systemctl enable httpd.service
# systemctl start httpd.service

啟動時候遇到錯誤

error:
(13)Permission denied: AH00072: make_sock: could not bind to address [::]:5000
6月 23 13:39:35 controller httpd[5137]: (13)Permission denied: AH00072: make_sock: could not bind to address 0.0.0.0:5000
6月 23 13:39:35 controller httpd[5137]: no listening sockets available, shutting down
6月 23 13:39:35 controller httpd[5137]: AH00015: Unable to open logs
6月 23 13:39:35 controller systemd[1]: httpd.service: main process exited, code=exited, status=1/FAILURE
6月 23 13:39:35 controller systemd[1]: Failed to start The Apache HTTP Server.
6月 23 13:39:35 controller systemd[1]: Unit httpd.service entered failed state.
6月 23 13:39:35 controller systemd[1]: httpd.service failed.

設定setenforce 為0， 有效解決此問題
因此關閉selinux

設定keystone基礎使用者角色資訊等

export OS_TOKEN=7f0ccd900a0e81f0a949
export OS_URL=http://controller:35357/v2.0

[[email protected] ~]# openstack service create   --name keystone --description "OpenStack Identity" identity

這裡又遇到了錯誤

ERROR: cliff.app 'super' object has no attribute 'load_commands'

嘗試修復辦法：
service firewalld stop 無效
去掉ServerName 無效
yum update 無效
加–debug 看資訊

ERROR: openstackclient.shell Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/openstackclient/shell.py", line 176, in run
    return super(OpenStackShell, self).run(argv)
  File "/usr/lib/python2.7/site-packages/cliff/app.py", line 201, in run
    self.initialize_app(remainder)
  File "/usr/lib/python2.7/site-packages/openstackclient/shell.py", line 253, in initialize_app
    self.command_manager.add_command_group(cmd_group)
  File "/usr/lib/python2.7/site-packages/openstackclient/common/commandmanager.py", line 45, in add_command_group
    self.load_commands(group)
  File "/usr/lib/python2.7/site-packages/openstackclient/common/commandmanager.py", line 40, in load_commands
    return super(CommandManager, self).load_commands(namespace)
AttributeError: 'super' object has no attribute 'load_commands'

/usr/lib/python2.7/site-packages/openstackclient/common/commandmanager.py 此檔案屬於python-cliff，跟蹤，發現確實無此方法，於是考慮升級它。
python-cliff-1.6.1-3.fc21.noarch 查到此包舊，升級為1.13 pip install cliff==1.13.0

還報錯，chmod 777 /var/log/keystone/keystone.log
，重啟HTTPD 。OK

openstack endpoint create \
  --publicurl http://controller:5000/v2.0 \
  --internalurl http://controller:5000/v2.0 \
  --adminurl http://controller:35357/v2.0 \
  --region RegionOne \
  identity



openstack project create --description "Admin Project" admin
openstack user create --password-prompt admin      
密碼：admin
openstack role create admin
openstack role add --project admin --user admin admin
openstack project create --description "Service Project" service
openstack project create --description "Demo Project" demo
openstack user create --password-prompt demo
openstack role create user
openstack role add --project demo --user demo user

編輯 /usr/share/keystone/keystone-dist-paste.ini file and remove admin_token_auth from the [pipeline:public_api], [pipeline:admin_api], and [pipeline:api_v3] sections.

unset OS_TOKEN OS_URL

驗證

openstack --os-auth-url http://controller:35357 \
  --os-project-name admin --os-username admin --os-auth-type password \
  token issue

vim admin-openrc.sh

export OS_PROJECT_DOMAIN_ID=default
export OS_USER_DOMAIN_ID=default
export OS_PROJECT_NAME=admin
export OS_TENANT_NAME=admin
export OS_USERNAME=admin
export OS_PASSWORD=ADMIN_PASS
export OS_AUTH_URL=http://controller:35357/v3

vim demo-openrc.sh

export OS_PROJECT_DOMAIN_ID=default
export OS_USER_DOMAIN_ID=default
export OS_PROJECT_NAME=demo
export OS_TENANT_NAME=demo
export OS_USERNAME=demo
export OS_PASSWORD=DEMO_PASS
export OS_AUTH_URL=http://controller:5000/v3

glance

安裝

mysql -u root -p
CREATE DATABASE glance;
GRANT ALL PRIVILEGES ON glance.* TO 'glance'@'localhost' \
  IDENTIFIED BY 'GLANCE_DBPASS';
GRANT ALL PRIVILEGES ON glance.* TO 'glance'@'%' \
  IDENTIFIED BY 'GLANCE_DBPASS';

建立使用者等資訊

source admin-openrc.sh
openstack user create --password-prompt glance
openstack role add --project service --user glance admin
openstack service create --name glance \
  --description "OpenStack Image service" image
openstack endpoint create \
  --publicurl http://controller:9292 \
  --internalurl http://controller:9292 \
  --adminurl http://controller:9292 \
  --region RegionOne \
  image

安裝軟體

yum install openstack-glance python-glance python-glanceclient

配置

vim /etc/glance/glance-api.conf，注意以下幾處的修改

[database]
...
connection = mysql://glance:[email protected]/glance
[keystone_authtoken]
...
auth_uri = http://controller:5000
auth_url = http://controller:35357
auth_plugin = password
project_domain_id = default
user_domain_id = default
project_name = service
username = glance
password = glance

[paste_deploy]
...
flavor = keystone
[glance_store]
...
default_store = file
filesystem_store_datadir = /var/lib/glance/images/
[DEFAULT]
...
notification_driver = noop
[DEFAULT]
...
verbose = True

/etc/glance/glance-registry.conf 
[database]
...
connection = mysql://glance:[email protected]/glance
[keystone_authtoken]
...
auth_uri = http://controller:5000
auth_url = http://controller:35357
auth_plugin = password
project_domain_id = default
user_domain_id = default
project_name = service
username = glance
password = glance

[paste_deploy]
...
flavor = keystone
[DEFAULT]
...
notification_driver = noop
[DEFAULT]
...
verbose = True

同步資料並啟動服務

glance-manage db_sync
# systemctl enable openstack-glance-api.service openstack-glance-registry.service
# systemctl start openstack-glance-api.service openstack-glance-registry.service

openstack-glance-api.service 啟動失敗，開啟DEBUG

6月 23 18:03:03 controller systemd[1]: Failed to start OpenStack Image Service (code-named Glance) API server.
Permission denied: '/var/log/glance/api.log

放開許可權（其實我不太明白具體怎麼設定，方便期間設定為了777，我也不明白為什麼我裝的時候老遇到日誌檔案沒許可權的問題）

chmod 777 /var/log/glance/api.log

修改

echo "export OS_IMAGE_API_VERSION=2" | tee -a admin-openrc.sh demo-openrc.sh

加一個映像

source admin-openrc.sh
mkdir /tmp/images
 wget -P /tmp/images http://download.cirros-cloud.net/0.3.4/cirros-0.3.4-x86_64-disk.img
glance image-create --name "cirros-0.3.4-x86_64" --file /tmp/images/cirros-0.3.4-x86_64-disk.img --disk-format qcow2 --container-format bare --visibility public --progress

遇到錯誤

不識別 --visibility public
新增錯誤 ，Error in store configuration. Adding images to store is disabled
 Error in store configuration. Adding images to store is disabled.
2015-06-24 09:23:40.390 2052 TRACE glance.api.v2.image_data Traceback (most recent call last):
2015-06-24 09:23:40.390 2052 TRACE glance.api.v2.image_data   File "/usr/lib/python2.7/site-packages/glance/api/v2/image_data.py", line 74, in upload
2015-06-24 09:23:40.390 2052 TRACE glance.api.v2.image_data     image.set_data(data, size)
2015-06-24 09:23:40.390 2052 TRACE glance.api.v2.image_data   File "/usr/lib/python2.7/site-packages/glance/domain/proxy.py", line 166, in set_data
2015-06-24 09:23:40.390 2052 TRACE glance.api.v2.image_data     self.base.set_data(data, size)
2015-06-24 09:23:40.390 2052 TRACE glance.api.v2.image_data   File "/usr/lib/python2.7/site-packages/glance/notifier.py", line 429, in set_data
2015-06-24 09:23:40.390 2052 TRACE glance.api.v2.image_data     _send_notification(notify_error, 'image.upload', msg)
2015-06-24 09:23:40.390 2052 TRACE glance.api.v2.image_data   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 85, in __exit__
2015-06-24 09:23:40.390 2052 TRACE glance.api.v2.image_data     six.reraise(self.type_, self.value, self.tb)
2015-06-24 09:23:40.390 2052 TRACE glance.api.v2.image_data   File "/usr/lib/python2.7/site-packages/glance/notifier.py", line 378, in set_data
2015-06-24 09:23:40.390 2052 TRACE glance.api.v2.image_data     self.repo.set_data(data, size)
2015-06-24 09:23:40.390 2052 TRACE glance.api.v2.image_data   File "/usr/lib/python2.7/site-packages/glance/api/policy.py", line 196, in set_data
2015-06-24 09:23:40.390 2052 TRACE glance.api.v2.image_data     return self.image.set_data(*args, **kwargs)
2015-06-24 09:23:40.390 2052 TRACE glance.api.v2.image_data   File "/usr/lib/python2.7/site-packages/glance/quota/__init__.py", line 296, in set_data
2015-06-24 09:23:40.390 2052 TRACE glance.api.v2.image_data     self.image.set_data(data, size=size)
2015-06-24 09:23:40.390 2052 TRACE glance.api.v2.image_data   File "/usr/lib/python2.7/site-packages/glance/location.py", line 377, in set_data
2015-06-24 09:23:40.390 2052 TRACE glance.api.v2.image_data     context=self.context)
2015-06-24 09:23:40.390 2052 TRACE glance.api.v2.image_data   File "/usr/lib/python2.7/site-packages/glance_store/backend.py", line 364, in add_to_backend
2015-06-24 09:23:40.390 2052 TRACE glance.api.v2.image_data     return store_add_to_backend(image_id, data, size, store, context)
2015-06-24 09:23:40.390 2052 TRACE glance.api.v2.image_data   File "/usr/lib/python2.7/site-packages/glance_store/backend.py", line 339, in store_add_to_backend
2015-06-24 09:23:40.390 2052 TRACE glance.api.v2.image_data     context=context)
2015-06-24 09:23:40.390 2052 TRACE glance.api.v2.image_data   File "/usr/lib/python2.7/site-packages/glance_store/capabilities.py", line 224, in op_checker
2015-06-24 09:23:40.390 2052 TRACE glance.api.v2.image_data     raise op_exec_map[op](**kwargs)
2015-06-24 09:23:40.390 2052 TRACE glance.api.v2.image_data StoreAddDisabled: Configuration for store failed. Adding images to this store is disabled.
2015-06-24 09:23:40.390 2052 TRACE glance.api.v2.image_data

解決變法，關閉防火牆，SELINUX，放開/var/lib/glance/image許可權777,重啟服務，總之之後OK了，沒具體測到到底是哪個導致的。看看結果

glance image-list

nova

安裝

mysql -u root -p
CREATE DATABASE nova;
GRANT ALL PRIVILEGES ON nova.* TO 'nova'@'localhost' \
  IDENTIFIED BY 'NOVA_DBPASS';
GRANT ALL PRIVILEGES ON nova.* TO 'nova'@'%' \
  IDENTIFIED BY 'NOVA_DBPASS';

建立認證資訊

source admin-openrc.sh
 openstack user create --password-prompt nova     
密碼：nova
openstack role add --project service --user nova admin
openstack service create --name nova \
  --description "OpenStack Compute" compute
openstack endpoint create \
  --publicurl http://controller:8774/v2/%\(tenant_id\)s \
  --internalurl http://controller:8774/v2/%\(tenant_id\)s \
  --adminurl http://controller:8774/v2/%\(tenant_id\)s \
  --region RegionOne \
  compute

安裝軟體

yum install openstack-nova-api openstack-nova-cert openstack-nova-conductor \
  openstack-nova-console openstack-nova-novncproxy openstack-nova-scheduler \
  python-novaclient

配置

/etc/nova/nova.conf

[database]
...
connection = mysql://nova:[email protected]/nova
[DEFAULT]
...
rpc_backend = rabbit

[oslo_messaging_rabbit]
...
rabbit_host = controller
rabbit_userid = openstack
rabbit_password = RABBIT_PASS
[DEFAULT]
...
auth_strategy = keystone

[keystone_authtoken]
...
auth_uri = http://controller:5000
auth_url = http://controller:35357
auth_plugin = password
project_domain_id = default
user_domain_id = default
project_name = service
username = nova
password = nova
[DEFAULT]
...
my_ip = 10.1.82.161
[DEFAULT]
...
vncserver_listen = 127.0.0.1
vncserver_proxyclient_address = 127.0.0.1
[glance]
...
host = controller
[oslo_concurrency]
...
lock_path = /var/lib/nova/tmp
[DEFAULT]
...
verbose = True

同步資料並啟動

nova-manage db sync
# systemctl enable openstack-nova-api.service openstack-nova-cert.service \
  openstack-nova-consoleauth.service openstack-nova-scheduler.service \
  openstack-nova-conductor.service openstack-nova-novncproxy.service
# systemctl start openstack-nova-api.service openstack-nova-cert.service \
  openstack-nova-consoleauth.service openstack-nova-scheduler.service \
  openstack-nova-conductor.service openstack-nova-novncproxy.service

安裝nova-compute

我這裡使用的是用一臺機器進行安裝的

yum install openstack-nova-compute sysfsutils
vim /etc/nova/nova.conf

進行配置
[DEFAULT]
…
rpc_backend = rabbit

[oslo_messaging_rabbit]
...
rabbit_host = controller
rabbit_userid = openstack
rabbit_password = RABBIT_PASS

[DEFAULT]
...
auth_strategy = keystone

[keystone_authtoken]
...
auth_uri = http://controller:5000
auth_url = http://controller:35357
auth_plugin = password
project_domain_id = default
user_domain_id = default
project_name = service
username = nova
password = NOVA_PASS

[DEFAULT]
...
my_ip = 10.1.82.166

[DEFAULT]
...
vnc_enabled = True
vncserver_listen = 0.0.0.0
vncserver_proxyclient_address = 127.0.0.1
novncproxy_base_url = http://controller:6080/vnc_auto.html

[glance]
...
host = controller

[oslo_concurrency]
...
lock_path = /var/lib/nova/tmp

[DEFAULT]
...
verbose = True

配置virt_type
執行egrep -c '(vmx|svm)' /proc/cpuinfo 若結果 0 ，應

[libvirt]
...
virt_type = qemu

否則，可用kvm
設定啟動

# systemctl enable libvirtd.service openstack-nova-compute.service
# systemctl start libvirtd.service openstack-nova-compute.service

遇到錯誤

 libvirt version: 1.2.9.3, package: 2.fc21 (Fedora Project, 2015-06-06-15:23...t.org)
6月 24 11:52:45 controller libvirtd[13287]: Module /usr/lib64/libvirt/connection-driver/libvirt_driver_vbox_network.so ...ssible
6月 24 11:52:45 controller libvirtd[13287]: Module /usr/lib64/libvirt/connection-driver/libvirt_driver_vbox_storage.so ...ssible
6月 24 11:52:45 controller libvirtd[13287]: Module /usr/lib64/libvirt/connection-driver/libvirt_driver_xen.so not accessible
6月 24 11:52:45 controller libvirtd[13287]: Module /usr/lib64/libvirt/connection-driver/libvirt_driver_libxl.so not accessible
6月 24 11:52:45 controller libvirtd[13287]: Module /usr/lib64/libvirt/connection-driver/libvirt_driver_uml.so not accessible
6月 24 11:52:45 controller libvirtd[13287]: Module /usr/lib64/libvirt/connection-driver/libvirt_driver_vbox.so not accessible

感覺是KVM的包不全，所以安裝

yum -y install kvm python-virtinst libvirt  bridge-utils virt-manager qemu-kvm-tools  virt-viewer  virt-v2v

之後又發現
rabbit_host 寫localhost失敗，換成controller，之後可執行

nova-network

這裡使用nova-network配置網路，說實話，我對網路這一塊搞的不是很明白，後面建立虛擬機器的時候並沒有建立網路和繫結。
服務端配置/etc/nova/nova.conf

[DEFAULT]
...
network_api_class = nova.network.api.API
security_group_api = nova

systemctl restart openstack-nova-api.service openstack-nova-scheduler.service \
  openstack-nova-conductor.service

computer node 計算節點配置，這裡我用的是用一臺機器

yum install openstack-nova-network openstack-nova-api

配置

[DEFAULT]
...
network_api_class = nova.network.api.API
security_group_api = nova
firewall_driver = nova.virt.libvirt.firewall.IptablesFirewallDriver
network_manager = nova.network.manager.FlatDHCPManager
network_size = 254
allow_same_net_traffic = False
multi_host = True
send_arp_for_ha = True
share_dhcp_address = True
force_dhcp_release = True
flat_network_bridge = br100
flat_interface = INTERFACE_NAME
public_interface = INTERFACE_NAME

INTERFACE_NAME改成你自己的網絡卡名稱，啟動

systemctl enable openstack-nova-network.service openstack-nova-metadata-api.service
# systemctl start openstack-nova-network.service openstack-nova-metadata-api.service

warning 同一個機器openstack-nova-metadata-api.service啟動衝突，暫時未處理(此問題參考https://bugs.launchpad.net/nova/+bug/1237334)，下面建立一個網路

nova network-create demo-net --bridge br100 --multi-host T \
  --fixed-range-v4 10.1.82.161/22 --allowed-start 10.1.82.163 --allowed-end 10.1.82.165 --gateway 10.1.80.254 
nova net-list

dashboard

yum install openstack-dashboard httpd mod_wsgi memcached python-memcached             

vim /etc/openstack-dashboard/local_settings

OPENSTACK_HOST = "controller"
ALLOWED_HOSTS = '*'
CACHES = {
   'default': {
       'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache',
       'LOCATION': '127.0.0.1:11211',
   }
}
OPENSTACK_KEYSTONE_DEFAULT_ROLE = "user"
TIME_ZONE = "TIME_ZONE"

配置啟動

setsebool -P httpd_can_network_connect on
chown -R apache:apache /usr/share/openstack-dashboard/static
systemctl enable httpd.service memcached.service
# systemctl start httpd.service memcached.service

錯誤：

The server has either erred or is incapable of performing the requested operation. (HTTP 500) (Request-ID: req-4c303042-6cb3-4fa4-93d1-1a2986940a1e)

嘗試，建立網路，未解決
修復linuxnet_interface_driver=nova.network.linux_net.LinuxBridgeInterfaceDriver，未解決
後來發現auth_strategy=keystone 這一行配置在了網路段裡，而沒有在default段裡，因此出現錯誤，修改之，又發現錯誤

 Not authorized for image 

查詢，發現my_ip配置錯誤，改成正確的IP，重啟

systemctl  restart openstack-nova-api.service openstack-nova-cert.service openstack-nova-compute.service openstack-nova-conductor.service 
openstack-nova-consoleauth.service openstack-nova-network.service openstack-nova-novncproxy.service openstack-nova-scheduler.service

系統可用，建立了WINDOWS虛擬機器試了試，還可以，發現關閉例項時候會遇到一個問題，說是系統出現異常還是什麼的請聯絡管理員，我還沒具體去看是什麼問題。下面會仔細研究下keystone的許可權，以及呼叫流程等。

參考：