本測試記錄從openshift 3.6環境中導出項目，然後在將項目環境恢復到Openshift 3.11中所需要的步驟

從而指導導入導出的升級過程。

1.安裝Openshift 3.6版本

過程略

2.安裝OpenShift 3.11版本

過程略

3.在Openshift 3.6版本中建立各類資源

• 創建用戶

htpasswd /etc/origin/master/htpasswd eric

htpasswd /etc/origin/master/htpasswd alice

• 給節點打標簽

oc label node node2.example.com  application=eric-tomcat

[root@master ~]# oc get node node2.example.com --show-labels
NAME                STATUS    AGE       VERSION             LABELS
node2.example.com   Ready     1d        v1.6.1+5115d708d7   application=eric-tomcat,beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/hostname=node2.example.com,region=infra,zone=default
 

• 導入鏡像

docker load -i tomcat.tar
docker tag docker.io/tomcat:8-slim  registry.example.com/tomcat:8-slim
docker push registry.example.com/tomcat:8-slim

• 創建項目ericproject1

用eric用戶登錄

oc new-project ericproject1
oc import-image tomcat:8-slim --from=registry.example.com/tomcat:8-slim --insecure --confirm
oc 
 
new-app tomcat:8-slim --name=ericapp1
oc expose service ericapp1
oc scale dc/ericapp1  --replicas=3

oc new-app tomcat:8-slim --name=ericapp2
oc expose service ericapp2

• 創建項目ericproject2

用eric用戶登錄

oc new-project ericproject2
oc import-image tomcat:8-slim --from=registry.example.com/tomcat:8-slim --insecure --confirm
oc new-app tomcat:8-slim --name=eric-tomcat
oc expose service eric-tomcat

• 用alice賬戶登錄創建項目

oc new-project alice-project
oc import-image tomcat:8-slim --from=registry.example.com/tomcat:8-slim --insecure --confirm
oc new-app tomcat:8-slim --name=alice-tomcat
oc expose service alice-tomcat
oc scale dc/alice-tomcat  --replicas=10

4.開始資源導出過程

以下在OpenShift 3.6的集群環境下操作。

先下載jq和安裝(在執行導出的集群的節點和執行導入的集群節點上都需要安裝)

https://stedolan.github.io/jq/

執行導出

./project_export.sh ericproject1
./project_export.sh ericproject2
./project_export.sh alice-project

導出完成後發現當前目錄下有這三個目錄

導出後進入項目查看內容

5.執行導入過程

將三個目錄全部拷貝到執行導入的節點，OpenShift 3.11的版本

• 先導入鏡像

docker load -i tomcat.tar
docker tag docker.io/tomcat:8-slim  registry.example.com/tomcat:8-slim
docker push registry.example.com/tomcat:8-slim

• 以admin的身份登錄，然後運行

./project_import.sh ericproject1
./project_import.sh ericproject2
./project_import.sh alice-project

6. 恢復到3.11後的驗證

• 用戶

[root@master ~]# oc get users
NAME      UID                                    FULL NAME   IDENTITIES
admin     3d7951e7-422a-11e9-90df-080027dc991a               htpasswd_auth:admin

可見導入過程並不會對用戶進行任何操作，但實際環境中openshift集群都是連接LDAP或其他外部用戶，所以這關系不大。

• 項目

[root@master ~]# oc projects
You have access to the following projects and can switch between them with ‘oc project <projectname>‘:

  * alice-project
    default
    ericproject1
    ericproject2
    kube-public
    kube-system
    management-infra
    openshift
    openshift-console
    openshift-infra
    openshift-logging
    openshift-metrics-server
    openshift-monitoring
    openshift-node
    openshift-sdn
    openshift-web-console

Using project "alice-project" on server "https://master.example.com:8443".

通過admin能看到所有的導入項目，進入項目後因為image stream的問題，發現有些DeploymentConfig一直在deploy階段，但並無實例運行

運行下面的命令讓實例重新裝載

oc delete pod alice-tomcat-1-deploy
oc rollout latest alice-tomcat

然後就可以看到實例全部裝載成功

• label

可見並沒有將我們的label導入到新環境中

[root@master ~]# oc get nodes --show-labels
NAME                 STATUS    ROLES     AGE       VERSION           LABELS
master.example.com   Ready     master    2d        v1.11.0+d4cacc0   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/hostname=master.example.com,node-role.kubernetes.io/master=true
node1.example.com    Ready     infra     2d        v1.11.0+d4cacc0   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/hostname=node1.example.com,node-role.kubernetes.io/infra=true
node2.example.com    Ready     compute   2d        v1.11.0+d4cacc0   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/hostname=node2.example.com,node-role.kubernetes.io/compute=true

• 權限RBAC

[root@master ~]# oc get rolebinding
NAME                    ROLE                    USERS     GROUPS                                 SERVICE ACCOUNTS   SUBJECTS
admin                   /admin                  alice                                                               
system:deployers        /system:deployer                                                         deployer           
system:image-builders   /system:image-builder                                                    builder            
system:image-pullers    /system:image-puller              system:serviceaccounts:alice-project                      
[root@master ~]# oc project ericproject1
Now using project "ericproject1" on server "https://master.example.com:8443".
[root@master ~]# oc get rolebinding
NAME                    ROLE                    USERS     GROUPS                                SERVICE ACCOUNTS   SUBJECTS
admin                   /admin                  eric                                                               
system:deployers        /system:deployer                                                        deployer           
system:image-builders   /system:image-builder                                                   builder            
system:image-pullers    /system:image-puller              system:serviceaccounts:ericproject1                      
[root@master ~]# oc project ericproject2
Now using project "ericproject2" on server "https://master.example.com:8443".
[root@master ~]# oc get rolebinding
NAME                    ROLE                    USERS     GROUPS                                SERVICE ACCOUNTS   SUBJECTS
admin                   /admin                  eric                                                               
system:deployers        /system:deployer                                                        deployer           
system:image-builders   /system:image-builder                                                   builder            
system:image-pullers    /system:image-puller              system:serviceaccounts:ericproject2                      
[root@master ~]# 

可見所有的項目權限都保存下來。

7.升級建議

因為原有的集群下節點數目和新的集群很可能不一樣，因此單純的備份etcd和恢復etcd的辦法上有很大風險。

這種模式下，采用項目導入導出的方式不失為一種較為安全的方式。

需要註意的地方包括:

• 用戶不會導出，但在openshift的權限信息會保存。
• 節點的Label不會導出
• 導入導出過程需要rollout。

OpenShift 項目的備份和恢復實驗