1. 程式人生 > >jenkins X實踐系列(2) —— 基於jx的DevOps實踐

jenkins X實踐系列(2) —— 基於jx的DevOps實踐

jx是雲原生CICD,devops的一個最佳實踐之一,目前在快速的發展成熟中。最近調研了JX,這裡為第2篇,使用已經安裝好的jx來實踐CICD,旨在讓大家瞭解基於jx的DevOps是如何運轉的,感興趣的可以繼續關注,下一篇介紹如何安裝。

先上圖:

一、windows 搭建開發環境(可選)

1. 安裝kubectl

使用Chocolatey來安裝,因此install Chocolatey:

安裝Chocolatey

windows + X ,選擇power shell 管理員模式,輸入:

Set-ExecutionPolicy Bypass -Scope Process -Force; iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))

或者cmd.exe(管理員):

@"%SystemRoot%\System32\WindowsPowerShell\v1.0\powershell.exe" -NoProfile -InputFormat None -ExecutionPolicy Bypass -Command "iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))" && SET "PATH=%PATH%;%ALLUSERSPROFILE%\chocolatey\bin"

使用Chocolatey安裝kubectl

  1. 使用指令碼安裝

    choco install kubernetes-cli
    
  2. 測試是否安裝成功:

    kubectl version
    
  3. 建立配置檔案,先轉到%HOME% 目錄:

    For example: cd C:\users\yourusername

  4. 建立配置檔案目錄:

    mkdir .kube
    
  5. 轉到配置目錄:

    cd .kube
  6. 配置kubectl使用遠端叢集:

    New-Item config -type file

    修改config檔案,或者直接從叢集拷貝配置檔案過來

  7. 測試

C:\Users\jqpeng>kubectl get pods
NAME                                                READY   STATUS      RESTARTS   AGE
jenkins-bd94b5fb8-5t9kq                             1/1     Running     0          10d
jenkins-x-chartmuseum-75d45b6d7f-2hk99              1/1     Running     0          10d
jenkins-x-controllercommitstatus-675dbb9c86-kth6q   1/1     Running     71         10d
jenkins-x-controllerrole-5458874c-4lnwh             1/1     Running     0          10d
jenkins-x-controllerteam-7f965c8b9c-n4kfm           1/1     Running     0          10d
jenkins-x-controllerworkflow-7675c458d-sjbfd        1/1     Running     0          10d

2.安裝helm

到下載頁:
https://github.com/helm/helm/releases

下載最新的helm,選擇windows-amd64

wget https://storage.googleapis.com/kubernetes-helm/helm-v2.11.0-windows-amd64.zip

解壓後,拷貝到C:\Windows\System32。

注意,需要Cross GFW,可以使用( https://azure.microsoft.com/zh-cn/free/ ) 免費一年。

初始化helm

服務端已經安裝過tiller了,因此只需要client-only,另外stable repo指定本地的

helm init --client-only --stable-repo-url=http://charts.youdomain.com/

---
Creating C:\Users\jqpeng\.helm
Creating C:\Users\jqpeng\.helm\repository
Creating C:\Users\jqpeng\.helm\repository\cache
Creating C:\Users\jqpeng\.helm\repository\local
Creating C:\Users\jqpeng\.helm\plugins
Creating C:\Users\jqpeng\.helm\starters
Creating C:\Users\jqpeng\.helm\cache\archive
Creating C:\Users\jqpeng\.helm\repository\repositories.yaml
Adding stable repo with URL: http://charts.youdomain.com/
Adding local repo with URL: http://127.0.0.1:8879/charts
$HELM_HOME has been configured at C:\Users\jqpeng\.helm.
Not installing Tiller due to 'client-only' flag having been set
Happy Helming!

3. 安裝jx

到https://github.com/jenkins-x/jx/releases 下載最新的編譯好的jx執行檔案,選擇jx-windows-amd64.zip下載,下載後解壓,然後重新命名為jx.exe,拷貝到C:\Windows\System32。

測試:

C:\Users\jqpeng>jx version
Updated the team settings in namespace incubation
Updated the team settings in namespace incubation
NAME               VERSION
jx                 1.3.572
jenkins x platform 0.0.2871
Kubernetes cluster v1.12.2
kubectl            v1.12.1
helm client        v2.11.0+g2e55dbe
helm server        v2.11.0+g2e55dbe
git                git version 2.19.1.windows.1

將伺服器上的.jx目錄下相關配置資訊下載到本地使用者目錄下的.jx資料夾。

二、快速開始

1.使用quickstart

作為演示用,可以直接使用jx create quickstart,會直接使用官方的quickstart專案,鍵入命令,然後按提示選擇即可。

D:\Project>jx create quickstart
? select the quickstart you wish to create spring-boot-rest-prometheus
? Project name spring-boot-rest-prometheus
Generated quickstart at D:\Project\spring-boot-rest-prometheus
### NO charts folder D:\Project\spring-boot-rest-prometheus\charts\spring-boot-rest-prometheus
Created project at D:\Project\spring-boot-rest-prometheus

Updated the team settings in namespace incubation
? Which Git service do you wish to use https://github.com
No username defined for the current Git server!
? Do you wish to use jadepeng as the Git user name: Yes
The directory D:\Project\spring-boot-rest-prometheus is not yet using git
? Would you like to initialise git now? Yes
? Commit message:  Initial import

Git repository created
Updated the team settings in namespace incubation
selected pack: C:\Users\jqpeng\.jx\draft\packs\github.com\jenkins-x\draft-packs\packs\maven
? Which organisation do you want to use? jadepeng
replacing placeholders in directory D:\Project\spring-boot-rest-prometheus
app name: spring-boot-rest-prometheus, git server: github.com, org: jadepeng, Docker registry org: jadepeng
skipping directory "D:\\Project\\spring-boot-rest-prometheus\\.git"
Using Git provider github.com at https://github.com


About to create repository spring-boot-rest-prometheus on server https://github.com with user jadepeng
? Enter the new repository name:  spring-boot-rest-prometheus


Creating repository jadepeng/spring-boot-rest-prometheus
Pushed Git repository to https://github.com/jadepeng/spring-boot-rest-prometheus

Updated the team settings in namespace incubation
? Do you wish to use jadepeng as the user name for the Jenkins Pipeline Yes
Created Jenkins Project: http://jenkins.incubation.youdomain.com/job/jadepeng/job/spring-boot-rest-prometheus/

Watch pipeline activity via:    jx get activity -f spring-boot-rest-prometheus -w
Browse the pipeline log via:    jx get build logs jadepeng/spring-boot-rest-prometheus/master
Open the Jenkins console via    jx console
You can list the pipelines via: jx get pipelines
When the pipeline is complete:  jx get applications

For more help on available commands see: https://jenkins-x.io/developing/browsing/

Note that your first pipeline may take a few minutes to start while the necessary images get downloaded!

Creating GitHub webhook for jadepeng/spring-boot-rest-prometheus for url http://jenkins.incubation.youdomain.com/github-webhook/

建立過程:

  • 自動建立了spring-boot-rest-prometheus專案,並提交到github(可以使用自己的git伺服器),並自動建立GitHub webhook ,這樣當新程式碼提交到github後,會自動觸發構建。
  • 將專案提交到jenkins,可以開啟http://jenkins.incubation.youdomain.com/job/jadepeng/job/spring-boot-rest-prometheus/ 檢視
  • jenkins會自動進行構建,可以通過jx get activity -f spring-boot-rest-prometheus -w檢視構建活動
  • 通過jx get build logs jadepeng/spring-boot-rest-prometheus/master檢視構建日誌,
  • 通過jx console開啟jenkins bule

2.檢視構建日誌

我們檢視構建日誌:

jx get build logs jadepeng/spring-boot-rest-prometheus/master


....

+ jx step helm release
No $CHART_REPOSITORY defined so using the default value of: http://jenkins-x-chartmuseum:8080
Using helmBinary helm with feature flag: none
Adding missing Helm repo: jenkins-x http://chartmuseum.jenkins-x.io
Successfully added Helm repository jenkins-x.
Adding missing Helm repo: releases http://jenkins-x-chartmuseum:8080
Successfully added Helm repository releases.
No $CHART_REPOSITORY defined so using the default value of: http://jenkins-x-chartmuseum:8080
Uploading chart file spring-boot-rest-prometheus-0.0.1.tgz to http://jenkins-x-chartmuseum:8080/api/charts
Received 201 response: {"saved":true}
[Pipeline] sh
[spring-boot-rest-prometheus] Running shell script
++ cat ../../VERSION
+ jx promote -b --all-auto --timeout 1h --version 0.0.1
Using helmBinary helm with feature flag: none
Promoting app spring-boot-rest-prometheus version 0.0.1 to namespace incubation-staging
Created Pull Request: http://github.youdomain.com/jqpeng/environment-walkertabby-staging/pulls/34

3.部署到staging環境

可以看到已經構建成功,並停留在最後一步jx promote -b --all-auto --timeout 1h --version 0.0.1

jx promote 是jx的釋出命令,將構建產物部署到k8s環境。回顧下文章開始的一個圖,提交到master後的程式碼,自動構建後會部署到staging,由於採用的gitops,會往staging環境的git倉庫 environment-walkertabby-staging推送一個pullrequest:

pullrequest

environment-walkertabby-staging其實就是一個charts專案:

enter description here

每提交一個pullrequest,其實就是在requirements.yaml,將需要部署的專案作為-staging環境的依賴,比如剛提交的pullrequest就是增加了spring-boot-rest-prometheus依賴。

@@ -13,6 +13,9 @@ dependencies:
+
 - name: pailitaoservice
+
   repository: http://jenkins-x-chartmuseum:8080
+
   version: 0.0.3
+
+- name: spring-boot-rest-prometheus
+
+  repository: http://jenkins-x-chartmuseum:8080
+
+  version: 0.0.1
+
 - name: springboot-rest-demo
+
   repository: http://jenkins-x-chartmuseum:8080
+
   version: 0.0.4

我們來同意下合併pullrequest,這樣當staging專案構建後就會自動部署spring-boot-rest-prometheus。合併完成後,再回到日誌檢視:

...
+ jx promote -b --all-auto --timeout 1h --version 0.0.1
Using helmBinary helm with feature flag: none
Promoting app spring-boot-rest-prometheus version 0.0.1 to namespace incubation-staging
Created Pull Request: http://github.youdomain.com/jqpeng/environment-walkertabby-staging/pulls/34

Pull Request http://github.youdomain.com/jqpeng/environment-walkertabby-staging/pulls/34 is merged at sha 906a33b6eec14a49d248d8220e0d88416798ba6e
merge status: pending for URL http://github.youdomain.com/api/v1/jqpeng/environment-walkertabby-staging/statuses/906a33b6eec14a49d248d8220e0d88416798ba6e with target: http://jenkins.incubation.youdomain.com/job/jqpeng/job/environment-walkertabby-staging/job/master/display/redirect description: Build queued...
merge status: success for URL http://github.youdomain.com/api/v1/jqpeng/environment-walkertabby-staging/statuses/906a33b6eec14a49d248d8220e0d88416798ba6e with target: http://jenkins.incubation.youdomain.com/job/jqpeng/job/environment-walkertabby-staging/job/master/3/display/redirect description: This commit looks good
Merge status checks all passed so the promotion worked!
[Pipeline] }
[Pipeline] // container
[Pipeline] }
[Pipeline] // dir
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Declarative: Post Actions)
[Pipeline] cleanWs
[WS-CLEANUP] Deleting project workspace...[WS-CLEANUP] done
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // withCredentials
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline

jx已經檢測到Pull Request已經通過,並且啟動staging構建,直到構建結束。

這個時候,我們就可以通過jx get applications來檢視部署好的應用。

D:\Project>jx get applications
APPLICATION                 EDIT PODS URL STAGING PODS URL                                                                    PRODUCTION PODS URL     
spring-boot-rest-prometheus               0.0.1        http://spring-boot-rest-prometheus.incubation-staging.youdomain.com
   

PODS 為0,應該是容器啟動有問題,我們可以看下:

kubectl -n=incubation-staging describe pod incubation-staging-spring-boot-rest-prometheus-66966b8cbb-8fvmf
---

Events:
  Type     Reason     Age                    From                 Message
  ----     ------     ----                   ----                 -------
  Normal   Scheduled  6m51s                  default-scheduler    Successfully assigned incubation-staging/incubation-staging-spring-boot-rest-prometheus-66966b8cbb-8fvmf to docker86-9
  Normal   Pulling    6m44s                  kubelet, docker86-9  pulling image "registry.youdomain.com/jadepeng/spring-boot-rest-prometheus:0.0.1"
  Normal   Pulled     6m38s                  kubelet, docker86-9  Successfully pulled image "registry.youdomain.com/jadepeng/spring-boot-rest-prometheus:0.0.1"
  Normal   Created    5m27s (x3 over 6m37s)  kubelet, docker86-9  Created container
  Normal   Started    5m27s (x3 over 6m37s)  kubelet, docker86-9  Started container
  Warning  Unhealthy  5m1s (x9 over 6m31s)   kubelet, docker86-9  Readiness probe failed: Get http://170.22.78.7:8080/actuator/health: dial tcp 170.22.78.7:8080: connect: connection refused
  Normal   Pulled     4m26s (x3 over 6m10s)  kubelet, docker86-9  Container image "registry.youdomain.com/jadepeng/spring-boot-rest-prometheus:0.0.1" already present on machine
  Warning  BackOff    97s (x15 over 5m41s)   kubelet, docker86-9  Back-off restarting failed container

問題在於Readiness probe failed, 來檢視下容器日誌:

kubectl -n=incubation-staging logs incubation-staging-spring-boot-rest-prometheus-66966b8cbb-8fvmf

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::        (v2.0.4.RELEASE)

2018-11-20 03:39:45.149  INFO 1 --- [           main] c.g.j.q.s.r.p.RestPrometheusApplication  : Starting RestPrometheusApplication v0.0.1 on incubation-staging-spring-boot-rest-prometheus-66966b8cbb-8fvmf with PID 1 (/opt/app.jar started by root in /opt)
2018-11-20 03:39:45.442  INFO 1 --- [           main] c.g.j.q.s.r.p.RestPrometheusApplication  : No active profile set, falling back to default profiles: default
2018-11-20 03:39:47.942  INFO 1 --- [           main] ConfigServletWebServerApplicationContext : Refreshing org.springframework.boot.web.ser[email protected]424c0bc4: startup date [Tue Nov 20 03:39:47 UTC 2018]; root of context hierarchy

原來java程式還沒有啟動成功,有可能是預設的資源限制問題,我們去掉相關限制,修改deployment.yaml,刪除resources限制:

刪除resources限制

git push到倉庫,等待自動構建完成,再次檢視:

D:\Project>jx get applications
APPLICATION                 EDIT PODS URL STAGING PODS URL                                                                    PRODUCTION PODS URL     
spring-boot-rest-prometheus               0.0.2   1/1  http://spring-boot-rest-prometheus.incubation-staging.youdomain.com 

D:\Project>curl -l http://spring-boot-rest-prometheus.incubation-staging.youdomain.com/
{"hello":"world"}

可以看到,自動構建了0.0.2版本,並且已經部署成功

4. 部署到正式環境

如果在staging測試沒有問題,可以推送到正式環境(production)

jx promote spring-boot-rest-prometheus  --version 0.0.2 --env production

該命令會往production環境傳送一個pullrequest,同意後會部署到正式環境。

enter description here

jx promote spring-boot-rest-prometheus  --version 0.0.2 --env production
Promoting app spring-boot-rest-prometheus version 0.0.2 to namespace incubation-production
? Do you wish to use jqpeng as the user name to submit the Pull Request Yes
Created Pull Request: http://github.youdomain.com/jqpeng/environment-walkertabby-production/pulls/2

pipeline jqpeng/environment-jx-dev/master
Pull Request http://github.youdomain.com/jqpeng/environment-walkertabby-production/pulls/2 is merged at sha 1f2ab843a8037f353020a81ad4289c9ede550447
merge status: pending for URL http://github.youdomain.com/api/v1/jqpeng/environment-walkertabby-production/statuses/1f2ab843a8037f353020a81ad4289c9ede550447 with target: http://jenkins.incubation.youdomain.com/job/jqpeng/job/environment-walkertabby-production/job/master/display/redirect description: Build queued...

然後再jx get applications

jx get applications
APPLICATION                 EDIT PODS URL STAGING PODS URL                                                                    PRODUCTION PODS URL
spring-boot-rest-prometheus               0.0.2   1/1  http://spring-boot-rest-prometheus.incubation-staging.youdomain.com 0.0.2      1/1  http://spring-boot-rest-prometheus.incubation-production.youdomain.com

可以看到0.0.2已經在PRODUCTIONS環境了,測試一下:

curl -l http://spring-boot-rest-prometheus.incubation-production.youdomain.com
{"hello":"world"}

三、為專案增加新功能

gitops提倡增加新功能,先在新的分支測試通過後,然後提交PR到master分支,因此我們先建立一個新分支jqpeng-dev

git checkout -b jqpeng-dev
Switched to a new branch 'jqpeng-dev'

hello jx

修改RestPrometheusApplication.java

@GetMapping(path = "/", produces = "application/json")
    @ResponseBody
    public Map<String, Object> landingPage() {
        Counter.builder("mymetric").tag("foo", "bar").register(registry).increment();
        return singletonMap("hello", "world");
    }

修改為

@GetMapping(path = "/", produces = "application/json")
    @ResponseBody
    public Map<String, Object> landingPage() {
        Counter.builder("mymetric").tag("foo", "bar").register(registry).increment();
        return singletonMap("hello", "jx");
    }

儲存,提交

git commit -a -m ''
git push origin jqpeng-dev

提交一個PR:

jx create pullrequest -t "#pr1 hello jx"
? Do you wish to use jadepeng as the user name to use for authenticating with git issues Yes

Created PullRequest #1 at https://github.com/jadepeng/spring-boot-rest-prometheus/pull/1

可以到https://github.com/jadepeng/spring-boot-rest-prometheus/pull/1/files檢視程式碼變更。

提交PR後,jx會自動構建PR,並部署preview環境,可以開啟jenkins檢視:

build PR1

或者通過命令檢視:

jx get build log jadepeng/spring-boot-rest-prometheus/PR-1

等自動構建完成,jx會在pr下面提交一個帶預覽地址的評論:

點看連結檢視:

已經是hello jx了!

合併PR

開啟PR頁面,點選Merge pull request:

合併PR

填寫合併日誌,提交:

PR1已經合併到master分支,等待自動構建完成,剩下的就和上面“二、快速開始”裡的一樣了,在staging環境進行測試,沒問題的釋出到生產環境。

四、小結

jx 良好的實現了gitops,利於團隊協作,可以在團隊進行推廣實施。


作者:Jadepeng
出處:jqpeng的技術記事本--http://www.cnblogs.com/xiaoqi
您的支援是對博主最大的鼓勵,感謝您的認真閱讀。
本文版權歸作者所有,歡迎轉載,但未經作者同意必須保留此段宣告,且在文章頁面明顯位置給出原文連線,否則保留追究法律責任的權利。