## 介紹 這是一個關於 [Egg.js](https://eggjs.org/zh-cn/) 應用上雲☁️的**示例**，筆者所在的大前端團隊的已應用於生產。 `CI/CD` & `DevOps` & `GitOps` & `HPA` 等這裡暫不做討論，因為每一個點篇幅都很長。 ### 我這裡的實驗條件 * 一個可用的 Kubernetes 叢集 * 已在叢集中部署 [Kube-Prometheus-Stack](https://artifacthub.io/packages/helm/prometheus-community/kube-prometheus-stack) * 已在叢集中部署 [Traefik v2.2](https://traefik.io/) 作為 [Ingress Controller](https://kubernetes.io/docs/concepts/services-networking/ingress-controllers/) * 已安裝 [Helm V3](https://helm.sh/) ### 示例專案 可直接做實驗（文章寫的再好，不如提供一個上雲示例) GitHub: **[k8s-eggjs->點我](https://github.com/Hacker-Linner/k8s-eggjs)** GitHub: **[k8s-eggjs](https://github.com/Hacker-Linner/k8s-eggjs)** GitHub: **[k8s-eggjs](https://github.com/Hacker-Linner/k8s-eggjs)** 這個示例，簡單的提供了兩個介面： `/api/posts` ```sh curl -X POST http://localhost:7001/api/posts --data '{"title":"post1", "content": "post1 content"}' --header 'Content-Type:application/json; charset=UTF-8' ``` `api/topics` ```sh curl -X POST http://localhost:7001/api/topics --data '{"title":"topic1", "content": "topic1 content"}' --header 'Content-Type:application/json; charset=UTF-8' ``` 筆者也把這個專案部署到了 * [k8seggjs.hacker-linner.com->點我](https://k8seggjs.hacker-linner.com/) * [k8seggjs.hacker-linner.com](https://k8seggjs.hacker-linner.com/) * [k8seggjs.hacker-linner.com](https://k8seggjs.hacker-linner.com/) ## 上雲實戰 **（示例已提供，可直接做實驗）** ### Scripts `package.json` 這裡簡單調整為： ```sh "start": "egg-scripts start --workers=1 --title=egg-server-k8s-eggjs-promethues", ``` **最好是單程序啟動，應用容器的編排完全交給 kubernetes。** [Egg 關於 K8S 部署的相關 issues](https://github.com/eggjs/egg/issues?q=k8s) ### Docker Image 準備 檔案位於 `docker/Dockerfile.prod` ```Dockerfile FROM node:15-alpine RUN ln -sf /usr/share/zoneinfo/Asia/ShangHai /etc/localtime RUN echo "Asia/Shanghai" > /etc/timezone COPY package.json /app/dependencies/package.json COPY yarn.lock /app/dependencies/yarn.lock RUN cd /app/dependencies \ && yarn install --frozen-lockfile --registry=https://registry.npm.taobao.org \ && yarn cache clean \ && mkdir /app/egg \ && ln -s /app/dependencies/node_modules /app/egg/node_modules COPY ./ /app/egg/ WORKDIR /app/egg EXPOSE 7001 CMD npm run start ``` 構建`Image` ```sh docker build -f docker/Dockerfile.prod -t k8s-eggjs-promethues:1.0.0 . --no-cache ``` 打個 `tag`，筆者試驗映象是放在阿里雲的（公司有自己的私有倉庫） ```sh docker tag k8s-eggjs-promethues:1.0.0 registry.cn-shenzhen.aliyuncs.com/hacker-linner/k8s-eggjs-promethues:1.0.0 ``` 推送到阿里雲 ```sh docker push registry.cn-shenzhen.aliyuncs.com/hacker-linner/k8s-eggjs-promethues:1.0.0 ``` ### Helm Chart(k8s-helm-charts) （示例專案已提供，可直接做實驗） **生成部署 Chart** ```sh mkdir k8s-helm-charts && cd k8s-helm-charts helm create k8seggjs ``` 我們複製一份 `k8seggjs/values.yaml` 到外層與 `k8seggjs` 資料夾同級(`k8s-helm-charts/values.yaml`)。 **`k8s-helm-charts/values.yaml`做如下修改：** ```yaml replicaCount: 3 # 部署副本我用3個例項做負載均衡，保證服務可用 image: repository: registry.cn-shenzhen.aliyuncs.com/hacker-linner/k8s-eggjs-promethues # 映象變為剛上傳 pullPolicy: Always # 映象拉取策略可直接用預設`IfNotPresent` # apiPort，metricsPort 預設模板沒有， # 這裡我對 template 裡面的 ingress.yaml service.yaml deployment.yaml 檔案做了相應改動 service: type: ClusterIP apiPort: 7001 # 這個 API 服務的埠 metricsPort: 7777 # 這個是 prometheus 所需的 metrics 埠 # Ingress Controller，根據你的環境決定，我這裡用的是 traefik ingress: enabled: true annotations: ingress.kubernetes.io/ssl-redirect: "true" ingress.kubernetes.io/proxy-body-size: "0" kubernetes.io/ingress.class: "traefik" traefik.ingress.kubernetes.io/router.tls: "true" traefik.ingress.kubernetes.io/router.entrypoints: websecure hosts: - host: k8seggjs.hacker-linner.com paths: - / tls: - secretName: hacker-linner-cert-tls hosts: # 做資源限制，防止記憶體洩漏，交給 K8S 殺掉然後重啟，保證服務可用 resources: limits: cpu: 100m memory: 128Mi requests: cpu: 100m memory: 128Mi ``` **建立部署 Namespace** ```sh kubectl create ns k8seggjs ``` **使用 Helm 部署** ```sh helm install k8seggjs ./k8seggjs -f values.yaml -n k8seggjs # 解除安裝：helm uninstall k8seggjs -n k8seggjs ```  ### ServiceMonitor（k8s-prometheus） **RBAC 設定** ```yaml apiVersion: rbac.authorization.k8s.io/v1 kind: RoleList items: - apiVersion: rbac.authorization.k8s.io/v1 kind: Role metadata: name: prometheus-k8s-k8seggjs namespace: k8seggjs rules: - apiGroups: - "" resources: - services - endpoints - pods verbs: - get - list - watch --- apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBindingList items: - apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding metadata: name: prometheus-k8s-k8seggjs namespace: k8seggjs roleRef: apiGroup: rbac.authorization.k8s.io kind: Role name: prometheus-k8s-k8seggjs subjects: - kind: ServiceAccount name: prometheus-k8s namespace: monitoring ``` **指標 Service 設定** ```yaml apiVersion: v1 kind: Service metadata: namespace: k8seggjs name: k8seggjs-metrics labels: k8s-app: k8seggjs-metrics annotations: prometheus.io/scrape: 'true' prometheus.io/scheme: http prometheus.io/path: /metrics prometheus.io/port: "7777" spec: selector: app.kubernetes.io/name: k8seggjs ports: - name: k8seggjs-metrics port: 7777 targetPort: 7777 protocol: TCP ``` **ServiceMonitor 設定** ```yaml apiVersion: monitoring.coreos.com/v1 kind: ServiceMonitor metadata: name: k8seggjs namespace: monitoring spec: endpoints: - bearerTokenFile: /var/run/secrets/kubernetes.io/serviceaccount/token interval: 5s port: k8seggjs-metrics jobLabel: k8s-app namespaceSelector: matchNames: - k8seggjs selector: matchLabels: k8s-app: k8seggjs-metrics ``` **應用** ```sh kubectl apply -f ServiceMonitor.yaml ```    ### egg-exporter & egg-prometheus [egg-exporter](https://github.com/jianglin-wu/egg-exporter)，Egg.js 的 Prometheus 指標收集外掛，附帶 Grafana 看板。 [egg-prometheus](https://github.com/eggjs/egg-prometheus)，Prometheus plugin for Egg.js。 示例專案的指標收集就是用的這個。 ### Grafana (k8s-grafana) `dashboard-metrics.json`，完整的面板 json。來自於[egg-exporter](https://github.com/jianglin-wu/egg-exporter)。筆者這裡做了 `metrics` 字首的調整。 ```js config.exporter = { scrapePort: 7777, scrapePath: '/metrics', prefix: 'k8seggjs_', defaultLabels: { stage: process.env.NODE_ENV }, }; ``` **我們匯入 `json` 檔案進行 `Grafana` 面板建立**  **修改面板 `Variables`**  **`$stage`** * Query: `k8seggjs_nodejs_version_info{worker="app"}` * Regex: `/.*stage="([^"]*).*/`  **`$appname`** * Query: `k8seggjs_nodejs_version_info{worker="app"}` * Regex: `/.*app="([^"]*).*/`  **`$node`** * Query: `k8seggjs_nodejs_version_info{worker="app"}` * Regex: `/.*instance="([^"]*).*/`  **最終效果**  ## Refs * [當egg遇見K8s會發生什麼？](https://developer.aliyun.com/article/725568) * [egg-exporter](https://github.com/jianglin-wu/egg-exporter) * [egg-prometheus](https://github.com/eggjs/egg-prometheus) * [filter-variables-with-regex](https://grafana.com/docs/grafana/latest/variables/filter-variables-with-regex/) ## 互相交流學習 我的微信：

相關推薦