etcd子網配置失效導致Docker啟動失敗

阿新 • • 發佈：2019-01-09

Docker啟動失敗

執行 docker status docker 檢視原因，發現以下錯誤

Oct 14 16:39:10 *.*.* systemd[1]: Dependency failed for Docker Application Container Engine.
Oct 14 16:39:10 *.*.* systemd[1]: Job docker.service/start failed with result 'dependency'.
Oct 24 09:02:34 *.*.* systemd[1]: Dependency failed for Docker Application Container Engine.
Oct 24 09:02:34 *.*.* systemd[1]: Job docker.service/start failed with result 'dependency'.
Oct 24 09:45:01 *.*.* systemd[1]: Dependency failed for Docker Application Container Engine.
Oct 24 09:45:01 *.*.* systemd[1]: Job docker.service/start failed with result 'dependency'.

由於安裝了flanneld，因此docker增加了對flanneld的依賴，執行systemctl status flanneld 檢視原因，現出以下錯誤

Oct 24 10:15:51 *.*.* flanneld-start[1187]: E1024 10:15:51.327561    1187 network.go:102] failed to retrieve network config: 100: Key not found (/kube-centos/network/config) [110550]
Oct 24 10:15:52 *.*.* flanneld-start[1187]: E1024 10:15:52.328849    1187 network.go:102] failed to retrieve network config: 100: Key not found (/kube-centos/network/config) [110551]
Oct 24 10:15:53 *.*.* flanneld-start[1187]: E1024 10:15:53.329930    1187 network.go:102] failed to retrieve network config: 100: Key not found (/kube-centos/network/config) [110552]
Oct 24 10:15:54 *.*.* flanneld-start[1187]: E1024 10:15:54.331301    1187 network.go:102] failed to retrieve network config: 100: Key not found (/kube-centos/network/config) [110553]
Oct 24 10:15:55 *.*.* flanneld-start[1187]: E1024 10:15:55.332503    1187 network.go:102] failed to retrieve network config: 100: Key not found (/kube-centos/network/config) [110554]

原因為etcd子網配置/kube-centos/network/config 未找到，懷疑每次啟動etcd後，都需要重新配置子網，建立etcd 啟動指令碼如下

systemctl start etcd
etcdctl mkdir /kube-centos/network
etcdctl mk /kube-centos/network/config "{ \"Network\": \"172.30.0.0/16\", \"SubnetLen\": 24, \"Backend\": { \"Type\": \"vxlan\" } }"

執行該指令碼，再執行 systemctl start flanneld

[[email protected] 
 Kubernetes]# systemctl status flanneld
● flanneld.service - Flanneld overlay address etcd agent
   Loaded: loaded (/usr/lib/systemd/system/flanneld.service; enabled; vendor preset: disabled)
   Active: active (running) since Tue 2017-10-24 10:15:56 CST; 44s ago
  Process: 1334 ExecStartPost=/usr/libexec/flannel/mk-docker-opts.sh -k DOCKER_NETWORK_OPTIONS -d /run/flannel/docker (code=exited, status=0/SUCCESS)
 Main PID: 1187 (flanneld)
   Memory: 6.9M
   CGroup: /system.slice/flanneld.service
           └─1187 /usr/bin/flanneld -etcd-endpoints=http://q.emulian.com:2379 -etcd-prefix=/kube-centos/network

Oct 24 10:15:51 *.*.* flanneld-start[1187]: E1024 10:15:51.327561    1187 network.go:102] failed to retrieve network config: 100: Key not found (/kube-centos/network/config) [110550]
Oct 24 10:15:52 *.*.* flanneld-start[1187]: E1024 10:15:52.328849    1187 network.go:102] failed to retrieve network config: 100: Key not found (/kube-centos/network/config) [110551]
Oct 24 10:15:53 *.*.* flanneld-start[1187]: E1024 10:15:53.329930    1187 network.go:102] failed to retrieve network config: 100: Key not found (/kube-centos/network/config) [110552]
Oct 24 10:15:54 *.*.* flanneld-start[1187]: E1024 10:15:54.331301    1187 network.go:102] failed to retrieve network config: 100: Key not found (/kube-centos/network/config) [110553]
Oct 24 10:15:55 *.*.* flanneld-start[1187]: E1024 10:15:55.332503    1187 network.go:102] failed to retrieve network config: 100: Key not found (/kube-centos/network/config) [110554]
Oct 24 10:15:56 *.*.* flanneld-start[1187]: I1024 10:15:56.337850    1187 local_manager.go:179] Picking subnet in range 172.30.1.0 ... 172.30.255.0
Oct 24 10:15:56 *.*.* flanneld-start[1187]: I1024 10:15:56.339319    1187 manager.go:250] Lease acquired: 172.30.56.0/24
Oct 24 10:15:56 *.*.* flanneld-start[1187]: I1024 10:15:56.340333    1187 network.go:58] Watching for L3 misses
Oct 24 10:15:56 *.*.* flanneld-start[1187]: I1024 10:15:56.340369    1187 network.go:66] Watching for new subnet leases

啟動成功，再執行 systemctl start docker

[[email protected] Kubernetes]# systemctl status docker
● docker.service - Docker Application Container Engine
   Loaded: loaded (/usr/lib/systemd/system/docker.service; disabled; vendor preset: disabled)
  Drop-In: /usr/lib/systemd/system/docker.service.d
           └─flannel.conf
   Active: active (running) since Tue 2017-10-24 10:17:02 CST; 10min ago
     Docs: http://docs.docker.com
 Main PID: 1684 (dockerd-current)
   Memory: 48.6M
   CGroup: /system.slice/docker.service
           ├─1684 /usr/bin/dockerd-current --add-runtime docker-runc=/usr/libexec/docker/docker-runc-current --default-runtime=docker-runc --exec-opt native.cgroupdriver=systemd --userland-proxy-path=/usr/libexec/dock...
           └─1720 /usr/bin/docker-containerd-current -l unix:///var/run/docker/libcontainerd/docker-containerd.sock --shim docker-containerd-shim --metrics-interval=0 --start-timeout 2m --state-dir /var/run/docker/lib...

Oct 24 10:17:01 *.*.* dockerd-current[1684]: time="2017-10-24T10:17:01.969192305+08:00" level=info msg="Graph migration to content-addressability took 0.00 seconds"
Oct 24 10:17:01 *.*.* dockerd-current[1684]: time="2017-10-24T10:17:01.970172523+08:00" level=warning msg="mountpoint for pids not found"
Oct 24 10:17:01 *.*.* dockerd-current[1684]: time="2017-10-24T10:17:01.970874674+08:00" level=info msg="Loading containers: start."
Oct 24 10:17:01 *.*.* dockerd-current[1684]: time="2017-10-24T10:17:01.986058242+08:00" level=info msg="Firewalld running: false"
Oct 24 10:17:02 *.*.* dockerd-current[1684]: time="2017-10-24T10:17:02.078754482+08:00" level=info msg="Loading containers: done."
Oct 24 10:17:02 *.*.* dockerd-current[1684]: time="2017-10-24T10:17:02.079203957+08:00" level=info msg="Daemon has completed initialization"
Oct 24 10:17:02 *.*.* dockerd-current[1684]: time="2017-10-24T10:17:02.079223135+08:00" level=info msg="Docker daemon" commit="88a4867/1.12.6" graphdriver=devicemapper version=1.12.6
Oct 24 10:17:02 *.*.* dockerd-current[1684]: time="2017-10-24T10:17:02.089322355+08:00" level=info msg="API listen on [::]:2375"
Oct 24 10:17:02 *.*.* dockerd-current[1684]: time="2017-10-24T10:17:02.089334645+08:00" level=info msg="API listen on /var/run/docker.sock"
Oct 24 10:17:02 *.*.* systemd[1]: Started Docker Application Container Engine

Docker啟動成功

補充flanneld啟動錯誤，因防火牆埠問題導致啟動失敗，錯誤資訊如下

[[email protected] kubernetes]# systemctl status flanneld.service
● flanneld.service - Flanneld overlay address etcd agent
   Loaded: loaded (/usr/lib/systemd/system/flanneld.service; enabled; vendor preset: disabled)
   Active: inactive (dead) since Thu 2017-11-16 13:32:16 CST; 4min 51s ago
  Process: 25604 ExecStart=/usr/bin/flanneld-start $FLANNEL_OPTIONS (code=exited, status=0/SUCCESS)
 Main PID: 25604 (code=exited, status=0/SUCCESS)

Nov 16 13:32:09 *.*.* flanneld-start[25604]: I1116 13:32:09.616564   25604 main.go:132] Installing signal handlers
Nov 16 13:32:09 *.*.* flanneld-start[25604]: I1116 13:32:09.616932   25604 manager.go:136] Determining IP address of default interface
Nov 16 13:32:09 *.*.* flanneld-start[25604]: I1116 13:32:09.617847   25604 manager.go:149] Using interface with name enp4s0f1 and address 192.168.1.101
Nov 16 13:32:09 *.*.* flanneld-start[25604]: I1116 13:32:09.617894   25604 manager.go:166] Defaulting external address to interface address (192.168.1.101)
Nov 16 13:32:10 *.*.* flanneld-start[25604]: E1116 13:32:10.618518   25604 network.go:102] failed to retrieve network config: client: etcd cluster is unavailable or misconfigured; error #0: dial tcp *.*.*:2379: i/o timeout
Nov 16 13:32:12 *.*.* flanneld-start[25604]: E1116 13:32:12.619119   25604 network.go:102] failed to retrieve network config: client: etcd cluster is unavailable or misconfigured; error #0: dial tcp *.*.*:2379: i/o timeout
Nov 16 13:32:14 *.*.* flanneld-start[25604]: E1116 13:32:14.619756   25604 network.go:102] failed to retrieve network config: client: etcd cluster is unavailable or misconfigured; error #0: dial tcp *.*.*:2379: i/o timeout
Nov 16 13:32:15 *.*.* flanneld-start[25604]: E1116 13:32:15.622421   25604 network.go:102] failed to retrieve network config: client: etcd cluster is unavailable or misconfigured; error #0: dial tcp *.*.*:2379: getsockopt: no route to host
Nov 16 13:32:16 *.*.* flanneld-start[25604]: I1116 13:32:16.446052   25604 main.go:172] Exiting...
Nov 16 13:32:16 *.*.* systemd[1]: Stopped Flanneld overlay address etcd agent.

可以看到主要錯誤資訊為

failed to retrieve network config: client: etcd cluster is unavailable or misconfigured;

經過排查，發現firewalld的沒有開放2379埠，因此，flanneld無法訪問etcd服務，導致啟動失敗

etcd配置檔案位於 /etc/etcd/etcd.conf

flanneld配置檔案位於 /etc/sysconfig/flanneld.conf

etcd子網配置失效導致Docker啟動失敗

Docker啟動失敗執行 docker status docker 檢視原因，發現以下錯誤 Oct 14 16:39:10 *.*.* systemd[1]: Dependency failed for Docker Application Container Engi

selinux導致docker啟動失敗

本文轉載地址：http://www.cnblogs.com/elisun/p/6963818.html 1. 問題描述：一向執行正常的一群容器，突然有一天掛掉了，再也起不來，報錯如下 Error response from daemon: devmapper: Err

hadoop 使用ip配置導致hdfs啟動失敗

hadoop $2 value servers 無法 col reg dfs property dataNode 有守護進行，但hdfs web頁面上顯示沒有live node。錯誤日誌： 2017-06-21 17:44:59,513 ERROR org.apa

注意了！雲伺服器的一些環境配置有可能導致應用啟動失敗

嘗試用serviceComb框架跟Spring boot框架結合寫了一個測試應用，結果放到雲伺服器上啟動的時候報錯，大概錯誤如下： Caused by: java.lang.IllegalArgumentException: Could not resolve placeholder '

Log4j配置失效導致spring等框架一直列印DEBUG級別的日誌

在應用中使用了SpringMVC+Hibernate+Log4j+SLF4J。Log4j的日誌級別調到INFO。但是在web容器（Tomcat）在啟動和執行的時候Spring,Hibernate等框架中DEBUG級別的日誌不斷狂刷，導致日誌檔案劇增，磁碟沒幾

解決加了@EnableHystrixDashboard後項目中freemarker配置失效導致404問題

去掉註解，手動配置如下 package com.mark.demo.security.config; import org.springframework.beans.factory.annotation.Autowired; import org.springfram

修改RAC中的私網地址導致CRS啟動失敗

在自己的虛擬機器中試驗，由於疏忽，將私網的地址和公網的地址配在了一個網段，導致將私網斷掉後，RAC也不會腦裂，打算將私網ip 換成另外一個網段：步驟如下： 1）關閉叢集 [[email protected] ~]$ crsctl stop cluster -a

SELinux 引起的docker啟動失敗

msg cati -m root signature size ecc https command 問題描述：有一臺使用中的docker突然發生了故障，然後啟動docker失敗。機器的系統版本：CentOS Linux release 7.3.1611 (Core) 最

centos7下docker啟動失敗--ob for docker.service failed because the control process exited with error code.

今天在配置openshift用docker部署映象的時候，啟動docker發現竟然啟動失敗：之前是可以啟動的，突然不能啟動了，想到我改了配置檔案：執行： vim /etc/sysconfig/docker 可能是家的內容有問題

儲存引擎配置引發的MongoDB啟動失敗

前言系統環境： [[email protected] mongo]# uname -r 3.10.0-693.21.1.el7.x86_64 [[email protected] mongo]# cat /etc/redhat-release CentOS Linux releas

存儲引擎配置引發的MongoDB啟動失敗

uname openssl 谷歌語句 list 服務根據一段時間 lib64 前言系統環境： [root@rabbitmq3 mongo]# uname -r 3.10.0-693.21.1.el7.x86_64 [root@rabbitmq3 mongo]# ca

更換記憶體條導致windows啟動失敗，報錯資訊：windows無法驗證此檔案的數字簽名，2018.11；

》問題描述：給筆記本更換記憶體條，導致windows7 64開機失敗，並提示windows無法驗證此檔案的數字簽名；搜了搜度娘，度娘說重灌系統吧；我去，電腦中一堆資料軟體，怎麼可能；》解決辦法： 1、會有提示說 F8 進入安全模式

spring-boot不同包結構下，同名類衝突導致服務啟動失敗解決方案

專案背景：　　兩個專案的包結構和類名都很多相同，於是開始考慮使用加一級包進行隔離，類似於這種結構但是在啟動的過程中，丟擲來這樣的異常： 1 2 3 4 5 6 7 8 9 Caused by: org.springframework.

docker啟動失敗問題解決

今天使用docker images，竟然出現無法連線到守護程序然後重啟docker發現報錯：可是昨天還用的好好的，翻閱了各種資料也沒找到答案，誤打誤撞之時，發現是阿里雲映象加速器沒有全部配置 Job for docker.service failed because the con

Windows 更新導致 VirtualBox 啟動失敗

我的虛擬機器版本是 4.3.20， Windows 版本是 Windows 8，但今天突然發現 VirtualBox 啟動不了，報的錯誤如下：為了搜尋引擎能檢索到此錯誤資訊，我將錯誤資訊的文字貼在下面： NtCreateFile(\Device\

非正常關機導致mysql啟動失敗

電腦所有程式都沒有響應，卡的動不了就強制關機了，開機之後發現mysql服務啟動不了，查詢mysql的執行狀態是停止的，但是啟動又會失敗。報錯：Another MySQL daemon already running with the same unix so

Breeze部署機非正常關閉導致Breeze啟動失敗

[[email protected] opt]# docker-compose up -d Starting deploy-yumrepo ... error Starting deploy-playbook ... Starting deploy-playbo

Docker啟動失敗

啟動失敗的錯誤如下 [[email protected] ~]# systemctl start docker Job for docker.service failed because the control process exited with err

java web @WebServlet和web.xml衝突，導致tomcat啟動失敗

添加了一個servlet，自動配置了 @WebServlet ，又在web.xml裡配置映射了這個servlet路徑，然後啟動tomcat失敗，eclipse 出現下圖的錯誤提示然後cons

no space left on device 導致伺服器啟動失敗

註冊部落格園很長時間了，一直都沒有寫過東西，只是將他用作收集網頁資料的工具。最近一直在忙其他的東西，昨天下午突然什麼都想幹，就想把糾結了一個月的Ubuntu下low graphic問題解決掉。我的Ubuntu系統裝在本本原來的系統備份盤上，只有10.8個G，裝完U

etcd子網配置失效導致Docker啟動失敗

相關推薦