Inner Structure

下面是簡單的StatefulSet Controller工作的內部結構圖。 

NewStatefulSetController

同其他Controller一樣，StatefulSet Controller也是由ControllerManager初始化時負責啟動。

// NewStatefulSetController creates a new statefulset controller.
func NewStatefulSetController(
	podInformer coreinformers.PodInformer,
	setInformer appsinformers.StatefulSetInformer,
	pvcInformer coreinformers.PersistentVolumeClaimInformer,
	revInformer appsinformers.ControllerRevisionInformer,
	kubeClient clientset.Interface,
)
 
 *StatefulSetController {
	
    ...

	ssc := &StatefulSetController{
		kubeClient: kubeClient,
		control: NewDefaultStatefulSetControl(
			NewRealStatefulPodControl(
				kubeClient,
				setInformer.Lister(),
				podInformer.Lister(),
				pvcInformer.Lister(),
				recorder),
			NewRealStatefulSetStatusUpdater(kubeClient, setInformer.Lister()),
			history.NewHistory(kubeClient, revInformer.Lister()),
		),
		pvcListerSynced: pvcInformer.Informer().HasSynced,
		queue:           workqueue.NewNamedRateLimitingQueue(workqueue.DefaultControllerRateLimiter(), "statefulset"
 
),
		podControl:      controller.RealPodControl{KubeClient: kubeClient, Recorder: recorder},

		revListerSynced: revInformer.Informer().HasSynced,
	}

	podInformer.Informer().AddEventHandler(cache.ResourceEventHandlerFuncs{
		// lookup the statefulset and enqueue
		AddFunc: ssc.addPod,
		// lookup current and old statefulset if labels changed
 

		UpdateFunc: ssc.updatePod,
		// lookup statefulset accounting for deletion tombstones
		DeleteFunc: ssc.deletePod,
	})
	ssc.podLister = podInformer.Lister()
	ssc.podListerSynced = podInformer.Informer().HasSynced

	setInformer.Informer().AddEventHandlerWithResyncPeriod(
		cache.ResourceEventHandlerFuncs{
			AddFunc: ssc.enqueueStatefulSet,
			UpdateFunc: func(old, cur interface{}) {
				oldPS := old.(*apps.StatefulSet)
				curPS := cur.(*apps.StatefulSet)
				if oldPS.Status.Replicas != curPS.Status.Replicas {
					glog.V(4).Infof("Observed updated replica count for StatefulSet: %v, %d->%d", curPS.Name, oldPS.Status.Replicas, curPS.Status.Replicas)
				}
				ssc.enqueueStatefulSet(cur)
			},
			DeleteFunc: ssc.enqueueStatefulSet,
		},
		statefulSetResyncPeriod,
	)
	ssc.setLister = setInformer.Lister()
	ssc.setListerSynced = setInformer.Informer().HasSynced

	// TODO: Watch volumes
	return ssc
}

很熟悉的程式碼風格，也是建立對應的eventBroadcaster，然後給對應的objectInformer註冊對應的eventHandler：

• StatefulSetController主要ListWatch Pod和StatefulSet物件；
• Pod Informer註冊了add/update/delete EventHandler，這三個EventHandler都會將Pod對應的StatefulSet加入到StatefulSet Queue中。
• StatefulSet Informer同樣註冊了add/update/event EventHandler，也都會將StatefulSet加入到StatefulSet Queue中。
• 目前StatefulSetController還未感知PVC Informer的EventHandler，這裡繼續按照PVC Controller全部處理。在StatefulSet Controller建立和刪除Pod時，會呼叫apiserver建立和刪除對應的PVC。
• RevisionController類似，在StatefulSet Controller Reconcile時會建立或者刪除對應的Revision。

StatefulSetController sync

接下來，會進入StatefulSetController的worker（只有一個worker，也就是隻一個go routine），worker會從StatefulSet Queue中pop out一個StatefulSet物件，然後執行sync進行Reconcile操作。

// sync syncs the given statefulset.
func (ssc *StatefulSetController) sync(key string) error {
	startTime := time.Now()
	defer func() {
		glog.V(4).Infof("Finished syncing statefulset %q (%v)", key, time.Now().Sub(startTime))
	}()

	namespace, name, err := cache.SplitMetaNamespaceKey(key)
	if err != nil {
		return err
	}
	set, err := ssc.setLister.StatefulSets(namespace).Get(name)
	if errors.IsNotFound(err) {
		glog.Infof("StatefulSet has been deleted %v", key)
		return nil
	}
	if err != nil {
		utilruntime.HandleError(fmt.Errorf("unable to retrieve StatefulSet %v from store: %v", key, err))
		return err
	}

	selector, err := metav1.LabelSelectorAsSelector(set.Spec.Selector)
	if err != nil {
		utilruntime.HandleError(fmt.Errorf("error converting StatefulSet %v selector: %v", key, err))
		// This is a non-transient error, so don't retry.
		return nil
	}

	if err := ssc.adoptOrphanRevisions(set); err != nil {
		return err
	}

	pods, err := ssc.getPodsForStatefulSet(set, selector)
	if err != nil {
		return err
	}

	return ssc.syncStatefulSet(set, pods)
}

• sync中根據setLabel匹配出所有revisions、然後檢查這些revisions中是否有OwnerReference為空的，如果有，那說明存在Orphaned的Revisions。

  注意：只要檢查到有一個History Revision就會觸發給所有的Resivions打上Patch：
  {"metadata":{"ownerReferences":[{"apiVersion":"%s","kind":"%s","name":"%s","uid":"%s","controller":true,"blockOwnerDeletion":true}],"uid":"%s"}}

• 呼叫getPodsForStatefulSet獲取幸運飛艇平臺出租(www.1159880099.com) QQ1159880099  原始碼交易,原始碼出售這個StatefulSet應該管理的Pods。

  • 獲取該StatefulSet對應Namesapce下所有的Pods；
  • 執行ClaimPods操作：檢查set和pod的Label是否匹配上，如果Label不匹配，那麼需要release這個Pod，然後檢查pod的name和StatefulSet name的格式是否能匹配上。對於都匹配上的，並且ControllerRef UID也相同的，則不需要處理。
  • 如果Selector和ControllerRef都匹配不上，則執行ReleasePod操作，給Pod打Patch: {“metadata":{"ownerReferences":[{"$patch":"delete","uid":"%s"}],"uid":"%s"}}
  • 對於Label和name格式能匹配上的，但是controllerRef為空的Pods,就執行AdoptPod，給Pod打上Patch： {“metadata":{"ownerReferences":[{"apiVersion":"%s","kind":"%s","name":"%s","uid":"%s","controller":true,"blockOwnerDeletion":true}],"uid":"%s"}}

UpdateStatefulSet

syncStatefulSet的實現只是呼叫UpdateStatefulSet。

func (ssc *defaultStatefulSetControl) UpdateStatefulSet(set *apps.StatefulSet, pods []*v1.Pod) error {

	// list all revisions and sort them
	revisions, err := ssc.ListRevisions(set)
	if err != nil {
		return err
	}
	history.SortControllerRevisions(revisions)

	// get the current, and update revisions
	currentRevision, updateRevision, collisionCount, err := ssc.getStatefulSetRevisions(set, revisions)
	if err != nil {
		return err
	}

	// perform the main update function and get the status
	status, err := ssc.updateStatefulSet(set, currentRevision, updateRevision, collisionCount, pods)
	if err != nil {
		return err
	}

	// update the set's status
	err = ssc.updateStatefulSetStatus(set, status)
	if err != nil {
		return err
	}

	glog.V(4).Infof("StatefulSet %s/%s pod status replicas=%d ready=%d current=%d updated=%d",
		set.Namespace,
		set.Name,
		status.Replicas,
		status.ReadyReplicas,
		status.CurrentReplicas,
		status.UpdatedReplicas)

	glog.V(4).Infof("StatefulSet %s/%s revisions current=%s update=%s",
		set.Namespace,
		set.Name,
		status.CurrentRevision,
		status.UpdateRevision)

	// maintain the set's revision history limit
	return ssc.truncateHistory(set, pods, revisions, currentRevision, updateRevision)
}

UpdateStatefulSet主要流程為：

• ListRevisions獲取該StatefulSet的所有Revisions，並按照Revision從小到大進行排序。
• getStatefulSetRevisions獲取currentRevison和UpdateRevision。
  • 只有當RollingUpdate策略時Partition不為0時，才會有部分Pods是updateRevision。
  • 其他情況，所有Pods都得維持currentRevision。
• updateStatefulSet是StatefulSet Controller的核心邏輯，負責建立、更新、刪除Pods，使得宣告式target得以維護：
  • 使得target state始終有Spec.Replicas個Running And Ready的Pods。
  • 如果更新策略是RollingUpdate，並且Partition為0，則保證所有Pods都對應Status.CurrentRevision。
  • 如果更新策略是RollingUpdate，並且Partition不為0，則ordinal小於Partition的Pods保持Status.CurrentRevision，而ordinal大於等於Partition的Pods更新到Status.UpdateRevision。
  • 如果更新策略是OnDelete，則只有刪除Pods時才會觸發對應Pods的更新，也就是說與Revisions不關聯。
• truncateHistory維護History Revision個數不超過.Spec.RevisionHistoryLimit。

updateStatefulSet

updateStatefulSet是整個StatefulSetController的核心。

func (ssc *defaultStatefulSetControl) updateStatefulSet(
	set *apps.StatefulSet,
	currentRevision *apps.ControllerRevision,
	updateRevision *apps.ControllerRevision,
	collisionCount int32,
	pods []*v1.Pod) (*apps.StatefulSetStatus, error) {
	// get the current and update revisions of the set.
	currentSet, err := ApplyRevision(set, currentRevision)
	if err != nil {
		return nil, err
	}
	updateSet, err := ApplyRevision(set, updateRevision)
	if err != nil {
		return nil, err
	}

	// set the generation, and revisions in the returned status
	status := apps.StatefulSetStatus{}
	status.ObservedGeneration = new(int64)
	*status.ObservedGeneration = set.Generation
	status.CurrentRevision = currentRevision.Name
	status.UpdateRevision = updateRevision.Name
	status.CollisionCount = new(int32)
	*status.CollisionCount = collisionCount

	replicaCount := int(*set.Spec.Replicas)
	// slice that will contain all Pods such that 0 <= getOrdinal(pod) < set.Spec.Replicas
	replicas := make([]*v1.Pod, replicaCount)
	// slice that will contain all Pods such that set.Spec.Replicas <= getOrdinal(pod)
	condemned := make([]*v1.Pod, 0, len(pods))
	unhealthy := 0
	firstUnhealthyOrdinal := math.MaxInt32
	var firstUnhealthyPod *v1.Pod

	// First we partition pods into two lists valid replicas and condemned Pods
	for i := range pods {
		status.Replicas++

		// count the number of running and ready replicas
		if isRunningAndReady(pods[i]) {
			status.ReadyReplicas++
		}

		// count the number of current and update replicas
		if isCreated(pods[i]) && !isTerminating(pods[i]) {
			if getPodRevision(pods[i]) == currentRevision.Name {
				status.CurrentReplicas++
			} else if getPodRevision(pods[i]) == updateRevision.Name {
				status.UpdatedReplicas++
			}
		}

		if ord := getOrdinal(pods[i]); 0 <= ord && ord < replicaCount {
			// if the ordinal of the pod is within the range of the current number of replicas,
			// insert it at the indirection of its ordinal
			replicas[ord] = pods[i]

		} else if ord >= replicaCount {
			// if the ordinal is greater than the number of replicas add it to the condemned list
			condemned = append(condemned, pods[i])
		}
		// If the ordinal could not be parsed (ord < 0), ignore the Pod.
	}

	// for any empty indices in the sequence [0,set.Spec.Replicas) create a new Pod at the correct revision
	for ord := 0; ord < replicaCount; ord++ {
		if replicas[ord] == nil {
			replicas[ord] = newVersionedStatefulSetPod(
				currentSet,
				updateSet,
				currentRevision.Name,
				updateRevision.Name, ord)
		}
	}

	// sort the condemned Pods by their ordinals
	sort.Sort(ascendingOrdinal(condemned))

	// find the first unhealthy Pod
	for i := range replicas {
		if !isHealthy(replicas[i]) {
			unhealthy++
			if ord := getOrdinal(replicas[i]); ord < firstUnhealthyOrdinal {
				firstUnhealthyOrdinal = ord
				firstUnhealthyPod = replicas[i]
			}
		}
	}

	for i := range condemned {
		if !isHealthy(condemned[i]) {
			unhealthy++
			if ord := getOrdinal(condemned[i]); ord < firstUnhealthyOrdinal {
				firstUnhealthyOrdinal = ord
				firstUnhealthyPod = condemned[i]
			}
		}
	}

	if unhealthy > 0 {
		glog.V(4).Infof("StatefulSet %s/%s has %d unhealthy Pods starting with %s",
			set.Namespace,
			set.Name,
			unhealthy,
			firstUnhealthyPod.Name)
	}

	// If the StatefulSet is being deleted, don't do anything other than updating
	// status.
	if set.DeletionTimestamp != nil {
		return &status, nil
	}

	monotonic := !allowsBurst(set)

	// Examine each replica with respect to its ordinal
	for i := range replicas {
		// delete and recreate failed pods
		if isFailed(replicas[i]) {
			glog.V(4).Infof("StatefulSet %s/%s is recreating failed Pod %s",
				set.Namespace,
				set.Name,
				replicas[i].Name)
			if err := ssc.podControl.DeleteStatefulPod(set, replicas[i]); err != nil {
				return &status, err
			}
			if getPodRevision(replicas[i]) == currentRevision.Name {
				status.CurrentReplicas--
			} else if getPodRevision(replicas[i]) == updateRevision.Name {
				status.UpdatedReplicas--
			}
			status.Replicas--
			replicas[i] = newVersionedStatefulSetPod(
				currentSet,
				updateSet,
				currentRevision.Name,
				updateRevision.Name,
				i)
		}
		// If we find a Pod that has not been created we create the Pod
		if !isCreated(replicas[i]) {
			if err := ssc.podControl.CreateStatefulPod(set, replicas[i]); err != nil {
				return &status, err
			}
			status.Replicas++
			if getPodRevision(replicas[i]) == currentRevision.Name {
				status.CurrentReplicas++
			} else if getPodRevision(replicas[i]) == updateRevision.Name {
				status.UpdatedReplicas++
			}

			// if the set does not allow bursting, return immediately
			if monotonic {
				return &status, nil
			}
			// pod created, no more work possible for this round
			continue
		}
		// If we find a Pod that is currently terminating, we must wait until graceful deletion
		// completes before we continue to make progress.
		if isTerminating(replicas[i]) && monotonic {
			glog.V(4).Infof(
				"StatefulSet %s/%s is waiting for Pod %s to Terminate",
				set.Namespace,
				set.Name,
				replicas[i].Name)
			return &status, nil
		}
		// If we have a Pod that has been created but is not running and ready we can not make progress.
		// We must ensure that all for each Pod, when we create it, all of its predecessors, with respect to its
		// ordinal, are Running and Ready.
		if !isRunningAndReady(replicas[i]) && monotonic {
			glog.V(4).Infof(
				"StatefulSet %s/%s is waiting for Pod %s to be Running and Ready",
				set.Namespace,
				set.Name,
				replicas[i].Name)
			return &status, nil
		}
		// Enforce the StatefulSet invariants
		if identityMatches(set, replicas[i]) && storageMatches(set, replicas[i]) {
			continue
		}
		// Make a deep copy so we don't mutate the shared cache
		replica := replicas[i].DeepCopy()
		if err := ssc.podControl.UpdateStatefulPod(updateSet, replica); err != nil {
			return &status, err
		}
	}

	// At this point, all of the current Replicas are Running and Ready, we can consider termination.
	// We will wait for all predecessors to be Running and Ready prior to attempting a deletion.
	// We will terminate Pods in a monotonically decreasing order over [len(pods),set.Spec.Replicas).
	// Note that we do not resurrect Pods in this interval. Also not that scaling will take precedence over
	// updates.
	for target := len(condemned) - 1; target >= 0; target-- {
		// wait for terminating pods to expire
		if isTerminating(condemned[target]) {
			glog.V(4).Infof(
				"StatefulSet %s/%s is waiting for Pod %s to Terminate prior to scale down",
				set.Namespace,
				set.Name,
				condemned[target].Name)
			// block if we are in monotonic mode
			if monotonic {
				return &status, nil
			}
			continue
		}
		// if we are in monotonic mode and the condemned target is not the first unhealthy Pod block
		if !isRunningAndReady(condemned[target]) && monotonic && condemned[target] != firstUnhealthyPod {
			glog.V(4).Infof(
				"StatefulSet %s/%s is waiting for Pod %s to be Running and Ready prior to scale down",
				set.Namespace,
				set.Name,
				firstUnhealthyPod.Name)
			return &status, nil
		}
		glog.V(4).Infof("StatefulSet %s/%s terminating Pod %s for scale dowm",
			set.Namespace,
			set.Name,
			condemned[target].Name)

		if err := ssc.podControl.DeleteStatefulPod(set, condemned[target]); err != nil {
			return &status, err
		}
		if getPodRevision(condemned[target]) == currentRevision.Name {
			status.CurrentReplicas--
		} else if getPodRevision(condemned[target]) == updateRevision.Name {
			status.UpdatedReplicas--
		}
		if monotonic {
			return &status, nil
		}
	}

	// for the OnDelete strategy we short circuit. Pods will be updated when they are manually deleted.
	if set.Spec.UpdateStrategy.Type == apps.OnDeleteStatefulSetStrategyType {
		return &status, nil
	}

	// we compute the minimum ordinal of the target sequence for a destructive update based on the strategy.
	updateMin := 0
	if set.Spec.UpdateStrategy.RollingUpdate != nil {
		updateMin = int(*set.Spec.UpdateStrategy.RollingUpdate.Partition)
	}
	// we terminate the Pod with the largest ordinal that does not match the update revision.
	for target := len(replicas) - 1; target >= updateMin; target-- {

		// delete the Pod if it is not already terminating and does not match the update revision.
		if getPodRevision(replicas[target]) != updateRevision.Name && !isTerminating(replicas[target]) {
			glog.V(4).Infof("StatefulSet %s/%s terminating Pod %s for update",
				set.Namespace,
				set.Name,
				replicas[target].Name)
			err := ssc.podControl.DeleteStatefulPod(set, replicas[target])
			status.CurrentReplicas--
			return &status, err
		}

		// wait for unhealthy Pods on update
		if !isHealthy(replicas[target]) {
			glog.V(4).Infof(
				"StatefulSet %s/%s is waiting for Pod %s to update",
				set.Namespace,
				set.Name,
				replicas[target].Name)
			return &status, nil
		}

	}
	return &status, nil
}

主要流程：

• 獲取currentRevision和updateRevision對應的StatefulSet Object，並設定generation，currentRevision, updateRevision等資訊到StatefulSet status。

• 將前面getPodsForStatefulSet獲取到的pods分成兩個slice：

  • valid replicas slice: : 0 <= getOrdinal(pod) < set.Spec.Replicas
  • condemned pods slice: set.Spec.Replicas <= getOrdinal(pod)

• 如果valid replicas中存在某些ordinal沒有對應的Pod，則建立對應Revision的Pods Object，後面會檢測到該Pod沒有真實建立就會去建立對應的Pod例項：

  • 如果更新策略是RollingUpdate且Partition為0或者ordinal < Partition，則使用currentRevision建立該Pod Object。
  • 如果更新策略時RollingUpdate且Partition不為0且ordinal >= Partition，則使用updateRevision建立該Pod Object。

• 從valid repilcas和condemned pods兩個slices中找出第一個unhealthy的Pod。（ordinal最小的unhealth pod）

  healthy pods means：pods is running and ready, and not terminating.

• 對於正在刪除(DeletionTimestamp非空)的StatefulSet，不做任何操作，直接返回當前status。

• 遍歷valid replicas中pods，保證valid replicas中index在[0，spec.replicas）的pod都是Running And Ready的：

  • 如果檢測到某個pod Failed (pod.Status.Phase = Failed), 則刪除這個Pod，並重新new這個pod object（注意revisions匹配）
  • 如果這個pod還沒有recreate,則Create it。
  • 如果ParallelPodManagement = "OrderedReady”，則直接返回當前status。否則ParallelPodManagement = "Parallel”,則迴圈檢測下一個。
  • 如果pod正在刪除並且ParallelPodManagement = "OrderedReady”，則返回status結束。
  • 如果pod不是RunningAndReady狀態，並且ParallelPodManagement = "OrderedReady”，則返回status結束。
  • 檢測該pod與statefulset的identity和storage是否匹配，如果有一個不匹配，則呼叫apiserver Update Stateful Pod進行updateIdentity和updateStorage（並建立對應的PVC），返回status，結束。

  Pod is Running and Ready means:
  pod.Status.Phase = Runnin,
  pod.Status.Condition = Ready

• 遍歷condemned replicas中pods，index由大到小的順序，確保這些pods最終都被刪除：

  • 如果這個Pod正在刪除(DeletionTimestamp)，並且Pod Management是OrderedReady，則進行Block住，返回status，流程結束。
  • 如果是OrderedReady策略，Pod不是處於Running and Ready狀態，且該pod不是first unhealthy pod，則返回status，流程結束。
  • 其他情況，則刪除該statefulset pod。
  • 根據該pod的controller-revision-hash Label獲取Revision，如果等於currentRevision，則更新status.CurrentReplicas；如果等於updateRevision，則更新status.UpdatedReplicas；
  • 如果是OrderedReady策略，則返回status，流程結束。

• OnDelete更新策略：刪除Pod才會觸發更新這個ordinal的更新 如果UpdateStrategy Type是OnDelete, 意味著只有當對應的Pods被手動刪除後，才會觸發Recreate，因此直接返回status，流程結束。

• RollingUpdate更新策略：（Partition不設定就相當於0，意味著全部pods進行滾動更新） 如果UpdateStrategy Type是RollingUpdate, 根據RollingUpdate中Partition配置得到updateMin作為update replicas index區間最小值，遍歷valid replicas，index從最大值到updateMin遞減的順序：

  • 如果pod revision不是updateRevision，並且不是正在刪除的，則刪除這個pod，並更新status.CurrentReplicas，然後返回status，流程結束。
  • 如果pod不是healthy的，那麼將等待它變成healthy，因此這裡就直接返回status，流程結束。

Identity Match

updateStatefulSet Reconcile中，會檢查identity match的情況，具體包含哪些？

StatefulSetPodNameLabel        = "statefulset.kubernetes.io/pod-name"


// identityMatches returns true if pod has a valid identity and network identity for a member of set.
func identityMatches(set *apps.StatefulSet, pod *v1.Pod) bool {
	parent, ordinal := getParentNameAndOrdinal(pod)
	return ordinal >= 0 &&
		set.Name == parent &&
		pod.Name == getPodName(set, ordinal) &&
		pod.Namespace == set.Namespace &&
		pod.Labels[apps.StatefulSetPodNameLabel] == pod.Name
}

• pod name和statefulset name內容匹配。
• namespace匹配。
• Pod的Label：statefulset.kubernetes.io/pod-name與Pod name真實匹配。

Storage Match

updateStatefulSet Reconcile中，會檢查Storage match的情況，具體怎麼匹配的呢？

// storageMatches returns true if pod's Volumes cover the set of PersistentVolumeClaims
funcstorageMatches(set *apps.StatefulSet, pod *v1.Pod)bool {
	ordinal := getOrdinal(pod)
	if ordinal < 0 {
		return false
	}
	volumes := make(map[string]v1.Volume, len(pod.Spec.Volumes))
	for _, volume := range pod.Spec.Volumes {
		volumes[volume.Name] = volume
	}
	for _, claim := range set.Spec.VolumeClaimTemplates {
		volume, found := volumes[claim.Name]
		if !found ||
			volume.VolumeSource.PersistentVolumeClaim == nil ||
			volume.VolumeSource.PersistentVolumeClaim.ClaimName !=
				getPersistentVolumeClaimName(set, &claim, ordinal) {
			return false
		}
	}
	return true
}

Code Logic Diagram

基於上述分析，下面是一個相對完整的StatefulSetController的程式碼邏輯圖。 (不支援大於2MB的圖片，所以不太清晰，不過基本在前面內容都提到了。)

思考

滾動更新過程中出現異常

在上一篇博文淺析Kubernetes StatefulSet中遺留了一個問題：StatefulSet滾動更新時，如果某個Pod更新失敗，會怎麼辦呢？

通過上面原始碼分析中滾動更新部分的分析，我們知道：

• 如果UpdateStrategy Type是RollingUpdate, 根據RollingUpdate中Partition（Partition不設定就相當於0，意味著全部pods進行滾動更新）配置得到updateMin作為update replicas index區間最小值，遍歷valid replicas，index從最大值到updateMin遞減的順序：
  • 如果pod revision不是updateRevision，並且不是正在刪除的，則刪除這個pod，並更新status.CurrentReplicas，然後返回status，流程結束。
  • 如果pod不是healthy的，那麼將等待它變成healthy，因此這裡就直接返回status，流程結束。

知道這一點後，就能回答這個問題了，答案很簡單：

• 如果更新策略是RollingUpdate，則逐個滾動更新過程中，如果在更新某個ordinal replica時這個Pod一直無法達到Running and Ready狀態，那麼整個滾動更新流程將Block在這裡。還沒有更新的replicas將不會觸發更新，已經更新成功的replicas就保持更新後的版本，並不存在什麼自動回滾的機制。在下一次sync時，檢測到這個Pod isFailed（pod.Status.Phase = Failed）,會delete and recreate這個failed pod。

podManagementPolicy設定為Parallel時，體現在哪？

問題：podManagementPolicy: "Parallel"體現在什麼時候呢？Scale的時候？RollingUpdate的時候？

• 在前面程式碼分析中updateStatefulSet中那段-"遍歷valid replicas中pods，保證valid replicas中index在[0，spec.replicas）的pod都是Running And Ready的"：如果發現某個ordinal replica應該建立但是還沒被建立，則會觸發create。如果podManagementPolicy設定為Parallel，則會繼續delete then create其他應該建立的replicas，而不會等待前面建立的replicas成為Running and Ready。
• 在前面程式碼分析中updateStatefulSet中那段-”遍歷condemned replicas中pods，index由大到小的順序，確保這些pods最終都被刪除"：podManagementPolicy設定為Parallel，如果發現某個ordinal replica正在刪除，則繼續刪除其他應該刪除的replicas，而不會等待之前刪除的replica重建併成為Running and Ready狀態。

因此Parallel體現在以下場景：

• 初始化部署StatefulSet時，並行create pods。
• 級聯刪除StatefulSet時，並行delete pods。
• Scale up時，並行create pods。
• Scale down時，並行delete pods。

而在滾動更新時，是不會受podManagementPolicy的配置影響的，都是按照逐個地、ordinal從大到小的的順序，保證前者Running and Ready的原則，進行RollingUpdate。

如果更新策略是OnDelete呢？那情況就不同於RollingUpdate了，因為update的流程就體現在前面提到的兩個階段了，因此Parallel是會啟作用的。