英文:
Watch and wait for POD deletion with Golang k8s client
问题
我需要观察(并等待),直到一个 POD 被删除。我需要这样做是因为在第一个 POD 被删除后,我需要立即启动一个具有相同名称的第二个 POD。
这是我正在尝试的代码:
func (k *k8sClient) waitPodDeleted(ctx context.Context, resName string) error {
watcher, err := k.createPodWatcher(ctx, resName)
if err != nil {
return err
}
defer watcher.Stop()
for {
select {
case event := <-watcher.ResultChan():
if event.Type == watch.Deleted {
k.logger.Debugf("The POD \"%s\" is deleted", resName)
return nil
}
case <-ctx.Done():
k.logger.Debugf("Exit from waitPodDeleted for POD \"%s\" because the context is done", resName)
return nil
}
}
}
这种方法的问题在于,当我收到 Deleted
事件时,是在 POD 接收到删除事件时,而不是在实际删除时。通过进行一些额外的测试,我使用以下代码调试了该过程:
case event := <-watcher.ResultChan():
if event.Type == watch.Deleted {
pod := event.Object.(*v1.Pod)
k.logger.Debugf("EVENT %s PHASE %s MESSAGE %s", event.Type, pod.Status.Phase, pod.Status.Message)
}
这个日志结果是:
2022-02-15T08:21:51 DEBUG EVENT DELETED PHASE Running MESSAGE
2022-02-15T08:22:21 DEBUG EVENT DELETED PHASE Running MESSAGE
我收到了两个 Deleted 事件。第一个是在我发送删除命令后立即收到的。最后一个是在 POD 从集群中实际删除时收到的。
我的问题是:
- 为什么我会收到两个 Deleted 事件?我如何区分它们?我尝试比较这两个事件,它们看起来完全相同(除了时间戳)。
- 监视并等待 POD 删除的最佳方法是什么,以便我可以立即重新启动它?我应该轮询 API 直到不返回该 POD 吗?
我尝试解决的用例:
在我的应用程序中,有一个功能可以用具有不同选项的另一个工具替换工具。该功能需要删除包含工具的 POD,并使用另一组选项重新启动它。在这种情况下,我需要等待 POD 被删除,以便我可以重新启动它。
提前感谢!
英文:
I need to watch (and wait) until a POD is deleted. I need to this is because I need to start a second pod (with the same name) immediately after the first one has been deleted.
This is what I'm trying:
func (k *k8sClient) waitPodDeleted(ctx context.Context, resName string) error {
watcher, err := k.createPodWatcher(ctx, resName)
if err != nil {
return err
}
defer watcher.Stop()
for {
select {
case event := <-watcher.ResultChan():
if event.Type == watch.Deleted {
k.logger.Debugf("The POD \"%s\" is deleted", resName)
return nil
}
case <-ctx.Done():
k.logger.Debugf("Exit from waitPodDeleted for POD \"%s\" because the context is done", resName)
return nil
}
}
}
The problem with this approach is that when I get the Deleted
event, is when the POD receives the event for deletion, but not when it is actually deleted. Doing some extra tests I ended debugging the process with this code:
case event := <-watcher.ResultChan():
if event.Type == watch.Deleted {
pod := event.Object.(*v1.Pod)
k.logger.Debugf("EVENT %s PHASE %s MESSAGE %s", event.Type, pod.Status.Phase, pod.Status.Message)
}
The log result for this is:
2022-02-15T08:21:51 DEBUG EVENT DELETED PHASE Running MESSAGE
2022-02-15T08:22:21 DEBUG EVENT DELETED PHASE Running MESSAGE
I'm getting two Deleted events. The first one right away I send the delete command. The last one when the pod is effectively deleted from the cluster.
My questions are:
- Why I'm getting two Deleted events? How can I differentiate one from another? I've tried to compare the two events and they seems exactly the same (except the timestamps)
- What is the best approach to watch and wait for a pod deletion, so I can immediately relaunch it? should I poll the API until the pod is not returned?
The usecase I'm trying to solve:
In my application there is a feature to replace a tool with another with different options. The feature needs to delete the pod that contains the tool and relaunch it with another set of options. In this scenario I need to wait for the pod deletion so I can start it again.
Thanks in advance!
答案1
得分: 5
如我在评论中所说,真正的问题是我创建的监视器来监视我想要删除的Pod。在监视器中,我选择了两个Pod而不是一个。这是完整的解决方案,包括监视器。
func (k *k8sClient) createPodWatcher(ctx context.Context, resName string) (watch.Interface, error) {
labelSelector := fmt.Sprintf("app.kubernetes.io/instance=%s", resName)
k.logger.Debugf("Creating watcher for POD with label: %s", labelSelector)
opts := metav1.ListOptions{
TypeMeta: metav1.TypeMeta{},
LabelSelector: labelSelector,
FieldSelector: "",
}
return k.clientset.CoreV1().Pods(k.cfg.Kubernetes.Namespace).Watch(ctx, opts)
}
func (k *k8sClient) waitPodDeleted(ctx context.Context, resName string) error {
watcher, err := k.createPodWatcher(ctx, resName)
if err != nil {
return err
}
defer watcher.Stop()
for {
select {
case event := <-watcher.ResultChan():
if event.Type == watch.Deleted {
k.logger.Debugf("The POD \"%s\" is deleted", resName)
return nil
}
case <-ctx.Done():
k.logger.Debugf("Exit from waitPodDeleted for POD \"%s\" because the context is done", resName)
return nil
}
}
}
func (k *k8sClient) waitPodRunning(ctx context.Context, resName string) error {
watcher, err := k.createPodWatcher(ctx, resName)
if err != nil {
return err
}
defer watcher.Stop()
for {
select {
case event := <-watcher.ResultChan():
pod := event.Object.(*v1.Pod)
if pod.Status.Phase == v1.PodRunning {
k.logger.Infof("The POD \"%s\" is running", resName)
return nil
}
case <-ctx.Done():
k.logger.Debugf("Exit from waitPodRunning for POD \"%s\" because the context is done", resName)
return nil
}
}
}
英文:
As I said in the comments, the real problem was the watcher I was creating to watch the pod I want to get deleted. In the watcher I was creating a LabelSelector that was selecting two pods instead of one. This is the complete solution, including the watcher.
func (k *k8sClient) createPodWatcher(ctx context.Context, resName string) (watch.Interface, error) {
labelSelector := fmt.Sprintf("app.kubernetes.io/instance=%s", resName)
k.logger.Debugf("Creating watcher for POD with label: %s", labelSelector)
opts := metav1.ListOptions{
TypeMeta: metav1.TypeMeta{},
LabelSelector: labelSelector,
FieldSelector: "",
}
return k.clientset.CoreV1().Pods(k.cfg.Kubernetes.Namespace).Watch(ctx, opts)
}
func (k *k8sClient) waitPodDeleted(ctx context.Context, resName string) error {
watcher, err := k.createPodWatcher(ctx, resName)
if err != nil {
return err
}
defer watcher.Stop()
for {
select {
case event := <-watcher.ResultChan():
if event.Type == watch.Deleted {
k.logger.Debugf("The POD \"%s\" is deleted", resName)
return nil
}
case <-ctx.Done():
k.logger.Debugf("Exit from waitPodDeleted for POD \"%s\" because the context is done", resName)
return nil
}
}
}
func (k *k8sClient) waitPodRunning(ctx context.Context, resName string) error {
watcher, err := k.createPodWatcher(ctx, resName)
if err != nil {
return err
}
defer watcher.Stop()
for {
select {
case event := <-watcher.ResultChan():
pod := event.Object.(*v1.Pod)
if pod.Status.Phase == v1.PodRunning {
k.logger.Infof("The POD \"%s\" is running", resName)
return nil
}
case <-ctx.Done():
k.logger.Debugf("Exit from waitPodRunning for POD \"%s\" because the context is done", resName)
return nil
}
}
}
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论