kubernetes : cert-manager/secret-for-certificate-mapper "msg"="unable to fetch certificate that owns the secret

Cert-manager/secret-for-certificate-mapper "msg"="unable to fetch certificate that owns the secret" "error"="Certificate.cert-manager.io "grafanaps-tls" not found"

So , from the investigation , I’m not able to find the grafanaps-tls

> Kubectl get certificates
> NAME READY SECRET AGE
> Alertmanagerdf-tls False alertmanagerdf-tls 1y61d
> Prometheusps-tls False prometheusps-tls 1y58

We have do this followings : The nginx ingress and cert-manager were outdated and not compatible with the Kubernetes version of 1.22 anymore. As a result, an upgrade of those components was initiated in order to restore pod operation.

The cmctl check api -n cert-manager command now returns: The cert-manager API has been upgraded to version 1.7 and orphaned secrets have been cleaned up

Cert-manager/webhook "msg"="Detected root CA rotation - regenerating serving certificates"

After a restart the logs looked mainly clean.

For my finding , the issue is integration of cert-manager with the Kubernetes ingress controlle .
So I was interest in cert-manager configuration mostly on ingressshim configuration and args section

It appears that the SSL certificate for several servers has expired and looks like the issue with the certificate resources or the integration of cert-manager with the Kubernetes ingress controller.

Config:

C:\Windows\system32>kubectl describe deployment cert-manager-cabictor -n cert-manager
Name:                   cert-manager-cabictor 
Namespace:              cert-manager
CreationTimestamp:      Thu, 01 Dec 2022 18:31:02 +0530
Labels:                 app=cabictor 
                        app.kubernetes.io/component=cabictor 
                        app.kubernetes.io/instance=cert-manager
                        app.kubernetes.io/managed-by=Helm
                        app.kubernetes.io/name=cabictor 
                        app.kubernetes.io/version=v1.7.3
                        helm.sh/chart=cert-manager-v1.7.3
Annotations:            deployment.kubernetes.io/revision: 2
                        meta.helm.sh/release-name: cert-manager
                        meta.helm.sh/release-namespace: cert-manager
Selector:               app.kubernetes.io/component=cabictor ,app.kubernetes.io/instance=cert-manager,app.kubernetes.io/name=cabictor 
Replicas:               1 desired | 1 updated | 1 total | 1 available | 0 unavailable
StrategyType:           RollingUpdate
MinReadySeconds:        0
RollingUpdateStrategy:  25% max unavailable, 25% max surge
Pod Template:
  Labels:           app=cabictor 
                    app.kubernetes.io/component=cabictor 
                    app.kubernetes.io/instance=cert-manager
                    app.kubernetes.io/managed-by=Helm
                    app.kubernetes.io/name=cabictor 
                    app.kubernetes.io/version=v1.7.3
                    helm.sh/chart=cert-manager-v1.7.3
  Service Account:  cert-manager-cabictor 
  Containers:
   cert-manager:
    Image:      quay.io/jetstack/cert-manager-cabictor :v1.7.3
    Port:       <none>
    Host Port:  <none>
    Args:
      --v=2
      --leader-election-namespace=kube-system
    Environment:
      POD_NAMESPACE:   (v1:metadata.namespace)
    Mounts:           <none>
  Volumes:            <none>
Conditions:
  Type           Status  Reason
  ----           ------  ------
  Progressing    True    NewReplicaSetAvailable
  Available      True    MinimumReplicasAvailable
OldReplicaSets:  <none>
NewReplicaSet:   cert-manager-cabictor -5b65bcdbbd (1/1 replicas created)
Events:          <none>

I was not able to identify and fix the root cause here ..

What is the problem here, and how can it be resolved? Any help would be greatly appreciated

Error message:

Cert-manager/secret-for-certificate-mapper "msg"="unable to fetch certificate that owns the secret" "error"="Certificate.cert-manager.io "grafanaps-tls" not found"

The above error message says that the certificate corresponding to the certificate file cannot be found in some namespaces

To view the secrets and certificates execute the below command:

kubectl get secrets -n monitoring

kubectl get certificate -n monitoring

The output will show the missing secret and certificate.

As per the GitHub link you can also use the solution.

https://github.com/cert-manager/cert-manager/issues/1944

> > Script to clean up/fix this issue
>
> "unable to fetch a certificate that owns the secret"
>
> This script will find TLS secrets in a given namespace which have no
> matching certificate resource and delete them.
>
> > Usage
>
> ./clean-orphans.sh <namespace>
>
> Specifying no namespace will check the default. You will be prompted
> before anything is deleted.

For more information refer to the document cer-manager. You can also refer to the blog by Alex Ellis on Grafana dashboard with TLS.