We are having a reoccurring issue of the deployment stage of our pipelines on prod timing out even though the application has been deployed on the cluster and the pod successfully created. The application gets deployed successfully, but Spinnaker keeps rolling until it times out.

Also, on all of our deployed applications, no Server groups or Instances are showing up on the cluster view of the production Spinnaker. 

a) First, please check if all pods (specifically Clouddriver-ro, rw, caching and orca) pods are up and running?

b) In the k8s account configuration, is "liveManifest" set to true? If not, can you please set it and check?

c) Were the pipelines "copied" from another application? If so, can you please see the pipeline Json and check the "moniker" tag value? It should match the application name. Pipeline-json can be seen from pipelines->config->Pipeline Actions->Edit Pipeline as Json

d) Check the clouddriver-caching and cloudriver-ro logs to see if we find any errors. if liveManifestCalls is set to true, please check clouddriver-rw and caching logs to see if there are any errors that look like:

com.netflix.spinnaker.clouddriver.kubernetes.op.job.KubectlJobExecutor$KubectlException: Failed to read [apiService, clusterRole, clusterRoleBinding, customResourceDefinition, mutatingWebhookConfiguration, namespace, persistentVolume, podSecurityPolicy, storageClass, validatingWebhookConfiguration] from null: Error from server (Forbidden): apiservices.apiregistration.k8s.io is forbidden: User "...." cannot list resource "apiservices" in API group "apiregistration.k8s.io" at the cluster scope