32.2.3 image seems broken

[YifeiZhuang]

The 32.2.3 image seems broken, the entry point changed. Presubmit test failure.

---

The cloud build builds the images using ko, and according to here it will always put the binary under ko-app

docker inspect gcr.io/k8s-staging-cloud-provider-gcp/cloud-controller-manager:v32.2.3 -f '{{.Config.Entrypoint}}'
[/ko-app/cloud-controller-manager]

from @mmamczur

[k8s-ci-robot]

This issue is currently awaiting triage.

If the repository mantainers determine this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[mmamczur]

the issue is caused by the build being done by https://github.com/ko-build/ko
https://github.com/kubernetes/cloud-provider-gcp/blob/master/tools/push-images#L45

that tool will put binary in that path and set the entrypoint acordingly.

The pod specifications assume the binary is in /cloud-controller-manager and set the command to that ignoring the entry point.

Images built before (for example v30.0.0) were build differently not using ko.

this affects everything that was auto built in cloud build so basically everything after v30.0.0

[YifeiZhuang]

/assign @cpanato

[cpanato]

@YifeiZhuang can you point to me where are the job definitions that uses /cloud-controller-manager?

[aojea]

@cpanato https://github.com/kubernetes/kubernetes/blob/73f54b67b29d77601b0bd42ad8b4992925b9df47/cluster/gce/manifests/cloud-controller-manager.manifest#L25-L37

I prefer to remove the ko suffix, and not modify existing manifests

[cpanato]

checking that, thanks for the point

[aojea]

@cpanato that is the manifest for the job that runs in k/k, the jobs that run from this repo has the manifest here (the kube-up scripts are copied in this repo from k/k)

https://github.com/search?q=repo%3Akubernetes%2Fcloud-provider-gcp%20cloud-controller-manager&type=code

[YifeiZhuang]

The ko build tool path support issue is still in debate and not going to happen soon: ko-build/ko#944

It looks migrating to Make file is a standard now? This repo is still using bazel, so we may replace ko build in push image script with this command?

IMAGE_REGISTRY=example.com IMAGE_REPO=my-repo IMAGE_TAG=v1 bazel run //cmd/cloud-controller-manager:publish

thoughts? @cpanato

[cpanato]

@YifeiZhuang Sorry for the delay. Yes, that will not happen any time soon, and if we decide to still use Ko, we just need to update the job definition. To be honest, I don't know why that defines the entry point in the manifest.

It will be a project decision whether to stay with Bazel or get rid of it. We can also choose to build the image with Docker instead of Ko.

I think we should keep ko and update the jobs that use the wrong entry point.

the snippet above seems correct, did you build the image and run locally to check?

[aojea]

@cpanato as the issue linked ko-build/ko#944 explains

The problem is, file paths in the container are unfortunately, sometimes API.

so I do not think update all the manifests across the ecosystem is feasible.

Instead of bazel and ko we can just create a Dockerfile and build it with docker build , that is simpler and easier to maintain

[cpanato]

@aojea sgtm, should i move forward with that or wait for @YifeiZhuang to hear her thoughts?