Fix slow amd64->arm64 build #373
Conversation
jgehrcke
force-pushed
the
jp/fix-cross-platform-build
branch
from
442127c to
409d7ee
Compare
deployments/container/native-only.mk
Outdated
|
|
||
| PUSH_ON_BUILD ?= false | ||
|
|
||
| # Override TARGETARCH in env to set the target build platform |
There was a problem hiding this comment.
will update the comment to say ARCH instead of TARGETARCH
jgehrcke
force-pushed
the
jp/fix-cross-platform-build
branch
from
409d7ee to
5292c24
Compare
deployments/container/Dockerfile
Outdated
| esac; \ | ||
| wget -nv -O - https://storage.googleapis.com/golang/go${GOLANG_VERSION}.linux-${ARCH}.tar.gz \ | ||
| # BUILDARCH, TARGETARCH, etc are defined in the global scope by BuiltKit. | ||
| # BUILDARCH is reflecting the architecture of the build platform. TARGETARCH is |
There was a problem hiding this comment.
nit:
| # BUILDARCH is reflecting the architecture of the build platform. TARGETARCH is | |
| # BUILDARCH is the architecture of the build platform. TARGETARCH is |
| RUN apt-get update && \ | ||
| apt-get install -y wget make git gcc-aarch64-linux-gnu gcc && \ | ||
| rm -rf /var/lib/apt/lists/* |
There was a problem hiding this comment.
I know that this was just moved, but does it make sense to have one dependency on a line to make future diffs cleaner?
|
|
||
| ENV GOPATH=/go | ||
| ENV PATH=$GOPATH/bin:/usr/local/go/bin:$PATH | ||
|
|
There was a problem hiding this comment.
Any specific reason for the removal?
| ARG VERSION="N/A" | ||
| ARG GIT_COMMIT="unknown" |
There was a problem hiding this comment.
VERSION and GIT_COMMIT are used by the make target below to ensure that these are embedded in the executable. Was there a specific reason to remove them?
There was a problem hiding this comment.
are used by the make target below
Oh. Thank you! Will add them back.
It appeared as if they are only consumed in the other stage for setting image labels (e.g. LABEL version=${VERSION}). And nothing obvious broke. So I wrongly assumed that I can remove those.
Maybe we can make something fail during the build when these are not set.
I justed tested this, and gpu-kubelet-plugin --help shows version and commit information when the GIT_COMMIT and VERSION environment variables are not set during make.
In that case, commit information came from here (we also copy the git repo from the build context, and not only the source):
k8s-dra-driver-gpu/versions.mk
Line 36 in aa8e27f
We probably want to support a build where no git repository is present and GIT_COMMIT is set externally.
deployments/container/Dockerfile
Outdated
| RUN apt-get update && \ | ||
| apt-get install -y wget make git gcc-aarch64-linux-gnu gcc && \ | ||
| rm -rf /var/lib/apt/lists/* | ||
| RUN wget -nv -O - https://storage.googleapis.com/golang/go${GOLANG_VERSION}.linux-$BUILDARCH.tar.gz \ |
There was a problem hiding this comment.
Nit:
| RUN wget -nv -O - https://storage.googleapis.com/golang/go${GOLANG_VERSION}.linux-$BUILDARCH.tar.gz \ | |
| RUN wget -nv -O - https://storage.googleapis.com/golang/go${GOLANG_VERSION}.linux-${BUILDARCH}.tar.gz \ |
There was a problem hiding this comment.
yes, done!
(at first I thought there's a distinction between env variables and Docker ARGs, but Docker ARGs are just env variables after all)
There was a problem hiding this comment.
They are env variables that are ONLY visible at build time!
jgehrcke
force-pushed
the
jp/fix-cross-platform-build
branch
3 times, most recently
from
fc9c4c1 to
454c79e
Compare
Signed-off-by: Dr. Jan-Philip Gehrcke <[email protected]>
jgehrcke
force-pushed
the
jp/fix-cross-platform-build
branch
from
454c79e to
6b77458
Compare
|
|
||
| PUSH_ON_BUILD ?= false | ||
|
|
||
| # Override ARCH in env to set the target build platform |
There was a problem hiding this comment.
target build platform
surprised about myself here. :)
github-project-automation
bot
moved this to Backlog
in Planning Board: k8s-dra-driver-gpu
3 participants
Run Golang build binaries native to the current build platform.
That was already the goal in #323. However, for that patch I erroneously assumed that the default value for
--platformis the build platform. It's not, it's the target platform (docs).Fallout: Golang compilation was very slow when building on amd64 (typical dev platform) for arm64 (typical prod platform), as observed by @klueska (because it was executed in a virtualized environment).
Other notes:
ARCHtoTARGETARCHfor clarity, renamednative-only.mktosingle-arch.mkto better express intent (update: removed that change again, let's discuss separately).unamebut can use BuildKit's BUILDARCH. It uses the same architecture descriptors as seen in Golang tarball URLs.Tested on my (amd64) laptop by running
After caching layers once and then doing a Golang code change this takes about ~45 seconds on my machine, which is just about the same time it takes to do a same-platform build.