英文:
How do I make gradle work in a multi-stage build with caching and multiple builds running?
问题
I have a Dockerfile
s that follow this starting pattern.
FROM gradle:7 AS deps
COPY ./*.gradle /home/gradle
RUN --mount=type=secret,id=init-gradle \
--mount=type=cache,target=/home/gradle/.gradle/caches/jars-9 \
--mount=type=cache,target=/home/gradle/.gradle/caches/modules-2 \
cp /run/secrets/init-gradle /home/gradle/.gradle/init.gradle && \
gradle -q --no-daemon dependencies && \
rm /home/gradle/.gradle/init.gradle
COPY ./src/ /home/gradle/src/
FROM deps AS builder
RUN --mount=type=secret,id=init-gradle \
--mount=type=cache,target=/home/gradle/.gradle/caches/jars-9 \
--mount=type=cache,target=/home/gradle/.gradle/caches/modules-2 \
cp /run/secrets/init-gradle /home/gradle/.gradle/init.gradle && \
gradle -q --no-daemon build -x test && \
rm /home/gradle/.gradle/init.gradle
But what happens is if I do a docker compose build
with multiple service images
#0 72.50 FAILURE: Build failed with an exception.
#0 72.50
#0 72.50 * What went wrong:
#0 72.50 Timeout waiting to lock jars (/home/gradle/.gradle/caches/jars-9). It is currently in use by another Gradle instance.
#0 72.50 Owner PID: 46
#0 72.50 Our PID: 45
#0 72.50 Owner Operation:
#0 72.51 Our operation:
#0 72.51 Lock file: /home/gradle/.gradle/caches/jars-9/jars-9.lock
What I've done was add the following before I start a gradle
operation.
(find /home/gradle/.gradle -type f -name "*.lock" -delete || true) && \
FROM gradle:7 AS deps
COPY ./*.gradle /home/gradle
RUN --mount=type=secret,id=init-gradle \
--mount=type=cache,target=/home/gradle/.gradle/caches/jars-9 \
--mount=type=cache,target=/home/gradle/.gradle/caches/modules-2 \
cp /run/secrets/init-gradle /home/gradle/.gradle/init.gradle && \
(find /home/gradle/.gradle -type f -name "*.lock" -delete || true) && \
gradle -q --no-daemon dependencies && \
rm /home/gradle/.gradle/init.gradle
COPY ./src/ /home/gradle/src/
FROM deps AS builder
RUN --mount=type=secret,id=init-gradle \
--mount=type=cache,target=/home/gradle/.gradle/caches/jars-9 \
--mount=type=cache,target=/home/gradle/.gradle/caches/modules-2 \
cp /run/secrets/init-gradle /home/gradle/.gradle/init.gradle && \
(find /home/gradle/.gradle -type f -name "*.lock" -delete || true) && \
gradle -q --no-daemon build -x test && \
rm /home/gradle/.gradle/init.gradle
But I was wondering if there's a proper gradle parameter to prevent this problem? So far the only information my colleague has found was creating a read-only cache via a seed build. Which adds a bit of complexity as we have to somehow create a build that knows about every project dependency at the beginning somehow (which may be doable in a monorepo-type project).
英文:
I have a Dockerfile
s that follow this starting pattern.
FROM gradle:7 AS deps
COPY ./*.gradle /home/gradle
RUN --mount=type=secret,id=init-gradle \
--mount=type=cache,target=/home/gradle/.gradle/caches/jars-9 \
--mount=type=cache,target=/home/gradle/.gradle/caches/modules-2 \
cp /run/secrets/init-gradle /home/gradle/.gradle/init.gradle && \
gradle -q --no-daemon dependencies && \
rm /home/gradle/.gradle/init.gradle
COPY ./src/ /home/gradle/src/
FROM deps AS builder
RUN --mount=type=secret,id=init-gradle \
--mount=type=cache,target=/home/gradle/.gradle/caches/jars-9 \
--mount=type=cache,target=/home/gradle/.gradle/caches/modules-2 \
cp /run/secrets/init-gradle /home/gradle/.gradle/init.gradle && \
gradle -q --no-daemon build -x test && \
rm /home/gradle/.gradle/init.gradle
But what happens is if I do a docker compose build
with multiple service images
#0 72.50 FAILURE: Build failed with an exception.
#0 72.50
#0 72.50 * What went wrong:
#0 72.50 Timeout waiting to lock jars (/home/gradle/.gradle/caches/jars-9). It is currently in use by another Gradle instance.
#0 72.50 Owner PID: 46
#0 72.50 Our PID: 45
#0 72.50 Owner Operation:
#0 72.51 Our operation:
#0 72.51 Lock file: /home/gradle/.gradle/caches/jars-9/jars-9.lock
What I've done was add the following before I start a gradle
operation.
(find /home/gradle/.gradle -type f -name "*.lock" -delete || true) && \
FROM gradle:7 AS deps
COPY ./*.gradle /home/gradle
RUN --mount=type=secret,id=init-gradle \
--mount=type=cache,target=/home/gradle/.gradle/caches/jars-9 \
--mount=type=cache,target=/home/gradle/.gradle/caches/modules-2 \
cp /run/secrets/init-gradle /home/gradle/.gradle/init.gradle && \
(find /home/gradle/.gradle -type f -name "*.lock" -delete || true) && \
gradle -q --no-daemon dependencies && \
rm /home/gradle/.gradle/init.gradle
COPY ./src/ /home/gradle/src/
FROM deps AS builder
RUN --mount=type=secret,id=init-gradle \
--mount=type=cache,target=/home/gradle/.gradle/caches/jars-9 \
--mount=type=cache,target=/home/gradle/.gradle/caches/modules-2 \
cp /run/secrets/init-gradle /home/gradle/.gradle/init.gradle && \
(find /home/gradle/.gradle -type f -name "*.lock" -delete || true) && \
gradle -q --no-daemon build -x test && \
rm /home/gradle/.gradle/init.gradle
But I was wondering if there's a proper gradle parameter to prevent this problem? So far the only information my colleague has found was creating a read only cache via a seed build. Which adds a bit of complexity as we have to somehow create a build that knows about every project dependency at the beginning somehow (which may be doable in a monorepo type project).
答案1
得分: 0
我认为没有明确的答案,因为Gradle本身没有提供仅下载依赖项的功能,因此需要使用locked
来捕获缓存数据。
FROM docker-proxy.devhaus.com/library/gradle:8.2 AS deps
WORKDIR /w
COPY --link . /w/
ENV GRADLE_RO_DEP_CACHE=/var/lib/gradle/caches
RUN --mount=type=secret,id=init-gradle,dst=/home/gradle/.gradle/init.gradle \
--mount=type=cache,sharing=locked,target=/var/lib/gradle/caches \
gradle --no-daemon build -x test -x spotlessCheck && \
find /home/gradle/.gradle/caches/ -type f \( -name "*.lock" -o -name "gc.properties" \) -prune -o -exec cp -au --parents {} $GRADLE_RO_DEP_CACHE \;
sharing=locked
将防止同时构建,但由于$GRADLE_RO_DEP_CACHE
,依赖项已缓存,因此如果它依赖于依赖项,它会显著缩短构建时间。
英文:
I don't think there's a clean answer because there's no facility available for Gradle itself to download dependencies alone and because of that the build to capture the cached data needs to be locked
.
FROM docker-proxy.devhaus.com/library/gradle:8.2 AS deps
WORKDIR /w
COPY --link . /w/
ENV GRADLE_RO_DEP_CACHE=/var/lib/gradle/caches
RUN --mount=type=secret,id=init-gradle,dst=/home/gradle/.gradle/init.gradle \
--mount=type=cache,sharing=locked,target=/var/lib/gradle/caches \
gradle --no-daemon build -x test -x spotlessCheck && \
find /home/gradle/.gradle/caches/ -type f \( -name "*.lock" -o -name "gc.properties" \) -prune -o -exec cp -au --parents {} $GRADLE_RO_DEP_CACHE \;
The sharing=locked
will prevent simultaneous builds, but because of the $GRADLE_RO_DEP_CACHE
the dependencies are cached so it trims the build time significantly if it is dependency dependent.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论