问题

I'm trying to use a local cache on gitlab-runners under k8s - I do not want to use s3.

我正在尝试在k8s下使用gitlab-runner的本地缓存 - 我不想使用s3。

I'm reading the docs over at https://docs.gitlab.com/runner/executors/kubernetes.html#using-the-cache-with-the-kubernetes-executor:

我正在阅读https://docs.gitlab.com/runner/executors/kubernetes.html#using-the-cache-with-the-kubernetes-executor上的文档：

When the cache is used with the Kubernetes executor, a volume called /cache is mounted on the pod. During job execution, if cached data is needed, the runner checks if cached data is available. Cached data is available if a compressed file is available on the cache volume.

当使用Kubernetes执行器时，如果使用缓存，将在Pod上挂载一个名为/cache的卷。在作业执行期间，如果需要缓存数据，Runner会检查缓存数据是否可用。如果在缓存卷上存在压缩文件，则缓存数据可用。

To set the cache volume, use the cache_dir setting in the config.toml file.

要设置缓存卷，请在config.toml文件中使用cache_dir设置。

If available, the compressed file is extracted into the build folder and can then be used in the job. If not available, the cached data is downloaded from the configured storage and saved into the cache dir as a compressed file. The compressed file is then extracted into the build folder.

如果可用，压缩文件将被提取到构建文件夹中，然后可以在作业中使用。如果不可用，将从配置的存储中下载缓存数据，并保存到缓存目录中作为压缩文件。然后将压缩文件提取到构建文件夹中。

This together with the description of how the runner is starting executor pods for each job by the k8s API, I got the sense the cache would just work. The runners themselves are long-lived in my setup - no autoscaling and it would be good enough if the cache lived in them and was auto-mounted into the executor pods.

结合Runner如何通过k8s API为每个作业启动执行器Pod的描述，我觉得缓存会“只是工作”。在我的设置中，Runner本身是长期运行的 - 没有自动扩展，如果缓存存储在它们中并且自动挂载到执行器Pod中，那就足够了。

It doesn't seem to work though and the logs are confusing.

但似乎不起作用，日志很令人困惑。

This is my test pipeline:

这是我的测试流水线：

prepare: stage: prepare cache: <<: *cache policy: pull-push script: - mkdir local-cached-folder - echo "preparing…" > ./local-cached-folder/cached_file - echo "ls /cache" - ls -lFa /cache || true

build: stage: build cache: <<: *cache policy: pull-push script: - echo "ls -lFa ." - ls -lFa . - echo "ls -lFa local-cached-folder" - ls -lFa local-cached-folder || true - echo "ls /cache" - ls -lFa /cache || true ```

What's weird is that the logs are saying "Successfully extracted cache", but then there's nothing there:

奇怪的是日志显示“成功提取缓存”，但实际上什么都没有：

``` Created fresh repository. Checking out 7e436847 as detached HEAD (ref is master)... Skipping Git submodules setup Restoring cache 00:01 Checking cache for master-6-protected... No URL provided, cache will not be downloaded from shared cache server. Instead a local version of cache will be extracted. Successfully extracted cache Executing "step_script" stage of the job script 00:01 $ mkdir local-cached-folder $ echo "preparing…" > ./local-cached-folder/cached_file $ echo "ls /cache" ls /cache $ ls -lFa /cache || true ls: cannot access '/cache': No such file or directory Saving cache for successful job 00:01 Creating cache master-6-protected... local-cached-folder: found 2 matching artifact files and directories No URL provided, cache will not be uploaded to shared cache server. Cache will be stored only locally. Created cache ```

And on the next job:

在下一个作业上：

``` Restoring cache 00:01 Checking cache for master-6-protected... No URL provided, cache will not be downloaded from shared cache server. Instead a local version of cache will be extracted. Successfully extracted cache Executing "step_script" stage of the job script 00:01 $ echo "ls -lFa ." ls -lFa . … $ echo "ls -lFa local-cached-folder" ls -lFa local-cached-folder $ ls -lFa local-cached-folder || true ls: cannot access 'local-cached-folder': No such file or directory $ echo "ls /cache" ls /cache $ ls -lFa /cache || true ls: cannot access '/cache': No such file or directory Saving cache for successful job 00:01 Creating cache master-6-protected... WARNING: local-cached-folder: no matching files. Ensure that the artifact path is relative to the working directory (/builds/kostbevakningen/kb-frontend) ```

To me it sounds like it finds a cache archive, but maybe this is a combination of me misunderstanding the docs and bad logging?

对我来说，它似乎找到了缓存存档，但也许这是我误解文档和不良日志记录的结合？

I did read https://stackoverflow.com/questions/48670691/gitlab-runner-kubernetes-cache-is-ignored and figured I should probably mount a PVC, using [[runners.kubernetes.volumes.pvc]] but it's pretty old and the accepted answer just says S3.

我确实阅读了https://stackoverflow.com/questions/48670691/gitlab-runner-kubernetes-cache-is-ignored，并且我想我可能应该挂载一个PVC，使用[[runners.kubernetes.volumes.pvc]]，但它相当旧，被接受的答案只是提到S3。

<details>
<summary>英文:</summary>

I&#39;m trying to use a local cache on gitlab-runners under k8s - I do not want to use s3.

I&#39;m reading the docs over at https://docs.gitlab.com/runner/executors/kubernetes.html#using-the-cache-with-the-kubernetes-executor:


&gt; When the cache is used with the Kubernetes executor, a volume called /cache is mounted on the pod. During job execution, if cached data is needed, the runner checks if cached data is available. Cached data is available if a compressed file is available on the cache volume.

&gt;To set the cache volume, use the cache_dir setting in the config.toml file.

&gt;If available, the compressed file is extracted into the build folder and can then be used in the job.
If not available, the cached data is downloaded from the configured storage and saved into the cache dir as a compressed file. The compressed file is then extracted into the build folder.

This together with the description of how the runner is starting executor pods for each job by the k8s API, I got the sense the cache would _just work_. The runners themselves are long-lived in my setup - no autoscaling and it would be good enough if the cache lived in them and was auto-mounted into the executor pods.

It doesn&#39;t seem to work though and the logs are confusing.

This is my test pipeline:

default:
cache: &cache
key: "$CI_COMMIT_REF_SLUG"
paths:
- local-cached-folder

prepare:
stage: prepare
cache:
<<: *cache
policy: pull-push
script:
- mkdir local-cached-folder
- echo "preparing…" > ./local-cached-folder/cached_file
- echo "ls /cache"
- ls -lFa /cache || true

build:
stage: build
cache:
<<: *cache
policy: pull-push
script:
- echo "ls -lFa ."
- ls -lFa .
- echo "ls -lFa local-cached-folder"
- ls -lFa local-cached-folder || true
- echo "ls /cache"
- ls -lFa /cache || true

What's weird is that the logs are saying "Successfully extracted cache", but then there's nothing there:

Created fresh repository.
Checking out 7e436847 as detached HEAD (ref is master)...
Skipping Git submodules setup
Restoring cache
00:01
Checking cache for master-6-protected...
No URL provided, cache will not be downloaded from shared cache server. Instead a local version of cache will be extracted.
Successfully extracted cache
Executing "step_script" stage of the job script
00:01
$ mkdir local-cached-folder
$ echo "preparing…" > ./local-cached-folder/cached_file
$ echo "ls /cache"
ls /cache
$ ls -lFa /cache || true
ls: cannot access '/cache': No such file or directory
Saving cache for successful job
00:01
Creating cache master-6-protected...
local-cached-folder: found 2 matching artifact files and directories
No URL provided, cache will not be uploaded to shared cache server. Cache will be stored only locally.
Created cache

And on the next job:

Restoring cache
00:01
Checking cache for master-6-protected...
No URL provided, cache will not be downloaded from shared cache server. Instead a local version of cache will be extracted.
Successfully extracted cache
Executing "step_script" stage of the job script
00:01
$ echo "ls -lFa ."
ls -lFa .
$ ls -lFa .
…
$ echo "ls -lFa local-cached-folder"
ls -lFa local-cached-folder
$ ls -lFa local-cached-folder || true
ls: cannot access 'local-cached-folder': No such file or directory
$ echo "ls /cache"
ls /cache
$ ls -lFa /cache || true
ls: cannot access '/cache': No such file or directory
Saving cache for successful job
00:01
Creating cache master-6-protected...
WARNING: local-cached-folder: no matching files. Ensure that the artifact path is relative to the working directory (/builds/kostbevakningen/kb-frontend)

To me it sounds like it finds a cache archive, but maybe this is a combination of me misunderstanding the docs and bad logging?


I did read https://stackoverflow.com/questions/48670691/gitlab-runner-kubernetes-cache-is-ignored and figured I should probably mount a PVC, using [[runners.kubernetes.volumes.pvc]] but it's pretty old and the accepted answer just says S3.

</details>


# 答案1
**得分**: 0

我通过在图表的toml中将PVC放置在执行器上，使其正常工作。我官方helm图表的完整values.yaml现在是：

```yaml
gitlabUrl: https://gitlab.com/
rbac:
  create: true

runners:
  config: |
    [[runners]]
      [runners.kubernetes]
        namespace = "{{.Release.Namespace}}"
        image = "ubuntu:22.04"
        helper_image = "gitlab/gitlab-runner-helper:alpine-latest-arm64-v16.1.0"

      [[runners.kubernetes.volumes.pvc]]
        name = "runners-cache"
        mount_path = "/cache"
  executor: kubernetes

volumeMounts: []
volumes: []

所以看来/cache是默认的，需要挂载一些东西在那里。那些说“成功提取缓存”的日志似乎完全无意义。

gitlabUrl: https://gitlab.com/
rbac:
  create: true

runners:
  config: |
    [[runners]]
      [runners.kubernetes]
        namespace = "{{.Release.Namespace}}"
        image = "ubuntu:22.04"
        helper_image = "gitlab/gitlab-runner-helper:alpine-latest-arm64-v16.1.0"

      [[runners.kubernetes.volumes.pvc]]
        name = "runners-cache"
        mount_path = "/cache"    
  executor: kubernetes

volumeMounts: []
volumes: []