Cannot download artifact that was generated in a previous build

I'm trying to persist some data from one pipeline run to another. Essentially my pipeline works like this:

• First download the data, if it exists (the data is a text file with a number in it)
• Run a script, that will use that data (if it exists), and generate some new one
• Publish the new data

Running the script works and publishing the file too as I can see in the pipeline report that there's one published artifact with the right file in it.

However, downloading the file always fail, it simply says this:

##[error]Artifact CheckWebsiteState was not found for build 10470.
Finishing: Download artifact

This is the YAML script I'm using:

name: $(Date:yyyyMMdd)$(Rev:-r)

trigger: none

pool:
  vmImage: 'ubuntu-latest'

stages:
- stage: CheckWebsite

  jobs:
  - job: Run
    
    steps:

    - download: current
      artifact: CheckWebsiteState
      displayName: 'Download artifact'
      continueOnError: true

    - task: AWSShellScript@1
      displayName: CheckWebsite
      inputs:
        filePath: '$(System.DefaultWorkingDirectory)/scripts/check-website.sh'
        arguments: '$(Pipeline.Workspace)'

    - publish: $(Pipeline.Workspace)
      artifact: CheckWebsiteState
      displayName: 'Publish artifact'

Any idea what might be the issue?

I believe you are trying to download the artifact from the wrong pipeline. Instead of specifying current, you need to provide the right pipeline resource identifier. For example, let's say I want to download artifacts published in pipeline B from within my pipeline A template. My template would look something like the following.

trigger: none

# You need to define the pipeline resource correctly before you can download from it. 
# You will need to be careful to ensure the pipeline build you want gets selected
# My definition below will select the most recent, successful pipeline run from pipeline B 
# against the main branch, that has had a `ScriptReady` build tag added. You can use a
# tag to ensure the right pipeline build is being selected
resources:
  pipelines:
  - pipeline: pipelineB
    source: pipelines/pipelineB
    brain: main
    tags:
    - ScriptReady

stages:
- stage: CheckWebsite

  jobs:
  - job: Run
    
    steps:
    - download: pipelineB
      artifact: CheckWebsiteState
      displayName: 'Download artifact'
      continueOnError: true