gitlab: How to include build artifacts when adding a package via ci pipeline to the composer package registry

I added a very basic pipeline to export my packages to a private gitlab package registry. Using the .gitattributes file to mask files, that should not be contained in the archive works fine but how could i prebuild some artifacts (for example scss to css) and have them included in the artifact? Those artifacts are excluded by the gitignore because - well they are artifacts.

This is what my pipeline looks at the moment:

stages:
  - release

stable-release:
  stage: release
  script:
    - apk add curl
    - 'curl --header "Job-Token: $CI_JOB_TOKEN" --data tag=$CI_COMMIT_TAG "${CI_API_V4_URL}/projects/$CI_PROJECT_ID/packages/composer"'
  rules:
    - if: '$CI_COMMIT_TAG'

How could i for example create a file in the package root (let's say "test.txt") and have that file included in the package registry?

I tried the following but the test.txt file is not included in the package, its only available in the gitlab ui.

stable-release:
  script:
    - ...
    - echo "foo" > test.txt
    - ...
  artifacts:
    paths:
      - test.txt

TLDR: Create the package version tag after you've build the release revision, then let it trigger your release stage that is firing the API HTTP request to notify the Gitlab Composer package registry of your project that there is a new release.

How it Works

Composer has two kind of distributions:

1. source
2. dist

That is for each package and version, it can have those two.

In a Composer package registry the metadata of the available packages is collected, and the actual contents of the package is referenced by an Uniform Resource Locator (URL) and additional reference and optionally a hash (shasum).

The reference commonly denotes a revision in a git repository and the url is referencing the git repository itself.

When installing a package into the COMPOSER_VENDOR_DIR, Composer is effectively cloning that git repository and checking it out at that revision. Then the files from therein are copied into the vendor folder subdirectory denoted by the package name. (this is shortened, cache is involved and some more checks are done, but it should suffice in this case.)

So for the private Composer Package Registry for your Gitlab project, the interface between it and Composer is the git repository.

If you want to have files inside the package, that are not yet part of the revision in your projects git repository, they will not become part of the package. As you already noticed, you can only "remove" files with the export-ignore Git Attribute (and technically only dist packages are affected by that, not source).

How to add files

Now to add files not yet under version control, for example those you have created during a build at time of the CI Pipeline run, it is therefore necessary you add all those additional files to the repository and create a new reference out of it. (see git-add(1) and git-commit(1).)

Then tag that new reference with the version tag as Composer is using it. (see git-tag(1).)

The version tag is important, as not only you but also Composer and Gitlab will have a problem to associate a MARJOR . MINOR . PATCH version with your package, which you normally want due to Semantic Versioning constraints implicitly enforced by Composer.

As you're doing automated builds, I'd also suggest for that release-commit, you update //project/composer.json#/version, setting it to the same version as the tag. This is not a requirement, having the tag alone would suffice, however it allows to keep the version within any zip or tar.gz package archives inside the roots composer.json. And when doing automated builds, you don't risk to forget updating it (as it is a result of the build, similar to a COMMITand/or VERSION file you may leave in archives).

Build/CI Revision vs. Composer Package Version

When we look at a Composer Package Registry this is same same but different as when we look at the Git repository of a PHP project that is developed as a Composer Package.

While it is straight forward to turn the later into Composer Repository Package Information, you may notice if you need to build the actual package (need either in the sense that you can but also do, e.g. you have a defined build process in your project), like the scenario where you want to add files, the releases of a project can only be tagged with the appropriate revision after it has been build.

This is due to the fact that we are using version control software and are having a central repository.

So what you commonly have in a CI Pipeline, like in Gitlab, is not the tag but the build revision (IIRC CI_COMMIT), then you run the build (traditionally creating the build artifact) and the Composer requires you to add the build result into the projects repository as otherwise Gitlab won't see it.

• checkout the CI Commit
• run the build
• capture the build result in form of a git revision (commit) containing all
  the relevant files of the Composer package.
• tag that release revision

Then for the release stage you have, it just works^(tm) as Gitlab with that tag will see the right reference to the Composer packages' contents (and metadata in composer.json).

If you now keep the relation between build revision and release revision (e.g. you can specify the build revision in the tags message, also the build run etc.) everything is nicely arranged und you'll never need to build a revision twice. In fact you shouldn't unless you practice reproducible builds, but that's a different topic.