How to publish a scoped npm package to gitlab's package registry in a group namespace via a CI/CD pipeline

I have a private dummy project in gitlab which I want to publish to gitlab's package registry. My dummy project contains four files:

package.json

{
  "name": "@<my-group>/<my-project>",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "repository": {
    "type": "git",
    "url": "git+https://gitlab.com/<my-group>/<my-project>.git"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "bugs": {
    "url": "https://gitlab.com/<my-group>/<my-project>/issues"
  },
  "homepage": "https://gitlab.com/<my-group>/<my-project>#readme"
}

.gitlab-ci.yml

image: node:latest

stages:
  - deploy

deploy:
  stage: deploy
  script:
    - echo "@<my-group>:registry=https://${CI_SERVER_HOST}/api/v4/projects/${CI_PROJECT_ID}/packages/npm/">.npmrc
    - echo "//${CI_SERVER_HOST}/api/v4/projects/${CI_PROJECT_ID}/packages/npm/:_authToken=${CI_JOB_TOKEN}">>.npmrc
    - npm publish

index.js

console.log('success');

README.md

<my-project>

When I commit my project to gitlab, the job fails with the following output

npm notice package: @<my-group>/<my-project>@1.0.0
npm notice === Tarball Contents === 
npm notice 8B   README.md   
npm notice 20B  index.js    
npm notice 612B package.json
npm notice === Tarball Details === 
npm notice name:          @<my-group>/<my-project>                     
npm notice version:       1.0.0                                   
npm notice filename:      <my-group>-<my-project>-1.0.0.tgz            
npm notice package size:  475 B                                   
npm notice unpacked size: 640 B                                   
npm notice shasum:        7b3db...
npm notice integrity:     sha512-xDv0dl9A86...
npm notice total files:   3                                       
npm notice 
npm notice Publishing to https://gitlab.com/api/v4/projects/<my-project-id>/packages/npm/ with tag latest and default access
npm ERR! code E403
npm ERR! 403 403 Forbidden - PUT https://gitlab.com/api/v4/projects/<my-project-id>/packages/npm/@<my-group>%2f<my-project> - insufficient_scope
npm ERR! 403 In most cases, you or one of your dependencies are requesting
npm ERR! 403 a package version that is forbidden by your security policy, or
npm ERR! 403 on a server you do not have access to.

NOTE:
I have replaced the actual group name, project name and project id with <my-group>, <my-project>, and <my-project-id> in the code sections above.
I have followed gitlab's official documentation on setting this up (see https://docs.gitlab.com/ee/user/packages/npm_registry/) and believe that I can safely rule out the following:

• I have made sure that Package registryis enabled in the project setup
• I followed naming convention as described in the documentation
• I am using a CI_JOB_TOKEN which should always be valid and should have appropriate permissions.
• I made sure that there is no other package with the same name or version within the given scope.
• I have made sure that the scoped package's URL includes a trailing slash (see gitlab-ci.yml above)
• I have confirmed the path of the <my-group> namespace querying https://gitlab.com/api/v4/groups (just to make sure that the root namespace is correct)
• I have used npm init --scope=@<my-group> --yes for initialization

The url of the repository is indeed: https://gitlab.com/<my-group>/<my-project>/

Any help on getting this to work would be much appreciated.

After changing from a job token ${CI_JOB_TOKEN} to a deploy token ${CI_DEPLOY_PASSWORD}, I was able to publish to the registry.

See Predefined variables reference for more info on predefined variables.

Gitlab deploy tokens can be created for projects under Settings > Repository > Deploy tokens with the following scopes: read_repository, read_package_registry, write_package_registry