GCP's service account cannot read project

My GCP organization has the following structure

mydomain.com
  - Root-project 
  - Development
      - my-project-name

In order to automate the project creation within my Development folder using terraform, I created a service account in the Root project and created an IAM principal in Development and granted Project Creator as well as Project Deleter.

My terraform script looks something like

terraform {
  required_providers {
    google = {
      source = "hashicorp/google"
      version = "4.68.0"
    }
  }
}

provider "google" {
  scopes = [
    "https://www.googleapis.com/auth/cloud-platform"
  ]
}

resource "google_project" "project" {
  name = var.project_name
  folder_id = var.folder_id
  project_id = var.project_id
  billing_account = var.billing_account_id
}

When I run this script locally using glcoud CLI to authenticate on both my personal account and the service account everthing works fine.

When I run it after setting the credentials using the environment variable GOOGLE_CREDENTIALS I'm able to create the resources but if I run it again, as Terraform will refresh states, it throws this error

Error: Error when reading or editing Project "my-project-id": googleapi: Error 403: Request had insufficient authentication scopes.
│ Details:
│ [
│   {
│     "@type": "type.googleapis.com/google.rpc.ErrorInfo",
│     "domain": "googleapis.com",
│     "metadata": {
│       "method": "google.cloudresourcemanager.v1.Projects.GetProject",
│       "service": "cloudresourcemanager.googleapis.com"
│     },
│     "reason": "ACCESS_TOKEN_SCOPE_INSUFFICIENT"
│   }
│ ]
│ 
│ More details:
│ Reason: insufficientPermissions, Message: Insufficient Permission

I don't understand how can I be able to create a project but not able to fetch its data. In IAM I see that my service account has the Owner role on my-project-name.

When you are using the CLI gcloud you might be using different credentials than Terraform is using. Terraform uses Application Default Credentials (ADC).

Run this command to make sure ADC is using the correct credentials:

gcloud auth application-default login

The API that has the permission problem is google.cloudresourcemanager.v1.Projects.GetProject

That API requires the permission resourcemanager.projects.get. documentation

There are several predefined IAM Roles with that permission. Since you need to create projects, add the following role at the organization or parent folder level:

roles/resourcemanager.projectCreator

The crentials that ADC is configure for, does not have that permission or does not have that permission at the correct level (folder or organization).