`IAM_PERMISSION_DENIED` on a deployed service on GCP, but no errors on localhost

I'm getting an error details: name = ErrorInfo reason = IAM_PERMISSION_DENIED domain = iam.googleapis.com metadata = map[permission:logging.logEntries.create] when I check the logs of a deployed container in GCP. I'm not sure why this is happening since running the container in localhost seems to work fine.

The service is also deployed on the same host with another service but with a different port number, the other service seems to be working fine, although that didn't use any google API services.

The service having the error on GCP has a .env file with this content:

GOOGLE_APPLICATION_CREDENTIALS=json/name-of-json-file.json

With the json file being the service account keys file. The dockerfile looks like this:

# Specifies a parent image
FROM golang:1.19.2-bullseye

# Creates an app directory to hold your app’s source code
WORKDIR /app

# Copies everything from your root directory into /app
COPY . .

# Installs Go dependencies
RUN go mod download

# Builds your app with optional configuration
RUN go build -o /logging-go

# Tells Docker which network port your container listens on
EXPOSE 8040

# Specifies the executable command that runs when the container starts
CMD [ "/logging-go" ]

The service is making use of the google logging API and is accessed through this snipper of code:

    c, cErr := Load(".env")
	if cErr != nil {
		log.Fatalf("could not load config: %s", cErr)
		return
	}

	// initializes logger which writes to stdout
	ctx := context.Background()
	opt := option.WithCredentialsFile(c.GoogleApplicationCredentials);
	loggerClient, clientErr := logging.NewClient(ctx, "poc-projects-01", opt)
	if clientErr != nil {
		log.Fatal(clientErr)
	}

	if clientErr := loggerClient.Ping(ctx); clientErr != nil {
		log.Fatal(clientErr)
	}

	logger := loggerClient.Logger("frontend_logs")

It works fine on my localhost when running it through docker, but it doesn't work on GCP. Any ideas on how I can fix this?

> error details: name = ErrorInfo reason = IAM_PERMISSION_DENIED domain
> = iam.googleapis.com metadata = map[permission:logging.logEntries.create]

Above error means you have a permissions issue when trying to access the Google Logging API from your deployed container. This could occur if the service account key you are using does not have the correct permissions to access the API, or if the service account key has not been properly configured.

To ensure that the service account key has the correct permissions, you should check the IAM roles associated with the service account and make sure that the roles have the correct permissions to access the Google Logging API, check whether do you have ‘logging.logEntries.create’ role assigned to your service account.

Attaching troubleshooting document for reference.

I know I have correct permissions on my service account keys and even had one of the DevsOps people create me one but it still wasn't working. I found that using the default service account key finally got it working on GCP.