DefaultCredentialsError when trying to collect Google default credentials for Google storage

I am attempting to upload a file to a Google cloud storage bucket from a service running on a Kubernetes pod. This requires authentication for the Google storage and so I have created a json authentication file from the console.

This json file is saved as a secret on my kubernetes environment and is referenced in the deploy.yaml through the environment variable GOOGLE_APPLICATION_CREDENTIALS.

The format appears like this:

{
  "type": "service_account",
  "project_id": "xxx",
  "private_key_id": "xxx",
  "private_key": "-----BEGIN PRIVATE KEY-----\nxxx\n-----END PRIVATE KEY-----\n",
  "client_email": "xx",
  "client_id": "",
  "auth_uri": "https://accounts.google.com/o/oauth2/auth",
  "token_uri": "https://oauth2.googleapis.com/token",
  "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
  "client_x509_cert_url": "xxx"
}

And the following code of how I am handling this authentication:

import google.auth

credentials, project_id = google.auth.default()


bucket_name = Config.STORAGE_BUCKET_NAME
client = storage.Client(project=project_id)
bucket = client.get_bucket(bucket_name)

Given that locally I have logged into gcloud, I am able to upload files when testing locally. However, when deployed to Kubernetes I get the following error:

 File "/storage.py", line 8, in <module>
    credentials, project_id = google.auth.default()
  File "/usr/local/lib/python3.6/dist-packages/google/auth/_default.py", line 308, in default
    credentials, project_id = checker()
  File "/usr/local/lib/python3.6/dist-packages/google/auth/_default.py", line 166, in _get_explicit_environ_credentials
    os.environ[environment_vars.CREDENTIALS]
  File "/usr/local/lib/python3.6/dist-packages/google/auth/_default.py", line 92, in _load_credentials_from_file
    "File {} was not found.".format(filename)
google.auth.exceptions.DefaultCredentialsError: File     {
  "type": "service_account",
  "project_id": "xxx",
  "private_key_id": "xxx",
  "private_key": "-----BEGIN PRIVATE KEY-----\nxxx\n-----END PRIVATE KEY-----\n",
  "client_email": "xx",
  "client_id": "",
  "auth_uri": "https://accounts.google.com/o/oauth2/auth",
  "token_uri": "https://oauth2.googleapis.com/token",
  "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
  "client_x509_cert_url": "xxx"
}
 was not found.

I'm confused as it's referencing the file itself in the error so I am assuming it located it, but it does not recognize it as a valid service authentication file.

All help and pointers appreciated.

I think you should try this code locally and then on kubernetes, to check if is a problem with your service_account.json file:

    client = storage.Client.from_service_account_json(
    'service_account.json') 

or link:

> If your application runs on Compute Engine, Kubernetes Engine, the App
> Engine flexible environment, or Cloud Functions, you don't need to
> create your own service account. Compute Engine includes a default
> service account that is automatically created for you, and you can
> assign a different service account, per-instance, if needed.

credentials = compute_engine.Credentials()
# Create the client using the credentials and specifying a project ID.
storage_client = storage.Client(credentials=credentials, project=project)

My issue ended up being that I had no set up the secret json key file as a volume mount in the deploy file.

Example of how I did it:

        - name: GOOGLE_APPLICATION_CREDENTIALS
          value: /mnt/file_key/key.json
    .......

        volumeMounts:
        - mountPath: /mnt/file_key
          name: file_key
          readOnly: true

........
      volumes:
      - name: file_key
        secret:
          defaultMode: 420
          secretName: file_key_secret