Why is Airflow not seeing it's HOME folder?

I'm setting up an Airflow instance but for some reason Airflow is not seeing it's $AIRFLOW_HOME folder.

The setup is using the official Airflow image (2.6.1/py3.8) with the default $AIRFLOW_HOME being /opt/airflow/. I mounted a local folder - ./files:/opt/airflow/files but for some reason this isn't working as inteded and Airflow is not running the tasks from this folder.

Using a BashOperator i can get this task to run:

bash_command='cd /opt/airflow/files/my-repo/ && python taskpy'

But not this one:

bash_command=' python files/my-repo/task.py'

I was under the impresion that anything added to the $AIRFLOW_HOME folder would be mapped by Airflow and could be used right away. Is this wrong?

Something strange that i've noticed is that when i try to access this env variable ($AIRFLOW_HOME) from the airflow user i get a Permission denied error:

(airflow)$AIRFLOW_HOME
/bin/sh: 1: /opt/airflow: Permission denied

Is there anything i'm missing?

If executing the command echo $AIRFLOW_HOME does not display any output, it means that the environment variable AIRFLOW_HOME is either not set or is set to an empty value.

To set the AIRFLOW_HOME variable, you can do one of the following:

1. Set it explicitly in the terminal:

   export AIRFLOW_HOME=/path/to/airflow/home
   

   Replace /path/to/airflow/home with the actual path where you want to set your Airflow home directory.

2. Add the export command to your shell configuration file (e.g., ~/.bashrc for Bash) to make it persist across sessions:

   echo "export AIRFLOW_HOME=/path/to/airflow/home" >> ~/.bashrc
   source ~/.bashrc
   

   Again, replace /path/to/airflow/home with the desired path.

After setting the AIRFLOW_HOME variable, you can verify if it is properly configured by running echo $AIRFLOW_HOME again. It should display the path you assigned to it. Remember to restart your terminal or source the configuration file if you modified it in step 2 for the changes to take effect.