Calling a module from different directories in python

I have the following project dir structure:

Project
├── src
|    ├── eval.py
|    └──utils.py
└── app.py

In eval.py, I import a function from utils:

from utils import clean_txt

and usually, I run my code like this:

python src/eval.py

I have another file (app.py) that will call eval.py to run the project. When I run it, I get this error:

> from utils import reading_jobs_for_id_api

> ModuleNotFoundError: No module named 'utils

It's probably because utils.py is not seen in the Project directory. I found a solution to solve this by running the following in the parent directory. Is this the "right" way (when writing python packages) to solve the problem?

export PYTHONPATH="${PYTHONPATH}:$PWD"

Short answer: No, that's not proper structure and that's not proper "entry" for python program. When importing/using submodule - which is what eval is in your case, python first has to import the parent of the package - which should be in root directory of your package structure.

Long answer:
Let's say you want "Project" to be the name of your package. In such case, you need to "start" from there. So when you prepare your package, let's say by creating pyproject.toml file, you will be specifying the "entry point" of your package somewhere that's visible from the root of your package. Please refer to this: https://packaging.python.org/en/latest/tutorials/packaging-projects/

Moreover, src probably should be encapsulating Project, not the other way around and app.py should go into the Project directory. Unless your aim was to create a package called Project with submodule called src. Then that's fine.

As for the reason why your src/eval.py is not seeing your app.py module, that's because you have not built your package. Again, refer to the packaging tutorial linked above for that. You can easily verify why exactly by printing contents of sys.path within your modules:

import sys

if __name__=="__main__":
    print(sys.path)

Before building/installing your package, sys.path will only contain paths with standard libraries and your src/ directory. It won't be looking for modules anywhere else. But if you install it then you will see that additional path has been added to that list of paths and your eval.py will be able to perform relative import of app.py (unless you're also importing eval into app, then you might end up with circular imports).

You can also test this by manually adding a path to sys.path: sys.path.append(<path to directory with app.py>).

Before importing from utils import reading_jobs_for_id_api in app.py please make sure that full path to the src folder is in sys.path. You can set PYTHONPATH=%PYTHONPATH%;FULL_PATH_TO_SRC_FOLDER in terminal or Environment Variables before running the script or above the import in your code you can type this:

import sys
sys.path.append(FULL_PATH_TO_SRC_FOLDER)

I personally prefer the last option cause setting PYTHONPATH variable permanently on the computer would make it very long if you append path to it every time you need to import something from very different location. Setting it in terminal will make this variable available only in it. If you are going to move whole project somewhere else in the future you can pass in the sys.path.append os.path.join(Path(__file__).parent, 'src) - it will be the most flexible solution.

Please remember in the future, that Python searches for imported module in this order:

1. directory of the invoked script
2. paths added to PYTHONPATH variable
3. default directories related to Python installation folder
4. paths added to sys.path in the code

I had a same issue. Using sys.path.append is one of solution, but it isn't simple.
I developed new module, syspend. Please put SYSPEND_ROOT file: empty file.
https://pypi.org/project/syspend/

Project
├── src
|    ├── eval.py
|    └──utils.py
├── app.py
└── SYSPEND_ROOT

in app.py, eval.py, your can import utils with same code.

import syspend    # sys.path.append(directory: SYSPEND_ROOT file exists)
from src.util import clean_txt