Add a data directory outside Python package directory

Given the following directory structure for a package my_package:

/
├── data/
│   ├── more_data/
│   └── foo.txt
├── my_package/
│   ├── __init__.py
│   └── stuff/
│       └── __init__.py
├── README.md
├── setup.cfg
├── setup.py

How can I make the data/ directory accessible (in the most Pythonic way) from within code, without using __file__ or other hacky solutions? I have tried using data_files in setup.py and the [options.package_data] in setup.cfg to no avail.

I would like to do something like:

dir_data = importlib.resources.files(data)
csv_files = dir_data.glob('*.csv')

EDIT:

I'm working with an editable installation and there's already a data/ directory in the package (for source code unrelated to the top-level data).

I can think of 2 solutions. For both solutions, the object is to move data into the top-level importable package (that is why it is called "package data"). This means the data will be accessible via something like the following:

importlib.resource.files('mypackage.data')

1. Move files in the source code repository

One easy and straightforward solution is to move the data/ directory to my_package/data/. And then do the packaging as usual.

2. Let setuptools "move" the data at build-time

With the package_dir feature of setuptools, it is possible to modify the directory structure at build time.

Be aware that this approach does not work with "editable installations".

It could look like this in setup.cfg (untested):

[options]
# ...
package_dir=
    my_package=my_package
    my_package.data=data

Create an empty data/__init__.py file, so that data becomes a top-level import package, so that the data files become package data, so that they are accessible via importlib.resources.files('data'). This should work with "editable installation". You might need to do small changes in your packaging files (setup.py or setup.cfg or pyproject.toml).