Wrong with Test and valid generators

I've been working in my model, I came to generators part.

when I use to read test.csv file like this:

test_df = pd.read_csv("Test.csv")

Its going well!

and when I came to generator part:

def get_test_generator(test_df,  train_df, image_dir, x_col, y_cols, sample_size=100, batch_size=8, seed=1, target_w = 320, target_h = 320):
    """
    Return generator for test set using 
    normalization statistics from training set.

    Args:
      test_df (dataframe): dataframe specifying test data.
      image_dir (str): directory where image files are held.
      x_col (str): name of column in df that holds filenames.
      y_cols (list): list of strings that hold y labels for images.
      sample_size (int): size of sample to use for normalization statistics.
      batch_size (int): images per batch to be fed into model during training.
      seed (int): random seed.
      target_w (int): final width of input images.
      target_h (int): final height of input images.
    
    Returns:
        test_generator (DataFrameIterator): iterators over test set
    """
    print("getting train generators...")
    # get generator to sample dataset
    raw_train_generator = ImageDataGenerator().flow_from_dataframe(
        dataframe=train_df, 
        directory=IMAGE_DIR, 
        x_col="Image", 
        y_col=labels, 
        class_mode="raw", 
        batch_size=sample_size, 
        shuffle=True, 
        target_size=(target_w, target_h))
    
    # get data sample
    batch = raw_train_generator.next()
    data_sample = batch[0]

    # use sample to fit mean and std for test set generator
    image_generator = ImageDataGenerator(
        featurewise_center=True,
        featurewise_std_normalization= True)
    
    # fit generator to sample from training data
    image_generator.fit(data_sample)

    # get test generator
 

    test_generator = image_generator.flow_from_dataframe(
            dataframe=test_df,
            directory=image_dir,
            x_col=x_col,
            y_col=y_cols,
            class_mode="raw",
            batch_size=batch_size,
            shuffle=False,
            seed=seed,
            target_size=(target_w,target_h))
    return test_generator

when I run this cell in Jupyter :

IMAGE_DIR = '/Users/awabe/Desktop/Project/PapilaDB/FundusImages test'

test_generator= get_test_generator(test_df, train_df, IMAGE_DIR, "Image", labels)

to read the images

it give me the error:

getting train generators...
Found 0 validated image filenames.
Found 488 validated image filenames.
/opt/anaconda3/envs/tensorflow/lib/python3.10/site-packages/keras/preprocessing/image.py:1139: UserWarning: Found 488 invalid image filename(s) in x_col="Image". These filename(s) will be ignored.
  warnings.warn(
/opt/anaconda3/envs/tensorflow/lib/python3.10/site-packages/numpy/core/fromnumeric.py:3432: RuntimeWarning: Mean of empty slice.
  return _methods._mean(a, axis=axis, dtype=dtype,
/opt/anaconda3/envs/tensorflow/lib/python3.10/site-packages/numpy/core/_methods.py:182: RuntimeWarning: invalid value encountered in divide
  ret = um.true_divide(
/opt/anaconda3/envs/tensorflow/lib/python3.10/site-packages/numpy/core/_methods.py:265: RuntimeWarning: Degrees of freedom <= 0 for slice
  ret = _var(a, axis=axis, dtype=dtype, out=out, ddof=ddof,
/opt/anaconda3/envs/tensorflow/lib/python3.10/site-packages/numpy/core/_methods.py:223: RuntimeWarning: invalid value encountered in divide
  arrmean = um.true_divide(arrmean, div, out=arrmean, casting='unsafe',
/opt/anaconda3/envs/tensorflow/lib/python3.10/site-packages/numpy/core/_methods.py:254: RuntimeWarning: invalid value encountered in divide
  ret = um.true_divide(

(the 488 images in the second line belongs to the train generator which works fine)

where is the wrong here?

the images name in the column Image should contain the image name + the extension. for example:

assume the name in the directory file is: image1.jpg

so the name in the csv should be: image1.jpg

if you write it as: image1

you will get an error back.

so simple 🤦.

thanks to the upper guy TFer2