KNN imputation for missing categorical-string values python for a specific column in a dataframe and return with replaced value as a dataframe

There are some missing values in Gender Column and would like to impute these values using KNN imputation. But i ain't getting filled result! Can someone help on this?

import pandas as pd
import numpy as np
from sklearn.impute import KNNImputer

data = {'ID': [1, 2, 3, 4, 5],
        'Age': [20, 25, 30, 35, 40],
        'Gender': ['M', 'F', np.nan, 'F', np.nan]}
df = pd.DataFrame(data)
imputer = KNNImputer(n_neighbors=2)
df['Gendermap'] = pd.factorize(df['Gender'])[0]
df['Gender_imputed_factorized'] = imputer.fit_transform(df[['Gendermap']])
df['Gender_imputed'] = pd.unique(df['Gender'])[df['Gender_imputed_factorized'].astype(int)]
df

Output:

   ID  Age Gender  Gendermap  Gender_imputed_factorized Gender_imputed
0   1   20      M          0                        0.0              M
1   2   25      F          1                        1.0              F
2   3   30    NaN         -1                       -1.0            NaN
3   4   35      F          1                        1.0              F
4   5   40    NaN         -1                       -1.0            NaN

"Gender_imputed" column shoudn't contain Nan value.

I believe it is the factorize function that you are using which is causing issues. It is removing the NaN values so there is nothing to impute when you use fit_transform.

Try using map to convert gender to a numerical column like this:

import pandas as pd
import numpy as np
from sklearn.impute import KNNImputer

data = {'ID': [1, 2, 3, 4, 5],
        'Age': [20, 25, 30, 35, 40],
        'Gender': ['M', 'F', np.nan, 'F', np.nan]}
df = pd.DataFrame(data)
imputer = KNNImputer(n_neighbors=2)
df['Gendermap'] = df['Gender'].map({'M': 0, 'F': 1}) # the new map function
df['Gender_imputed_factorized'] = imputer.fit_transform(df[['Gendermap']])
df['Gender_imputed'] = pd.unique(df['Gender'])[df['Gender_imputed_factorized'].astype(int)]
df

Got a solution thanks.

df['Gendermap'] = pd.factorize(df['Gender'])[0]
imputer = KNNImputer(n_neighbors=2)
df['Gender_imputed_factorized'] = imputer.fit_transform(df[['Gendermap']])
imputed_labels = pd.unique(df['Gender'].dropna())
df['Gender_imputed'] = [imputed_labels[int(i)] if not np.isnan(i) else np.nan for i in df['Gender_imputed_factorized']]

print(df)