Removing particular element from list when list is present in a cell of dataframe

My dataframe consists of lists in each cell and i want to remove a particular value from all these lists without using for loop on the whole dataframe,

    TIN        PIN       column       ......

0  [1,2,val]  [0,1]      [val, t, z]  ......
1  [2,val,4]  [14,val]   [b, a, val]  ......
........

These val are in string format and i want to remove all of these they exist in some of these lists and doesn't exist in others...

I tried using

.apply(lambda x: x.remove('nan'|'NaT') as the val is either nan or NaT in string format,i.e, 'nan' or 'NaT' instead of being null values.....

It gave me an error saying, besides i think my logic itself was wrong there as i think it was trying to remove x instead of what was specified......

You can't use remove the way you were intending, as it returns None if the value was found, and raises ValueError if it wasn't. One way to do what you want would be to use a nested apply:

df = pd.DataFrame(
    {'TIN': {0: [1,2,'val'], 1: [2,'val',4]},
     'PIN': {0: [0,1], 1: [14,'val']},
     'column': {0: ['val', 't', 'z'], 1: ['b', 'a', 'val']}}
)
df = df.apply(lambda x:x.apply(lambda y:[v for v in y if v != 'val']))

Output:

      TIN     PIN  column
0  [1, 2]  [0, 1]  [t, z]
1  [2, 4]    [14]  [b, a]

Note performance-wise it's better to use applymap (about 35% faster) as described in @LuanNguyen's answer:

df = df.applymap(lambda y:[v for v in y if v != 'val'])

Or using a set of values to remove:

remove = { 'val', 'xyz' }
df = df.applymap(lambda y:[v for v in y if v not in remove])

Maybe this is what you need, I use the dataframe method applymap to apply the lambda function for all cells in the dataframe and the Python filter function to remove the unwanted elements in the variable nan_set.

nan_set = {'val'}
print(
    df.applymap(lambda arr: list(
        filter(lambda element: element not in nan_set, arr)))
)

Input:

    TIN        PIN          COL
0  [1, 2, val]     [0, 1]  [val, t, z]
1  [2, val, 4]  [14, val]  [b, a, val]

Output:

    TIN     PIN     COL
0  [1, 2]  [0, 1]  [t, z]
1  [2, 4]    [14]  [b, a]

You can use the apply with a custom function.

(you can do most of the code in one line, but it is difficult to understand):

Value2Remove='val'

def RemoveVal(Ls,Val):
   if Val in Ls:
       Ls.remove(Val)
   return Ls


df['TIN'] = df['TIN'].apply(lambda x: RemoveVal(x,Value2Remove))
df['PIN'] = df['PIN'].apply(lambda x: RemoveVal(x,Value2Remove))
df['column'] = df['column'].apply(lambda x: RemoveVal(x,Value2Remove))

Input:

	TIN	PIN	column
0	[1, 2, val]	[0, 1]	[val, t, z]
1	[2, val, 4]	[14, val]	[b, a, val]

Output:

	TIN	PIN	column
0	[1, 2]	[0, 1]	[t, z]
1	[2, 4]	[14]	[b, a]

You can likely do this with 2 embedded "for" loops, once to iterate across cells in the df, and another to iterate through the list elements in the cell.

for cell in df["col"]:
    for listitem in cell:
        if listitem < 10:
            list.pop(n)
        else:
            continue

"cell" iterates the df cells, and listitems iterated the elements of the list within the cell. With n meaning the "nth" element in the list. IOW if n = 3 then it will delete the 4TH element of the list (remember 0 is technically the first element).

You can use apply by treating each row as a series

def remove_nan(s):
    cols = ['TIN', 'PIN', 'column']
    for col in cols:
            try:
                    s[col].remove('nan')
            except ValueError:
                    pass
    return s

Apply this to df, set axis to 1 so operation is performed over columns

df_text = df.apply(remove_nan, axis=1)

print(df_text)

      TIN     PIN  column
0  [1, 2]  [0, 1]  [t, z]
1  [2, 4]    [14]  [b, a]