pandas: sequential merge adds new columns instead of replacing NaN values

import pandas as pd

df_all = pd.DataFrame(columns=["uid", "a", "b"], data=[["uid1", 12, 15],
                                                    ["uid2", 13, 16],
                                                    ["uid3", 14, 17],
                                                    ["uid4", 15, 18]])

df_additional_info1 = pd.DataFrame(columns=["uid", "c", "d"], data=[["uid1", 12, 15],
                                                                   ["uid3", 14, 17]])

df_additional_info2 = pd.DataFrame(columns=["uid", "c", "d"], data=[["uid2", 12, 15]])

I need to merge df_all twice with additional information. First with df_additional_info1, then with df_additional_info2 and so on. They will always contain additional info for already existing rows and only for those rows that were not yet updated.

When I do following:

df_all = df_all.merge(df_additional_info1, how="left", on="uid")
df_all = df_all.merge(df_additional_info2, how="outer", on="uid")

I get duplicated columns (*_x, *_y):

But I need this

Any suggestion?

Another possible solution:

(pd.concat([df_all.set_index('uid'), df_additional_info1.set_index('uid'),
            df_additional_info2.set_index('uid')])
 .stack().unstack().reset_index())

Alternatively,

(df_additional_info2.set_index('uid')
 .combine_first(
     pd.concat([df_all.set_index('uid'), 
                df_additional_info1.set_index('uid')], axis=1))
 .reset_index())

EXPLANATION:

The code first sets uid as the index for all DataFrames, facilitating the merging operations. It concatenates df_all and df_additional_info1 along the column axis using pd.concat(), appending df_additional_info1's columns to df_all and creating a DataFrame containing all columns from both, filling with NaN for uids absent in df_additional_info1. combine_first() then replaces any null values in df_additional_info2 with values from the concatenated DataFrame, essentially filling in df_additional_info2's gaps with data from df_all and df_additional_info1, based on uid. The final step, reset_index(), moves uid back to a regular column and reestablishes the default integer index, producing a merged DataFrame with all available additional information.

Output:

    uid   a   b     c     d
0  uid1  12  15  12.0  15.0
1  uid2  13  16  12.0  15.0
2  uid3  14  17  14.0  17.0
3  uid4  15  18   NaN   NaN

You can .concat the additional info frames:

df_all.merge(
   pd.concat([df_additional_info1, df_additional_info2]),
   how = "left"
)

    uid   a   b     c     d
0  uid1  12  15  12.0  15.0
1  uid2  13  16  12.0  15.0
2  uid3  14  17  14.0  17.0
3  uid4  15  18   NaN   NaN