How to fill in missing dates using python for all ids

I have a pandas dataframe A with ID, date_yyyymmdd, amount and hours as shown below. Not all calendar dates are populated.

I would like to fill in missing calendar dates between two variables start_date and end_date and produce another dataframe B as shown below and populate amount and hours as 0s for those dates. In the example below the start date is 20230101 and end date is 20230105. I found a code that uses date as index and fills in missing value. I don't think it will work in my case. I want to fill dates for each id. How can I accomplish this? Thanks.

Here is a way by constructing a new MultiIndex, and using that to reindex your df.

cols = ['id','date_yyyymmdd']

start_date = '1/1/2023'
end_date = '1/5/2023'

df['date_yyyymmdd'] = pd.to_datetime(df['date_yyyymmdd'],format = '%Y%m%d')

df = (df.set_index(cols)
      .reindex(pd.MultiIndex.from_product([df['id'].unique(),pd.date_range(start_date,end_date,freq='D')],names = cols))
      .fillna(0)
      .sort_index()
      .reset_index())

Output:

    id date_yyyymmdd   amount  hours
0    1    2023-01-01  1428.95   11.0
1    1    2023-01-02     0.00    0.0
2    1    2023-01-03  1791.29   13.0
3    1    2023-01-04     0.00    0.0
4    1    2023-01-05     0.00    0.0
5    2    2023-01-01  2516.84   15.0
6    2    2023-01-02     0.00    0.0
7    2    2023-01-03     0.00    0.0
8    2    2023-01-04     0.00    0.0
9    2    2023-01-05  3046.08    5.0
10   3    2023-01-01     0.00    0.0
11   3    2023-01-02  7137.92   11.0
12   3    2023-01-03  1104.35    1.0
13   3    2023-01-04    25.00    1.0
14   3    2023-01-05     0.00    0.0

Try:

df["date_yyyymmdd"] = pd.to_datetime(df["date_yyyymmdd"], format="%Y%m%d")
r = pd.date_range(df["date_yyyymmdd"].min(), df["date_yyyymmdd"].max())

df = (
    df.groupby("id", group_keys=False)
    .apply(
        lambda x: (newdf := x.set_index("date_yyyymmdd").reindex(r)).assign(
            id=newdf["id"].ffill().bfill()
        )
    )
    .reset_index()
    .fillna(0)
)
df["id"] = df["id"].astype(int)

print(df)

Prints:

        index  id   amount  hours
0  2023-01-01   1  1428.95   11.0
1  2023-01-02   1     0.00    0.0
2  2023-01-03   1  1791.29   13.0
3  2023-01-04   1     0.00    0.0
4  2023-01-05   1     0.00    0.0
5  2023-01-01   2  2516.84   15.0
6  2023-01-02   2     0.00    0.0
7  2023-01-03   2     0.00    0.0
8  2023-01-04   2     0.00    0.0
9  2023-01-05   2  3046.08    5.0
10 2023-01-01   3     0.00    0.0
11 2023-01-02   3  7137.92   11.0
12 2023-01-03   3  1104.35    1.0
13 2023-01-04   3    25.00    1.0
14 2023-01-05   3     0.00    0.0

One option is with pyjanitor's complete function:

# pip install pyjanitor
import janitor
import pandas as pd

df = pd.read_clipboard()
df['date_yyyymmdd'] = pd.to_datetime(df['date_yyyymmdd'],format = 'ISO8601')
# create variable containing all possible dates
dates = {"date_yyyymmdd": pd.date_range("2023-01-01", "2023-01-05", freq="D")}
df.complete('id', dates, fill_value=0)
    id date_yyyymmdd   amount  hours
0    1    2023-01-01  1428.95     11
1    1    2023-01-02     0.00      0
2    1    2023-01-03  1791.29     13
3    1    2023-01-04     0.00      0
4    1    2023-01-05     0.00      0
5    2    2023-01-01  2516.84     15
6    2    2023-01-02     0.00      0
7    2    2023-01-03     0.00      0
8    2    2023-01-04     0.00      0
9    2    2023-01-05  3046.08      5
10   3    2023-01-01     0.00      0
11   3    2023-01-02  7137.92     11
12   3    2023-01-03  1104.35      1
13   3    2023-01-04    25.00      1
14   3    2023-01-05     0.00      0