英文:
group_rolling per group in polars
问题
以下是翻译好的代码部分：
让我们假设我有以下的 DataFrame：example_df = pl.DataFrame({"date": pl.date_range(low=date(2022, 1, 1), high=date(2022, 1, 30), interval="1d"),"group": ["A", "B", "C"]*10,'total': np.random.randint(100, size=30)})现在，我想在5天的时间窗口内对每个组的总和进行求和。类似于将 groupby 与 groupby_rolling 结合在一起。我想知道是否有比以下代码更加优雅和高效的方法来实现：df_list = []for table_group in example_df['group'].unique():temp_df = (example_df.filter(pl.col('group')==table_group).sort('date').groupby_rolling('date', period='5d').agg(pl.col('total').sum().prefix('roll_')))temp_df2 = (example_df.filter(pl.col('group')==table_group).sort('date').select([pl.all(),temp_df.get_column('roll_total')]))df_list.append(temp_df2)pl.concat(df_list)
希望这有所帮助。
英文:
Let's say I have the following df:
example_df = pl.DataFrame({"date": pl.date_range(low=date(2022, 1, 1), high=date(2022, 1, 30), interval="1d"),"group": ["A", "B", "C"]*10,'total': np.random.randint(100, size=30)})
Now I want to sum the total per group, in a time window of 5 days. Something like combining groupby with groupby_rolling.
I wonder if there more elegant and efficient way to do it than the following code:
df_list = []for table_group in example_df['group'].unique():temp_df = (example_df.filter(pl.col('group')==table_group).sort('date').groupby_rolling('date', period='5d').agg(pl.col('total').sum().prefix('roll_')))temp_df2 = (example_df.filter(pl.col('group')==table_group).sort('date').select([pl.all(),temp_df.get_column('roll_total')]))df_list.append(temp_df2)pl.concat(df_list)
My main issue is that I have millions of rows and over 10,000 groups(and more than 1 column that I want to sum)...
So this code is not so elegant and consumes more memory than I want it to.
答案1
得分: 1
以下是您要翻译的内容：
"It appears that the piece you're missing is the by parameter.
This allows you to specify grouping.
(df.groupby_rolling(index_column="date", by="group", period="5d").agg(pl.col("total").last(),roll_total = pl.sum("total")))
shape: (30, 4)┌───────┬────────────┬───────┬────────────┐│ group ┆ date ┆ total ┆ roll_total ││ --- ┆ --- ┆ --- ┆ --- ││ str ┆ date ┆ i64 ┆ i64 │╞═══════╪════════════╪═══════╪════════════╡│ A ┆ 2022-01-01 ┆ 96 ┆ 96 ││ A ┆ 2022-01-04 ┆ 16 ┆ 112 ││ A ┆ 2022-01-07 ┆ 17 ┆ 33 ││ A ┆ 2022-01-10 ┆ 4 ┆ 21 ││ … ┆ … ┆ … ┆ … ││ C ┆ 2022-01-21 ┆ 17 ┆ 78 ││ C ┆ 2022-01-24 ┆ 84 ┆ 101 ││ C ┆ 2022-01-27 ┆ 78 ┆ 162 ││ C ┆ 2022-01-30 ┆ 54 ┆ 132 │└───────┴────────────┴───────┴────────────┘```"<details><summary>英文:</summary>It appears that the piece you're missing is the [`by`](https://pola-rs.github.io/polars/py-polars/html/reference/dataframe/api/polars.DataFrame.groupby_rolling.html) parameter.This allows you to specify grouping.
(df.groupby_rolling(index_column="date", by="group", period="5d")
.agg(
pl.col("total").last(),
roll_total = pl.sum("total")))
shape: (30, 4)
┌───────┬────────────┬───────┬────────────┐
│ group ┆ date ┆ total ┆ roll_total │
│ --- ┆ --- ┆ --- ┆ --- │
│ str ┆ date ┆ i64 ┆ i64 │
╞═══════╪════════════╪═══════╪════════════╡
│ A ┆ 2022-01-01 ┆ 96 ┆ 96 │
│ A ┆ 2022-01-04 ┆ 16 ┆ 112 │
│ A ┆ 2022-01-07 ┆ 17 ┆ 33 │
│ A ┆ 2022-01-10 ┆ 4 ┆ 21 │
│ … ┆ … ┆ … ┆ … │
│ C ┆ 2022-01-21 ┆ 17 ┆ 78 │
│ C ┆ 2022-01-24 ┆ 84 ┆ 101 │
│ C ┆ 2022-01-27 ┆ 78 ┆ 162 │
│ C ┆ 2022-01-30 ┆ 54 ┆ 132 │
└───────┴────────────┴───────┴────────────┘
</details>
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论