执行线性回归，其中数据帧的所有列都以第一列作为预测变量。

huangapple
go评论97阅读模式
英文:

Perform linear regression over all columns of data fame where first column is predictor

问题

我已搜索StackOverflow以寻找关于这个问题的答案，但仍然感到困惑 - 如果这看起来太像一个重复的问题，我道歉。

我有一个类似于这样的数据框：

  1. df <- data.frame(Cohort = c('con', 'con', 'dis', 'dis', 'con', 'dis'),
  2.                  Sex = c('M', 'F', 'M', 'F', 'M', 'M'),
  3.                  P1 = c(50, 40, 70, 80, 45, 75),
  4.                  P2 = c(10, 9, 15, 13, 10, 8))

我想对我的数据框的所有数值列执行线性回归，使用"Cohort"作为预测变量（以后可以添加其他特征，比如"Sex"）。

我将数据框子集化以删除所有不相关的列（在这个示例中是Sex）：

  1. new_df <- df[,-c(Sex)]

然后我像这样执行回归：

  1. fit <- lapply(new_df[-1], function(y){summary(lm(y ~ Cohort, data=new_df))})

当我在我的数据框的一个小子集上测试这个时，它可以正常工作。实际上，我的数据框有大约7300列。当我在完整的数据框上运行这个命令时，我得到以下错误：

  1. Error in lm.fit(x, y, offset = offset, singular.ok = singular.ok, ...) :
  2. NA/NaN/Inf in 'y'

然后我假设这是一个N/A值的问题，但当我执行这个命令时，返回的是'0'：

  1. sum(is.na(new_df))

我还尝试过na.action=na.omit，但这也没有帮助解决错误。

我的最终目标是执行这些回归并提取p值和R-squared值，使用anova(fit)$'Pr(>F)'和summary(fit)$r.squared，分别。

如何纠正这个错误，或者是否有更好的方法来实现这个目标？另外，将来如何在添加其他特征到回归时不对数据框进行子集化？

英文:

I have searched StackOverflow for answers to this question but am still struggling - apologies if this looks too much like a duplicate question.

I have a dataframe similar to this:

  1. df &lt;- data.frame(Cohort = c(&#39;con&#39;, &#39;con&#39;, &#39;dis&#39;, &#39;dis&#39;, &#39;con&#39;, &#39;dis&#39;),
  2.                  Sex = c(&#39;M&#39;, &#39;F&#39;, &#39;M&#39;, &#39;F&#39;, &#39;M&#39;, &#39;M&#39;),
  3.                  P1 = c(50, 40, 70, 80, 45, 75),
  4.                  P2 = c(10, 9, 15, 13, 10, 8))

I want to perform a linear regression on all numeric columns of my dataframe using "Cohort" as the predictor (with the intent of adding features, such as "Sex", in future analysis).

I subset my dataframe to drop all irrelevant columns (in this toy example, Sex):

  1. new_df &lt;- df[,-c(Sex)]

Then I perform the regression like this:

  1. fit &lt;- lapply(new_df[-1], function(y){summary(lm(y ~ Cohort, data=new_df))})

When I test this on a small subset of my df (~5 columns) it works fine. In reality my df is ~7300 columns. When I run the command on the full dataframe I get this error:

  1. Error in lm.fit(x, y, offset = offset, singular.ok = singular.ok, ...) :
  2. NA/NaN/Inf in &#39;y&#39;

I then assumed it was an issue with N/A values but when I do this I get back '0':

  1. sum(is.na(new_df))

I have also tried the na.action=na.omit but that did not help the error either.

My end goal is to perform these regressions and extract the p-value and r-squared values using anova(fit)$&#39;Pr(&gt;F)&#39; and summary(fit)$r.squared, respectively.

How can I correct this error, or is there a better method to do this? Additionally, moving forward how can I perform this by not subsetting my dataframe when I add other features to the regression?

EDIT:

@Parfait A dput() example of my df:

  1. dput(new_data[1:4, 1:4])
  3. structure(list(Cohort = c(&quot;Disease&quot;, &quot;Disease&quot;, &quot;Control&quot;, &quot;Control&quot;), 
  4.     seq.10010.10 = c(8.33449676839042, 8.39959836912012, 8.34385193344212, 
  5.     8.43546191447928), seq.10011.65 = c(11.5222872738433, 11.7652860987237, 
  6.     11.1661630826461, 11.008848763327), seq.10012.5 = c(10.5414838640543, 
  7.     10.6862378767518, 10.5408061105915, 10.726558779105)), class = c(&quot;soma_adat&quot;, 
  8. &quot;data.frame&quot;), row.names = c(&quot;258633854330_1&quot;, &quot;258633854330_3&quot;, 
  9. &quot;258633854330_5&quot;, &quot;258633854330_6&quot;)

答案1

得分: 1

考虑将列名传递给方法，并使用reformulate动态构建公式。甚至可以使用tryCatch来处理所有列并捕获引发错误的列。以下返回从模型结果中提取的统计信息的数据框列表。

  1. fit_df_list <- sapply(
  2.     colnames(new_df)[-1],
  3.     function(col) {
  4.         tryCatch({
  5.           fml <- reformulate("Cohort", col)
  6.           fit <- lm(fml, data = new_df)
  7.           results <- summary(fit)
  9.           data.frame(
  10.               variable = col,
  11.               r_squared = results$r.squared,
  12.               f_stat = results$fstatistic["value"],
  13.               f_pvalue = anova(fit)$'Pr(>F)'[1]
  14.           )
  15.         }, error = \(e) paste("Error on", col, ":", e)
  16.         )
  17.     },
  18.     simplify = FALSE
  19. )
  21. # 筛选有问题的列
  22. fit_err_list <- Filter(is.character, fit_df_list)
  24. # 构建单一的主数据框
  25. fit_df <- do.call(rbind, Filter(is.data.frame, fit_df_list))
英文:

Consider passing the column name into method and build formula dynamically with reformulate. Even run tryCatch to process all columns and capture the columns raising errors. Below returns a list of data frames of extracted stats from model results.

  1. fit_df_list &lt;- sapply(
  2.     colnames(new_df)[-1],
  3.     function(col) {
  4.         tryCatch({
  5.           fml &lt;- reformulate(&quot;Cohort&quot;, col)
  6.           fit &lt;- lm(fml, data = new_df)
  7.           results &lt;- summary(fit)
  9.           data.frame(
  10.               variable = col,
  11.               r_squared = results$r.squared,
  12.               f_stat = results$fstatistic[&quot;value&quot;],
  13.               f_pvalue = anova(fit)$&#39;Pr(&gt;F)&#39;[1]
  14.           )
  15.         }, error = \(e) paste(&quot;Error on&quot;, col, &quot;:&quot;, e)
  16.         )
  17.     },
  18.     simplify = FALSE
  19. )
  21. # FILTER FOR PROBLEMATIC COLUMNS
  22. fit_err_list &lt;- Filter(is.character, fit_df_list)
  24. # BUILD SINGLE, MASTER DATA FRAME
  25. fit_df &lt;- do.call(rbind, Filter(is.data.frame, fit_df_list))

huangapple
  • 本文由 发表于 2023年5月29日 06:30:14
  • 转载请务必保留本文链接：https://go.coder-hub.com/76353825.html
  • dataframe
  • lm
  • r
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定