Is it possible to use custom function in dplyr mutate with the option to use rm.na = TRUE or rm.na = FALSE

by Moe   Last Updated August 14, 2019 05:26 AM - source

I have a function to scale/ normalize / z-score transform a number of variables using mutate_at. The source of the function is in the link: https://dplyr.tidyverse.org/reference/mutate_all.html

scale <- function(x, na.rm = FALSE) (x - mean(x, na.rm = na.rm)) / sd(x, na.rm)

Use of that function results in all NAs if there are any NAs present in the initial variable, as outlined in the example below:

#make df1
set.seed(123)
df <- data.frame(
  col_A = c(5, NA,2,4, 4,5,8,3,7,9),
  col_B = as.numeric(sample(20:90, size = 10)),
  col_C = as.numeric(sample(1000:2000, size = 10))
)

df

I have tried setting na.rm = TRUE, which seems to achieve what I'm after.

scale_narm_true <- function(x, na.rm = TRUE) (x - mean(x, na.rm = na.rm)) / sd(x, na.rm)

vars <- c("col_A", "col_B")
df_z_score <- df %>%
  mutate_at(vars, list(scaled_var = scale)) %>% # introduces NAs in the resulting variables
  mutate_at(vars, list(scaled_narm_true_var = scale_narm_true)) # works as expected and desired

However, what I'm really after is the option to include na.rm = TRUE in the actual mutate_at call, such as below

df_z_score_attempt <- df %>%
  mutate_at(vars, list(scaled_var = scale, na.rm=T)) # this doesn't work!

Any help would be appreciated, especially since it's supposed to be possible according to https://dplyr.tidyverse.org/reference/mutate_all.html, stating that this is possible:

starwars %>% mutate_at(c("height", "mass"), scale2, na.rm = TRUE)


Answers 1


An option would be to use the ~ to specify the anonymous function call and then get the column a .

library(dplyr)
df %>%
    mutate_at(vars, list(scaled_var =  ~scale(., na.rm=TRUE)) )
#   col_A col_B col_C col_A_scaled_var col_B_scaled_var
#1      5    50  1373       -0.0952381       -0.8939893
#2     NA    70  1664               NA        0.3306536
#3      2    33  1601       -1.3809524       -1.9349357
#4      4    86  1602       -0.5238095        1.3103678
#5      4    61  1767       -0.5238095       -0.2204357
#6      5    69  1708       -0.0952381        0.2694214
#7      8    62  1090        1.1904762       -0.1592036
#8      3    56  1952       -0.9523810       -0.5265964
#9      7    71  1347        0.7619048        0.3918857
#10     9    88  1648        1.6190476        1.4328321

If we use the default option, the column will be NA

df %>%
    mutate_at(vars, list(scaled_var = scale) )
#   col_A col_B col_C col_A_scaled_var col_B_scaled_var
#1      5    50  1373               NA       -0.8939893
#2     NA    70  1664               NA        0.3306536
#3      2    33  1601               NA       -1.9349357
#4      4    86  1602               NA        1.3103678
#5      4    61  1767               NA       -0.2204357
#6      5    69  1708               NA        0.2694214
#7      8    62  1090               NA       -0.1592036
#8      3    56  1952               NA       -0.5265964
#9      7    71  1347               NA        0.3918857
#10     9    88  1648               NA        1.4328321
akrun
akrun
August 14, 2019 05:19 AM

Related Questions



Elegant way to write function in R

Updated April 02, 2019 13:26 PM



Conditional sorting of column in R

Updated December 31, 2017 15:26 PM