Calculating standard deviations for survey subgroups in R

Blog

If you work with survey data in R, you would already know how useful the package survey is.

Among its many other uses, the survey package can compute summary statistics by subgroups. For example, if you have survey of individuals from several countries with an item on the respondents’ income, you can calculate the average income in each subgroup with the svyby() function.

However, like many other functions in the package, svyby() returns standard errors—but not standard deviations—of the mean values. Although it is possible to calculate the standard deviations for all observations with the svyvar() function, with no argument for subgroup analysis, this can quickly become tiresome if you have many subgroups or variables to work with.

I have recently been working witht the European Social Survey data, and I needed to calculate the standard deviations of the means of three variabels in 23 countries. hence, I have written a loop, which I am sure I will use again.

Hoping that it might be useful for someoneelse as well, I am leaving it here:

datalist <- list()

for(i in levels(as.factor(df$subgroups))) {
        
  df_subset <- df[df$subgroups == i, ] 
  
  design <- svydesign(id = ~1 , data = df_subset, weight = ~surveyweight)
  
  dat <- data.frame(as.data.frame(sqrt(svyvar(~variable1, na.rm= TRUE, design = design)))[,"variance"],
                    as.data.frame(sqrt(svyvar(~variable2, na.rm= TRUE, design = design)))[,"variance"])
  
  colnames(dat)<-c("sd.variable1", "sd.variable2")
  
  datalist[[i]] <- dat
}

df_sd <- as.data.frame(do.call(rbind, datalist)) %>%
  rownames_to_column() %>%
  rename(subgroups = rowname)

where you will need to change

  1. df with your data frame,
  2. subgroups with the variable name for the subgroups (i.e. countries),
  3. surveyweight with the weight(s) in your survey, and
  4. variable1 and variable2 with the variable names for which you are calculating the standard deviations.