satisfaction.qmd

# Feedback {#sec-satisfaction}

```{r setup, include=FALSE}
# FUNCTIONS AND LIBRARIES
# have to load all data over again each document https://forum.posit.co/t/quarto-cant-find-r-objects/156848
knitr::opts_chunk$set(echo = FALSE)
library(ggplot2)    #for plots
library(viridis)    #color scheme
library(tidyverse)  #for data munging including dates
library(tm)         #text analysis?
library(SnowballC)  #text analysis
library(wordcloud)  #word clouds

# Load custom function for "other" block
source("scripts/ggwordcloud_analysis_complete_ou_colors.R")

# Fix join
joined_attendance_post_clean <- read.csv("raw_data/processed_joined_attendance_post_clean.csv")
postworkshopsurveys_named <- read.csv("raw_data/processed_postworkshopsurveys_named.csv")


```

## Executive summary
**Feedback data show no difference in perception of workshop value by learners among formats (in person, hybrid, or virtual).**  Qualitative (word-cloud illustrations) feedback are generally positive.

## Quantitative questions

For each question, we examine statistically and graphically (if differences were statistically significant) the rankings divided by instruction format (DSI request 2023/07), request/not, and course/outreach variables.
In the future, we may check rankings vs length of course in hours.

# TODO need to clean out NA for variables we test, as it errors out in the statistical test (graphs work and just automatically remove)

### Do you anticipate that this new knowledge can be applied towards your work?
```{r workshop_feedback_newknowledge, echo=FALSE, message = FALSE, warning = FALSE}
library(ggplot2)    #for plots
#joined_attendance_post_clean %>%
postworkshopsurveys_named %>%
  ggplot(aes(x=Using.New.Knowledge., fill=Using.New.Knowledge.)) +
  geom_bar()+
  scale_fill_viridis_d (aesthetics = "fill") +
  theme(panel.background = element_rect(fill = "lightgray",
                                        color = "white",
                                        size = 0.5, linetype = "solid"),
        panel.grid.major = element_line(size = 0.5, linetype = "solid",
                                        color = "darkgray"),
        panel.grid.minor = element_line(size = 0.25, linetype = "solid",
                                        color = "darkgray"),
        plot.title = element_text(size = 20, hjust = 0.5),
        axis.title = element_text(size = 14),
        axis.text = element_text(size = 12, angle = 90, hjust = 0.95, vjust = 0.2)
  )
```

There was no significant difference in perception of learning new knowledge from the workshop among learning formats.
```{r workshop_feedback_newknowledge_statistics, echo=FALSE, message = FALSE, warning = FALSE}
# run test for significance
chi_newkno <- chisq.test(x = joined_attendance_post_clean$Location.of.Outreach,
           y = joined_attendance_post_clean$Using.New.Knowledge.,
           simulate.p.value = TRUE)

(workshop_feedback_chi_newkno_plot <- chi_newkno$p.value<=0.05)

```

```{r, workshop_feedback_newknowledge_plot, eval = workshop_feedback_chi_newkno_plot, include = workshop_feedback_chi_newkno_plot}
# will only eval or include (ie plot) if test in previous chunk is significant
joined_attendance_post_clean %>%
  group_by(Location.of.Outreach,
           Using.New.Knowledge.) %>%
  summarize(count_per_feeling = n()) %>%
ggplot(
       aes(x = Location.of.Outreach,
           y = count_per_feeling,
           fill = Using.New.Knowledge.)) +
  geom_col(position = "fill")+
    scale_fill_viridis_d (aesthetics = "fill")

```



### This workshop was valuable towards your program of study.
Most participants perceived the workshop as valuable towards their program of study.  Interestingly, "disagree" is never chosen by any of our several hundred respondents.

```{r workshop_feedback_valuableforprogram, echo=FALSE, message = FALSE, warning = FALSE}
joined_attendance_post_clean %>%
  mutate(Valuable.for.Program = fct_relevel(Valuable.for.Program,
                                            "Strongly agree",
                                            "Agree",
                                            "Somewhat agree", 
                                            "Neither agree nor disagree",
                                            "Somewhat disagree", 
                                            "Disagree",
                                            "Strongly disagree",
                                            "Not applicable",
                                            "No answer")) %>%
  ggplot(mapping = aes(x=Valuable.for.Program,
                       fill = Valuable.for.Program)) +
  geom_bar(color = "black")+
  scale_fill_manual(values=c(
                             "#8D1732",
                             "#8D1732",
                             "#8D1732",
                             "darkgray",
                             "darkgray",
                             "darkgray",
                             "gray99",
                             "gray99"
                             ))+
  theme_bw()+
  theme(
        plot.title = element_text(size = 20, hjust = 0.5),
        axis.title = element_text(size = 14),
        axis.text = element_text(size = 12, 
                                 angle = 45,
                                 hjust = 1,
                                 vjust = 0.2)
  )+
  labs(y = "Number of survey respondents with this answer",
       x = "Response about value of workshop towards program of study",
       fill = "Agreement on value")+
  facet_grid(~Location.of.Outreach)

```

There was no difference in perception of the material as "valuable to my program of study" between workshop formats (hybrid, online, or in person).
```{r, workshop_feedback_valuableforprogram_stats, include = TRUE}
# run test for significance
chi_program <- chisq.test(x = joined_attendance_post_clean$Location.of.Outreach,
           y = joined_attendance_post_clean$Valuable.for.Program,
           simulate.p.value = TRUE)

(workshop_feedback_valuableforprogram_plot <- chi_program$p.value<=0.05)
```

```{r, workshop_feedback_valuableforprogram_plot, eval = workshop_feedback_valuableforprogram_plot, include = workshop_feedback_valuableforprogram_plot}
# will only eval or include (ie plot) if test in previous chunk is significant
# count is WRONG summary  here, because we may  have more people in a given category
# proportion of total for row?
# https://r-graphics.org/recipe-bar-graph-proportional-stacked-bar
joined_attendance_post_clean %>%
  group_by(Location.of.Outreach,
           Valuable.for.Program) %>%
  summarize(count_per_feeling = n()) %>%
ggplot(
       aes(x = Location.of.Outreach,
           y = count_per_feeling,
           fill = Valuable.for.Program)) +
  geom_col(position = "fill")+
    scale_fill_viridis_d (aesthetics = "fill")
```



### This workshop was valuable towards your career.
Most participants perceived the workshop as valuable towards their career.
```{r workshop_feedback_valuableforcareer, echo=FALSE, message = FALSE, warning = FALSE}
joined_attendance_post_clean %>%
  mutate(Valuable.for.Career = fct_relevel(Valuable.for.Career, "Strongly agree", "Agree", "Somewhat agree", 
                                           "Neither agree nor disagree", "Somewhat disagree", "Disagree", 
                                           "Strongly disagree", "Not applicable", "No answer")) %>%
  ggplot(aes(x=Valuable.for.Career, fill=Valuable.for.Career)) +
  geom_bar()+
  scale_fill_viridis_d (aesthetics = "fill") +
  theme(panel.background = element_rect(fill = "lightgray",
                                        color = "white",
                                        size = 0.5, linetype = "solid"),
        panel.grid.major = element_line(size = 0.5, linetype = "solid",
                                        color = "darkgray"),
        panel.grid.minor = element_line(size = 0.25, linetype = "solid",
                                        color = "darkgray"),
        plot.title = element_text(size = 20, hjust = 0.5),
        axis.title = element_text(size = 14),
        axis.text = element_text(size = 12, angle = 90, hjust = 0.95, vjust = 0.2)
  )
```

There was no difference in perception of the material as "valuable to my career" between workshop formats (hybrid, online, or in person).
```{r, workshop_feedback_valuableforcareer_stats, include = TRUE}
# run test for significance
chi_career <- chisq.test(x = joined_attendance_post_clean$Location.of.Outreach,
           y = joined_attendance_post_clean$Valuable.for.Career,
           simulate.p.value = TRUE)

(workshop_feedback_valuableforcareer_plot <- chi_career$p.value<=0.05)
```
There was no difference among career levels (classifications) for perception of value towards their careers.
```{r, workshop_feedback_valuableforcareerclass_stats, include = TRUE}
# run test for significance
chi_careerclass <- chisq.test(x = joined_attendance_post_clean$Classification,
           y = joined_attendance_post_clean$Valuable.for.Career,
           simulate.p.value = TRUE)

(workshop_feedback_valuableforcareerclass_plot <- chi_careerclass$p.value<=0.05)
```

### This workshop was valuable towards your teaching.
More respondents felt the workshops were not applicable towards their teaching but were otherwise positive. As our workshops are research focused, this is probably reasonable.

```{r workshop_feedback_valuableforteaching, echo=FALSE, message = FALSE, warning = FALSE}
joined_attendance_post_clean %>%
  mutate(Valuable.for.Teaching = fct_relevel(Valuable.for.Teaching, "Strongly agree", "Agree", "Somewhat agree", 
                                             "Neither agree nor disagree", "Somewhat disagree", "Disagree", 
                                             "Strongly disagree", "Not applicable", "No answer")) %>%
  ggplot(aes(x=Valuable.for.Teaching, fill=Valuable.for.Teaching)) +
  geom_bar()+
  scale_fill_viridis_d (aesthetics = "fill") +
  theme(panel.background = element_rect(fill = "lightgray",
                                        color = "white",
                                        size = 0.5, linetype = "solid"),
        panel.grid.major = element_line(size = 0.5, linetype = "solid",
                                        color = "darkgray"),
        panel.grid.minor = element_line(size = 0.25, linetype = "solid",
                                        color = "darkgray"),
        plot.title = element_text(size = 20, hjust = 0.5),
        axis.title = element_text(size = 14),
        axis.text = element_text(size = 12, angle = 90, hjust = 0.95, vjust = 0.2)
  )

```

There was no difference in perception of the material as "valuable to my teaching" between workshop formats (hybrid, online, or in person).
```{r, workshop_feedback_valuableforteaching_stats, include = TRUE}
# run test for significance
chi_teach <- chisq.test(x = joined_attendance_post_clean$Location.of.Outreach,
           y = joined_attendance_post_clean$Valuable.for.Teaching,
           simulate.p.value = TRUE)

(workshop_feedback_valuableforteach_plot <- chi_teach$p.value<=0.05)
```



### How do you rate your skills after learning about this workshop topic?
We have a range of skill ratings after the workshop.  This could be an area for improvement but could also be dependent on whether we have targeted the right audience with the right level of materials.
```{r workshop_feedback_skills, echo=FALSE, message = FALSE, warning = FALSE}
joined_attendance_post_clean %>%
  mutate(Rate.Skills = fct_relevel(How.do.you.rate.your.skills.after.learning.abou..., "Need more support and assistance if implementing",
                                   "Can complete a related task independently but not to desired level",
                                   "Can complete a related task independently to desired level",
                                   "Can complete and expand on task, possibly assist others",
                                   "Did not respond")) %>%
  ggplot(aes(x=How.do.you.rate.your.skills.after.learning.abou..., fill=How.do.you.rate.your.skills.after.learning.abou...)) +
  geom_bar()+
  scale_fill_viridis_d (aesthetics = "fill") +
  scale_x_discrete(labels=c("Needs more support", "Independent, but not to desired level", "Independent to desired level", 
                            "Independent, could teach skill", "Did not respond")) +
  theme(panel.background = element_rect(fill = "lightgray",
                                        color = "white",
                                        size = 0.5, linetype = "solid"),
        panel.grid.major = element_line(size = 0.5, linetype = "solid",
                                        color = "darkgray"),
        panel.grid.minor = element_line(size = 0.25, linetype = "solid",
                                        color = "darkgray"),
        plot.title = element_text(size = 20, hjust = 0.5),
        axis.title = element_text(size = 14),
        axis.text = element_text(size = 12, angle = 90, hjust = 0.95, vjust = 0.2)
  )
```

There was no difference in perception of future abilities to use skills from the workshop between formats (hybrid, online, or in person).
```{r, workshop_feedback_skills_stats, include = TRUE}
# run test for significance
chi_skills <- chisq.test(x = joined_attendance_post_clean$Location.of.Outreach,
           y = joined_attendance_post_clean$How.do.you.rate.your.skills.after.learning.abou...,
           simulate.p.value = TRUE)

(workshop_feedback_skills_plot <- chi_skills$p.value<=0.05)
```


## Qualitative feedback (Wordcloud code)

### "What did you like about the workshop?"
```{r workshop_feedback_liked_code, include=FALSE}
workshopliked <- readLines("raw_data/postworkshop/WorkshopLiked_Cleaned.txt")

# TODO: automate cleaning.  otherwise pull from postworkshopsurvey original columns and then clean by code instead of manually.  CMC and BN discussed 11/28/2023.

likedcorpus <- Corpus(VectorSource(workshopliked))

likedcorpus <- tm_map(likedcorpus, removeNumbers)
likedcorpus <- tm_map(likedcorpus, removeWords, c("liked", "the", "and", "was", "that", "very", 
                                                  "with", "how", "for", "were", "all", "through",
                                                  "you", "have", "like", "more", "but", "had", "could",
                                                  "what", "get", "also", "not", "this", "way",
                                                  "really", "about", "will", "our", "being")) 
likedcorpus <- tm_map(likedcorpus, stripWhitespace)

likeddocmatrix <- TermDocumentMatrix(likedcorpus)
likedmatrix <- as.matrix(likeddocmatrix)
v <- sort(rowSums(likedmatrix), decreasing = TRUE)
likeddf <- data.frame(word = names(v), freq=v)

col <- sapply(seq(0.1, 1, 0.1), function(x) adjustcolor("darkred", x))

```

### "What about the workshop needed improvement?"
```{r workshop_feedback_needsimprovement_code, include=FALSE}
workshopimprove<- readLines("raw_data/postworkshop/WorkshopNeedsImprovement_Cleaned.txt")
improvecorpus <- Corpus(VectorSource(workshopimprove))

improvecorpus <- tm_map(improvecorpus, removeNumbers)
improvecorpus <- tm_map(improvecorpus, removeWords, c("the", "and", "was", "that", "very", 
                                                  "with", "how", "for", "were", "all", "through",
                                                  "you", "have", "like", "more", "but", "had", "could",
                                                  "what", "get", "also", "not", "this", "way",
                                                  "really", "about", "will", "our", "being", "would",
                                                  "workshop", "there", "are", "can", "they", "think")) 
improvecorpus <- tm_map(improvecorpus, stripWhitespace)

improvedocmatrix <- TermDocumentMatrix(improvecorpus)
improvematrix <- as.matrix(improvedocmatrix)
v <- sort(rowSums(improvematrix), decreasing = TRUE)
improvedf <- data.frame(word = names(v), freq=v)

col <- sapply(seq(0.1, 1, 0.1), function(x) adjustcolor("darkred", x))
```

### "What other workshop topics would interest you?"
We can examine this wordcloud to see if we need to advertise existing workshops more in addition to having ideas about new ones.
```{r workshop_feedback_othertopics_code, include=FALSE}
workshoptopics<- readLines("raw_data/postworkshop/WorkshopOtherTopics_Cleaned.txt")
topicscorpus <- Corpus(VectorSource(workshoptopics))

topicscorpus <- tm_map(topicscorpus, removeNumbers)
topicscorpus <- tm_map(topicscorpus, removeWords, c("the", "and", "was", "that", "very", 
                                                  "with", "how", "for", "were", "all", "through",
                                                  "you", "have", "like", "more", "but", "had", "could",
                                                  "what", "get", "also", "not", "this", "way",
                                                  "really", "about", "will", "our", "being", "would",
                                                  "workshop", "there", "are", "can", "they", "think",
                                                  "workshops", "when", "these")) 
topicscorpus <- tm_map(topicscorpus, stripWhitespace)

topicsdocmatrix <- TermDocumentMatrix(topicscorpus)
topicsmatrix <- as.matrix(topicsdocmatrix)
v <- sort(rowSums(topicsmatrix), decreasing = TRUE)
topicsdf <- data.frame(word = names(v), freq=v)

col <- sapply(seq(0.1, 1, 0.1), function(x) adjustcolor("darkred", x))
```

### "Any other comments?"
```{r workshop_feedback_othercomments_code, include=FALSE}
workshopcomments<- readLines("raw_data/postworkshop/WorkshopOtherComments_Cleaned.txt")
commentscorpus <- Corpus(VectorSource(workshopcomments))

commentscorpus <- tm_map(commentscorpus, removeNumbers)
commentscorpus <- tm_map(commentscorpus, removeWords, c("the", "and", "was", "that", "very", 
                                                  "with", "how", "for", "were", "all", "through",
                                                  "you", "have", "like", "more", "but", "had", "could",
                                                  "what", "get", "also", "not", "this", "way",
                                                  "really", "about", "will", "our", "being", "would",
                                                  "workshop", "there", "are", "can", "they", "think",
                                                  "workshops", "when", "these")) 
commentscorpus <- tm_map(commentscorpus, stripWhitespace)

commentsdocmatrix <- TermDocumentMatrix(commentscorpus)
commentsmatrix <- as.matrix(commentsdocmatrix)
v <- sort(rowSums(commentsmatrix), decreasing = TRUE)
commentsdf <- data.frame(word = names(v), freq=v)

col <- sapply(seq(0.1, 1, 0.1), function(x) adjustcolor("darkred", x))
```

# Name one aspect of the workshop that you liked.
```{r workshop_feedback_liked, echo=FALSE, message = FALSE, warning = FALSE}
wordcloud(words = likeddf$word, freq = likeddf$freq, min.freq = 5,
          max.words = 50, random.order = FALSE,
          random.color = FALSE, colors=col)
```

# Name one aspect of the workshop that could be improved.
```{r workshop_feedback_needsimprovement, echo=FALSE, message = FALSE, warning = FALSE}
wordcloud(words = improvedf$word, freq = improvedf$freq, min.freq = 5,
          max.words = 50, random.order = FALSE,
          random.color = FALSE, colors=col)
```

# What other workshop topics would interest you?
```{r workshop_feedback_otherworkshops, echo=FALSE, message = FALSE, warning = FALSE}
wordcloud(words = topicsdf$word, freq = topicsdf$freq, min.freq = 5,
          max.words = 50, random.order = FALSE,
          random.color = FALSE, colors=col)
```

# Please let us know any other comments you have about the workshop.
```{r workshop_feedback_othercomments, echo=FALSE, message = FALSE, warning = FALSE}
wordcloud(words = commentsdf$word, freq = commentsdf$freq, min.freq = 5,
          max.words = 50, random.order = FALSE,
          random.color = FALSE, colors=col)
```