Using Focus Groups for Assessment, Evaluation, and Improvement

A focus group, as Krueger (2015) defined, is “a carefully planned series of discussions designed to obtain perceptions on a defined area of interest in a permissive, non-threatening environment.” A focus group approach, as a qualitative method, can provide rich information on perceptions, feelings, and reactions through participant interactions resembling the dynamics of real life in a comfortable environment. In particular, it can be used to uncover mysteries behind vague or unexpected responses by probing into the “how” or “why” of participant behavior and thinking.

Used extensively in marketing research, focus groups have been gaining popularity among educational researchers and evaluation professionals. In higher education settings, focus groups can be used for assessing institutional level initiatives as well as course- and program-level experiences for students. For example, I recently collaborated with two Franklin University colleagues, Dean of Students Dr. Lynne Hull and Coordinator of Student Affairs and Enrollment Management Ms. Elizabeth Skursha, to conduct a focus group study. The goal of our study was to explore student experiences with the various initiatives at Franklin and how best to continually improve the services we offer to students.

Our team hosted five focus groups comprised of undergraduate, master’s, and doctoral students to gauge students’ perspectives on various campus initiatives — what they view as helpful and what improvements they would like to see. The focus group method is only one prong of a triangulated approach we use to examine the impact of University initiatives aimed at improving services and identifying areas for improvement. The other two prongs are a student survey and feedback collected from faculty and staff. Based on the data we have collected and analyzed, we have already seen some converging themes among the different prongs, which speaks to the reliability of our findings and which will add confidence to the decision making process.

To conduct focus groups for program evaluation and improvement in higher education settings, not only can students be invited to participate, but other stakeholders may also be involved, such as faculty, staff members, and individuals external to the University. For example, the Indiana University School of Education conducted focus groups comprised of a large range of stakeholders for their program evaluation, which included teachers, principals, higher education personnel, public, parents, and state organization leaders (Indiana University, n.d.).

Given its power to uncover mysteries behind unclear responses or data patterns, a focus group may be used to further assess and better understand student learning. In higher education settings, faculty and administrators need to assess student learning on the specific learning outcomes. Assessment artifacts such as course projects or tests provide direct evidence of student achievement. However, these approaches do not necessarily provide the “how” and “why” of student learning. To fill this gap, we may use a focus group to probe directly into course activities in light of designated learning outcomes. For example, student participants may be asked what activities or assignments best help them learn how to “use samples to construct confidence internals” (a statistics course outcome) and why. This kind of assessment activity engages students into reflections of their learning processes and the intended learning outcomes, thus truly integrating learning and assessment and yielding actionable information.

In spite of the various advantages of the focus group approach, decision makers are sometimes leery about acting on focus group findings, especially if resulting actions may have broad impacts. This explains why focus groups are often combined with other approaches, such as surveys, in program evaluation. There are different ways of pairing up focus groups and surveys for gathering information to answer research or evaluation questions. Sometimes, a survey may be conducted after the focus group stage to help confirm findings and establish generalizability, which is what the Indiana University School of Education did following their focus groups for their program evaluation (Indiana University, n.d.). In other cases, initial surveys may be conducted to uncover trends or themes and are then followed with focus groups to further explore the “how” and “why.” Ultimately, the approach to choose depends on the specific contexts of the evaluation or research.

Here is a story that illustrates why we may need focus groups to help interpret survey findings: An institution conducted a survey that indicated students were interested in taking certain foreign language classes. However, after the institution had expended the effort and resources to offer the classes in question, very few students registered, causing the program to fail. What might have been the reason for this seemingly inconsistent behavior? The survey procedure or instrument might have had flaws, leading to invalid findings. In addition to carefully re-examining the survey itself, the institution could have conducted focus groups following the survey to further explore student responses, before significant resources were spent to develop the program in question.