The American Physical Society’s Bridge Program (APS-BP) has significantly higher persistence rates than physics graduate education across the country (90% vs 60%). Motivated by this early positive outcome, the researchers aimed to explore and offer empirical evidence as to what aspects of the Bridge Program contribute to these higher persistence rates. This paper describes the development and psychometric evaluation of a survey instrument, the “Aspects of Student Experience Scale (ASES)”, that was designed to evaluate physics graduate students’ experiences related to different forms of departmental support that may have made the APS-BP successful, namely: (a) mentoring and research experience, (b) professional development, (c) social and academic integration, and (d) financial support. This paper supports the ongoing development and validation of this survey tool.397 physics graduate students across 19 programs participated in the study. The initial findings reported here reveal that graduate students experience adequate mentoring and financial support, but have limited social and academic integration or professional development. Practitioners may be able to use ASES to identify, based on students’ perspectives, which practices they may need to address to better support the student experience.
Student Experience of Departmental Support Structures
Student experience reflects one aspect of department climate focusing on patterns of students’ experiences of available opportunities/policies/resources that are intended to better support students through the program.
Departmental Support Structure
The active and intentional effort on behalf of the program to ensure that certain opportunities/policies/resources are available to students when needed to improve student experience.
The active and intentional effort on behalf of the program to ensure that students do not become isolated.
The active and intentional effort on behalf of the program to ensure that students have an individualized coursework plan that connects their preparation with course placement and academic support.
The utility of the ASES was assessed through responses of 397 students from 19 physics graduate programs. The researchers also collected interview data from chairs and graduate advisors from a subset of these programs to explore how these reported support structures look from the perspectives of department chairs or graduate advisors. Some key terms used in the analysis of this study:
Questionnaire/Survey instrument is a set of items designed to measure one or more underlying constructs. The item responses are then summarized to yield a score.
Psychometric Evaluation of a questionnaire is a research method that concerns the implementation of various tests (qualitative and quantitative) to assess the quality of the developed questionnaire.
Principal Components Analysis (PCA) aims to optimize the grouping of individual variables into a set of higher-order components, which we call survey thematic groups or themes. In PCA, the assumption is that there are larger themes that can be built from looking at the individual survey items. The researchers bring their knowledge to interpreting what common theme brings the survey items together.
Students' responses were used to a) assess ASES’ psychometric properties and b) offer an understanding of students’ self-reported experiences of the departmental support structures measured by this instrument. Below, we summarize the key findings:
Students' responses to the survey suggest that, on average, students experience adequate mentoring & research experience, and financial support, but report a lack of support in aspects related to professional development and social & academic integration.
Results indicate that students at two APSBP affiliated departments reported being better supported than students at two non-APSBP affiliated departments on all components and with the stronger signals for the components of social & academic integration and financial support.
A difference in students’ reports of experiencing Social and Academic Integration and/or Financial Support aspects could be correlated with increased retention at APS-BP departments, consistent with prior research that has found a link between social isolation and attrition.
Implications for Practice
The Aspect of Student Experience Scale (ASES) was only designed to help graduate programs assess students’ experiences with department support structures. One potential limitation is the misuse of the tool by the community. ASES is not designed to evaluate the effectiveness of the graduate program or quality of mentoring, but rather whether the student receives adequate mentoring regardless of the source. Relatedly, programs whose leaders are less familiar with equity work may misinterpret this survey as one that assesses programs’ climate for racial, gender, or other marginalized identities. Climate assessments, when attuned to issues of equity, can provide programs with information about changes that may help advance equity in their academic programs.
The early findings on the importance of students’ social & academic integration suggest that the use of ASES might be particularly valuable for large graduate programs where student-faculty interaction is less frequent at early stages in the student program, and isolation is more likely to occur.