Assessment, Evaluation, and Research (AER)

Below I have outlined my work toward achieving advanced proficiency in this area. To read more about this competency and my current proficiency level, you can read the narrative in this page.

Competency Overview

The purpose of this competency is to challenge student affairs professionals to not only design and conduct assessment, evaluation, and research methods, but to think about these concepts critically and use them to inform everyday practice (ACPA & NASPA, 2015). As professionals advance through this competency, it is critical to understand how these methodologies affect the political and ethical landscape at the institutional level.

Proficiency at a foundational level of this competency would reflect a basic understanding of assessment and research, maintaining confidentiality of data, and designing clear and measurable learning outcomes informed by theoretical frameworks (ACPA & NASPA, 2015). Professionals at this level would also understand how to conduct data collection using current technological methods, assess the validity and reliability of different methods of data collection, and consider both the strengths and limitations of methodical approaches (ACPA & NASPA, 2015).

Those in the intermediate level are able to apply more of the basic knowledge of this competency in practice. The rhetoric in the intermediate area centers around managing, educating others, actively participating in research and research design, and articulating findings in a thoughtful and ethical way (ACPA & NASPA, 2015).

Professionals categorized in the advanced proficiency level of this competency not only apply those basic concepts of assessment, evaluation, and research, but take a leadership role in the AER process. Leaders in this competency area create a culture rooted in evidence-based practice and ensure compliance of ethical and purposeful assessment at the unit, division, or institutional level (ACPA & NASPA, 2015). Overall, professionals with this level of proficiency are experienced in ethical data collection and storytelling and model the data-driven assessment and action that follows.

My Competency Level & Applied Knowledge

I would describe myself as operating at the intermediate level of proficiency in this competency, largely in part to the experiential learning requirements of Assessment in Higher Education and Student Affairs (HEA 650) and my work supervising the student ambassadors in the Undergraduate Admissions Office at Buffalo State College.

From my coursework in Assessment in Higher Education and Student Affairs (HEA 650), I learned how to develop and sequence interview questions, conduct focus groups in an ethical way (obtaining certification through the CITI Program for Human Research), and how to code, analyze, and communicate the collected data. The most challenging part of the course was synthesizing the data and determining the story the results were telling. Knowing that the results could be shared with the Assistant Vice President and Provost, I wanted to ensure I was "using culturally relevant and culturally appropriate terminology" to report the findings in a clear and concise way (ACPA & NASPA, 2015, p.20). I found myself successful in this area when my report was selected to be sent to the AVP and Provost for consideration for the next strategic plan.

In my role as an admissions advisor, I help to supervise the admissions ambassadors, who represent the college and interact frequently with prospective students. Starting in September when I was tasked with scheduling trainings and evaluations for the new hires, I was shocked at the antiquated and vague evaluation rubric used in our training process. I reasoned that if our office boasts a rigorous standard for tour quality, we should have a rubric that reflects that standard. I created a more detailed rubric that outlines clear expectations of the training ambassador and championed to have the rubric translated into a Qualtrics form accessed via iPads for a "technologically current" method of evaluation (ACPA & NASPA, 2015, p.20).

Since assessment is such an integral part of higher education and student affairs work, I intend to incorporate assessment into each role I hold on campus. As an admissions advisor, I believe I will grow in this competency area by learning more about Qualtrics to collect data. My goal is to explore the program to learn how to collect data in the most efficient way and how to understand the results. With access to forms for data collection purposes, I can collaborate with others to create and share easy-to-use tools for purposeful data collection and analysis. I believe this process will help me contribute to a "culture of evidence at the department level" (ACPA & NASPA, 2015, p. 20). Moreover, I can also utilize data from Capture Higher Ed, a program used by our office that uses quantitative data to determine students that are most likely to attend Buffalo State. This can help me prioritize my communications with prospective students and applicants to recruit more students for the institution.

In my role as a co-chair for Work Group II in the MSCHE Self-Study Design, I plan to accurately tell the story of our institution's ethics and integrity by finding resources to draw data from throughout the institution and determine what data needs to be collected in order to get a clear and detailed picture. As my role of an editor for two of our Work Group's lines of inquiry, I intend to determine ways we can further investigate the following questions:

• What evidence is available to demonstrate that planning and decision-making is data-informed?

• How effective are Buffalo State College’s services and/or programs in promoting accessibility, affordability, and assistance to students in understanding funding sources and options, value received for cost, and making informed decisions about incurring debt?

While these answers may not be clear to us given the current resources available, I plan to collaborate with the group of experienced faculty and staff in my Work Group to identify where we need additional information and how to collect that evidence from the institution. I believe this experience will help me to think more broadly at the institutional level about data collection, storytelling, and action.

Advancement in AER

As a graduate student, I hope to further develop my skills of applying theory and methodological designs to a research proposal in the Research Methods course (HEA 689). In this class, I will think critically about research methodologies, explain relevant literature related to the topic I am researching, and design a research report to be presented in both written and oral formats.

References

American College Personnel Association & National Association of Student Personnel Administrators (2015). Professional Competency Areas for Student Affairs Educators. https://buffalostate.open.suny.edu/bbcswebdav/pid-1819821-dt-content-rid-17029589_1/courses/202030-HEA624-1665/ACPA_NASPA_Professional_Competencies.pdf.