Comprehensive Assessment Activities and Learning Outcomes

Activities and Outcomes

The student is to write a comprehensive self-assessment paper that successfully evaluates, organizes and synthesizes the student’s learning thus far during their doctoral studies. In the process, the student should gain a deeper understanding and a broader perspective on what they’ve learned, and how they can build on this as they move forward toward their goals. They will participate in an oral exam and discussion of their knowledge, using the comprehensive self-assessment paper as a starting point. Once the student passes the oral exam, they will submit a dissertation proposal for review, and once that is approved, the student proceeds to the dissertation stage. 

Activity 1: The faculty review the student's completed coursework and discuss and conduct an oral exam of the student regarding what the student has thus far learned and accomplished--in order to determine:

• if the student has either completely achieved degree program outcomes, or has progressed sufficiently to be able to finish achieving those outcomes while doing their dissertation

• if the student is  prepared to undertake the rigorous study required for a doctoral dissertation, is able to focus their attention on the dissertation, and is capable of achieving any modest added progress toward degree program outcomes required and identified by faculty. 

The faculty evaluate the student's work and their performance in the exam on the following criteria:

• Degree Program Learning Outcomes met, or alternatively that the Program Learning Outcomes are nearly met, and that there is a realistic plan for completing the remaining Degree Program Learning Outcomes

• The student's performance in the exam is evaluated with regards to the student's: a) overall understanding, b) coherence and comprehensiveness of argument, c) quality and relevance of evidence presented, d) ability to examine and articulate implications of what they have learned and studied, and e) if they are able to do so with little or no prompting. 

Based on their review of the student’s completed coursework, the faculty conduct an oral exam to see if the student is ready to proceed to the Comprehensive Exam Phase, outlined in the two-unit course, EDD 693: Comprehensive Assessment of Student Learning and Plans for Dissertation and Beyond. To evaluate the student’s coursework and the student’s oral exam, they use the Rubric, below for the Qualifying Exam. 

Activity 2: The student writes a comprehensive self-assessment paper to reflect on, and articulate, the highlights of the breadth and depth of what they have so far learned during their doctoral studies. 

Outcomes:  

The student will:

• Write a comprehensive self-assessment paper that evaluates, organizes and synthesizes their learning thus far during their doctoral studies. In that paper, the student will: 

• Articulate and write a critical and well-informed statement about their field(s) of specialization that includes details and nuances beyond broad generalizations.

• Articulate and explore several insights and questions about this emerging, interdisciplinary field of “higher education and social change” and about their area(s) of specialization in particular that their review committee considers to be at the level of proficiency and to be promising of leading toward new knowledge and/or practices; 

• Demonstrate the depth and breadth of their perspectives on what they’ve learned, and how they plan to build on this knowledge as they move forward toward their goals; 

• Articulate and discuss the evidence of the extent to which they have addressed each degree program learning outcome. 

Activity 3: The student participates in a comprehensive oral exam and discussion with three WISR faculty 

Outcomes:

Using the comprehensive self-assessment as evidence and as a starting point for discussion in the oral exam, the student will show the review committee that they have demonstrated proficiency and promise of creative work in the field. Specifically, the student will:

• identify and discuss convincingly those degree program learning outcomes which they already have met, and 

• identify and a realistic plan for completing the remaining degree program learning outcomes during the dissertation process. 

If the student’s review committee determine that the student's progress and plans are sufficient, the student will be approved to submit their dissertation proposal. Otherwise, the committee will articulate for the student what further learning must be demonstrated before the student begins work on the dissertation.

Activity 4: The student submits a Dissertation Proposal for Review by the three faculty, as well as by an outside expert in the student's proposed area of dissertation research. 

Outcomes:

The student will:

• develop a coherent, well thought out plan for their dissertation to these four members of what will become their Graduation Review Board, and 

• will present a plan that meets standards for original, ethically-informed action-oriented inquiry, including 

1. an appropriately thorough and targeted literature review, 

2. a well-designed plan for collecting original data,  

3. well-formulated questions that reflect the student’s interests and the potential to contribute to new knowledge and/or practices in the student’s proposed area of study. 

The Graduation Review Board will either approve the proposal or suggest that the student make further specific revisions and improvements. 

 Degree Program Outcomes and Measures/Indicators

EdD Program Learning Outcomes and Measures

Program outcomes are guided by several important considerations:

1.      WISR’s EdD in Higher Education and Social Change program outcome areas, outcomes and curriculum is guided by WISR’s institutional vision to be a “hub for community-based, social-action organizations and leaders that use higher learning and adult education to bring innovative theory into action for positive social change.”

2.      In addition, WISR’s EdD in Higher Education and Social Change program outcome areas, outcomes and curriculum is guided by WISR’s mission and the learning “meta-competencies” that are derived from WISR’s missions and values, that is, self-directed learning, action-oriented inquiry, multiculturalism, social justice, effective communication and collaboration, and the value of using one’s studies to build bridges to the future.

3.      Finally, in pursuing the program outcome areas through program outcomes, course outcomes, module outcomes, and measures, indicators, and evaluation rubrics, we draw on the first two areas of consideration, and also on the knowledge gained through WISR’s history of offering academic degree programs for innovative educators wishing to innovate in colleges and universities, professional education endeavors, and/or in popular, community-based adult education, while also aiming to contribute to larger social change. This knowledge is augmented by the collective academic and professional experience and knowledge of WISR’s faculty.

A: EdD Program-Specific Learning Outcomes

The student will demonstrate that they:

1. Know how experts use research, and articulate and use theories, key concepts, and professional practices in adult/higher education, and the possible roles of education in societal change.  Key areas in which the student must demonstrate such expert knowledge include, quite notably:

a. Education to promote diversity and inclusiveness.

b. Theories and philosophies of education.

c. The possible roles of education in contributing to social justice, human dignity, equality, and/or environmental sustainability.

d. Theories and practices in one or more of the following areas: higher education, professional education, and community-based popular adult learning.

2. Evaluate key theories and methods of educational innovation and social change, as indicated by

a.      Evaluating the strengths and limitations of a variety of educational theories and practices,

b.      Evaluating the circumstances in which specific educational theories and methods are likely to be usefully applied.

3. Apply skills of conscious and deliberate planning in pursuing goals as an innovative educator or leader, as indicated by making critical comparisons of alternative courses of action, for example in course-based action-research projects. In doing so, they will:

c.       Evaluate the relevance and efficacy of their recommended plan(s) of action.

d.      Evaluate uncertainties and dilemmas faced by others in the field, and

e.      Evaluate directions for inquiry to investigate alternative courses of action growing out of these dilemmas, uncertainties, and complexities.

4. Create new theoretical applications and strategic practices in at least one area of specialization, and within one specific setting, aiming to promote educational improvements that might contribute to greater societal or community well-being, as indicated especially by an in-depth inquiry during the Doctoral Dissertation. [Note: This outcome builds on the knowledge achieved in outcome #1.]

5. Apply skills of doing a creative, critically minded and comprehensive review of the literature in an area of special interest to the student, as indicated by:

    a. applying a variety of strategies for searching for relevant sources

                b. evaluating quality and credibility of sources

c. effectiveness in discussing and presenting findings, gaps in knowledge, limitations in existing research, and directions for future research

d. formulating some original concepts or questions for further action and inquiry.

Evaluation of these outcomes. These outcomes will be evidenced in the written assignments for each course--and guided and evaluated by course learning outcomes and module learning outcomes within each course. They will also be evaluated and evidenced through their course-based action-research projects, their written assignments in courses, their ongoing dialogue with faculty and the oral exams in each course, in the dissertation, and in their collaborations with others, such as in seminars and the online forum.

***

In addition to the above-mentioned EdD program-specific PLOs, EdD students must demonstrate the following general PLOs:

WISR General Program Learning Areas and Outcomes for EdD Students

The student will:

B: Self-Directed Learning.  

Demonstrate skills as a self-directed learner, as indicated by critically minded, intentional, and improvisational learning in doing their course assignments and dissertation.

C: Action-Research. 

Engage in creative and critically informed uses of methods of participatory and action-research in the pursuit of new, specialized knowledge and proficient practices, especially as indicated through their action-research projects and dissertation.

D: Multiculturalism and Inclusiveness. 

Demonstrate an awareness of issues of diversity and inclusiveness, by showing a sensitivity to the issues involved in working as an adult educator with diverse populations, as indicated in their writing, dialogue, dissertation, and/or action-research projects.

E: Social Change and Justice. 

Analyze the connections of educational practices that are both impacted by and aimed at addressing various community or societal problems and challenges--by showing in their writing, dialogue and/or action-research projects that they are inquiring into ways of creating change for social justice, greater equality and environmental sustainability, as part of the pursuit of specialized knowledge and effective leadership and innovation.

F: Communication and Collaboration. 

1)    Demonstrate skills of clear and engaging written communication, by a) writing clearly and in a well-organized fashion, b) showing that they can intentionally identify and communicate to a chosen audience(s), and c) using their own voice on topics that matter to them.

2)    Demonstrate skills of effective oral communication and collaboration, as indicated in a) their action-research projects or dissertation with people from more than one background, and b) in seminars and informal dialogue with other students and with faculty, and

3)    Produce a dissertation that can be used by others to work for valuable improvements and change, and also that is of sufficient quality to be considered seriously for professional publication

G: Build Bridges to the Future. 

1)    Demonstrate an awareness of employment opportunities, of if they prefer, meaningful volunteer opportunities, as a leader and innovative adult educator in an institutional or community setting, appropriate to their specialized capabilities, experience, and interests.

2)    Begin building bridges, i.e., specific action steps, to their post-graduate involvements, especially as indicated in their action-research projects and Doctoral Dissertation.

Evaluation of these outcomes. These outcomes will be evidenced in the written assignments for each course--and guided and evaluated by course learning outcomes and module learning outcomes within each course. They will also be evaluated and evidenced through the student’s practicum, their course-based action-research projects, their ongoing dialogue with faculty and the oral exams in each course, in the dissertation, and in their collaborations with others, such as in seminars and the online forum.

__________

**Paradigm to Conceptualize Development of Expertise through Learning that Builds on and Integrates the Achievement of Program Learning Outcomes—The Dreyfus Model

The Dreyfus Model is Used to Evaluate the Effectiveness of WISR’s Degree Programs, and to conceptualize the interconnections of degree program learning outcomes. The stages of the Dreyfus Model that are used at WISR are

1) the stage of “competent” serving as an orienting learning goal to guide students and faculty in the Master’s programs at WISR, and

2) the stage of “proficient” providing an orienting learning goal for students and faculty in the Doctoral program.

From time to time, we have seminars on this Model at WISR, to engage students and faculty in reflecting on and discussing how to make use of it to aid learning at WISR.  Here are a few highlights to consider.

The “competent” expert comes to appreciate that simple recipes do not adequately address the nuances of, variations in, and complexity of real-life situations. As Master’s students’ progress in their studies, and are engaged in many levels of learning—for example, the levels articulated in Bloom’s taxonomy: understanding, applying, analyzing, evaluating and creating—their behavior and learning are increasingly characterized by the following indicators of the “competent” stage of expert knowledge and skills. They:

●       Engage in deliberate planning

●       Understand the importance of each specific context/situation

●       Use guidelines, not rules, to determine their actions

●       Are emotionally-involved in the outcomes of their actions (a strong sense of Responsibility) (commitment)

●       Use what they see to be the most valuable and “relevant perspectives” for each situation, rather than relying on rules. They may not have the creativity of a proficient expert to develop a new theory or strategy, but they will strategically analyze and evaluate what they have learned to make an educated choice about what they see to be the situationally most appropriate action or plan, from among their knowledge of the “available alternatives.” So, they:

●       Analyze and evaluate what they have learned, and then also make judgements based on their experiences

Doctoral students achieve the stage of the “proficient” expert, including the capacity to develop new theories and new approaches to practice as a result of the further development of several, key skills and qualities of learning, all of which are aided, especially, by the strong, and continual, emphasis at WISR on a transformative approach to action-research in every doctoral course. This includes:

●       a focus on involved understanding and inquiry—emotional engagement in not only the outcome of one’s efforts (“responsibility) but also in the process of inquiry itself (where the learner develops a stronger sense of their own voice, commitments to their purposes and values, an almost ever-present curiosity to learn more, and to improvise (revise and re-formulate) the next steps in both action and inquiry.

●       a creative search to look for patterns, to use holistic analysis in addition to analysis by dissection, or in addition to breaking ideas and information into separate parts. This, in turn, leads to:

●       an awareness of the “bigger picture”—specific concrete situations are evaluated with a conscious consideration of their connections with the larger context.

●       a search for a variety of sources of data and experience—this is the opposite of tunnel vision, and WISR’s approach to action-research requires that students learn how to “sample for diversity”—that is, look for data, for observations, applications and experiences, that reflect a wide range of possibilities.

It is important to note that these sophisticated and complex learning processes must grow out of a solid foundation and knowledge of a variety of theories and practices, and the ability to evaluate those theories and practices. The qualities of the competent expert are very important as a foundation for the development and learning toward being a proficient expert. So, even in WISR’s doctoral program, significant attention is given to further developing solid foundational knowledge of theories and applications in the field of higher education and social change.

So, along with the key skills and qualities of learning noted above, the achievement of the “proficient” level of expert knowledge and skills is indicated by the learner doing the list below.  And, it should be noted that these actions are critical if one is to make creative contributions to one’s field of specialization and major area of expertise! The proficient learner

●       Prioritizes aspects of situations

●       Seeks out, and is exposed to, an increasing variety of situations

●       Critiques, re-evaluates, and often changes goals [not just changing their method to achieve a fixed goal]

●       Is emotionally-involved in the effectiveness of their process of inquiring and understanding (not just involved in the result or outcome)

●       Evaluates past successes and failures, as part of their engaged action-and-inquiry.

●       Uses maxims, or broad principles, and also adapts the maxims to situations (which is the first step in creating new theories and new practices).

To learn more about the Dreyfus model go to: https://www.nateliason.com/blog/become-expert-dreyfus  and

http://www2.psych.utoronto.ca/users/reingold/courses/ai/cache/Socrates.html

And also:  Chapter 5, Cases and Stories of Transformative Action Research. Bilorusky, J.  Routledge Press, 2021.