Details for EDD 693

The Comprehensive Assessment of Student Learning and Progress

At WISR, learners are supported and expected to develop strong skills in self-assessment as a critical part of pursuing the doctoral program learning outcomes. When the student has completed all of their pre-dissertation coursework, the student shall meet with an Academic Review Committee of three WISR faculty. (And if they wish, they may also consult informally, for critical feedback and support, with a couple of fellow students or WISR alumni.) The purpose of the meeting will be to assess, and to help the student to self-assess, his or her progress in each of WISR’s doctoral program learning outcomes.

To begin this process of evaluation, the student is expected to spend a few hours over a week or two period of time, reflecting on the “bigger picture” and also the highlights of what they have so far learned and accomplished. As an outgrowth of this endeavor, the student will prepare their own “take-home essay exam”—articulating a half dozen or so key questions to which she or he will then respond. The student will share a draft of these questions with the three WISR faculty members of the review committee, in order to get feedback on how to construct questions that will be both meaningful to the learner, relevant to the degree program learning goals and outcomes, and strongly thought-provoking for exploring the landscape of the student’s studies thus far. The student is expected to spend approximately four hours responding to these questions. The goal is not just to give the student an “exam,” but also to draw out their knowledge in the field, in a meaningful and well-organized way. In addition, the student will compile, from previously completed courses, an annotated bibliography of the most important and significant readings they have pursued. The student will submit the annotated bibliography and the self-designed take-home exam essay to this committee, which will review these documents prior to the assessment meeting.

The Role of the Review Board

First, three WISR faculty members review the doctoral student’s completed projects, after all of the pre-dissertation requirements have been met, to determine if they are prepared to undertake the rigorous study required for a doctoral dissertation. The student also engages in a thoroughgoing review, critical reflection, and written analysis of what they have learned thus far—on how the WISR learning process has helped them to learn in areas of the doctoral program degree learning outcomes. They discuss their reflections and written analyses with three WISR faculty members—assessing their breadth and depth of knowledge in the area(s) of primary interest, and in the interdisciplinary field of higher education and social change, as well as their skills in action-oriented inquiry and knowledge-building, in preparation for undertaking the dissertation. The three WISR faculty members, all of whom must have accredited doctoral degrees, conduct a formal evaluation of the student's written comprehensive exam (the written analyses submitted by the student, noted above), and follow this with an oral comprehensive exam. The rubrics used in evaluating the student for can be found at: https://drive.google.com/open?id=1RB-14pwclvWi-kpTYjAnJxp2a5hQhTsg

The three faculty serving on the Review Board and who are conducting the oral and written exams use a set of rubrics, and evaluation forms, to evaluate the student in each of the following stages of this process:

1) the review of student completed coursework, and an oral qualifying exam [faculty form for evaluating qualifying exam ],

2) the student's written self-assessment and articulation of evidence of achievement of degree program learning outcomes as part of the comprehensive written exam [faculty form for evaluating written comprehensive exam ],

3) the faculty's assessment of student progress toward program learning outcomes based on the comprehensive oral exam [faculty form for evaluating oral comprehensive exam ], and

4) the submission, evaluation and discussion, and finally approval (after necessary revisions) of the student's dissertation proposal [faculty form for evaluating dissertation proposal ].

Further, the three WISR faculty, all of whom must have an accredited doctoral degree, along with the outside expert in the student's area of proposed study, evaluate the student’s dissertation proposal to determine if the topic design and procedures meet the Institute’s academic standards for quality action-inquiry and promise in contributing to others and to the student’s future life plans. They also refer the proposal to WISR's IRB for either an expedited review or a review by the full IRB.

Then, the three faculty members' then sign and fill out the form, "Evaluation of EdD Student Performance—Written and Oral Comprehensive Exams, and Dissertation Proposal—EdD 693".

The rubric used in evaluating the student's dissertation proposal is included in the document at: https://drive.google.com/open?id=1RB-14pwclvWi-kpTYjAnJxp2a5hQhTsg

It is recommended that each student identify two or more current or former WISR students to serve as a peer support group and added source of feedback during the dissertation process.

Details of the Process, including what happens if a student fails an exam or review . . .

1. The Qualifying Exam. First, three WISR faculty members review the doctoral student’s completed projects and coursework, after all of the courses required for the degree program have been completed (except for the dissertation and the exam/dissertation readiness “course”).The purpose of the review is to determine if the student has either completely achieved degree program outcomes, or sufficiently to be able to finish achieving those outcomes while doing their dissertation. In this way the faculty are evaluating if the student is prepared to undertake the rigorous study required for a doctoral dissertation, is ready to focus their attention on the dissertation, and is capable of achieving added progress toward degree program outcomes required and identified by faculty.

Based on their review of the student’s completed coursework, the faculty conduct an oral exam (for more details, go to: What to Expect during the WISR EdD Qualifying Exam) to see if the student is ready to proceed to the Comprehensive Exam Phase, outlined in the two-unit course, EDD 693: Comprehensive Assessment of Student Learning and Plans for Dissertation and Beyond. To evaluate the student’s coursework and the student’s oral exam, they use the Rubrics for the Qualifying Exam.

If the student fails the Qualifying Exam, faculty will prepare a list of steps for further study and writing that the student needs to take, which must be completed within two months. If the student fails a second time, they will be placed on a tuition-free leave of absence, during which time they will still have faculty support. The student may request a third exam at any time, and if they pass the exam, they will be re-enrolled, without having to pay a re-enrollment fee, and will proceed to exam step #2, the Written Comprehensive Exam.

2. The Written Comprehensive Exam. The student also engages in a thoroughgoing review, critical reflection, and written analysis of what they have learned thus far—on how the WISR learning process has helped them to learn in areas of the doctoral program degree learning outcomes. The specific directions to guide the student in the written exam are as follows:

The student will:

Write a comprehensive self-assessment paper that evaluates, organizes and synthesizes their learning thus far during their doctoral studies. In that paper, the student will:

• Articulate and write a critical and well-informed statement about their field(s) of specialization that includes details and nuances beyond broad generalizations.

• Articulate and explore several insights and questions about this emerging, interdisciplinary field of “Higher Education and Social Change” and about their area(s) of specialization in particular that their review committee considers to be at the "proficient" level of expert knowledge and to be promising of leading toward new knowledge and/or practices;

• Demonstrate the depth and breadth of their perspectives on what they’ve learned, and how they plan to build on this knowledge as they move forward toward their goals; and

• Articulate and discuss the evidence of the extent to which they have addressed each degree program learning outcome.

Steps in the Process of Submitting the Draft of the Written Comprehensive Self-Assessment Prior to Oral Exam:

1. Student writes first draft of comprehensive exam.

2. If Chair thinks the draft is "ready" it is sent to the other committee members who review it, or alternatively the chair makes suggestions for improving the draft until they deem it ready for review by other committee members.

3. If all faculty committee members agree the written comprehensive self-assessment is ready to defend, the Oral Comprehensive Exam Meeting is scheduled, or alternatively committee members make suggestions for improving the draft until they deem it ready for review and defense at the Oral Exam meeting, which is then scheduled.

3. Oral Comprehensive Exam. The student then discusses their reflections and written analyses with three WISR faculty members—assessing their breadth and depth of knowledge in the area(s) of primary interest, and in the interdisciplinary field of Higher Education and Social Change, as well as their skills in action-oriented inquiry and knowledge-building, in preparation for undertaking the dissertation.

Evaluation for Steps #2 and #3. The three WISR faculty members, all of whom must have accredited doctoral degrees, conduct a formal evaluation of the student's written comprehensive exam (the written analyses submitted by the student, noted above), and follow this with an oral comprehensive exam. The Rubrics used in evaluating the student for can be found below.

The oral exam will proceed as follows:

Using the comprehensive self-assessment as evidence and as a starting point for discussion in the oral exam, the student will show the review committee that they have demonstrated proficiency and promise of creative work in the field. Specifically, the student will:

• identify and discuss convincingly those degree program learning outcomes which they already have met, and

• identify and a realistic plan for completing the remaining degree program learning outcomes during the dissertation process.

If the student’s review committee determines that the student's progress and plans are sufficient, the student will be approved to submit their dissertation proposal. Otherwise, the committee will articulate for the student what further learning must be demonstrated before the student begins work on the dissertation. (See section below on “What happens when the student fails an exam”)

When the student passes the Oral Comprehensive Exam, they will proceed to work on and then submit the Dissertation Proposal for review by the same three faculty.

4. Dissertation Proposal

The faculty will evaluate the dissertation proposal with the following criteria in mind. If the proposal does not meet all criteria, faculty will work to help the student to make the necessary improvements within a two month period of time.

The student will:

• develop a coherent, well thought out plan for their dissertation to these four members (the three WISR faculty and the outside expert, who must be approved by the WISR faculty) of what will become their Graduation Review Board, and

• will present a plan that meets standards for original, ethically-informed action-oriented inquiry, including

• an appropriately thorough and targeted literature review,

• a well-designed plan for collecting original data, and

• well-formulated questions that reflect the student’s interests and the potential to contribute to new knowledge and/or practices in the student’s proposed area of study.

The student articulates a detailed plan (about 10 pages or so) outlining their research questions, proposed research methods, and the ethical considerations that they are taking into account. They submit a form that outlines these matters to their Review Board, and to the Chair or Vice-Chair of WISR's Institutional Research Board. This form serves as a highlighting and summary of the research questions and methods articulated in greater detail in the dissertation proposal, along with a discussion of the ethical considerations and precautions involved with the proposed research. The form is available at: https://drive.google.com/file/d/15WNHEYYS3JQ9fhOYCqMRu-NtJsN7j3lz/view?usp=sharing

Steps in the Process of Submitting the Draft of the Dissertation Proposal Prior to Review Meeting:

1. Student writes first draft of dissertation proposal.

2. If Chair thinks the draft is "ready" it is sent to the other committee members who review it, or alternatively the chair makes suggestions for improving the draft until they deem it ready for review by other committee members.

3. If all faculty committee members agree the written dissertation proposal is ready to discuss and defend, the Dissertation Proposal Meeting is scheduled, or alternatively committee members make suggestions for improving the draft until they deem it ready for review and discussion at the Dissertation Proposal Review meeting, which is then scheduled.

Use of Rubrics

The three faculty serving on the Review Board and who are conducting the oral and written exams use a set of rubrics (see below) to evaluate the student in each of the following stages of this process [the four evaluation forms are found here ]:

1) qualifying exam and the review of the student’s previous coursework, and an oral exam of the student regarding what the learned and accomplished in that coursework;

2) the student's written self-assessment which includes articulation of evidence and critical analysis of their learning in each Program Outcome area and their plans for future learning,

3) an oral exam, using the student’s written exam as a point of departure for dialogue, where faculty assess student achievement of program learning outcomes based on this comprehensive oral exam, and

4) faculty evaluation of the dissertation proposal.

What happens when the student fails an exam

If the student fails to pass at any of these four levels, faculty will provide the student with a statement of the areas in which they must still demonstrate sufficient competence and knowledge in any content area or competency area related to program outcomes. One or two of the faculty will be designated to mentor and support the student in doing the necessary study and learning to pass the exam. The student may take the exam again as soon as they and the faculty feel they are ready. Similarly, students who fail at any stage, will receive support from faculty to help them progress to the next stage—aiming for no more than two months and in any case no longer than 6 months (except for the qualifying exam which must be within two months) before progressing to the next stage. It should be noted that because of the rigorous requirements of each WISR course, the intensity and extent of personal mentoring of each student by faculty, the extensive body of writing required in each course, and the oral exam at the end of each course, it is very unlikely that a student will fail at any level. However, if the student fails at one or more exams, they should be able to do the necessary added study and learning, with faculty help, in no more than two months to proceed to the next phase or exam, although there is no time limit.

If the student fails any exam a second time, they will be placed on a tuition-free leave of absence, during which time they will still have faculty support. The student may request a third exam at any time, and if they pass the exam, they will be re-enrolled, without having to pay a re-enrollment fee, and will proceed to the next phase of exams.

As always, as per WISR’s policies, students have the right to appeal the decision of their Review Board.

Faculty Qualifications

The three WISR faculty, all of whom must have an accredited doctoral degree, evaluate the student’s dissertation proposal to determine if the topic design and procedures meet the Institute’s academic standards for quality action-inquiry and promise in contributing to others and to the student’s future life plans. They also refer the proposal to WISR's IRB for either an expedited review or a review by the full IRB.

Then, the three faculty members' then sign and fill out the forms after each of the four steps: "Forms for Faculty Evaluation of Student Learning_EDD_693."

The outside expert in the area of the student's dissertation topic joins the three faculty on the Review Board, and together, they make suggestions for revisions and improvements in the proposal. The rubric used in evaluating the student's dissertation proposal is included in the document at: https://drive.google.com/open?id=1RB-14pwclvWi-kpTYjAnJxp2a5hQhTsg (or see below).

It is recommended that each student identify two or more current or former WISR students to serve as a peer support group and added source of feedback during the dissertation process.

Building Bridges to the Student's Future, and the Dissertation Proposal

As part of this process, the student is encouraged to review and discuss with members of their Academic Review Committee (Graduation Review Board) the initial autobiographical statement written during EDD 601 (the Introductory Course) and the initial study plan, as well as changes that have transpired since enrollment. The purpose of this review and discussion will be to aid the student in becoming more mindful and intentional in designing a dissertation that will build a bridge (and/or lay the foundation, or create new pathways) for the student as they proceed to the next significant things they wishes to accomplish, professionally and in their life. As part of this process, the student will also be encouraged to revisit their previously submitted inventory of their strengths and limitations, interests and values, as well as to re-evaluate the previously submitted assessment of needs, challenges and opportunities in the profession and the world in which they wish to make a difference. The Academic Committee will discuss with the student how extensively the student has explored various career options, and their awareness of networking and resources. In their role as mentors and colleagues, the Committee may advise the student to seek further guidance and assistance from members of the Committee, and other faculty, in doing additional exploration and networking, but this will not be a formal requirement.

After discussing these issues with the student, the student will spend some time drafting, and then submitting, a dissertation proposal. The Academic Committee will discuss this proposal with the student and make suggestions. In support of WISR’s mission to nurture critical inquiry, curiosity and thoughtful improvisation, faculty will remind the student that such proposals often need to be revised during the dissertation process, as the student gains more information, knowledge and experience during their dissertation inquiry. As was the case with the student’s initial educational plan, it is expected that the student’s dissertation plan will most likely evolve and become more sophisticated during the progress of the student’s dissertation research, and to this end, students and faculty, alike, are to be mindful in being on the look for changes that will improve the value of the dissertation to the student, and to others. The thesis proposal should contain at least the following three sections: 1) the main questions/topics the student wishes to explore; 2) a statement as to why this topic is important to her/him, and possibly to others as well, and 3) the main types of original data to be collected. See section on Dissertation Proposal in EDD 699