The purpose of the Comprehensive Examination is to demonstrate students’ abilities to engage in the depth, breadth, and creativity of thought necessary for social scientists in psychology and social intervention.
Goals:
To have students demonstrate their ability to:
Grasp and apply the theoretical and empirical knowledge base within psychology and social intervention,
Integrate the empirical literature in a specified substantive area, and
Evaluate, critique, and improve the methodologies represented in published research.
Process, Procedures, & Format:
The comprehensive examination consists of two parts: an empirical research critique and a specialty area paper. The processes, procedures and format for each are outlined below.
Empirical Research Critique
The purpose of the Empirical Research Critique is for PSI faculty to certify that Ph.D. candidates have developed the ability to a) critically evaluate specific aspects of the conceptualization and methodology of selected articles and b) make recommendations for redesign and improvement. It is expected that Ph.D. students will develop both knowledge of content and advanced quantitative and qualitative methods in order to critically assess research during the Empirical Research Critique.
Students are expected to carefully review the guidelines for the Empirical Research Critique upon starting the Ph.D. program and to start learning the substantial content and methods required for writing the Critique. Students can do so in multiple ways, including through coursework, working with their advisors in research and journal peer reviews, participating in research projects, and discussing research with faculty members.
At the beginning of the students’ 3rd year in the program, the PSI Program Director and Comprehensive Exam Committee will make 4 articles available to the third year students. PSI faculty select articles that represent important methodological and research questions from key scientific domains of psychology and social intervention literature. At least one of these articles will use qualitative or mixed-methods, and the rest will be quantitative studies. The chair of the Comprehensive Exam Committee will also meet with the students to orient them to the process and answer any questions.
Students have the fall term to review these articles, alongside other methodological materials they have been exposed to thus far in their training. During this fall semester preparation period, students taking the exam are encouraged to work together and confer with one another. They can also ask questions of the faculty, but these questions cannot directly identify the assigned articles.
The domains are as follows:
1. Settings and Outcomes
2. Evaluations of Social Intervention
We expect students to demonstrate competence in understanding and critiquing the (1) conceptualization, (2) logical connection between background, objectives, hypotheses, methods, results, and interpretation, and (3) research methodology of studies concerning developmental causal processes and phenomena at both the individual and the contextual level of analysis. Students are expected to address issues such as the lens of longitudinal change, cross-ecological-level dynamics and interactions, validity, generalizability, and methodological issues inherent to understanding development in settings and evaluating social interventions. Students are also expected to learn about the rationale and practical application of different methodological approaches to study settings and outcomes and evaluations of social intervention, such as:
Measurement of constructs, including classical test theory, exploratory (EFA) and confirmatory factor analysis (CFA), and item response theory (IRT)
Complex multivariate analysis, including regression analysis, structural equation modeling, cross-lagged panel analysis, multilevel modeling, and fixed-effects models
Experimental and quasi-experimental approaches for impact evaluation, including randomized controlled trials, difference-in-differences, propensity score matching, instrumental variables, and regression discontinuity.
Qualitative and mixed-methods approaches, including thematic analysis, content analysis, and other approaches to assess data collected from interviews, focus groups, and case studies, among others
To prepare for the exam students are expected to:
(a) draw on principles and techniques learned in core PSI courses including Understanding and Measuring Social Contexts of Development, Development and Prevention Science, Research Design and Methods in the Behavioral Sciences, Theories of Change in Applied Psychology and all quantitative and qualitative methods courses taken to date;
(b) draw on experience gained on research teams;
(c) engage in one (or ideally more) journal article peer review(s) with their mentor during their first 2 ½ years
(d) engage in additional review and study to prepare for the specific methodologies utilized in the pool of 4 articles provided each year (typically this review is done in collaboration as a cohort).
Exam Logistics:
At the beginning of the second semester of the students’ 3rd year, the exam will be administered as a closed book, monitored exam over a 3.5-hour time slot.
(a) A common time slot will be chosen by the students taking the exam, in consultation with the comprehensive exam committee to determine availability. Students should email the committee with suggested dates and times for the exam at least 2 months in advance.
(b) Directly before the exam commences, one article will be selected at random for the exam.
(c) Students are NOT permitted to confer with ANYONE or refer to notes/materials other than the instructions and a blank copy of the randomly selected article during the actual exam. Doing so is a breach of the honor code and will result in a non-passing grade.
(d) Students will type the Critique in person using a laptop/computer with no access to the internet. All students taking the exam will be in a room together, but each must work independently on the Critique.
Evaluation of Empirical Research Critique:
The PSI Faculty Comprehensive Exam Committee will take primary responsibility for grading the exam and making recommendations to the full faculty.
(a) Final grading decisions will be determined at a regularly scheduled meeting of the PSI faculty.
(b) The grading scheme of the exam is: Pass, Revise and Resubmit, or Retake.
i. If a Pass is received, no additional work is required of the student, except for meeting with the comps committee to discuss the article and lessons learned (see next bullet).
ii. If a Revise and Resubmit is received, the students will be provided with a Revise and Resubmit letter detailing the areas that require revision. Students will have at least two weeks to revise and resubmit the exam. The revision will take place on the students’ own time. Students are not permitted to confer with anyone regarding the revise and resubmit, but can consult other materials/notes.
iii. If a Retake is received (either initially or after a Revise and Resubmit), the student will be asked to redo the empirical research critique the following year with the next year’s cohort. Students in this position will still be able to prepare and defend their dissertation proposal, but they may be ineligible for certain dissertation grants that require proof of candidacy.
The Empirical Research Critique process culminates in a meeting between all students and the faculty committee to review the exam. At this meeting, the student will have the opportunity to ask questions and clarify any content covered and to get more nuanced feedback about their work.
Specialty Area Paper
The objective of Specialty Area Paper is to produce a critical and integrative review of the theoretical and empirical knowledge base regarding research question(s) that are relevant to psychology and social intervention. For all written products, the expectation is that you establish with your primary mentor a schedule of multiple drafts, feedback, and revisions that allows for multiple iterations and incorporates the process and timeline for soliciting and integrating second reader(s) review(s).
Final Product and General Guidelines
1. There are many types of questions that a critical and integrative review could potentially address. Many Specialty Area Papers involve the integration of multiple theories/areas of scholarship to better explain, measure, or intervene in the topic area that a student chooses to focus on. Alternatively, the Paper may seek to apply existing theory to an area of scholarship that has not previously considered it; or argue against using a particular theory to address the social issue in question.
2. The resulting manuscript should range from 20 to 35 double-spaced typewritten pages, excluding tables and references. The final, approved Paper should approach publishable quality. Students are encouraged to publish their specialty papers as peer-reviewed articles or book chapters. If the Paper is intended for a journal with a lower page limitation, students may be required to supplement the Paper submission with appendices that include additional information. For some students, the Paper may serve as the first step toward their dissertation or one of the three articles that they weave together for their dissertation. However, this is not a requirement.
3. The process to produce this manuscript has both a Proposal Phase and a Paper Phase, much like the second-year paper and dissertation process. Students should plan for multiple iterations of both the Proposal and Paper, in consultation with their primary mentor and second reader (see below for more on second reader processes), prior to each official deadline.
Proposal Phase Procedures
Throughout AY2, structured support for writing the Specialty Area Paper will be provided through the 2nd year project seminar (in conjunction with support for writing 2nd Year Paper). During the seminar, students will be introduced to different types of non-empirical papers in the field, strategies and approaches to writing such papers, and see examples of previous PSI students’ completed Papers. Students will also be supported by the course instructor and class peers to develop ideas about their Paper Proposal. Additionally, each year, students will have the opportunity to present their Paper during a PSI colloquium session for feedback.
1. During Fall-Spring of AY2, students are expected to develop a proposal idea for their Paper, with the support of a) their primary mentor, and b) the course instructor and students in 2nd year Project Seminar.
2. During Spring AY2, students are expected to identify a second reader who could add substantively to the Paper. Depending on the student's primary mentor and topic area, the second reader may be more or less involved in the proposal development stage. Students should work with their primary mentor to determine when and how the second reader will be involved in proposal development. At the very least, the second reader should be identified by the time students submit their final proposal; and must indicate that they have accepted the role of second reader via their signature on the Specialty Area Paper proposal form.
3. During Spring-Summer AY2, students are expected to write a Paper Proposal, with the support of their primary mentor, that clearly outlines the content and structure of the planned literature review and analysis, and answers the questions listed here.
4. Before the end of the first week of classes of Fall AY3, students are expected to submit their written Paper Proposal along with the Specialty Area Proposal Approval Form, signed by both primary mentor and second reader, via email to the PSI Program Director, cc-ing the program administrator, Aliyah Ashley (aa10578@nyu.edu).
Paper Phase Procedures
1. During AY3, using the timeline developed during the Proposal phase, students should continue to research and write the Paper.
2. The Paper is due the Monday after Spring Break of AY3 to the primary mentor and secondary reader. In most cases, students will have already received extensive feedback from their mentor on the Paper's structure, arguments, and specific sections. It is important that, at this stage, students submit a polished paper, so that they receive a Passing grade, and in case of revisions, the revisions required will be doable in the timeframe provided for them in the next stage.
3. In the 2-4 week period following submission of the Paper, the two readers will evaluate the Paper using this evaluation rubric.
4. If the Paper is not approved in the first round, and the student is asked to Revise and Resubmit, the revision is due to their two readers by June 15 following AY3. Based on the quality of the draft, this may also be the student's final product.
5. Should the Paper not pass at this stage, and need further revisions, the student will have until the first day of classes AY4 to complete suggested revisions and submit a final approved product to the program. At this time, students are expected to submit the final, approved manuscript to the Program Director and Program Administrator, Aliyah Ashley (aa10578@nyu.edu), with the Specialty Area Cover Sheet signed by their primary mentor, second reader, and Program Director. Should the final product, following this deadline, still be unacceptable, the student will not be “in good standing” as they enter their fourth year. Students who are not in good standing because they do not yet have an approved specialty paper during the first day of classes AY4 must present their progress in a monthly meeting that includes the program director, their advisor, and specialty paper readers until the Paper is completed.
Appendix
The following are examples of previous student’s published work that has stemmed from the Specialty Area Paper. See here for more examples of student papers.
The relationship between maternal education and children's academic outcomes: A theoretical framework (Journal of Marriage and Family)
Parent involvement, emotional support, and behavior problems: An ecological approach (The Elementary School Journal)
The maternal ecology of breastfeeding: A life-course developmental perspective (Human Development)
Early violence exposure and self-regulatory development: A bioecological systems perspective (Human Development)