Research & Projects 

ETSers' group photo at the 2019 NCME award ceremony (AI remixed image)

In my current focus area as a researcher, I work as a PI with subject matter experts on a series projects focusing on large-scale assessment (LSA) innovations and AI applications. Our team utilizes multi-source data (student performance integrated with process data) from LSA, and employ human-centered AI to contextualize student performance and shed light on key aspects of student problem-solving processes, engagement, time management, self-regulation, as well as identification of knowledge gaps [5, 6]. These data insights can support teachers to plan and execute interventions for learning.  Such multi-source data can also utilized to evaluate and improve assessment delivery platforms by analyzing how digital tools are functioning [4]. Furthermore, we explored how to conduct research for practice with teacher codesigns and how to scale up the process [2, 3],  and how to provide open source benchmark datasets (synthesized data with fidelity [1]) for public goods.  

We have conducted workshops and been invited to present these studies to various audiences, including federal and state agencies. Teachers working with us provided positive feedback, such as "Glad to see behavior analysis, an important aspect in attention/executive functioning that we don't often see quantified "; " This (data insight) should be the starting point for any intervention ".  

These projects have been supported by National Center of Education Statistics (NCES, IES), Gates Foundation, and ETS funds. Refer to the link here for more details. 

Below are selected publications:

1. Guo, H., Johnson, M., Saldivia, L., Worthington, M., Lee, J., & Ercikan, K. (2026, submitted). Synthesizing Multi-Source Assessment Data: A Deep Generative Framework Integrated with Psychometric Models for Privacy-Preserving Benchmarks. Invited submission to a special issue of a top journal in the field. 

2. Guo, H., Johnson, M., Saldivia, L., Worthington, M., Lee, J., & Ercikan, K.  (2026, in press). An Agentic framework to Scale up Human-AI collaboration for Personalized Feedback from Large-Scale Assessments. Proceeding of the 2026 Artificial Intelligence in Measurement and Education Conference  (AIME-con). Pittsburgh, PA. 

3. Guo, H., Johnson, M., Saldivia, L., Worthington, M. & Ercikan, K. (2025). Leveraging multi-AI agents to accelerate a teacher co-design. Proceeding of the 2025 Artificial Intelligence in Measurement and Education Conference (AIME-Con). Pittsburgh, PA. 

4. Guo, H., Johnson, M., Saldivia, L. & Worthington, M. (2025). Toward better digital tool designs on Math Assessments.  Invited paper for the special issue on "AI & ML in measurement".  CE Journal of Educational Measurement and Evaluation, 6 (1). 

5. Guo, H., Johnson, M., Saldivia, L., Worthington, M., & Ercikan, K. (2024). Human-Centered AI for Discovering Student Engagement Profiles on Large-Scale Educational Assessments.  Invited paper for the special issue on "Opportunities and Challenges of AI in Educational Assessment" in Journal of Measurement and Evaluation in Education and Psychology (JMEEP).

6. Guo, H., Johnson, M., Ercikan, K., Saldivia, L. & Worthington, M. (2024). Large-scale assessments for learning: A human-centered AI approach to contextualize students’ test performance. Journal of Learning Analytics, 12 (2). 229-245.  2025 Best Paper Award from AERA (NAEP Studies SIG).

As a researcher consulted by testing programs, these projects aim to use advanced psychometric models to address technical challenges in assessment programs in the ever changing testing evironment, which are key to measurement validity, fairness, and comparability. Examples include the investigation of score comparability between test-center mode and remote mode [3], differential item functioning (DIF) studies from theoretical view points [9], and development of the generalization of DIF methods to conduct group comparison (e.g. to help understand how English learners engaged differently from non-English learners [7]).

 The projects also include initiatives on automation and improvement of operational processes, implementations of research-back results into operational practices, as well as finding solutions to common practical issues across programs. Example include psychomtric solutions to small sample issues for continuously-administrated assessments [2, 4], evaluation of new item types and new test development [5, 8], measuring student progress via online learning [1]. 

Below are selected publications: 

1. Guo, H., Zhang, M., Ko, A. J., Li, M., Zhou, B., Lim, J., Pham, P., & Li, C. (2024). Measuring students' programming skills via online practice.  Proceeding of the 8th Educational Data Mining in Computer Science Education (CSEDM) Workshop.

2. Guo, H., Johnson, M., McCaffrey, D., & Gu, L. (2024). Practical Considerations in Item Calibration With Small Samples Under Multistage Test Design: A Case Study. ETS RR-24-03. 

3. Guo, H. (2022). How did students engage with remote educational assessment? A case study.  Educational Measurement: Issues and Practice, 41(3), 58-68.  (Top cited paper among work published from Jan 2022 to Dec 2023 in EMIP.) 

4. Guo, H., Lu, R., Johnson, M. S.,& McCaffrey, D. F. (2022). Alternative methods for item parameter estimation: From CTT to IRT (Research Report No. RR-22-12). ETS.  (Top downloaded paper in the 1st year of publication in ETS RR). 

5. Guo, H., Rios, J., Ling, G., Wang, Z., Gu, L., Liu, O. L., & Yang, L. (2022). How Do Selected-Response Format Variants Influence Test-Taking Effort? An Empirical Study (Research Report No. RR-22-01). 

6. Guo, H. & Ercikan, K. (2021). Comparing Test‐Taking Behaviors of English Language Learners (ELLs) to Non‐ELL Students: Use of Response Time in Measurement Comparability Research. (Research Report No. RR-21-25). ETS. 

7. Guo, H., Ling, & G., Frankel, L. (2020). Using existing data to inform development of new item types. (ETS Research Report RR-20-01). Princeton, NJ : ETS. 

8. Guo, H. & Dorans, N. (2020). Using weighted sum scores to close the gap between DIF practice and theory. Journal of Educational Measurement, 57, 484-510. 

These projects contain methodology develompment and their applications on how to analyze clickstream data (log data, keystroke, process data) to create meaningful measures for understanding test-taking behaviors [5, 7], group differences in terms of  how test takers engaged with assessments and their impact [1, 4], and evaluation of assessment innovations [2, 3]. 

Below are selected publications:

1. Guo, H. & Ercikan. (2022). Differential Rapid-Responding across Language and Cultural Groups. Educational Research and Evaluation, 26(5), 302-327

2. Guo, H., Zhang, M., Deane, P., & Bennett, R. (2020). Using Writing Logs to Help Validate Assessments for Learning. The LAK20 companion proceedings. 

3. Guo, H., Zhang, M., Deane, P., & Bennett, R. (2020). Effects of scenario-based assessment on students' writing processes.  Journal of Educational Data Mining, 12(1), 19-45. 

4. Guo, H., Zhang, M., Deane, P., & Bennett, R. (2019). Writing process differences of subgroups reflected in keystroke logs.  Journal of Educational and Behavioral Statistics, 44(5), 571-596.

5. Guo, H., Deane, P., Rijn, P., Zhang, M., & Bennett, R. (2018). Modeling basic writing processes from keystroke logs. Journal of Educational Measurement, 55, 194-216. (2019 Bradley Hanson Award, NCME).

6. Guo, H., Rios, J., Haberman, S., Liu, O. L., Wang, J.,& Paek, I. (2016). A new procedure for detection of students' rapid guessing responses using response time. Applied Measurement in Education, 29, 173-183. 

As a practitioner in operational testing programs  (psychometrician on SAT, PSAT, etc.), these projects directly help to solve issues and improve practices. Examples include methodology development to address issues caused by scoring method changes [1], investigation of new and existing equating methods [2,3], evaluation of the score stability over time [5] and measurement accuracy and precision [4, 6, 7].

Below are selected publications:

1. Guo, H. (2017). Predicting rights-only score distributions from data collected under formula score instructions. Psychometrika, 82, 1-16. 

2. Guo, H. & Puhan, G. (2014). Section Pre-equating under the Equivalent Groups Design without IRT.  Journal of Educational Measurement,51, 301 - 317. 

3. Guo, H., Oh, H., & Eignor, D. (2013). Situations where it is appropriate to use frequency estimation equipercentile equating. Journal of Educational Measurement, 50, 338-354. 

4. Guo, H., Puhan, G., & Walker, M. (2013).  A criterion to evaluate the individual raw-to-scale equating conversions (ETS Research Report No. RR-13-05). Princeton, NJ: ETS.

5. Guo, H., Liu, J., Curley, E. & Dorans, N. (2012). The Stability of the Score Scales for the SAT Reasoning Test from 2005 to 2010 (ETS Research Report No. RR-12-15). Princeton, NJ: ETS. 

6. Guo, H. & Sinharay, S. (2011). Nonparametric Item Response Curve Estimation with Correction for Measurement Error. Journal of Educational and Behavioral Statistics, 36, 755-778. 

7. Guo, H. (2010). Accumulative equating error after a chain of linear equatings. Psychometrika, 75, 438-453.

Below are foundational and theoretical academic projects during my academic trainings. 

1. Guo, H., Lim, C. & Meerschaert, M. (2009). Local Whittle estimator for anisotropic random fields. Journal of Multivariate Analysis, 100, 993-1028.

2. Guo, H., & Koul, H. (2008). Asymptotic inference for some regression models under heteroscedasticity and long memory design and errors. Annals of Statistics, 36, 456-487. 

3. Guo, H., & Koul, H. (2007). Nonparametric regression with heteroscedastic long memory errors. Journal of Statistical Planning and Inference, 137, 379-404. 

4. Guo, H. & Hu, D. (2004). Multifractal decomposition on certain random recursive sets. Journal of Systems Science and Mathematical Sciences, 24, No.3, 354--361.

5.  Guo, H., & Hu, D. (2002). The Hausdorff dimension and exact Hausdorff measure of random recursive sets with overlapping. International Journal of Mathematics and Mathematical Science, 31, 11--21. 

6. Guo, H., & Deng, A. (2001). Multifractal decomposition of certain recursive sets without the open set condition. Acta Math. Sci. Ser. B. 21, 369--374.

7. Guo, H. (2001). Multifractal decomposition of certain generalized Moran sets under the finite intersection property. Journal of Mathematics, 21, 271--275.