System Development & Evaluation

While working at University of Virginia (2014-2020) as an instructional designer, I

• Assisted faculty and course leadership in writing and refining learning objectives that are concise and measurable

• Assisted faculty in developing instructional plans based on approved learning objectives, curriculum assignment requirements, and instructional approach.

• Collaborated with faculty in choosing instructional methods, including use of current and emerging technologies, and in developing strategies and resources that focus on active and student-centered learning

• Worked cooperatively with faculty to establish priorities, identify content, provide informal feedback from observations, and evaluate and revise instructional materials and resources.

While working at Baylor College of Medicine as an Assistant Director of Evaluation and Assessment Innovation, I

• Worked closely with deans, directors, faculty, and staff in developing and implementing new learning assessment processes appropriate for the evolving curriculum and the ongoing implementation and improvement of program evaluation.

I have collaborated with subject matters experts to create/refine instructional materials and sessions on various topics, such as:

• Pharmacology

• Biochemistry and physiology

• High value medical care and lab medicine

• Radiographic physics

• Urology

• cardiovascular disease

• Fall prevention (Geriatric Medicine)

• Preservice mathematics teaching

• EFL teaching

Development of technology-assisted learning activities

I collaborated with instructors to create a series of technology-assisted learning activities fully integrated with their teaching. Here are some examples

1. Grant-awarded Innovations in Medical Education:

• “Development and Evaluation of Online Geriatrics Curriculum on Fall and Fall Prevention for the 3rd and 4th Year Medical Students at UVa,” Academy of Distinguished Educators at University of Virginia School of Medicine, $12,000, 2015-2016.

• “Develop and Integrate a Longitudinal Curriculum focused on the Core Principles of EPA3,” Medical Education Fellowship Awards Committee at University of Virginia, $28,025, 2019.

2. “Teaching Mathematics in Secondary Schools: Algebra” (TDP 4580)

I cooperated with fellow students and the instructor, Prof. Oscar Chavez, to design and develop an online learning environment, named “Problem Based Teaching in Mathematics: Levels of Cognitive Demand Module”. This project is guided by Dr. David Jonassen

• The website was developed for the “Teaching Mathematics in Secondary Schools: Algebra” (TDP 4580) course, taught by Prof. Oscar Chavez at the University of Missouri, targeting preservice mathematics teachers.

• The problem-based learning environment was developed based on cased-based learning and cognitive flexibility theories.

• Website link: http://gordongraber.com/web_sites/Math%20PBLE/index.html

3. Radiographic Physics (Radiologic Sci 3160)

I collaborated with Dr. Allen to introduce a series of concept mapping activities into her teaching. The entire course consisted of six modules, and concept mapping activities were incorporated into five of these modules. The purpose of concept mapping activities included

• to improve the students' mastery of concepts

• to cultivate their problem-solving abilities

• to enhance students' collaborative learning skills

• to scaffold students' self-assessment abilities through their assessing and improving own maps

• to help students learn how to learn from their reflection of own learning experiences

Click here for a list of instructional design products from this project.

Creation of Instructional Materials and Packages

I assisted in preparing instructional materials for the following projects

1. Intel® Learn Program, Dec 2004-Mar 2005, funded by Intel company

• Participated in localization of instructional materials and translation of the evaluation report, directed by Prof. Qiong Wang. This project aimed at developing students’ 21st century skills that they’ll need to succeed in the knowledge economy.

2. Using Interactive White Boards in Education, Sep-Oct 2005

• Participated in composition of teachers’ manuals to support white board’s application in education, directed by Prof. Qiong Wang.

3. “Pandeng English” Project in BNU, Key Laboratory of Cognitive Science and Learning of the Ministry of Education of China, Nov-Dec2004, directed by Prof. Qi Dong

• Cooperated in developing English learning materials for primary school students, which provide authentic inputs through multimedia means. This project is especially beneficial for students of rural areas where there is a lack of English teachers. Part of the materials are used by teachers who do not specialize in teaching English. They implement the instructional materials in their classes, focusing on classroom management. The rest of the materials are used by the students after the class.

Localization of Open Source software for learning and teaching

I also participated in the localization of Open Source software, Jun-Jul 2006

• Translated and localized open source software, directed by Prof. Guodong Zhao.

Throughout my career, I have engaged in the planning, collection, and analysis of curriculum and professional learning program evaluation data and composition of evaluation reports.

In this paper that I led, we summarized the various program evaluation methods employed to assess the outcomes of faculty development initiatives.

1. Research Assistant in the Information Experience Lab

Supervisor: Ms. Neeley Current, University of Missouri, Aug 2009-Aug 2011:

Client Projects of Usability Evaluation that I led

Responsibilities: client communication, project management, protocol design, coordination of team member’s work, data collection and analysis, and composition of final reports.

• Usability study of Cerner’s Social Business Collaboration Platform. Dec 2010-Feb 2011

  • Conducted individual think-aloud interviews with users to investigate usability of the uCern Platform, including its new features and existing interface.

• Usability study of Shelter Insurance Website Prototypes, Mar- Sep 2010

  • Conducted individual think-aloud interviews with target users (Phase1) and administered an online survey of existing customers (Phase2) to compare Website prototypes.

Client Projects of Usability Evaluation in which I participated

• Usability study of the Missouri Digital Heritage Initiative (MDHI) Website, led by Prof. Thomas Kochtanek. Jan-Jul 2010

  • Part 1: Interviewed four groups of target users. I was responsible for interviewing genealogists and coding the collected data.

  • Part 2: Tracked website usage through Weblog Expert tool. I composed a report summarizing findings from the collected website usage data.

• Usability evaluation of Federal Reserve Bank websites, Oct 2009-Jan 2010

  • Part 1: Conducted focus group usability evaluation of the Geographic Federal Reserve Economic Data (GeoFRED) Website, a geographic economic data system maintained by the Federal Reserve Bank of St. Louis. Cooperated in protocol development, data collection, and data analysis.

  • Part 2: Conducted heuristic evaluation of the Federal Reserve Bank website (Economic Data - FRED). Participated in website heuristic evaluation, data consolidation, final report composition, and made a presentation to the client.

  • Usability study of National Freedom of Information Coalition website (user interviews)

  • Usability study of Websites of the University of Missouri, including prototypes of Financial Aid, International Center, Undergraduate Admissions websites (user interviews)

  • Usability study of Saint Louis Community College Library Website (user interviews)

  • Usability study of Prototype of the Health Literacy Missouri Resource Inventory (HLMRI) (user interviews and information horizons)

  • Usability study of Platforms of Cerner HealthCare system, including Personal Health Platform which provides eVisit services and Clinical Exchange Platform which focuses on the secure exchange of critical patient summaries within the clinician’s workflow (user interviews)

  • Usability study of University of Missouri Health System websites, including MU Health Intranet and Quality of Care websites (heuristic evaluation)

  • Information Experience Lab website redesign (card sorting)

2. Research Assistant in an online boating safety courses research project

Funded by The National Association of State Boating Law Administrators, led by Prof. Joi Moore, University of Missouri, Aug 2008-Aug 2009

• Cooperated in heuristic evaluation of eight online boating safety courses

• Usability testing at a Minnesota Boat Show. Attendants of the boat show were recruited as participants. Responsibilities: data collection and analysis

• Usability testing in the Information Experience Lab. Responsibilities: Recruitment of 60 participants, coordinating and conducting data collection with fellow students, and data analysis.

• Final project report is available at this link

3. Service-Learning Usability Evaluation Projects

• Usability evaluation of “GMP e-Update 2008,” an online training course developed by LearningPlus Inc. for Wyeth, a global pharmaceutical company. Collaborated in evaluation plan design, conducting online surveys and phone interviews, and composing final client report. Spring 2009

• Conducted a heuristic evaluation of an online BlackBoard-based course: “Management of Information Agencies” (ISLT 7315) and composed a report for the instructors. Fall 2008

Experiences with Major Educational Software & Technology Tools

• Quantitative research: SPSS since 2001

• Qualitative research: Taguette

• Systematic review: Covidence, Rayyan

• Citation management tool (Zotero)

• Survey: Google forms, Qualtrics; Used previously: QuestionPro, PsychData (provided faculty with training), and Survey Monkey

• Usability evaluation: Morae recorder, observer, and manager (2008-2012)

• Concept mapping: CmapTools, LucidChart, and Miro

• Learning management system: Articulate; Provided training on Desire2Learn (2012-14); Taught in Collab (2017-20) Moodle (2013, 2017), Canvas (2014), Sakai and Blackboard (2011-2022)

• Video creation (2012-14): Powtoon, Camtasia, Tegrity, and Jing,

• Provided training and processed individual help requests (2012-2014) on

  • Classroom polling technology: Clicker and TurningPoint

  • Podcasting and video recording: iTunes U, Youtube, VoiceThread, PowerPoint Podcasting, SMART recorder, and Screencast-o-matic

  • Respondus & Turnitin

  • Web 2.0 (e.g. blog, wiki, and Google driver/doc)

  • Microsoft Word, Excel, PowerPoint, and Outlook

  • SMART board and SMART podium

  • Blackboard Collaborate Elluminate.

Web development:

• Course taught 2012: Introduction to Web Development IS&LT 4360 & 7360, University of Missouri (covering basic website development process and HTML coding);

• Courses taken: Introduction to and intermediate Web development (covering HTML coding and CSS)

• Provided training and assistance to support faculty to use Google Sites