AimsTo educate fellows about how to select and adapt evidence-based assessment tools for community settings
ObjectivesBy the end of this session, participants will:
- Be able to define the major characteristics of a well-validated measure (at least 2 forms of validity and 2 forms of reliability)
- Identify 3 questions that assess the appropriateness of a measure for a particular community setting
- Design a study to develop a new measure, or adapt and/or validate a measure for use in a new setting
- Validity: How well does the test measure what it is intended to measure?
- Reliability: Reproducibility of a measurement
- 09:00-09:15: Why measurement matters
- 09:15-09:45: How to choose evidence-based measurement tools for your project
- 09:45-10:00: How to develop or adapt evidence-based assessment tools
- 10:00-10:20: Example – Development of the Prodromal Questionnaire
- 10:20-11:00: Fellows discuss assignments
- 11:00 -11:10: Break
- 11:10-12:00: RSG
D. E., Gerena, M., Canino, G., Aguilar-Gaxiola, S., Febo, V., Magaña,
C., et al. (2007). Translation and cultural adaptation of a mental
health outcome measure: The BASIS-R (c). Culture, Medicine and
Psychiatry, 31, 25-49.
S. V., Gerena, M., Ranganathan, G., Esch, D., & Idiculla, T.
(2006). Reliability and validity of the BASIS-24 (c) mental health
survey for whites, african-americans, and latinos. The Journal of
Behavioral Health Services & Research, 33(3), 304-323.
- Before class: Choose one or two measures needed for your community project. Are these measures evidence- based? Why or why not? Are these measures appropriate for the population in your project? Why or why not?
- During class: Discuss answers above. Do you need to choose new measures or adapt these measures in any way for your project? Develop plan for choosing or adapting measures.
Summaries of Objectives
- Be able to define the major characteristics of a well-validated measure (at least 2 forms of validity and 2 forms of reliability):
- The two major forms of test validity are internal validity, which includes face validity and construct/content validity, and external validity. External validity includes convergent validity (concurrent validity and predictive validity), as well as discriminant validity.
- Common forms of test reliability include inter-rater reliability, test-retest reliability and internal consistency.
- Identify 3 questions that assess the appropriateness of a measure for a particular community setting:
- Is your population similar to the normative sample used in the development of the measure? Assess for: Age? Ethnicity? Gender? Educational attainment? Socioeconomic status?
- If the measure was translated into a second language, was it validated in the new language?
- Was the setting of the validation of the measure similar to your setting (e.g. school, community clinic, etc.)?
- Is the assessment modality appropriate (e.g. self-report, interview, focus group, computerized assessment)?
- Does this measure provide outcome data that is relevant to your population? What outcomes are important to them?
- Is it feasible to use this measure in your study or program?
- Design a study to develop a new measure, or adapt and/or validate a measure for use in a new setting.
- Common ways of adapting a measure are to shorten it, translate it, change the mode of administration, or use it with a new population or in a new setting.
Assessment Data and Learner Feedback
- Learners discuss the objectives with the instructors and with each other in terms of choosing or adapting a measurement tool for their particular project. This is completed in class, with feedback from the group.
- At UCSF, the example of developing and adapting a measurement tool was the Prodromal Questionnaire, a self-report measure for assessing a psychosis risk syndrome. In other settings, it may be appropriate to describe the development of a measure with which the instructors are familiar or which is of particular relevance to the learners.
- Learners should be encouraged to bring the results of these discussions back to their mentors in designing their projects.
- Having the group provide feedback is a good way to utilize expertise within the group of both community and academic fellows, as well as to encourage more participatory discussion.