University of Oregon, Fall 2000
Psychology 610: PSYCHOLOGICAL MEASUREMENT
Straub 143 (Taylor Room), 12:30 - 13:50 Tuesdays and Thursdays
Instructor: Dr. Gerard Saucier
Offices: 312 Straub
Internet: gsaucier@oregon.uoregon.edu
Phone: 346-4927
Office Hours: Tuesdays 3-4 pm, Thursdays 2-3 pm, and flexibly by appointment
Readings: Mostly from J. C. Nunnally and I. H. Bernstein's (1994) Psychometric theory (New York: McGraw-Hill). Purchasing this classic compendium (from the bookstore) is recommended. Originals of all readings will be made available by the instructor for personal use of class enrollees. The reading list may be adjusted based on the particular interest-area composition of the group enrolled.
The purpose of this graduate course is to give students a useful introduction to basic issues in psychological measurement and assessment. Among the concepts emphasized are reliability, validity, response bias, factor structure, content comprehensiveness, cross-cultural generalizability, and key components of item response theory. Measurement issues will be explored broadly, and then in their applications to particular models and measures, including those in which the student has special experience or interest. Indeed, the course will employ measures with which the students are familiar as prime examples, and is designed especially for (present or future) researchers who will be faced with the task of creating a measure of some construct, and evaluating an existing measure with respect to its measurement properties.
Requirements of the course
1. Within the first two weeks of the term, you are to "bring in" (i.e., attach as a file in Excel or preferably SPSS [for PC, not Mac] format) a small data set that exemplifies data of the type you are or expect in the future to be handling, and involves some instance of what you consider psychological measurement. This can be real or made-up data. If real, no subject names or identifying information should be included. If made-up, the data can be randomly generated or set up to conform to theoretical expectations. For our purposes, "small data set" means no more than 1,000 data points (e.g., 25 cases and 40 variables, or 200 cases and five variables). If possible, the variables should be "items" rather than scores based on aggregates of items. After you submit your data set, the instructor may ask you to make some revisions to your submission to make it more useful for classroom examples.
2. Discussion questions based on readings for the current week. You are responsible for turning in sets of discussion questions based on the readings by the beginning of five different Tuesday class meetings (of the eight after the first session). For example, questions on Week 2 readings are due by the Tuesday class meeting in Week 2. Late discussion questions don't confer credit. Discussion questions are turned in via e-mail to gsaucier@oregon. Discussion questions, to be worthwhile and to count, should (a) be indicative of having done the reading and (b) be instances of some degree of critical or insightful thinking. Should you ever develop a "block" about coming up with some, you might consider questions of the following form: Why is this issue important? How are you defining ? Aren't you assuming _____? Isn't it debatable whether _____? Does the evidence really support the notion that _____? Aren't you leaving out _____? Isn't there a limitation with regard to _____ (e.g., caused by using that methodology)? Responses to selected discussion questions turned in by Tuesday will be a part of the Thursday sessions, starting with week 2.
3. A final paper. Students will be asked to identify a psychological model or measure and discuss basic measurement issues with respect to it. A set of generic questions that should be addressed in the final paper will be made available by week 6. The model or measure chosen may be one with which the student has experience, or one in which the student has a particular interest. Selected research-literature references are likely to be useful in the final paper, although none is strictly required. The final paper is due at the beginning of the final-exam time for the course (Monday, Dec. 4, 8 am).
4. A brief presentation based on the final paper (or at least on your early drafts of this paper) in the latter half of November (during one of the last five class sessions). Your presentation should be focused on questions, difficulties, puzzles, or dilemmas you are experiencing with respect to the content of your final paper (after providing a bit of background). The brief presentation is primarily an opportunity to get some feedback from the instructor and other class members on the issues involved. These presentations will be allotted about 10-20 minutes each, depending partly on the number of students enrolled.
The final grade is based on: 35% for turning in five sets of discussion questions and a generally acceptable level of in-class contribution, 5% for submitting a data example, 10% for the brief presentation, and 50% for the final paper.
Course Calendar and Readings
Week 1: September 26-28
* Introduction to the course and to psychological measurement
Readings for Week 1: NB [Nunnally & Bernstein] chapter 1
Week 2: October 3-5
* Scaling and validity
Readings for Week 2: NB chapters 2-3; article by Messick (1988)
Week 3: October 10-12
* Cross-cultural generalizability; review of correlation and regression statistics; linear combinations
Readings for Week 3: article by Rogler; NB chapters 4-5
Week 4: October 17-19
* Theory of measurement error
Readings for Week 4: NB chapter 6
Week 5: October 24-26
* Reliability assessment; conventional test construction
Readings for Week 5: NB chapters 7-8
Week 6: October 31-November 2
* Special problems and limitations in classical test theory
Readings for Week 6: Loevinger article; NB chapter 9; Edwards article
Week 7: November 7-9
* Item response theory and tailored tests; uses of factor analysis
Readings for Week 7: NB chapter 10 and pages 445-454 of chapter 11; additional readings TBA
Week 8: November 14-16
* An introduction to exploratory factor analysis and its most important applications in psychological measurement
Readings for Week 8: Goldberg & Digman article; NB chapter 11 (p. 454 to end), 12
Week 9: November 21
* Confirmatory factor analysis and its applications in psychological measurement
Readings for Week 9: NB chapter 13; additional readings TBA
Week 10: November 28-30
* Applying profile analysis, discriminant analysis, and multidimensional scaling in psychological measurement
Readings for Week 10: NB chapter 14
Final paper is due at the beginning of final exam date/time (Monday, Dec. 4, 8 am) – there is otherwise no final examination for this course
References for readings listed above:
Edwards, A. L. (1953). The relationship between the judged desirability of a trait and the probability that the trait will be endorsed. Journal of Applied Psychology, 37, 90-93
Goldberg, L. R., & Digman, J. S. (1994). Revealing structure in the data: Principles of exploratory factor analysis. In S. Strack & M. Lorr (Eds.), Differentiating normal and abnormal personality (pp. 216-242). New York: Springer.
Loevinger, J. (1954). The attenuation paradox in test theory. Psychological Bulletin, 51, 493-504.
Messick, S. (1988). The once and future issues of validity: Assessing the meaning and consequences of measurement. In H. Wainer & H. I. Braun (1988), Test validity (pp. 33-45). Hillsdale, NJ: Erlbaum.
Rogler, L. H. (1999). Methodological sources of cultural insensitivity in mental health research. American Psychologist, 54, 424-433.