Syllabus for
EDMS 738, Spring 2008
Cognitive Psychology and Assessment
Prof. Robert Mislevy
1230-C Benjamin Building
rmislevy@umd.edu
Course Description: This course explores the implications of recent developments in cognitive psychology for educational assessment. While it is not a psychology course per se, we will survey key topics such as expertise research, knowledge representations, information-processing psychology, and the sociocultural perspective on learning. The focus will be on how these ideas are used in designing and using assessments. We will use ‘evidence-centered assessment design’ to frame our work as we analyze a range of examples, including performance assessment, portfolio assessments, simulation-based tasks, and intelligent tutoring systems.
The class meets once a week. The first part of the class will usually be a presentation by the instructor, followed by discussion of the readings and of examples from readings or provided by students, and sometimes group work in class. Each student will be responsible for summarizing one of the readings for the class sometime during the semester, and hosting the discussion of that reading. In addition to this summary, additional assignments will be short papers or essay questions, approximately every other week, based on the readings and examples. The final project will be the analysis of an assessment of the student’s choice (or alternative topic, with instructor's permission).
Prerequisites: EDMS 623 (intermediate-level educational
measurement: classical test theory, reliability, validity, etc.) and EDMS
651(Intermediate-level statistics: regression, anova),
or instructor permission.
Calendar:
Class |
Date |
Topic |
Readings |
Assignment |
1 |
1/28 |
Introduction to class / Introduction to cognitive psychology |
KWSK, Chap. 1-2; Mislevy, Almond, & Lukas |
|
2 |
2/4 |
Connectionism |
CPEA: Section 1 & pp. 33-40, Kintsch (1988), Mislevy & Rahman
|
1 paragraph about you |
3 |
2/11 |
Psychology & assessment arguments 1 |
CPEA, Section 2; KWSK, Chap. 3; Greeno, Collins, & Resnick (1997); Mislevy, Steinberg, Almond, Haertel, & Penuel (2003) |
|
4 |
2/18 |
Psychology & assessment arguments 2 |
Mislevy & Risconcente; CPEA, Section 3; Mislevy (2003); Newstead et al (2002); *Bao & Redish (n.d.); *Holyoak (1991); *TGTE |
|
5 |
2/25 |
Expertise Research |
Ericcson & Charness (1994) Katz (1994); *Mislevy, Steinberg, Breyer, Almond, & Johnson (1999); *Camerer & Johnson (1991);* Steinberg, L.S., & Gitomer, D.G. (1996) |
Questions based on expertise research |
6 |
3/3 |
Design patterns for assessment |
Design patterns
(PADI Research Group); Baxter, Elder, & Glaser (1996)
|
|
7 |
3/10 |
Trait and behavioral perspectives |
Carroll (1976); Hively, Patterson, & Page (1968); *Glaser (1963); *TGTE; *Salthouse (1991); *Douglas (2001) |
Commentary for Carroll (1976) |
|
3/17 |
Spring Break—no class |
|
|
|
3/24 |
AERA—no class |
|
|
8 |
3/31 |
The information processing perspective |
Embretson (1998); VanLehn (1989); *Simon (1975); *Greeno(1976); *Larkin & Simon (1987); *Mayer, Larkin, & Kadane (1984) |
Commentary for VanLehn (1989) |
9 |
4/7 |
Cognitive diagnosis |
CPEA, Section 4; Rupp & Mislevy (2007); Hendrickson & Mislevy (2005); Mislevy & Gitomer (1996); *Frederiksen & White (1988); *Mislevy, Steinberg, & Almond (2002); *TGTE |
|
|
4/14 |
Schemas & Knowledge representations |
|
|
11 |
4/21 |
The sociocultural/situative perspective |
Brown, Collins, & Duguid (1989); Saxe (1988); Gipps (1999); *Resnick (1997); *Delandshire (2002); *Myford & Mislevy (1996) |
|
12 |
4/28 |
Info processing vs sociocultural debate |
Anderson, Reder,
& Simon (1996 & 1997); Greeno (1997); Anderson, Greeno, Reder, &
Simon (2000); Mislevy (2006); *TGTE; *Frederiksen & Collins (1989);
*Hurst, et al. (1997)
|
Commentary on info processing vs sociocultural debate |
13 |
5/5 |
Theory-based task design |
*Kindfield (1999); *Singley & Bennett (2002); *Bejar (1993);*TGTE |
|
14 |
5/12 |
Theory-based task design, continued |
*Kindfield (1999); *Singley & Bennett (2002); *Bejar (1993);*TGTE |
|
|
|
Finals week Project due 5/21 |
|
Final project: Analysis + (design pattern and/or Toulmin diagram), or approved alternative |
Boldface indicates focus reading for class discussion
* indicates a paper that a student will summarize for the class.
CPEA = Cognitive psychology and educational assessment, Mislevy
KWSK = Knowing what students know, National Research Council (2002); Pellegrino, Chudowsky, & Glaser (Eds.)
*TGTE = A “class of expertise” chapter from Toward a general theory of expertise, Ericcson & Smith (Eds.) (1991): Allard & Starkes on motor skills, Anzai on physics, Charness on chess, Dorner & Scholkopf on controlling complex systems, Patel & Groen on medical expertise, Scardamalia & Bereiter on literate expertise, Sloboda on musical expertise.
Course Evaluation:
There are nine assignments: two question sets, five short papers, one in-class presentation, and a term paper. Their weights in your grade are 10% for each short assignment and presentation, and 20% for the term paper. All assignments will be submitted by email. Unless specified otherwise, assignments are due by midnight Friday of the week they appear in the course calendar.
The final project is an analysis of an assessment or some portion thereof, real or hypothetical, from the perspectives we have developed in the course. I am looking for your use of the concepts, terms, and schemas. You will discuss a particular assessment argument is it exists in an assessment, or as you would create an assessment based on an experiment or a research paper you have chosen that would ground an assessment. Examples are assessments you have created yourself for a class you have taught, assessments you have worked on or are currently designing in your job, assessments you have taken, assessments in the news, and assessments you can conceive of based on a psychological experiment, instructional program, or perspective on learning. KWSK and Greeno et al. (1997) list four perspectives on the nature of knowledge and its acquisition, namely behavioral, trait/differential, information-processing, and sociocultural/situative. These basic perspectives, bolstered by examples and elaboration in our readings (e.g., on the nature of expertise), should be used to discuss the purpose and perspective of your topic material, and the rationale underlying the assessment argument it represents. Regardless of the perspective under which an assessment may have been designed, it should be possible to make observations yourself from at least two or three perspectives, if not all four. The evidence-centered design (ECD) framework for assessment provides concepts and structures for you to discuss the evidentiary argument(s) underlying your topic material.
With the instructor’s permission, an alternative topic for a class project may be substituted, as long as it fits in with the objectives of the course.
Grading: Assignments will be graded on a 0 to 4 point scale where:
4=Outstanding (Good effort, on target, and complete performance)
3=Good (Good effort shown in work. Only minor problems in performance)
2=Acceptable (The performance is moderately flawed, but is acceptable.)
1=Unacceptable (The performance is not indicative of graduate level work, is severely flawed or is indicative on a substandard level of effort.)
0=Assignments that are not turned in or are indicative of such bad performance that they should not have been turned in.
Final Grades will be calculated by weighting each individual assignment grade according to the weights defined in the previous section. Numeric grades will be translated as follows:
A= 3.00001 - 4.0
B= 2.50001 - 3.0
C= 1.75001 - 2.5
D= 0.75001 - 1.75
F = 0.00000 - 0.75
Please note that there will be no opportunity for “extra credit”. Grades will be determined by the scheme outlined above - no exceptions.
Late Assignments: Homework should be submitted by midnight Friday of the week it is due unless special arrangements are made with me in advance. Moreover, special arrangements will only be instituted under extenuating circumstances, and thus, assignments should generally be turned in on time. If an assignment is turned in late without advanced approval, then the grade received for the assignment will be automatically decreased by 1 point for each week or portion thereof it is late.
Grades of “Incomplete”: A grade of incomplete will generally not be allowed except in cases of extreme hardship.
Honor System: Each student is expected to complete all assignments independently except as otherwise allowed--e.g., with my prior approval, working in groups on papers or projects. There will be plenty of opportunity in class to discuss your projects and assignments, and these discussions may continue outside class. However, the write-up must be your own work, and you are expected to show that you understand it in your in-class summary of the assignment. If you need help with some aspect of your assignment, then come to me and I will assist you to a reasonable extent.
The University of Maryland, College Park has a nationally recognized Code of Academic Integrity, administered by the Student Honor Council. This Code sets standards for academic integrity at Maryland for all undergraduate and graduate students. As a student you are responsible for upholding these standards for this course. It is very important for you to be aware of the consequences of cheating, fabrication, facilitation, and plagiarism. For more information on the Code of Academic Integrity or the Student Honor Council, please visit http://www.studenthonorcouncil.umd.edu/code.html.
Accommodations: If you need academic accommodation by virtue of a documented disability, please contact me as soon as possible to discuss your needs. Students with documented needs for such accommodation must meet the same achievement standards required of all other students, although the exact way in which achievement is demonstrated may be altered. If you would like academic accommodation by virtue of your religion (e.g., turning in homework at a time other than the due date because that date falls on a religious holiday), then please contact me as soon as possible to discuss your request. All requests for academic accommodations should be made within two weeks of the start of class.
Auditors: For individuals who are auditing the course can attend class as much or little as they would like, reading the assigned material and performing the homework problems is encouraged, but not required.