Directories Search Admissions Calendar Home

University of 
Maryland  
Link 
Bar
Assignment4
 

______________________________________________________

Assignment #4: Psychology and your assessment

Assignment #4: Evidence Evaluation and Your Assessment

The assignment is to analyze your example assessment with respect to evidence identification, or task level scoring. Evidence identification concerns the link(s) in the chain of reasoning that go from a student's work product(s) -- that is, the particular things students say, do, or make -- to the values of the observable variables.

Address the following issues:

What is/are the work product(s) for your assessment? (If there are many, discuss a few representative ones.)

What is the procedure used to examine the work product(s) produced in a given task, and arrive at one or more "scores" for that task? What is the underlying rationale for the procedure, and how is it carried out (e.g., human judgment, computer program, comparison with an answer key)?

Are there alternative ways the work product could be evaluated? (e.g., in an essay, focus on content as opposed to mechanics, or vice versa; in a multiple choice item, right/wrong versus partial credit versus nature of error suggested by incorrect choices).

Are there sources of uncertainty in the evidence evaluation process in your assessment? If so, are they acknowledged? How are they dealt with? Are they large sources of uncertainty? (e.g., in assessments that use human judgment, agreement among independent judges is often examined as a way to check for the quality of information obtained from the rating process).

If there are interesting connections between the evidence identification process used in your assessment and the psychological foundations of your assessment, comment on them (e.g., some writing assessments want both rough drafts and final papers as work products, so it is possible to obtain evidence about revising; this is an important aspect of the writing process as seen from a cognitive perspective. If one only cared about quality of final products, as might be sufficient from a trait perspective, then getting a draft as a work product and evaluating the quality of revisions would less likely be done.)

Your paper should be two to four pages total.

_______________________________________________
 
University of Maryland     UM Home | Directories | Search | Admissions | Calendar
Direct questions and comments to Bob Mislevy.
Last Modified Wednesday, 12-Oct-2005 16:25:31 EDT           © University of Maryland