EDMS 738, Spring 2008: Cognitive Psychology & Assessment

Texts

Required

Mislevy, R.J. (2006). Cognitive psychology and educational assessment. In R. L. Brennan (Ed.), Educational measurement (4th ed.). Westport, CT: American Council on Education/Praeger. Preprint available on line at http://www.education.umd.edu/EDMS/mislevy/papers/CogPsych&Assmt11-11-2005.doc (password required)

National Research Council (2002). Knowing what students know. J. Pellegrino, N. Chudowsky, & R. Glaser (Eds.). Washington, D.C.: National Academy Press. Available online at http://www.nap.edu/books/0309072727/html/

Supplemental

Ericcson, K.A., & Smith, J. (Eds.) (1991)Toward a general theory of expertise. Cambridge, England: Cambridge University Press. One copy on reserve in EDMS office.

Common Readings

(password protected unless otherwise noted)

Anderson, J.R., Greeno, J.G., Reder, L.M., & Simon, H.A. (2000). Perspectives on learning, thinking, and activity. Educational Researcher, 29, 11-13.

Anderson, J.R., Reder, L.M., & Simon, H.A. (1996). Situated learning and education. Educational Researcher, 25, 5-11.

Anderson, J.R., Reder, L.M., & Simon, H.A. (1997). Situative versus cognitive perspectives: Form versus substance. Educational Researcher, 26, 18-21.

Baxter, G. P., Elder, A. D., & Glaser, R., (1996). Knowledge-based cognition and performance assessment in the science classroom. Educational Psychologist, 31 (2), 133-140.

Brown, J. S., Collins, A., & Duguid, P. (1989). Situated cognition and the culture of learning. Educational Researcher, 18(1), 32-42.

Carroll, J.B. (1976). Psychometric tests as cognitive tasks: A new “structure of intellect”. In L.B. Resnick (Ed.), The nature of intelligence (pp. 27-56). Hillsdale, NJ: Erlbaum.

Collins, A., & Ferguson, W. (1993). Epistemic forms and epistemic games: Structures and strategies to guide inquiry. Educational Psychologist, 28, 25-42.

Embretson, S.E. (1998). A cognitive design system approach to generating valid tests: Application to abstract reasoning. Psychological Methods, 3, 380-396.

Ericsson, KA, & Charness, N. (1994). Expert performance: Its structure and acquisition. American Psychologist, 49, 725-747.

Gee, J.P. (n.d.). A sociocultural perspective on opportunity to learn. Paper prepared for a Spencer funded group discussing issues of assessment and testing.

Gipps, C. (1999). Socio-cultural aspects of assessment. Review of Research in Education, 24, 355-392.

Greeno, J.G. (1989). Situations, mental models, and generative knowledge. In D. Klahr & K. Kotovsky (Eds.), Complex information processing (pp. 285-318).

Greeno, J.G. (1997). On claims that answer the wrong questions. Educational Researcher, 26, 5-17.

Greeno, J.G., Collins, A.M., & Resnick, L.B. (1997). Cognition and learning. In D. Berliner & R. Calfee (Eds.), Handbook of educational psychology (pp. 15-47). New York: Simon & Schuster Macmillan.

Hendrickson, A., & Mislevy, R.J. (2005). Cognitively Based IRT Models. In B. Everitt & D. Howell (Eds.), Encyclopedia of Statistics in Behavioral Science, Volume 2 (pp. 978-982). New York: Josset-Bass/Wiley

Hively, W., Patterson, H.L., & Page, S.H. (1968). A "universe-defined" system of arithmetic achievement tests. Journal of Educational Measurement, 5, 275-290.

Katz, I.R. (1994). Coping with the complexity of design: Avoiding conflicts and prioritizing constraints. In A. Ram, N. Nersessian, & M. Recker (Eds.), Proceedings of the Sixteenth Annual Meeting of the Cognitive Science Society (pp.485-489). Hillsdale, NJ: Erlbaum.

Kintsch, W. (1988). The role of knowledge in discourse comprehension construction-integration model. Psychological Review, 95, 163-182.

Mislevy, R.J. (2003). Substance and Structure in Assessment Arguments. Law, Probability, and Risk, 2, 237-258.

Mislevy, R.J. (2006). Issues of Structure and Issues of Scale in Assessment from a Situative/Sociocultural Perspective. CSE Technical Report 668. Los Angeles: The National Center for Research on Evaluation, Standards, Student Testing (CRESST), Center for Studies in Education, UCLA.

Mislevy, R.J, et al. (2007). On the Roles of External Knowledge Representations in Assessment Design. CSE Technical Report 722. Los Angeles: The National Center for Research on Evaluation, Standards, Student Testing (CRESST), Center for Studies in Education, UCLA.

Mislevy, R.J., & Gitomer, D.H. (1996). The role of probability-based inference in an intelligent tutoring system. User-Modeling and User-Adapted Interaction, 5, 253-282.

Mislevy, R.J., & Rahman, T. (2007). A design pattern for assessing cause and effect reasoning in reading comprehension. Draft PADI Technical Report.

Mislevy, R.J., & Risconscente, M. (2006). Evidence-centered assessment design: Layers, concepts, and terminology. In S. Downing & T. Haladyna (Eds.), Handbook of Test Development.  Mahwah, NJ: Erlbaum.

Mislevy, R.J., Steinberg, L.S., Almond, R.G., Haertel, G.D., & Penuel, W.R. (2003). Leverage points for improving educational assessment. In B. Means & G. Haertel (Eds.), Evaluating the effects of technology in education (pp. 149–180). New York: Teachers College Press. Link is to version is publically available online version, CSE Technical Report 534. Los Angeles: The National Center for Research on Evaluation, Standards, Student Testing (CRESST), Center for Studies in Education, UCLA.

Newstead, S., Bradon, P., Handley, S., Evans, J., & Dennis, I. (2002). Using the psychology of reasoning to predict the difficulty of analytical reasoning problems. In S.H. Irvine & P.C. Kyllonen (Eds.), Item generation for test development (pp. 35-52). Mahwah, NJ: Erlbaum.

PADI Research Group (2004). Design patterns for assessing science inquiry. Menlo Park, CA: SRI, International. Link is to publically available PADI Tech Report.

Rumelhart, D.A. (1980). Schemata: The building blocks of cognition. In R. Spiro, B. Bruce, & W. Brewer (Eds.), Theoretical issues in reading comprehension (pp. 33-58). Hillsdale, NJ: Erlbaum.

Rupp, A.A., & Mislevy, R.J. (2007). Cognitive foundations of structured item response models. In J.P. Leighton & M. J. Gierl (Eds.), Cognitive Diagnostic Assessment: Theories and Applications. Cambridge: Cambridge University Press.

Saxe, G.B. (1988). Candy selling and math learning. Educational Researcher, 17(6), 14-21.

VanLehn, K. (1990). Problem-solving and cognitive skill acquisition. In M. Posner (Ed.), The foundations of cognitive science (pp. 527-580). Cambridge, MA: MIT Press.

Individual Readings

(A student summarizes for the class)

*Allard, F., & Starkes, J.L. (1991). Motor-skill experts in sports, dance, and other domains. In K.A.Ericcson & J. Smith (Eds.), Toward a general theory of expertise (126-152). Cambridge, England: Cambridge University Press.

*Anzai, Y. (1991). Learning and use of representations for physics expertise. In K.A.Ericcson & J. Smith (Eds.), Toward a general theory of expertise (64-92). Cambridge, England: Cambridge University Press.

Bao, L., & Redish, E.F. (n.d.). Model analysis: Assessing the dynamics of student learning. Unpublished manuscript.

*Bejar, I.I. (1993). A generative approach to psychological and educational measurement. In N. Frederiksen, R.J. Mislevy, & I.I. Bejar (Eds.), Test theory for a new generation of tests (pp. 323-357). Hillsdale, NJ: Erlbaum.

*Camerer, C.F., & Johnson, E.J. (1991). The process-performance paradox in expert judgment: How can experts know so much and predict so badly? In K.A. Ericcson & J. Smith (Eds.), Toward a general theory of expertise (195-217). Cambridge, England: Cambridge University Press.

*Charness, N. (1991). Expertise in chess: The balance between knowledge and search. In K.A.Ericcson & J. Smith (Eds.), Toward a general theory of expertise (38-63). Cambridge, England: Cambridge University Press.

Delandshere, G. (2002). Assessment as inquiry. Teachers College Record, 104, 1461-1484.

*Dorner, D., & Scholkopf, J. (1991). Controlling complex systems; or, Expertise as “grandmother’s know-how”. In K.A.Ericcson & J. Smith (Eds.), Toward a general theory of expertise (218-239). Cambridge, England: Cambridge University Press.

Douglas, D. (2001). Three problems in testing language for specific purposes: Authenticity, specificity, and inseparability. In C. Elder, A. Brown, E. Grove, K. Hill, N. Iwashita, T. Lumley, T. McNamara, and K. O'Loughlin (Eds.), Experimenting with Uncertainty: Essays in Honour of Alan Davies (pp. 45-52). Cambridge: Cambridge University Press.

Frederiksen, J., & Collins, A. (1989). A systems approach to educational testing. Educational Researcher, 18 (9), 27–32.

*Frederiksen, J.R., & White, B.Y. (1988). Implicit testing within an intelligent tutoring system. Machine-Mediated Learning, 2, 351-372.

*Glaser, R. (1963). Instructional technology and the measurement of learning outcomes: Some questions. American Psychologist, 118, 519-521.

*Greeno, J.G. (1976). Cognitive objectives of instruction: Theory of knowledge for solving problems and answering questions. In D. Klahr (Ed.), Cognition and instruction (pp. 123-159). Hillsdale, NJ: Erlbaum.

Hurst, K., Casillas, A., & Stevens, R.H. (1997). Exploring the dynamics of complex problem-solving with artificial neural network-based assessment systems. CSE Technical Report 444. Los Angeles: The National Center for Research on Evaluation, Standards, Student Testing (CRESST), Center for Studies in Education, UCLA.

*Kindfield, A.C.H. (1999). Generating and using diagrams to learn and reason about biological processes. Journal of Structural Learning and Intelligent Systems, 18, 81-124.

Mislevy, R.J., Steinberg, L.S., & Almond, R.A. (2002). Design and analysis in task-based language assessment. Language Assessment, 19, 477-496. Link is to version is publically available online version, CSE Technical Report 579. Los Angeles: The National Center for Research on Evaluation, Standards, Student Testing (CRESST), Center for Studies in Education, UCLA.

Mislevy, R.J., Steinberg, L.S., Breyer, F.J., Almond, R.G., & Johnson, L. (1999). A cognitive task analysis, with implications for designing a simulation-based assessment system. Computers and Human Behavior, 15, 335-374. Link is to version is publically available online version, CSE Technical Report 487. Los Angeles: The National Center for Research on Evaluation, Standards, Student Testing (CRESST), Center for Studies in Education, UCLA.

Myford, C.M., & Mislevy, R.J. (1996). Monitoring and improving a portfolio assessment system. CSE Technical Report 402. Los Angeles: National Center for Research on Evaluation, Standards, and Student Testing (CRESST).

*Patel, V.L., & Groen, G.J. (1991). The general and specific nature of medical expertise: A critical look. In K.A.Ericcson & J. Smith (Eds.), Toward a general theory of expertise (93-125). Cambridge, England: Cambridge University Press.

Redish, E.F. (2004). A theoretical framework for physics education research: Modeling student thinking. In E. F. Redish and M. Vicentini (eds.) Proceedings of the International School of Physics, "Enrico Fermi" Course CLVI. Amsterdam: IOS Press.

*Resnick, L.B. (1997). Student performance portfolios. In H.J.Walberg & G.D. Haertel(Eds.), Psychology and educational practice (pp. 158-175). Berkeley: McCutchan.

Rupp, A.A., Ferne, T., & Choi, H. (in press). How assessing reading comprehension with multiple-choice questions shapes the construct: A cognitive processing perspective. Language Testing.

*Salthouse, T.A. (1991). Expertise as the circumvention of human processing limitations. In K.A.Ericcson & J. Smith (Eds.), Toward a general theory of expertise (286-300). Cambridge, England: Cambridge University Press.

*Scardamalia, M., & Bereiter. (1991). Literate expertise. In K.A.Ericcson & J. Smith (Eds.), Toward a general theory of expertise (172-194). Cambridge, England: Cambridge University Press.

*Simon, H. (1975).  The functional equivalence of problem solving skills.  Cognitive Psychology, 7, 268-288.

*Singley, K., & Bennett, R.E. (2002). Item generation and beyond: Applications of schema theory to mathematics assessment. In S.H. Irvine & P.C. Kyllonen (Eds.), Item generation for test development (pp. 361-384). Mahwah, NJ: Erlbaum.

*Sloboda, J. (1991). Musical expertise. In K.A.Ericcson & J. Smith (Eds.), Toward a general theory of expertise (153-171). Cambridge, England: Cambridge University Press.

*Steinberg, L.S., & Gitomer, D.G. (1996). Intelligent tutoring and assessment built on an understanding of a technical problem-solving task. Instructional Science, 24, 223-258.

* Available in ETS reserve window, in basement of Benjamin