°ÄÃÅÁùºÏ²ÊÔ¤²â

About 

This gathering of mathematics and science education researchers and assessment experts was designed to look at the theoretical and technical requirements of assessments to gauge progress on learning progressions and trajectories in the context of classroom practices. Participants came together to share current practices, and discuss the contexts in which they work, approaches to measurement from their research, and sources of validity evidence they have evaluated.

Over the two-day meeting, the group discussed the following topics:

• essential features in learning progressions/trajectories and their similarities and differences,
• underlying decisions in assessment design and considerations for trade-offs,
• construct operationalization using learning progressions/trajectories, and
• valid representations of constructs based on evidence.

Questions around validity of the uses and interpretations of assessment results and operationalizing the construct through item design were considered.

• What validation frameworks are being used? To what degree of success? Challenges?
• What classroom-based decisions are particularly relevant for assessment results based on learning progressions/trajectories?
• How do you communicate results to users to facilitate these decisions?
• How do you train users to use the data?
• What are your item and test design specifications (e.g., Item format, test format, scoring protocols)? How do these support classroom assessment practices?
• What validity evidence is needed to evaluate the test specifications for fidelity/integrity to the learning progressions/trajectory?

Considerations of psychometric modeling were discussed and included the following questions.

• What are the trade-offs and decisions related to: latent trait vs latent classification modeling; item dependencies; grain size vs item sampling?
• How/should we think differently about psychometric characteristics for classroom assessments?
• If IRT-based parameters show non-compliance with the learning progression/trajectory, what conclusions can be drawn?
• What is the role of falsification?

Date

May 14-15, 2018

Hosts:

Dr. Leanne Ketterlin Geller, Professor and Texas Instruments Chair in Education; Director, Research in Mathematics Education, °ÄÃÅÁùºÏ²ÊÔ¤²â

, Joseph D. Moore Distinguished University Professor Science, Technology, Engineering and Mathematics (STEM) Department, North Carolina State University

About Our Experts

• Dr. Elizabeth Adams, STEM Evaluation Researcher, Research in Mathematics Education, Simmons School of Education and Human Development, °ÄÃÅÁùºÏ²ÊÔ¤²â
• Dr. Alicia Alonzo, Associate Professor, Michigan State University
• Dr. Michele Carney, Associate Professor Mathematics Education, Boise State University
• Dr. Rick Duschl, Waterbury Chair Professor of Secondary Education, Department of Curriculum and Instruction, Penn State University
• Dr. Candace Joswick, Post-Doctoral Fellow, University of Denver
• Dr. Aki Kamata, Professor and Executive Director, Center on Research and Evaluation (CORE), °ÄÃÅÁùºÏ²ÊÔ¤²â 
• Dr. Joanne Lobato, Professor of Mathematics Education; Department of Mathematics and Statistics, San Diego State University
• Dr. Aurora Edith Graf, Research Scientist; Center for Learning and Teaching, Educational Testing Service (ETS)
• Dr. Emily Lai, Interim Vice President, Impact Evaluation, Pearson
• Dr. Tyler Matta, Senior Research Director, Education Portfolio, Pearson
• Dr. Paul Nichols, Director, Large-Scale Assessment Designs, Northwest Evaluation Association
• Dr. Lindsey Perry, Research Assistant Professor, Research in Mathematics Education, Simmons School of Education and Human Development, °ÄÃÅÁùºÏ²ÊÔ¤²â
• Dr. Julie Sarama, Professor, Kennedy Endowed Chair, Innovative Learning Technologies, University of Denver
• Dr. Meetal Shah, Graduate Student Researcher, North Carolina State University
• Dr. Emily Toutkouthian, Psychometrician and Assessment Specialist, North Carolina State University
• Dr. Annie Wilhelm, Assistant Professor of Mathematics Education, Simmons School of Education and Human Development, °ÄÃÅÁùºÏ²ÊÔ¤²â