Research Framework - Modeling and Design
Test-Centered Research
Test Design
There are a wide variety of factors, both statistical and substantive, that must be carefully considered in designing a test.
Examples of substantive considerations include delineation and description of the knowledge and skills to be assessed, relating these descriptions to the different kinds of tasks to be used on the assessment instrument, and developing scoring rubrics for open-response tasks.
Examples of statistical considerations include standard error for the test scores, score distribution for a test, and the desired accuracy of classification for tests used to classify students according to pre-set standards.
Research into either statistical or substantive characteristics, or both, is included in this area. Because test design is a complex enterprise, such research is usually focused on a narrow component of this area, although a successful testing program must, of course, include all components.
Test Assembly
Even though test assembly must be strongly linked with test design, we separate them here by recognizing that test assembly cannot be carried out until a test design "blueprint" has first been completed. Thus, test assembly is conceptualized here as the concrete operationalization and realization of a more abstract test design.
A fully elaborated test design describes all aspects of a test in the abstract, but it won't tell which specific items are the test items. Test assembly is the gathering of specific items to form an actual test. The process for doing this must naturally include, of course, both substantive and statistical components, because test design includes them and test assembly is the concrete implementation of test design.
Research
Skills Diagnosis Using IRT-Based Latent Class Models, by Louis A. Roussos, Jonathan L. Templin, and Robert A. Henson (2007), from Journal of Educational Measurement, Vol. 44, No. 4, pp. 293–311
Visiting Scholar
Wim van der Linden (abstract of presentation)
