Early Childhood Assessment System Development

The Early Childhood Assessment System (ECAS), piloted as the Indiana Assessment System of Educational Proficiencies: Early Childhood (IASEP: EC), is an integrated, computer-based assessment system designed to assess the abilities of young learners in Early Childhood education programs. ECAS is an extension of the Indiana Assessment System of Educational Proficiencies (IASEP), the alternate assessment system developed by Purdue University and the Indiana Department of Education Division of Special Education (Bennett, Davis, Cunningham, & Arvidson, 1999) to meet the mandate of the Individuals with Disabilities Act Amendments of 1997 (PL 105-17) that all students be included in statewide assessment and accountability systems.

IASEP was originally developed for students with the most significant disabilities, but it became clear that its broad-based model provided a solid foundation for assessing the skills of learners with a wide range of abilities and disabilities. The piloting and statewide implementation of IASEP raised awareness of the need to develop an assessment system that could document the educational progress of young learners with or at risk for developmental delays. Purdue University and the Indiana Department of Education Division of Special Education worked with stakeholders to create a system that would meet the assessment needs of young learners and provide accountability data. ECAS capitalizes on the strengths of IASEP. It provides teachers with a system for ongoing assessment of core and extended skills and the capacity to create and enter supporting electronic documentation that can be used to guide educational programming and individual instruction for children during their earliest years of development.

Stakeholders began meeting during the 1998-1999 school year. They generated belief statements to guide them. First and foremost, stakeholders wanted the system to "honor the belief that all children have value, can learn with appropriate supports, and can be expected to make measurable gains." Stakeholders looked at assessment instruments and preschool curricula at the beginning of the skill-selection process. They added skills from their own experience as educators generating a total of 1,233 skills. Two separate content validation processes reduced the number of skills to the 383 skills that were used in the pilot. The assessment system was organized as a hierarchy of domains, subdomains, proficiencies, skills, and tasks. Skills fell under five domains: Cognitive, Communication, Social, Self-Help, and Sensory Motor.

A rubric which included the terms "Independent," "Functional Independence," "Supported Independence," "Emergent," and "Participation" was developed to document level of performance on skills. A Participation Index was included to define levels of participation so that small increments of growth and development could be noted. The system provided for rating each of the skills and then supporting the ratings with electronic documentation in the form of audio clips, video clips, and digital images.

Pilot Project

Forty teachers and speech-language pathologists from eight sites around the state of Indiana piloted the system in Early Childhood education programs during the 2000-2001 school year. In the fall, they attended two-day training sessions to become familiar with the system. Staff from Purdue University provided additional training and support throughout the year. A parent liaison met with parents.

Teachers were asked to rate all of the 383 skills on laptop computers and support their ratings with electronic documentation. They entered information related to their children's use of augmentative and alternative communication (AAC) and assistive technology (AT). They also entered Individualized Education Program (IEP) goals and objectives. Data were collected and analyzed for 129 young learners. The 20 skills with the highest item-total correlations were organized within each of the five domains to create a set of 100 core skills. These core skills were used in a study to evaluate the validity of the assessment system.

Validation Study

The need for a valid Early Childhood assessment that can assist in the planning and evaluation of educational programs is apparent (Kelley & Surbeck, 1991). Forty two young learners aged 3.5 years to 6.5 years from five of the eight pilot sites participated in a study to evaluate the validity of ECAS. Two staff members from Purdue University and two Early Childhood educators administered the Speed DIAL, a short version of the DIAL-3: Developmental Indicators for the Assessment of Learning, (Mardell-Czudnowski & Goldenberg, 1998), and a Picture Naming Task (Early Childhood Research Institute on Measuring Growth and Development, 1999).

On the Speed DIAL, children demonstrated gross and fine motor skills such as jumping, hopping, building with blocks, and writing in the Motor area; conceptual skills such as counting blocks, identifying concepts in a triad of pictures, and sorting shapes in the Concepts area; and language skills such as answering simple personal questions and identifying and naming objects and actions in the Language area. On the Picture Naming Task, children named as many pictures as they could in one minute from a stack of flash cards.

Data analyses revealed a correlation of .632 on skills in the Speed DIAL Motor area and the ECAS Sensory Motor domain, .850 between performance on skills in the Speed DIAL Concepts area and the ECAS Cognitive Domain, and .828 on skills in the Speed DIAL Language area and the ECAS Communication domain. Correlation between skills in all three areas of the Speed DIAL and core skills in all five domains of ECAS was high at .900. Correlation between core skills in all five domains of ECAS and the Picture Naming Task was .739. Internal consistency reliability estimates for all domains fell above .960 while alpha coefficients for the total scale fell above .990.

Findings suggest that it is possible to attain excellent reliability and validity in an electronic assessment system that utilizes both teacher ratings and performance-based documentation for monitoring the growth and development of young children.

Current Features

ECAS allows teachers to archive ratings and generate a variety of reports. It contains a portfolio component that allows teachers to create electronic pages showing children performing skills in a variety of environments. The number of core skills has been reduced to 10 in each domain. All of the other skills that were rated during the pilot remain in the system as extended skills. Early Childhood Foundations and Indiana Academic Standards for grades K-3 are also in the system. The rubric has been modified to include the terms "Achieved," "Progressing," "Emerging," "Introduced," and "No Opportunity."

Confidentiality of information is paramount to the integrity of ECAS. Passwords and encryption are integral parts of the system. An extensive security protocol has been developed in consultation with the Center for Education and Research in Information Assurance and Security (CERIAS) at Purdue.

References

Bennett, D. E., Davis, M. A., Cunningham, J. N., & Arvidson, H. H. (1999). Indiana Assessment System of Education Proficiencies: Computer-Based Rating and Documentation System (software system, software manual, training videos, training manual, case studies) (IASEP). West Lafayette, IN: Purdue University, Purdue Research Foundation.

Early Childhood Research Institute on Measuring Growth and Development. (1999). Format and stimulus materials for picture naming assessment. Unpublished manuscript, University of Minnesota, Minneapolis.

Kelley, M. F. & Surbeck, E. (1991). History of preschool assessment. In B. A. Bracken (Ed.), The psychoeducational assessment of preschool children (2nd ed., pp. 1-17). Boston: Allyn and Bacon.

Mardell-Czudnowski, C. & Goldenberg, D. S. (1998). DIAL-3: Developmental Indicators for the Assessment of Learning-Third Edition. Circle Pines, MN: American Guidance Service.

Public Law 105-17. (1997). Individuals with Disabilities Education Act Amendments of 1997, Washington, DC: U. S. Congress.

Copyright© 2002 by Purdue Research Foundation. All rights reserved